Imagine the Kladde assessed a tender with 8 points in the spring, and in the summer someone asks why. Since then, you’ve changed the prompt twice and switched the model once. Without versions, the question can no longer be answered. A prompt changes the behavior of the application as much as code does, often more. That’s why it gets what code has long had: versions, a change log and pull requests. In this lesson, you put version 2 next to version 1, switch between them through configuration, and every stored assessment says which version and which model produced it.
What a version is
A prompt version is everything the model receives besides the data of the individual request. For the assessment, that is:
- the text of the system message,
- the examples,
- the way the messages are put together (
TenderMessage,TenderAssessmentPrompt), - the JSON schema that Microsoft.Extensions.AI derives from
TenderAssessment.
If one of these parts changes, the model may answer the same tender differently. So the Kladde’s rule is:
A published version never changes. A version is published as soon as an assessment has been stored with it. Anyone who wants to change something, even just a comma, creates a new version.
The stored versions from T3.1 help here: There is one per version, and if one of them turns red, you’ve changed a published version, even
if you only touched shared code such as TenderMessage. The test doesn’t check the schema, because it isn’t in the messages. A change to
TenderAssessment affects all versions at once and is therefore a bigger decision with a measurement of its own.
Two things don’t belong to the prompt version but are recorded anyway:
- the model: The same prompt behaves differently on a different model. The Kladde stores the model next to the version.
- the settings of the call: In the Kladde, the temperature is fixed at 0 in the code. If it should ever change along with the prompt, it belongs in the header of the prompt file.