Understanding and generating images

Travelers upload photos of hotel rooms; zimmerfoto.jpg stands for one of them. This lesson shows three things: putting a photo into a prompt and asking the model about it, generating new images with a model, and the status of the Azure Vision service. The code is not run against Azure; this is how Microsoft Learn shows it. Results, images and answers are not printed.

Understanding images in the prompt

Learn describes vision-enabled chat models: large multimodal models that evaluate images and answer questions about them. As current examples, the page lists the GPT-5 series, the GPT-4.1 series, the GPT-4o series and the o series. The model behind reise-chat can therefore be vision-enabled if it belongs to one of these series (availability in your region: open). You call the same chat interface; the content of the message is a list of text parts and image parts.

Limits that Learn names:

• An image may be at most 20 MB, and a call may contain at most 10 images.
• Formats: JPEG, PNG, GIF (first frame only) and WEBP.
• The image comes as a publicly reachable address or as Base64 text. Addresses behind private endpoints, in a VNet or behind a firewall do not work, even if the file is in your own storage.
• The "low resolution" setting saves compute, but can make the recognition of objects and text worse.
• A maximum for the length of the answer (max_tokens) must be set, otherwise the output is cut off.

The point about the address concerns Nordlicht directly: Photos of guests should not be publicly on the net just so that a model can fetch them. Learn's C# example works with an address; for files, Learn names the Base64 option in the REST description of the same page. How to write it in C#, Learn does not show there: open.

Learn's C# example

Excerpt (lesson 2.3 used the Foundry SDK; this example uses the package Azure.AI.OpenAI on Learn, a different entry point). The names YOUR_IMAGE_URL and the deployment name are placeholders from Learn; instead of gpt-4, the deployment reise-chat stands here:

string deploymentName = "reise-chat";

var chatClient = openAIClient.GetChatClient(deploymentName);

var imageUrl = "YOUR_IMAGE_URL";

var textPart = ChatMessageContentPart.CreateTextPart("Describe this picture:");
var imgPart = ChatMessageContentPart.CreateImagePart(imageUrl);

var chatMessages = new List<ChatMessage>
{
    new SystemChatMessage("You are a helpful assistant."),
    new UserChatMessage(textPart, imgPart)
};

ChatCompletion chatCompletion = await chatClient.CompleteChatAsync(chatMessages);

Console.WriteLine($"{chatCompletion.Content[0].Text}");

The system message sets the role (lesson 2.2), the user message carries text and image. The call waits for the answer and prints its text. Learn describes the output only as a description of the image.

For Nordlicht, a sensible question would be: "Which furniture and what condition of the room can be seen?" The answer is no substitute for a check: Whether there is a defect is judged by a person. The helper does not ask questions about people in the photo.

Generating images

For new images, Learn names the models of the gpt-image series. The earlier model dall-e-3 was retired according to Learn: Existing deployments no longer work, new ones are not possible. Learn names, for example, gpt-image-2 as generally available, older models of the series only with access by request ("limited access preview"). What applies on the day of writing changes quickly: open for the time at which you read.

The images come back as Base64 data. Excerpt from Learn's C# quickstart; the deployment name here is reise-bild (new for this case), on Learn gpt-image-1:

ImageClient chatClient = openAIClient.GetImageClient("reise-bild");

var imageGeneration = await chatClient.GenerateImageAsync(
        "a happy monkey sitting in a tree, in watercolor",
        new ImageGenerationOptions()
        {
            Size = GeneratedImageSize.W1024xH1024
        }
    );

Console.WriteLine(imageGeneration.Value.ImageUri);

Three oddities of this Learn page: The listing has no using Azure.Identity;, which DefaultAzureCredential needs. The text below it speaks of Base64 data as output, but the code prints an address. And the page pins an old prerelease version of the package. What the current package returns is open.

Safeguards according to Learn: The image interfaces have a content filter. If the service recognizes the prompt as harmful, it returns no image. Photorealistic images of minors are blocked by default. In addition, there is monitoring for abuse. Whether Nordlicht has to label generated images is a question for legal counsel: open. In the pilot, the helper generates no images for customers.

Azure Vision: Image Analysis

The service Azure Vision in Foundry Tools offers Image Analysis with captions, tags, object detection, text recognition (OCR) and more. Learn marks it as deprecated and announces its retirement: After that, calls fail. The date is on the page; the page gives it differently in two places, which one applies is open. As replacements, Learn names: Document Intelligence for text recognition, the Face service for faces, GPT models in Foundry and Content Understanding (lesson 2.9) for other image tasks. For Nordlicht this means: A new image application builds on the vision-enabled model from above or on Content Understanding, not on Image Analysis. The exam may still ask for the capabilities.

Responsibility with photos

Hotel photos can show people. The Learn pages read on vision-enabled models set no rules on recognizing people; this lesson therefore makes no claim on it: open. For Nordlicht, the limit of the pilot is: no recognizing or naming of people, only the condition of the room, and a person checks.

Further reading

• Use vision-enabled chat models
• Azure OpenAI image generation models
• Migrate from Azure Vision in Foundry Tools - Image Analysis