Copilot SDK sends image as text path instead of base64 image block (BYOM/Anthropic)

[KennyTurtle]

Summary

When a user attaches an image in a Copilot Studio agent conversation using BYOM (Bring Your Own Model) with an Anthropic/Claude provider, the Copilot SDK produces a text content block with a file path reference instead of a proper image content block. Since Dracarys passes the payload through to the LLM provider, the model receives a string of text rather than the actual image.

Current Behavior

The SDK produces:

{
  "type": "text",
  "text": "Image file at path /app/uploads/artworks-2bcW4Nnk6qFo73cq-ypWMug-t500x500.jpg"
}

Expected Behavior

The SDK should produce a valid Anthropic image content block with the actual image bytes:

{
  "type": "image",
  "source": {
    "type": "base64",
    "media_type": "image/jpeg",
    "data": "<base64 encoded image bytes>"
  }
}

Or via URL if the image is hosted:

{
  "type": "image",
  "source": {
    "type": "url",
    "url": "https://..."
  }
}

Details

• Scenario: BYOM (Bring Your Own Model) with Anthropic/Claude provider
• Model in payload: Sonnet46
• Payload format: Anthropic Messages API
• max_tokens: 8192
• temperature: 0.1
• Image file referenced: /app/uploads/artworks-2bcW4Nnk6qFo73cq-ypWMug-t500x500.jpg
• Observed at: 2026-03-28T00:41:58.703Z
• Impact: Vision/multimodal capabilities are completely broken — the model cannot see attached images, it only receives a file path string as text

Context

In the BYOM flow, Dracarys orchestrator passes the SDK-constructed payload through to the Anthropic API without transforming content blocks. The SDK is responsible for producing a valid payload for the target LLM provider, including encoding images in the format required by that provider's API (Anthropic Messages API in this case).

Repro

1. Create a Copilot Studio agent configured with BYOM (Anthropic/Claude provider)
2. Deploy to a Dracarys container
3. Upload any image in the chat
4. Ask "what is this image"
5. Inspect the payload sent to the LLM — image is a text block with a file path, not an image block with base64 data