Skip to main content
ModelRunner
microsoft avatar

Florence-2 Large Caption API

microsoft/florence-2-large/caption

Generate a concise one-sentence caption describing any photo — no prompt needed.

caption
0.01
OpenAPI

Model Input

Input

URL of the image to caption.

You need to be logged in to run this model and view results.
Log in

Model Output

Output

the microsoft logo

Generated in 1.92 seconds
Logs (1 lines)

Model Example Requests

Examples

Florence-2 Large Caption API

Florence-2 Large Caption is a image-to-text AI model by microsoft. On ModelRunner it runs through a REST API or via MCP from any AI assistant, at $0.01 per response.

POST https://queue.modelrunner.run/microsoft/florence-2-large/caption

cURL

# Submit a request to the queue. Input fields go at the top level of the
# body. The optional reserved "metadata" object holds your own flat string
# tags — stored on the request, never sent to the model; filter later with
# GET https://queue.modelrunner.run/requests?metadata=<url-encoded JSON>.
curl -X POST https://queue.modelrunner.run/microsoft/florence-2-large/caption \
  -H "Authorization: Key $MRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://media.modelrunner.ai/0SCFZDccUa0l5Luk-microsoft-avatar.png",
    "metadata": {
      "project": "my-project"
    }
  }'
# → { "request_id": "...", "status_url": "...", "response_url": "..." }

# Poll status_url until "COMPLETED", then fetch the result
curl "https://queue.modelrunner.run/microsoft/florence-2-large/caption/requests/$REQUEST_ID/status" \
  -H "Authorization: Key $MRUN_API_KEY"
curl "https://queue.modelrunner.run/microsoft/florence-2-large/caption/requests/$REQUEST_ID" \
  -H "Authorization: Key $MRUN_API_KEY"

JavaScript

import { modelrunner } from "@modelrunner/client";

const result = await modelrunner.subscribe("microsoft/florence-2-large/caption", {
  input: {
    "image_url": "https://media.modelrunner.ai/0SCFZDccUa0l5Luk-microsoft-avatar.png"
  },
});
console.log(result);

Python

import os
import requests

headers = {"Authorization": f"Key {os.environ['MRUN_API_KEY']}"}

submitted = requests.post(
    "https://queue.modelrunner.run/microsoft/florence-2-large/caption",
    headers=headers,
    json={
      "image_url": "https://media.modelrunner.ai/0SCFZDccUa0l5Luk-microsoft-avatar.png"
    },
).json()

# Poll submitted["status_url"] until "COMPLETED", then:
result = requests.get(submitted["response_url"], headers=headers).json()

Input parameters

NameTypeRequiredDescription
image_urlstring (uri)yesURL of the image to caption.

Machine-readable: OpenAPI schema · llms.txt

Use Florence-2 Large Caption from Claude & Cursor (MCP)

Point Claude Code, Claude Desktop, Cursor, or any MCP client at the ModelRunner MCP server and Florence-2 Large Caption becomes a tool your assistant can call directly — it authorizes via OAuth (no API key in config) and runs this model with the run_model tool using the endpoint microsoft/florence-2-large/caption.

MCP client config (Claude Desktop, Cursor)

{
  "mcpServers": {
    "modelrunner": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://mcp.modelrunner.run/mcp"]
    }
  }
}

Claude Code

claude mcp add --transport http modelrunner https://mcp.modelrunner.run/mcp

Then ask your assistant, for example: “Run microsoft/florence-2-large/caption on ModelRunner to generate text”. MCP setup guide.

Model Details

Model Details

Florence-2 Large Caption looks at a photo and returns a single concise sentence describing what it shows. Give it one image URL and you get back a short, natural caption of the whole scene — what is in it and roughly what is happening — with no prompt, no question, and nothing to tune. It is zero-shot: one image in, one caption string out. Built on Microsoft's Florence-2 vision-language foundation model, it is a fast, deterministic way to turn an image into searchable, human-readable text for alt-text, indexing, or a quick scene summary.

## Best for - Generating alt-text or accessibility captions for photos - Auto-describing images for search, tagging, or content indexing - Getting a quick one-line summary of what a picture shows - Bulk-captioning a dataset of images where a short, consistent description per image is enough

## Choose another model when - You want labeled bounding boxes around the objects in the image — use the Florence-2 Large object-detection variant - You want to read or transcribe text printed in the image — use an OCR model - You want to ask a specific question about the image or get a longer, detailed answer — use a visual question-answering model such as Moondream - You need a long, multi-paragraph description rather than one sentence — use a detailed-captioning model

## Tips - Feed a clear, reasonably high-resolution photo; the caption summarizes the dominant subject and setting, so a clean composition yields a sharper description. - There is nothing to prompt or configure — every run on the same image is deterministic, so you can cache results. - The output is a plain caption sentence (not a list of objects); use the object-detection variant if you need structured detections.

## Limitations - Produces one short sentence focused on the main subject — fine detail, small objects, and secondary elements are often omitted. - May misread unusual scenes, rare objects, or text-heavy images; it describes, it does not transcribe.

To run via the ModelRunner JavaScript client: ```js import { modelrunner } from "@modelrunner/client";

const result = await modelrunner.subscribe("microsoft/florence-2-large/caption", { input: { image_url: "https://media.modelrunner.ai/example-scene.png", }, }); // result.data is a caption string, e.g. "a green volkswagen beetle parked in front of a yellow building" ```