Skip to main content
ModelRunner
microsoft avatar

Florence-2 Large Object Detection API

microsoft/florence-2-large/object-detection

Detect and label every object in a photo and get back an annotated image with bounding boxes drawn on it.

0.01
OpenAPI

Model Input

Input

URL of the image to detect objects in.

You need to be logged in to run this model and view results.
Log in

Model Output

Output

Loading
Generated in 4.054 seconds
Logs (1 lines)

Model Example Requests

Examples

Example output 1Example output 2Example output 3

Florence-2 Large Object Detection API

Florence-2 Large Object Detection is a image-to-image AI model by microsoft. On ModelRunner it runs through a REST API or via MCP from any AI assistant, at $0.01 per image.

POST https://queue.modelrunner.run/microsoft/florence-2-large/object-detection

cURL

# Submit a request to the queue. Input fields go at the top level of the
# body. The optional reserved "metadata" object holds your own flat string
# tags — stored on the request, never sent to the model; filter later with
# GET https://queue.modelrunner.run/requests?metadata=<url-encoded JSON>.
curl -X POST https://queue.modelrunner.run/microsoft/florence-2-large/object-detection \
  -H "Authorization: Key $MRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://media.modelrunner.ai/zfTdkQ5gByXOZ8Fq7K2Wa.png",
    "metadata": {
      "project": "my-project"
    }
  }'
# → { "request_id": "...", "status_url": "...", "response_url": "..." }

# Poll status_url until "COMPLETED", then fetch the result
curl "https://queue.modelrunner.run/microsoft/florence-2-large/object-detection/requests/$REQUEST_ID/status" \
  -H "Authorization: Key $MRUN_API_KEY"
curl "https://queue.modelrunner.run/microsoft/florence-2-large/object-detection/requests/$REQUEST_ID" \
  -H "Authorization: Key $MRUN_API_KEY"

JavaScript

import { modelrunner } from "@modelrunner/client";

const result = await modelrunner.subscribe("microsoft/florence-2-large/object-detection", {
  input: {
    "image_url": "https://media.modelrunner.ai/zfTdkQ5gByXOZ8Fq7K2Wa.png"
  },
});
console.log(result);

Python

import os
import requests

headers = {"Authorization": f"Key {os.environ['MRUN_API_KEY']}"}

submitted = requests.post(
    "https://queue.modelrunner.run/microsoft/florence-2-large/object-detection",
    headers=headers,
    json={
      "image_url": "https://media.modelrunner.ai/zfTdkQ5gByXOZ8Fq7K2Wa.png"
    },
).json()

# Poll submitted["status_url"] until "COMPLETED", then:
result = requests.get(submitted["response_url"], headers=headers).json()

Input parameters

NameTypeRequiredDescription
image_urlstring (uri)yesURL of the image to detect objects in.

Machine-readable: OpenAPI schema · llms.txt

Use Florence-2 Large Object Detection from Claude & Cursor (MCP)

Point Claude Code, Claude Desktop, Cursor, or any MCP client at the ModelRunner MCP server and Florence-2 Large Object Detection becomes a tool your assistant can call directly — it authorizes via OAuth (no API key in config) and runs this model with the run_model tool using the endpoint microsoft/florence-2-large/object-detection.

MCP client config (Claude Desktop, Cursor)

{
  "mcpServers": {
    "modelrunner": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://mcp.modelrunner.run/mcp"]
    }
  }
}

Claude Code

claude mcp add --transport http modelrunner https://mcp.modelrunner.run/mcp

Then ask your assistant, for example: “Run microsoft/florence-2-large/object-detection on ModelRunner to generate image”. MCP setup guide.

Model Details

Model Details

Florence-2 Large Object Detection finds the objects in a photo and returns the same image with labeled bounding boxes drawn over each detection. Give it one image URL and you get back an annotated picture marking every object it found — people, vehicles, animals, and common everyday items — each box tagged with a category label. It needs no prompt and no tuning: one image in, one annotated image out. Built on Microsoft's Florence-2 vision-language foundation model, it covers a broad open vocabulary of everyday object categories in a single pass, which makes it a quick way to see and verify what is in an image.

## Best for - Annotating a photo with labeled boxes around every detected object for review or QA - Getting a fast visual inventory of what appears in a scene (people, cars, animals, objects) - Producing before/after detection overlays to sanity-check coverage on a dataset - Lightweight scene understanding where a drawn, human-readable result matters more than raw coordinates

## Choose another model when - You want to read or transcribe text in the image rather than detect objects — use an OCR model - You want a written caption or description of the scene instead of boxes — use an image captioning model - You want to detect only one specific named thing via a text query — use an open-vocabulary / grounding detector - You want to segment objects into pixel masks rather than boxes — use an image segmentation model

## Tips - Feed a clear, reasonably high-resolution photo; small or heavily occluded objects are easier to detect with more pixels. - The returned image is a standard PNG with boxes and labels already rendered, so you can display or compare it directly against the original. - There are no thresholds or prompts to set — every run on the same image is deterministic.

## Limitations - Detects from a fixed open vocabulary of common categories; rare or domain-specific objects may be missed or mislabeled. - Very small, blurry, or overlapping objects can be merged or dropped.

To run via the ModelRunner JavaScript client: ```js import { modelrunner } from "@modelrunner/client";

const result = await modelrunner.subscribe("microsoft/florence-2-large/object-detection", { input: { image_url: "https://media.modelrunner.ai/example-scene.png", }, }); ```