Python Client

Overview

ModelRunner provides a unified Python SDK to call any supported model with a consistent interface. Use it in scripts, services, and notebooks with both async and sync workflows.

For scripts and notebooks, never hardcode secrets. Use environment variables to manage your key securely.

Installation

pip install modelrunner-ai

pip3 install modelrunner-ai

poetry add modelrunner-ai

Configure credentials

Configure the client with a single key. Environment variables are recommended.

export MODELRUNNER_KEY=your-api-key

Get your credentials from your ModelRunner account. Keep them server-only and out of version control.

Call a model

Leverage the queue for long-running tasks. Optionally listen to queue updates.

import asyncio
import modelrunner_ai

async def main():
    response = await modelrunner_ai.submit_async(
        "bytedance/sdxl-lightning-4step",
        arguments={"prompt": "two friends cooking together"}
    )

    logs_index = 0
    async for event in response.iter_events(with_logs=True):
        if isinstance(event, modelrunner_ai.Queued):
            print("Queued. Position:", event.position)
        elif isinstance(event, (modelrunner_ai.InProgress, modelrunner_ai.Completed)):
            new_logs = event.logs[logs_index:]
            for log in new_logs:
                print(log["message"])
            logs_index = len(event.logs)

    result = await response.get()
    print(result["output"])

asyncio.run(main())

For a synchronous script, submit the request and block on the handle:

import modelrunner_ai

handle = modelrunner_ai.submit(
    "bytedance/sdxl-lightning-4step",
    arguments={"prompt": "two friends cooking together"}
)

result = handle.get()  # blocks until the request settles
print(result["output"])

Use submit() + handle.get(), not run(). run() returns the queue envelope — status, request_id, response_url, and friends — as soon as the request is accepted, so it does not wait for the inference and result["output"] raises KeyError.

Tag requests with metadata

Attach your own flat string map to a request — job ids, environments, batch labels — by passing metadata alongside arguments. It’s supported on run, submit, and submit_async:

import modelrunner_ai

handle = modelrunner_ai.submit(
    "bytedance/sdxl-lightning-4step",
    arguments={"prompt": "two friends cooking together"},
    metadata={"project": "onboarding-demo", "env": "prod"},
)

result = handle.get()  # blocks until the request settles
print(result["output"])

It’s also supported on subscribe and stream, and on every _async variant. metadata is sent as a reserved top-level sibling of your input fields — never nested inside arguments, and never forwarded to the model. It’s stored on the request so you can filter your history by it later; the client itself has no read-back or filtering API, so read tags back through the request lifecycle metadata filter or the MCP list_my_requests tool.

Requires modelrunner-ai 0.3.0 or newer. Earlier versions did not accept the argument at all — passing it raises TypeError: run() got an unexpected keyword argument 'metadata'.

Limits, enforced locally before the request is dispatched — every violation is reported at once, so a batch of bad tags surfaces in one error:

Constraint	Value
Max keys	16
Key length	1–64 characters
Values	strings only, ≤512 characters

metadata is reserved at the top level of the request body. If a model’s own input schema declares a field named metadata, sending it this way is rejected by that model’s validation — it’s treated as a request tag, not model input.

Get called back with webhooks

Instead of polling a handle, pass a webhook_url and ModelRunner POSTs the result to you when the request settles. Nothing is lost if your process restarts mid-request, which is what makes this the right choice for long video and training jobs.

import modelrunner_ai

handle = modelrunner_ai.submit(
    "bytedance/sdxl-lightning-4step",
    arguments={"prompt": "two friends cooking together"},
    webhook_url="https://example.com/hooks/modelrunner",
    webhook_events=["completed"],  # optional — this is the default
)

Both arguments work on submit, submit_async and subscribe. See the webhooks guide for the events, the retry schedule, and the full payload shape.

Requires modelrunner-ai 0.3.0 or newer. In earlier versions this argument was accepted and silently ignored — no callback was ever sent, and no error was raised.

Verify a delivery

Every delivery is signed. Fetch your signing secret once and keep it in your server environment:

secret = modelrunner_ai.get_webhook_secret()

Then verify each delivery against the raw request body:

import os

from fastapi import FastAPI, Request, Response
from modelrunner_ai import WebhookVerificationError, verify_webhook

app = FastAPI()
SECRET = os.environ["MODELRUNNER_WEBHOOK_SECRET"]

@app.post("/hooks/modelrunner")
async def hook(request: Request):
    try:
        payload = verify_webhook(
            SECRET,
            request.headers,
            await request.body(),  # raw bytes, before any JSON parsing
        )
    except WebhookVerificationError:
        # A missing header, a stale timestamp, or a signature that doesn't
        # match. Treat every case the same way.
        return Response(status_code=401)

    handle_result(payload)
    return Response(status_code=200)

The signature covers the delivered bytes, so a body that has been parsed and re-serialized will not verify. In Flask the raw body is request.get_data(); in Django it is request.body.

Read billingStatus, not status. A generation that failed at the provider still arrives as status: "COMPLETED", with billingStatus: "failed" and a populated error. Code that treats status alone as success will report every failure as a success.

if payload["billingStatus"] in ("charged", "partial"):
    print(payload["output"])

Two more things your endpoint must do, both easy to get wrong:

Return 2xx directly. Redirects are not followed, so a 301 — a missing trailing slash, an http→https upgrade — is recorded as a failed attempt and you see nothing but silence.
Deduplicate on the webhook-id header. Delivery is at-least-once and that id is stable across retries of the same delivery.

Rotate the secret

secret = modelrunner_ai.rotate_webhook_secret()

The previous secret keeps verifying for 24 hours, and deliveries are signed with both during that window — so you can roll the new value out without dropping anything. Pass a list to bridge the gap:

payload = verify_webhook(
    [os.environ["WEBHOOK_SECRET_NEXT"], os.environ["WEBHOOK_SECRET_CURRENT"]],
    request.headers,
    await request.body(),
)

Rotating twice inside that window ends it early and breaks receivers still holding the original secret, so this call is never retried automatically. Rotate once, deploy, then rotate again if you need to.

Async callers have get_webhook_secret_async and rotate_webhook_secret_async; verify_webhook is synchronous in both cases, since it only does local HMAC work.

Upload files

Upload local files to ModelRunner storage and receive a temporary URL you can pass to model inputs (for example, image or audio URLs).

import modelrunner_ai

input_image = modelrunner_ai.upload_file("./image.jpg")
print(input_image)

handle = modelrunner_ai.submit("swook/inspyrenet", arguments={"image_path": input_image})
result = handle.get()
print(result["output"])

Getting started

Wrappers

Platform features

SDKs & recipes

Overview

Installation

Configure credentials

Call a model

Tag requests with metadata

Get called back with webhooks

Verify a delivery

Rotate the secret

Upload files

​Overview

​Installation

​Configure credentials

​Call a model

​Tag requests with metadata

​Get called back with webhooks

​Verify a delivery

​Rotate the secret

​Upload files

Overview

Installation

Configure credentials

Call a model

Tag requests with metadata

Get called back with webhooks

Verify a delivery

Rotate the secret

Upload files