Serverless GPUs

GPUs & pricing

Rates are per second of worker time on a single GPU worker — including the idle-timeout seconds a worker waits for the next job before winding down. Availability reflects live capacity and changes throughout the day.

How it works

1. 1

   Create a deployment

   Pick a template, choose a GPU, set your scaling limits. Your endpoint is live in minutes — no cluster, no drivers, no capacity planning.

2. 2

   Submit jobs via the queue API

   Every deployment gets a private endpoint on the same queue API as catalog models: POST a job, poll the status, fetch the result.

3. 3

   Watch workers and logs live

   The dashboard streams worker states, queue depth, and a live log tail. Usage is metered per second and settles hourly.

curl -X POST https://queue.modelrunner.run/{username}/{alias} \
  -H "Authorization: Key $MRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "prompt": "..." }'
# → { "request_id": "...", "status_url": "...", "response_url": "..." }

Start from a template

Frequently asked questions

What are serverless GPUs?

GPU workers that start when jobs arrive and scale back to zero when idle. You create a deployment — a template plus a GPU and scaling limits — and get a private queue endpoint; ModelRunner runs the workers, streams the logs, and meters the seconds.

How does serverless GPU billing work?

Per second of worker time, at the rate shown for each GPU. Deployments scale to zero when idle, so an endpoint that receives no jobs costs nothing to keep. Usage is metered live and settles into hourly charges, and you can set a monthly spend cap per deployment.

How do I submit jobs to a deployment?

Every deployment gets an endpoint of the form {username}/{alias} on queue.modelrunner.run — the same queue API as catalog models. POST a job with your API key, poll the status URL, and fetch the result when it completes.

Can I watch my GPUs while they run?

Yes. The dashboard shows live workers (state, GPU, region, uptime), queue depth, a streaming log tail, per-request history, and hourly usage for every deployment.