# Gemini 3.5 Flash > Fast, general-purpose text model served over an OpenAI-compatible chat completions endpoint, with tool calling, JSON mode and streaming. ## Overview - **Endpoint**: `https://queue.modelrunner.run/google/gemini-3.5-flash` - **Model ID**: `google/gemini-3.5-flash` - **Category**: text-to-text - **Kind**: inference - **Tags**: gemini, gemini-3.5, flash, llm, text-to-text, chat, chat-completions, openai-compatible, streaming, tool-calling, json-mode ## Pricing - **Input tokens**: $2.1 per 1M - **Cached input tokens**: $0.21 per 1M - **Output tokens**: $12.6 per 1M ## Request Lifecycle This model runs on the ModelRunner **asynchronous queue API** — a single POST does not return the output. Every call requires an `Authorization: Key $MODEL_RUNNER_KEY` header. Run three steps: 1. **Submit** — `POST https://queue.modelrunner.run/google/gemini-3.5-flash` with a JSON body holding the input fields at the top level. The body may also include a reserved top-level `metadata` object — a flat string map (max 16 keys, key ≤64 / value ≤512 chars) stored on the request for your own tagging. It is never sent to the model; filter your request history with `GET https://queue.modelrunner.run/requests?metadata=` (exact key=value matches, AND-ed). The response carries request handles only (no output yet): ```json { "status": "IN_QUEUE", "request_id": "<21-char id>", "status_url": "https://queue.modelrunner.run/google/gemini-3.5-flash/requests//status", "response_url": "https://queue.modelrunner.run/google/gemini-3.5-flash/requests/", "cancel_url": "https://queue.modelrunner.run/google/gemini-3.5-flash/requests//cancel" } ``` 2. **Poll status** — `GET ` until `status` is `COMPLETED`. Possible values are `IN_QUEUE`, `IN_PROGRESS`, `COMPLETED`, `FAILED`, `CANCELLED`. A `FAILED` request responds with HTTP 400 and an `error` field. 3. **Read result** — `GET `. Returns the finished request, including the generated `output`: ```json { "id": "", "status": "COMPLETED", "output": ..., "input": ... } ``` The JavaScript and Python SDKs below perform steps 2–3 for you. In any language without an SDK (Swift, Go, Kotlin, etc.) you must implement the polling loop and the final result fetch yourself — see the cURL example for the full flow. ### Input Schema - **`seed`** (`integer`, _optional_): Best-effort determinism hint. - **`stop`** (`unknown`, _optional_): Up to 4 stop sequences. - **`tools`** (`array`, _optional_): OpenAI-format tool definitions the model may call. - **`top_p`** (`number`, _optional_): Nucleus sampling probability mass. - Range: `0` to `1` - **`stream`** (`boolean`, _optional_): Return the reply as a Server-Sent Events stream of deltas terminated by \`data: \[DONE\]\`. - Default: `false` - **`messages`** (`array`, _required_): OpenAI-style conversation history. Each item is an object with a \`role\` (\`system\`, \`user\`, \`assistant\` or \`tool\`) and \`content\`. - **`max_tokens`** (`integer`, _optional_): Upper bound on generated tokens. - Range: `1` to `"+inf"` - **`temperature`** (`number`, _optional_): Sampling temperature. Lower is more deterministic. - Range: `0` to `2` - **`tool_choice`** (`unknown`, _optional_): \`auto\`, \`none\`, \`required\`, or a specific tool. - **`response_format`** (`object`, _optional_): Set \`{"type":"json_object"}\` for JSON mode. ### Output Schema - **`id`** (`string`, _optional_): - **`model`** (`string`, _optional_): - **`usage`** (`object`, _optional_): - **`object`** (`string`, _optional_): - **`choices`** (`array`, _optional_): - **`created`** (`integer`, _optional_): ## Default Example **Input** ```json { "messages": [ { "role": "user", "content": "Think carefully and thoroughly, then answer: design a rate limiter for a multi-tenant API. Compare token bucket, leaky bucket, sliding window log and sliding window counter across accuracy, memory, burst behaviour and distributed coordination. Then recommend one and justify it in about 400 words." } ], "max_tokens": 8000 } ``` **Output** ```json { "id": "chatcmpl-nbUa8L9o0gwaThyHr5cHh", "model": "google/gemini-3.5-flash", "usage": { "total_tokens": 2444, "prompt_tokens": 58, "completion_tokens": 1194, "completion_tokens_details": { "reasoning_tokens": 1192 } }, "object": "chat.completion", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "### Multi-Tenant Rate Limiter Architecture\n\nIn a multi-tenant API, a rate limiter must isolate tenants to prevent a single noisy neighbor from degrading service for others. \n\n```\n [ Incoming Request ] (with Tenant ID / API Key)\n |\n [ API Gateway ]\n |\n (Check cache / Fetch Tenant Tier)\n |\n [ Distributed Rate Limiter Middleware ]\n |\n +-----------------+-----------------+\n | (Execute Lua Script in Redis) |\n v v\n [ Redis Cluster ] [ Dynamic Config DB ]\n (Tenant-specific rate keys) (Tenant limits/tiers cache)\n |\n +-------+-------+\n | |\n [Allow] [Deny]\n | |\n[Backend API] [429 Too Many Requests]\n```\n\n#### Key Architecture Components:\n1. **Tenant Identification**: The API Gateway extracts the `Tenant ID` (from JWT claims, API keys, or IP addresses).\n2. **Dynamic Limit Resolution**: Tenants map to tiers (e.g., *Free*, *Basic*, *Enterprise*). Limits (e.g., 100 req/min) are fetched from a fast-access cache (Redis or in-memory) rather than hardcoded.\n3. **Key Scheme**: Keys are structured as `rate:{tenant_id}:{tier}:{endpoint_hash}` to allow granular control.\n4. **Distributed Storage**: A centralized, low-latency in-memory data store like Redis manages the state.\n\n---\n\n### Algorithm Comparison\n\n| Dimension | Token Bucket | Leaky Bucket | Sliding Window Log | Sliding Window Counter |\n| :--- | :--- | :--- | :--- | :--- |\n| **Accuracy** | **High** (Quantized to update frequency) | **High** (Strictly enforces average rate) | **Extremely High** (Precise to the millisecond) | **Medium-High** (Approximated; up to ~5% error on window boundaries) |\n| **Memory** | **Very Low** (Stores 2 values: `last_tokens` & `last_updated_timestamp`) | **Low** (Stores queue size or virtual finish time) | **Very High** (Stores every request timestamp in the current window) | **Low** (Stores current and previous window counters) |\n| **Burst Behaviour** | **Excellent** (Allows sudden bursts up to the max bucket capacity) | **Poor** (Smooths out bursts; forces artificial latency or drops) | **Excellent** (Enforces limit strictly while allowing bursts) | **Good** (Allows bursts up to window limit, but may block edge cases) |\n| **Distributed Coordination** | **Easy** (Can be executed atomically in Redis via a single Lua script) | **Hard** (Requires distributed queue locking or complex time tracking) | **Hard** (High Redis network/CPU overhead using `ZREMRANGEBYSCORE`) | **Easy** (Simple atomic increments using Redis `INCRBY` and `EXPIRE`) |\n\n---\n\n### Recommendation & Justification\n\nFor a multi-tenant API, **Token Bucket** (implemented via Redis and Lua scripting) is the recommended algorithm. \n\n#### Justification (~380 words)\n\n1. **Handling Multi-Tenant Traffic Patterns:**\n API traffic is inherently bursty. Tenants performing batch migrations, webhooks, or UI load spikes require systems that accommodate bursts. The **Leaky Bucket** algorithm is unsuitable here because it forces a steady stream of output, introducing artificial latency and degrading the tenant's user experience. The **Token Bucket** permits instant bursts up to the designated bucket capacity while guaranteeing that the long-term rate does not exceed the allowed threshold.\n\n2. **Extreme Memory Efficiency for Scale:**\n In a multi-tenant system scaling to tens of thousands of active tenants and endpoints, memory footprint is a critical driver of infrastructure cost. **Sliding Window Log** scales memory linearly ($O(N)$) with the number of requests, making it highly vulnerable to memory exhaustion attacks or high-throughput tenants. **Token Bucket** maintains a constant $O(1)$ memory footprint, requiring only two fields per tracking key: `available_tokens` (float) and `last_updated_time` (integer). \n\n3. **Seamless Distributed Coordination:**\n In a distributed environment (e.g., multiple API Gateway instances behind a load balancer), rate-limiting state must be synchronized. Token Bucket excels here. By using a **lazy evaluation** pattern in a Redis Lua script, the rate limiter computes token replenishment on-demand when a request arrives:\n $$\\text{tokens} = \\min(\\text{capacity}, \\text{current\\_tokens} + \\text{time\\_elapsed} \\times \\text{refill\\_rate})$$\n This approach avoids background cron jobs or daemon processes to refill buckets. Executing this calculation inside a Redis Lua script guarantees **atomicity**, eliminating race conditions (double-spending tokens) without requiring expensive distributed locks.\n\n4. **Dynamic Tiering and Customization:**\n Multi-tenant platforms require flexible, tier-based limits that can change dynamically. Token Bucket parameters (capacity and refill rate) can be passed directly from the API gateway's cached tier configuration into the Redis Lua script on each evaluation. This enables instant plan upgrades (e.g., from *Free* to *Enterprise*) without needing to reinitialize or migrate existing Redis keys. \n\nUltimately, Token Bucket provides the optimal balance of burst resilience, deterministic memory consumption, and straightforward distributed implementation necessary to power a scalable, multi-tenant API.", "extra_content": { "upstream": { "thought_signature": "AY89a18c0wmi52XeJ6D7e+cjbJQFsllEwSW0IYRK08e540BMtBmSRDnI5ww4ZojTHc+4YuNGGuG02ocxbyDaYxjNfwnwEtoyy4J/VEnUDRoKCaAmB7UgTzpaMigKGF3kJoa7iBGqu4U8AcDVN8HmmBlDrRs6KkkMOqeZKEhELHLq77meCjTQ5swG4nsYrWFN/XF0vwZuyK8h2UWn6830j6CQ36PdWubphhq9qwoLuQcBA7LtlUHZ1O+pqDSJBcwtFEgsbJbLyU8qwBKXzCvF1/tFqxkml17UVIPMre+8v8hZfOb/WAxMrB+MVQkXU/DTNPs3IcH9hWiMN5WR1YsGxYbwh+sKmfI1Yf0nDjp5+IyY6aIChLNdT+MjMIKe8e2u3Kkkf3zbb/7aTqf6pm+LLuX5HdQW08OKSZ8a3CGQJk7AEaJ8f9y8s+mjcy/ilhJ+2qz7Sudyt1sx0BI2ANMQzKO4q/Ut+2WQLceeoy2/69Aub4+ILv6nVsQAx2mlKrlEI3CaAZHBU/3z0o56+C7lzM2v36oUZKHJW2WJIXp+mMA7rznwkh6jUmJroyaWhCNzGeNPNoXXt9W5fwpkCUgs7nONZKtQDOZ35toCxkJHDCD0kQZUXqtXHF0Vg98cD55045sBccb97QRDrGKUXwzWVj98OOi2JFErG6AAF98dKAIoTwpbpmabp/nMVNEmgtZ1g/oLJtFd1n2HRa1USYsyYK7xBmFb40H4vGkXPdwr2xScMewRuS/dmVsJIENOBrHtJQ3UVunfyJNpWq3cHyk+5zKd3MwdIRdUzC/0E9IdA55TIw04jSSfSQSSA/FLSJnoq7Y1d+bQ5J5BX4CroCrR4lCRXmJPjorTxt950kmQ3OyJ7MYaM3yBKNlq4G2TKHz85iQVwi9lpdhWG/ecaBU5U7LAmYlqTx0khFX0q8wOfU0Cjx4M8TcImcIcVFWcTpgZqN2PLoId/LanU4zhmQK1hTuPV3HgJOzqnffSLiEX3xPsDWO8bIw6DTm1CXRSE16chBkN5p4UODDzqipwwd/GB6Z415iTsAo+ijEfE/50qIsti0NPMMknoo04Wg4750DcJWGbhT9QFEqas4cRl0Y47/Xq/G1oB6XP/V5mh0dQ1JpaRHsgNu2YUkiB90Dd8NXBoFKzhbRBwmVNr2hYWtzTyDrlV5UfM1kSxIFZcEsTaM3f7zl0QRqxAJ1uC40/aDHMlXxNlfKlkSbo/8LtZcIPAdPfSetiX9DlLVYgteik02k+NN0gJkAmzd9Exyf9fsh+j4zYQKh7LUaVtIvJulXZyFugXtwmsIhStQ0rmIJrO7X2pLni3ZKeeLctsLmskJoDoFWQ7etFVodZvzo+AbfwqKitjVijtJfszQEN6IOkGRngyQ7OkzDsOpHAekWgaBUwEAeEBtV9wSPwmPOQhnHP45v1V/YwweG/p1qdgRXIFwI0Y0ml8UWFxD9dzsGk1XEBcVez5F2oLw1ObzySOddhr8Y/iwaK+Qgvb1Dp2lEtu7T1Fx367ky5n9EEPl6lubcqD5SrA8DbbYZY8nzLBjQE4mbqgMvgOggfunPRuCgsH6O1Uf8XaIgwNmahm8hA6lkcIOj5X5+6NaGBMF0J2uD6nWvCDEzlb8KH5Nn/d5u9Aj9uNlmtkQGqN1mAeq1RKPcTzNkS+ZQw25ULyG7YE4ewAUB6DUE/B7/6PIs8NpcQYMtlGwSLllkkGLpzLwiLSvrpSK8b3F0O8PElNbFhLZfdA5XZFTjhEgwubwlOSwXHgkZJWIEWRMidxN5crx9vRQK+LJQ9ewbvu0tgQZIVsy34E1AIzXNCb1WZgTMRHKAN5HBvt2O9SUIGnK7AXpMIwyBF7bOV7UWJ+jwpZNHtzBdaMDd8ffJ30Ijd61GuDCNPJZaPLG7TdTinTCbjQ5dbZVHaNAclbAxBqDXfWCByURWi2Ch9bruJd8xLw8aUPk6jlhdcKMp/4tiSyRWRoesqkD3hvobwPv8WPfNWjR4qx0D5DX1OUrDiKyOwF9T1dg/3V8GJBdBXJLyJjLwIZrovc4qhIVI0b9n2dg4bqQCKJ62CbgbYHXQpo6j576aPo6jrHYG609dJZ067UmydeAQB+h8VpBxqx3YKblHfBKnmWaAskcjDv2Oi1pOwUaCBTC4uvNBY9rwYeyUTc8kg4RpAUdKIKNiF3vYB1c77pO6gB0Gk3A5YP1zjqMLLsTLAYi8ek3+I60qBZ6CKTXTDwmSQL8uTnFdCwdYd5tWqeQrwgeNulnslhxZY7CjJavygDmEVuoQRT64l8Fv0wNp633GtSvT2MHzmVaj4Ps2LuOssb0YbgTZRzTyqwOxjFxFk4aXeuL5BKQI7BFs3iunlRXkEg3bzN+B/JIO71X7bsqQ4X97zzh/Rg5/WXOjr2B/qTjw3xuHH9OWLTPu7kNhSRLq0yDevxe8csTfmrrmIw6STbmNelkhBP0w3wxf54Dnib9vHJakwg2pBaTQ18ivjXORSU+itybBTuVJSRs8T4/KkZFQQgNSGNiZ+YlAd4ujUI3wd5X6CLUvJJekKLfAdAXwilmf16tTeSYOqO2L094AuzOnYaCPi9W6a/x9vPAKYTkwOXuHqrRreBSvOkr7dkHo0xN05UGvD0VfrhN6dUx11GbbV6Rvkmw5sTHDJfV4nDNl4cD1cpuP3AGaACRMtBC1k8DSpmxyPm4JdRAyYLHxCkjV+pF+aCb2sxQaHCrNcOE6Np7H9dyMo5w1jtW0X8fTX2c1BhjJZfVlFm8Ar1J2QI8ePVoTjp6HaVAEl17MWOicY0KKkUEbIyVRVoHvx7dQc1G3sAoId4AyQ/h/fSOdR9iPnlTlTarkthJyTBPX8KA3MkJQU3PRdrCexqME7j5EXgbNXPReg3Yc3yuDM91FHB9a+M2JvqryqtWx9n6M8gwfok/ns7FQmZq0J5rV6cBrQEbTkO5mx/jhzbee2NfTrIiH6SsMKaoReppkUMKH/apKo8kUTUQ1G3LRtGqPM7/UTk3e9zD1I92QN44+WqMAD0tDyGN0kJnHG8XIHXb8/EEzVzgVeCi+XWkGz9ibmhM+NiiStGlDZtRn3CjvlihSvyEyvwA2/p4fDD0rnx8ZUdkMemNaZBmx2L7bfe+1wBBuE4GH5KkqnSZaMuYQqdt3eLc/ELX+MYmusDGRPwuOzOEXVkQRbRmItc3Sr+iYCR7DvtTfTxf7fxh/hFU7NXqg3S3Ip3a8otkrzoYZIxA3hz+vh7JinS57wVVl/7B0xXrWrJi/yy38PhvbDatXGtzTBiHoDCkFr0929rNzwt2uqUvgbsb3lXsZ2UBddyNQSKsQGkt+qISsVYNraJ/gXtJStBqt8veIyo7+6AYFIvMQ6cDrAYlQyC5btUJarLK6CEYOdeiTGnLaDva/aFrrREgH9PTIaXtuO6M7OVpdkSfqTUZ6F10I+4W2yF8WvRW50x2DjI5ebHIuLM8LOsF6C6KRmYHrPTANWy2dgH6Br4W7Yta61cVM8XoCJSPvlw2aPaRiAW69jV7IqeXZ5v1OZ0M9WS/pScFCJMoSVmCk7+NDIEyHOD++Y+CrE1w1O5013odffFkcMkjVEhmNJOz4gjHpGl+knnByYIzo6hsK82paCq2eYgeyIPVilfAlXC/C+KJbuAkvxZ3xummkp4nZEgotQiNvyfu6xdXw4dVwY3UWihNuMPuwn0IIvAhnZBB28mZH0rRwX8VLMyttzz5G7lP+Z6rpNlBGreA/FObIn46I5IUFNuT3IbkxLl1Wy2Ay2B1mQY8+FLwPTPKDKm04raoEFxcXbT6iiuccnhXjUs8xAMUnOvwnlXgGxhky5nBfLKVwYPXk5kImWjM1yTKREBo4ZZItCSe1PBlfcImo60P56haKlZSyjS5DsFdEidzGKA7ur1h0jINQpBLMc3tGIYA7UdVefM0xfxQJ0JKg1ruJCJQvnf3nraUTjAaVfmfBze6WXsek3xDf9z7DQVYLH5GI6hCArKKi0Aww2tX8h4cPXXRfQ6FsuUdUWYf9P1rMVbovUNoYzjpqe5NXZNtOf5qDDpEH9YuXFSYlK6NLDSd+pyWi87N3mNWi3gOYiSszsjxHkUxvmt8Q7tZY6tJOzzYGM4wBbTF3ulfzZVYLx0qrw15e+i0lrIHtAfpvAsM6FpU4OS2sxCPI0nCqo0VzFfdlK69nyY8TIu1DHpfnxf9FdmAM5S5gVYKH4xSWSn6LIUeMids9sW6jV1lZsQwP36DH1B4pFBcusCfNcyQxah9DB7BCKKzmlv/lWTuQxg0JBhgrjF6NIZVsDCNt9/W87JuRkYB9Plo9qGa/VOaEi3T0idNgVpY8cg8oVpZd8rpbRehKgQLEO54lE/HSq2b6V5jcdD4nF7srBO2CtX7gZhND88KiO/9PUxD9+jx3QtTPwxEdc0IYN+NYZNOBDM7POkmCeh9IP6XPi9nYVUmjSeuiJKoh9jOB5amR/WQM7gUjOHJ8XLrjpL1TC1b5RajC33Zoj6/8cfrfsbtpv5kos+GSZs8vObybtby9TNYuD31hDd8cuajBHQKQE7Ia83bxFjFPtk8su/RfJX0Eva3pVnflCQYjzZFKOgIiZpge33nQkDYYK1MBjxh+rlOz0IramvJ0oBxrZ/RK24swyTbwIYV3sFKHb0zSZH329hdkYfoQUoJTTC1HFTkGC61cNWVuL5jGiw3NTrizpBlTPW160vtItK8amcW/1S1T9KPj7laftiang1el/GTdVfATW0bUt+WFlonvXGXI80kfomrFt48b4bYM5wpUmzwNU7JX7gIiDZJsJtWBzXvj+mVF5WPxsN6+Hch1CLKK4QFMpro6xIzC0kZdEu+s8OJ6zEnd+4g39sGYeaM6RsJBIm/Ln986R65L+Jog1clbFjQqNB96cUj4yv7o/K4k8ifoNAqI4WDnj8brpmlXXW58Fax+CwQtCi1oozwuAkAk2wJdBGk1mra1wuc62E01vfTqyEKQ1jP5Y8qN4LGYgDROsTPZqG/OnIV/H/wM+POgxlG26MUG4hVIIQih+SDtGuYhCmKVK+QU4qSSo85/i9FYAVEgYQF8U2tw4TZ1/ZJneGlo3D0H8NXB2NkH+PdMnjIWMIWrjIF14mObT8aD2HaYI4vrcJ5umDCJBFkDWLTP6IQhytH2bbqjkSUR7qbLE1NCKResnIEf2LUR/yBMeiXdcgVD72b/qTZ4NSUcaC23okAPmWaZR80Tx8hb/5sBQ13XfWjz4ZDPProEKAUtZb05NiufY/UBWMqezNxcyiPPOsIVIC99se7YB/xe+mcneWCR5Nup1P3JdAPF0Xz0Nqr3mE1vA/gInYHBkn21ikjrDz5SmwwTeVH/OskZFREiMlZVBA0T3LU8P9m+3ci5EkpZxZDFJ3NG6CCZ87IhA/dNxW43q6YrEozV4XHyReHRIfCZP/lZEQhjAZ4qxGXBJltL6Ri5tgACPV37Yn2KcbnCYzuM28HfWfqI+FABqt/5WglhKZ8Y9jUeLmUTtEg00qzE9rWrZuwSggNktIu620XWrZLfvBb3+ijeXx8FIIoE4v35/a+nL+YwtKWwmGHbpLAOqK4ic9CS+l+pyJmdMnaDSD4/FGuGtuhVk9NqqDfIi3NNNEj79KEsOfyCGRPJKR0FE4QOOZcINm78T4HOwJrKeHAzHn7fPTW/dAziRQy5t6hB07c5Cm775b8Qf8au+UDR5+9BsA5EiQPYN2Jbjogs9D556nnpJbkULHfoANFyCxdgTzRUcUuTQwz1784H+TUgy70kalEQZlhDqR30ozVjtZlUFkcZ1sKgpUlv3n9wqdeHgU6h6fhmLOqzr9ixsjcizR3D2GVbVt+1yuKUslySNuyeFJW/vf3ILh5t0SoFYjRCnP8mdVSeiHxdcKzhWXD5mEpLRRi5DxNyS9FkRez2KpA7PEp/vL8IEjs6oaK6mAzp12SYq7iUPOTJqviCp49HTjBVgJx5FRox9sZdth0pdBPQ4VPTtLbUcWUM2b0LsgB9vA+hiEZwC0wLADopFhEazrdY5u0S7aO8W2lhuxtKfxj1RQbQWf/Uc2Itv1QobhWQHIdaDboGqq27U3+tNMonN7lUIXsQIgYDq1x2RTABLMlLzoN+mg+V68ylvL2IHmUS3yy8amnuAhzgONSwiGYLYLD9dzvCVDK1Gh/qlCi/bomZJrENcQRhYTL2MjjpFIUAP20xWbBqhNdMV7T4taaE3klVe3O1GbUyVEK1ewmeASxwTA1NDSmqH6tPSJ6s0PtJo6gv5PS4PChdiwT4v7+3qv486zJ2AhR71u2/0Jj4kGfzS1s6JrJFebNaxgUCMqerYHK16SldqzAIq9R4fZfZAcHWDKBB33Zssn3VqbCOymR/DkpJvOiuTGP90w/DU5XvKPS2JU9ZdWZhRPLM/041ZcgURS+CoD1VRcnZ3no9Ol4CTz7nf9w1gvu+Kle22Mq+2pep1Ap+QAIvIu76+l6r+yEznJWGzI+ZH7h+4xCUGK90SWanjruBFr8q1XFIJ1ZMKcpLXpWrMAsD4Sgzmlgrt9fRAiPWktu+mj6Dw18fjpqh9ufhQOwWw4+qcF+nrNL0iPFfdQPNgdDulnZ1BQcM7p39sGHQWfyljusexUj6kMiejsi7M8jvJPLBBX7RD4JVe4WC75UFCNPKzZ9Acc2tjDwIhbiFurLXDkSQJx7u3+FTyUwtSpQEY2Rl5HWdnlOYOa0oyjNEiCwWRv+nL8JGDHYzSbzvYWb3S0/wZYIe0f3RLzZZ+NxBxGvSzVjfUvHZXkDo7TuKgMqJVxcx8k5xHgJ0P1cEOMuyHrmjF8wwiIAb3UvzwhUKHdX5i9INAty2c5wRBvlF+DWfJOyj/F/Ef0k7k6YxBXCwXwG1XEyUUfz38t9h3KR1CHnpwnqvZYA1Pn7l/vnzhKn4AxUrvE5pXPknuS1xtoFPn4FjDD95d+CYiQbXkqjkCP1RcWJSmx/a41Zi2FBgQ2wCi/uWQxm5BqhaVLCKGJCn3db3HR6QDooitx/omjF93Z+t34h7Z3154N573E1JgVs9538k12CDZdxyWgPFqTydkK7io9lTT/Ju6sxbxnQiV/rYKIZ0yVKQcxwZjc33SbrINEBWbeUVB4Tv92zspJg==" } } }, "logprobs": null, "finish_reason": "stop" } ], "created": 1786738119 } ``` ## Usage Examples ### cURL The queue API is asynchronous: submit the request, poll `status_url` until it is `COMPLETED`, then read the result from `response_url`. Requires `jq`. ```bash # 1. Submit the request (returns request handles, not the output) SUBMIT=$(curl --silent --request POST \ --url https://queue.modelrunner.run/google/gemini-3.5-flash \ --header "Authorization: Key $MODEL_RUNNER_KEY" \ --header "Content-Type: application/json" \ --data '{ "messages": [ { "role": "user", "content": "Think carefully and thoroughly, then answer: design a rate limiter for a multi-tenant API. Compare token bucket, leaky bucket, sliding window log and sliding window counter across accuracy, memory, burst behaviour and distributed coordination. Then recommend one and justify it in about 400 words." } ], "max_tokens": 8000 }') STATUS_URL=$(echo "$SUBMIT" | jq -r '.status_url') RESPONSE_URL=$(echo "$SUBMIT" | jq -r '.response_url') # 2. Poll until the request leaves the queue / in-progress state while true; do STATUS=$(curl --silent --url "$STATUS_URL" \ --header "Authorization: Key $MODEL_RUNNER_KEY" | jq -r '.status') echo "Status: $STATUS" case "$STATUS" in COMPLETED) break ;; FAILED|CANCELLED) echo "Request $STATUS"; exit 1 ;; esac sleep 1 done # 3. Read the finished request, including the generated output curl --silent --url "$RESPONSE_URL" \ --header "Authorization: Key $MODEL_RUNNER_KEY" ``` ### JavaScript ```javascript import { modelrunner } from "@modelrunner/client"; const result = await modelrunner.subscribe("google/gemini-3.5-flash", { input: { "messages": [ { "role": "user", "content": "Think carefully and thoroughly, then answer: design a rate limiter for a multi-tenant API. Compare token bucket, leaky bucket, sliding window log and sliding window counter across accuracy, memory, burst behaviour and distributed coordination. Then recommend one and justify it in about 400 words." } ], "max_tokens": 8000 } }); console.log(result.data); ``` ### Python ```python import asyncio import modelrunner_ai async def main(): response = await modelrunner_ai.submit_async( "google/gemini-3.5-flash", arguments={ "messages": [ { "role": "user", "content": "Think carefully and thoroughly, then answer: design a rate limiter for a multi-tenant API. Compare token bucket, leaky bucket, sliding window log and sliding window counter across accuracy, memory, burst behaviour and distributed coordination. Then recommend one and justify it in about 400 words." } ], "max_tokens": 8000 } ) result = await response.get() print(result["output"]) asyncio.run(main()) ``` ## Additional Resources - [Playground](https://modelrunner.ai/models/google/gemini-3.5-flash) - [OpenAPI Schema](https://modelrunner.ai/models/google/gemini-3.5-flash/openapi.json) - [LLM Instructions](https://modelrunner.ai/models/google/gemini-3.5-flash/llms.txt)