How to Run AI on Cloudflare Workers

How do you run AI on Cloudflare Workers?

Cloudflare Workers AI lets you call machine-learning models from a Worker that runs at the edge, close to your users, without managing any GPU or server. You bind the AI service to your Worker, send it a prompt or input, and get inference back — all inside Cloudflare's network, billed per request rather than per idle hour.

This matters because traditional AI hosting means renting a GPU box that costs money whether or not anyone is using it. Workers AI flips that to a serverless model: there is nothing to keep warm, it scales to zero, and your code already sits in hundreds of locations worldwide, so latency to the user is low.

What you can run at the edge

Workers AI ships a catalogue of ready models you invoke by name, covering the common needs of an AI feature.

• Text generation with open large language models for chat, summarization and rewriting.
• Embeddings for semantic search and retrieval, which pair naturally with Cloudflare's Vectorize vector database.
• Image and vision tasks such as classification and captioning.
• Speech to text for transcription workloads.

You do not download or host these models yourself. They run on Cloudflare's GPUs; your Worker just sends inputs and receives outputs, which keeps your code tiny and your deploy fast.

What a request looks like

The flow is deliberately simple. In your Worker you reference the bound AI service, pick a model by its identifier, and pass your input. The Worker awaits the result and returns it to the caller. Because Workers run on a request-response basis, you wire AI into an HTTP endpoint, a scheduled trigger or a queue consumer with the same few lines. Keys and bindings are configured in your Worker settings, so nothing sensitive ends up in client code. Pair Workers AI with KV for caching, R2 for file storage and Vectorize for embeddings, and you have an entire AI backend that never provisions a server.

When the edge beats a traditional server

Workers AI shines for short, stateless inference where low latency and scale-to-zero matter: a chat endpoint, an embedding generator behind search, a classification call inside a form. You pay per request, so a feature with spiky or low traffic costs almost nothing when idle, which is impossible with an always-on GPU instance.

It is less suited to very large frontier models, long-running multi-minute jobs, or workloads that need a specific framework and full control of the runtime. Workers have execution-time and memory limits, and you are choosing from the provided model catalogue rather than running arbitrary checkpoints. When you need that level of control, a dedicated GPU host or a managed inference service is the better fit — but for a great many AI features, the edge is faster to ship and cheaper to run.

Watch the limits and the cost model

Two things catch people out. The first is the execution and memory ceiling on a Worker: a single inference call is fine, but chaining several model calls or processing a large payload in one request can bump the limit, so push heavy or multi-step work onto a queue consumer rather than cramming it into one request. The second is billing. Workers AI is priced by usage measured in the units each model consumes, so a chatty endpoint without caching can cost more than you expect even though there is no idle charge. Put a cache in front of identical or near-identical requests and the per-request cost drops sharply, because the cheapest inference is the one you never run. Knowing both ceilings up front lets you design around them instead of discovering them in production.

A sensible edge AI architecture

A clean pattern: a Worker receives the request, checks KV for a cached answer, calls Workers AI on a miss, stores the result and responds. For search, generate embeddings with Workers AI, store them in Vectorize, and query Vectorize at request time to find relevant context before a generation call. This keeps the whole pipeline inside Cloudflare, with no origin server to maintain and no cold GPU to pay for.

Prefer it built and managed for you?

Edge AI is powerful but the right-fit decision — edge versus dedicated GPU — is easy to get wrong and expensive to undo. Talk to BSH Technologies and we will help you decide where your inference belongs and build it cleanly. Take a look at our cloud engineering services to see how we design serverless and edge architectures that stay cheap under real load.