How to Build an App on the Ollama API

Build on Ollama by calling its local HTTP API at port 11434 from your code

Ollama runs a local server, and that server is the foundation for building real applications, not just chatting in a terminal. Once Ollama is installed and a model is pulled, it exposes an HTTP API at http://localhost:11434 with endpoints for chat, completion, and embeddings. Any language that can make an HTTP request can drive it, which means you can build a private, local-first AI feature into a web app, a script, or a backend service without an API key, a cloud account, or a per-token bill. The model runs on your machine and answers your code directly.

The endpoints you will actually use

Ollama's API is small and focused, and three endpoints cover the vast majority of what you build.

• /api/chat — the workhorse. Send a list of messages with roles, get back an assistant reply. This is what you use for conversational features and most instruct-model tasks.
• /api/generate — a simpler single-prompt completion endpoint, handy when you have one prompt and want one response without managing a message history.
• /api/embeddings — turns text into vectors, which is what you need for semantic search and retrieval-augmented generation over your own documents.

Streaming makes it feel fast

By default the chat and generate endpoints stream their response token by token, the same way a hosted assistant types its answer out live. For any user-facing feature this matters enormously, because a reply that appears progressively feels responsive even when the full answer takes a few seconds to complete. Your code reads the stream and updates the interface as tokens arrive. If you would rather have the whole response at once — for a batch job or a simple script — you set streaming off and receive a single complete reply instead.

Because Ollama's request format closely mirrors the OpenAI API, code written against that shape often needs only its base URL changed to point at your local server. Prototyping locally and switching to or from a cloud provider later becomes a one-line decision.

Getting structured output you can rely on

Applications usually need data, not prose. When you want the model to return JSON your code can parse, ask for it explicitly in the prompt and use Ollama's support for a structured format so the response conforms to a schema rather than wandering into freeform text. Pair that with validation on your side — never trust the model's output blindly — and you have a reliable bridge between a language model's flexibility and the strict shapes your code expects. This pattern is what turns a chat toy into a dependable component of a real system.

From localhost to something real

Building against localhost is the easy and correct way to start, but a few realities matter as you go further. Ollama processes requests for a model sequentially, so for many concurrent users you front it with a queue or run multiple instances. If you deploy it on a server for a team, you secure the endpoint behind authentication rather than exposing the open port, because the API has no built-in auth of its own. And you keep the model choice configurable, so swapping a faster or smarter model later does not mean rewriting your application. None of this is hard; it just rewards thinking past the demo.

Building retrieval on the embeddings endpoint

The embeddings endpoint is the quiet enabler of the most useful local applications. By turning your documents into vectors and storing them, you can find the passages most relevant to a question and feed them to the chat endpoint, so the model answers from your own material rather than its generic training. The pattern is straightforward: embed each document chunk once and keep the vectors; at query time, embed the question, retrieve the closest chunks, and include them in the prompt. Everything — the documents, the vectors, and the model — stays on your hardware, which makes Ollama a complete foundation for private, grounded question answering without a single external call.

Handle failure like any other dependency

Treat the local model as a service that can be slow or unavailable, because occasionally it will be. Set timeouts so a stuck request does not hang your application, handle the case where the model is still loading into memory on the first call after startup, and validate every response before acting on it. These are the same defensive habits you would apply to any network dependency, and applying them here is what separates a fragile prototype from a component you can rely on.

Prefer it built and managed for you?

Wiring a model into an application is the fun part; making it concurrent, secured, and production-ready is where the engineering lives. BSH Technologies builds applications on local model APIs like Ollama — streaming interfaces, structured output, queuing, and the security an exposed endpoint demands. If you have an AI feature to build on private infrastructure, talk to BSH Technologies or explore our AI & automation services.