How to Set Up a Vector Database for Free

You can set up a vector database for free in minutes

Setting up a vector database for free is genuinely a few-minutes job: enable the pgvector extension on a PostgreSQL instance you already run, or start ChromaDB locally with a single import. Both store embeddings — the numerical vectors that capture meaning — and find the nearest ones to a query, which is the engine behind semantic search and RAG. Neither costs anything to begin with, and both scale far enough to prove real value before you spend anything at all.

The choice between them is mostly about durability and who else uses the data. Chroma is the fastest path for a local, single-developer prototype; pgvector is the better foundation for anything that needs backups, multiple users, or metadata filtering at scale. You can start with either and migrate later, because the concepts — vectors, dimensions, distance metrics, indexes — are identical across both.

Option A: pgvector on PostgreSQL

If you have Postgres — locally, in Docker, or on a free tier from Supabase or Neon — pgvector turns it into a vector store without a new service to operate, back up, or secure. That consolidation is a genuine advantage: you reuse the operational knowledge and tooling you already have rather than adopting a separate system just for vectors.

• Enable the extension on your database in a single statement.
• Create a table with a vector column whose dimension matches your embedding model — for example 384 for all-MiniLM-L6-v2, or 1536 for many hosted embedding APIs.
• Insert each row with its vector plus ordinary columns for source, text, and any metadata you want to filter on.
• Add an HNSW index once you pass a few thousand rows so similarity queries stay fast.

Option B: ChromaDB locally

ChromaDB is an open-source vector database designed to run with almost no setup, which makes it ideal when you just want to validate an idea quickly. Install it, create a collection, and add documents with their embeddings — Chroma can even call an embedding function for you, so you write very little glue code.

Chroma persists to disk when you point it at a directory, so your prototype survives a restart instead of vanishing into memory the moment the process ends.

Step: Get the embedding dimension right

The single most common setup error is a dimension mismatch, and it trips up nearly everyone the first time. Every embedding model outputs vectors of a fixed length, and your column or collection must match it exactly. Insert a 384-dimensional vector into a column expecting 1536 and the database rejects it — or, in some configurations, silently misbehaves in ways that are maddening to debug later.

• Check your embedding model's output dimension before creating the table or collection.
• Use the same model for stored documents and incoming queries; vectors from different models live in incompatible spaces and are not comparable.
• If you switch embedding models later, every stored vector must be regenerated, so keep the source text to rebuild from.

Step: Run your first similarity query

With data loaded, query by embedding a piece of text and asking the store for its nearest neighbours. In pgvector you order by the distance operator that matches your metric — cosine is the usual default for text — and limit to the top few. In Chroma you call the collection's query method with the query text or vector, and it returns the closest documents with their distances.

• Match the distance metric to your embedding model; cosine, dot product, and Euclidean are not interchangeable and the wrong one degrades results silently.
• Return the stored text and metadata, not just the vector, so the results are actually useful to act on.
• Treat the distance as a confidence signal — a very weak best match usually means no good answer exists.

Step: Know the free-tier limits

Free is perfect for learning, but plan ahead so the ceiling does not surprise you. Free Postgres tiers cap storage, connections, and compute; local Chroma is bounded by your machine's memory and has no built-in replication. Both are excellent for a prototype and neither is a production deployment on its own. When real load arrives you will want managed hosting, backups, and monitoring — the unglamorous infrastructure that keeps a service reliable once people depend on it.

Prefer it built and managed for you?

BSH Technologies sets up and operates vector infrastructure that scales past the free-tier ceiling — tuned indexes, correct metrics, metadata filtering, and a re-embedding plan for model upgrades. If semantic search is on your roadmap, talk to BSH Technologies or explore our AI & automation services.