pgvector or a Dedicated Vector Database?
A team building an internal knowledge-search tool has 600,000 document chunks, expects about 25 queries/second at peak, and every query must filter to the requesting employee's department (dozens of departments, each owning a modest, roughly even slice of the corpus). They already run Postgres for the rest of the product. A senior engineer proposes Pinecone "because that's what everyone uses for RAG."
- Would you recommend
pgvectoror a dedicated vector database here, and why? - Which single input, if changed enough, would flip your recommendation, and to what new value?
- What operational cost is the team implicitly signing up for if they adopt a dedicated vector database at this scale?
1. Recommendation
pgvector. At 600k vectors and 25 QPS, corpus size and query volume
are both comfortably inside what a well-sized Postgres instance with
an HNSW index handles without strain — this is far below the point
where index-build time or RAM footprint become the bottleneck. The
filtering requirement reinforces the choice rather than complicating
it: department-scoped filtering is a normal indexed
WHERE department_id = ? predicate that the Postgres query planner
already reasons about jointly with the vector index scan. A dedicated
vector database would need first-class filtered-ANN support to match
that correctness and efficiency, which is solving a problem pgvector
gets for free from the relational engine. There is also no new system
to keep consistent with the source-of-truth document store — the
vectors live in the same transaction as the documents they describe.
2. What would flip it
Scale or QPS moved far enough: tens of millions of chunks (where HNSW build time and in-RAM graph size stop being affordable on a single instance or its read replicas) or QPS in the thousands (where a single Postgres primary, even with read replicas, becomes the bottleneck regardless of corpus size). A requirement for horizontally-sharded, distributed ANN across multiple regions would also flip it — that is exactly what dedicated systems like Milvus are purpose-built for and what a single relational database is not.
3. Implicit operational cost
A dedicated vector database is a new system: a separate cluster or managed service to provision and monitor, a new failure mode to build on-call runbooks for, and — critically — a sync pipeline to keep the vector store's copy of the data eventually consistent with the document store, including a story for what happens when that pipeline lags or fails (stale or missing vectors served during an outage). "Everyone uses it for RAG" is not a reason; the team should be able to say what bottleneck the new system removes that Postgres could not handle at this scale, and here there isn't one.
Share this question