Embeddings and Vector Databases

Embeddings and Vector Databases

To build systems where AI can search and retrieve relevant information, we need a way to represent the meaning of text mathematically. That's what embeddings are — and vector databases are the infrastructure built around them.

What is an Embedding?

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Text with similar meaning produces similar vectors — vectors that are "close" to each other in mathematical space.

For example:

• "The cat sat on the mat" and "A feline rested on the rug" would have similar embeddings
• "Quantum mechanics" and "How to bake bread" would have very different embeddings

Embeddings are produced by neural networks trained specifically for this task (like OpenAI's text-embedding-3, or open-source models like BGE or Nomic Embed). You pass in text, you get back a vector of typically 768 to 3,072 numbers.

The key property: semantic similarity = vector proximity.

Vector Databases

A vector database stores embeddings and lets you search them efficiently — specifically, finding the vectors nearest to a query vector. This is called approximate nearest-neighbour (ANN) search.

Popular vector databases:

• pgvector: a PostgreSQL extension — adds vector storage and search to a standard Postgres database
• Pinecone: managed, cloud-native vector database
• Weaviate: open-source, supports multiple modalities
• Chroma: lightweight, designed for AI applications
• Qdrant: open-source, high-performance

For most projects starting out, pgvector is a great choice — it extends a database you probably already have, and avoids adding new infrastructure.

The typical query: "give me the 5 chunks most semantically similar to this question." That's a vector similarity search.

Practical Example

Say you have 10,000 customer support tickets. You want to find tickets similar to a new one:

1. When each ticket arrives, generate its embedding and store it in pgvector
2. When a new ticket comes in, embed it
3. Run a nearest-neighbour query: "find the 5 stored tickets with the most similar vector"
4. Return those tickets to the support agent as "similar previous issues"

No keyword matching. No full-text search index tuning. Just semantic similarity — it works even when the wording is completely different but the meaning is the same.

Embeddings also power recommendation systems, duplicate detection, content moderation, and anomaly detection. They're one of the most practical and underused techniques in applied AI.