Echoes of the Age of Exploration: Vector Databases and Why Most Explorers Died
Echoes of the Age of Exploration: Vector Databases and Why Most Explorers Died
In the Age of Exploration, hundreds of expeditions set sail. Most explorers died - from scurvy, shipwrecks, hostile encounters, or simply getting lost. The few who survived became legends and their routes became trade networks that lasted centuries. The vector database landscape looks eerily similar.
The Current Gold Rush
Count the vector databases that launched in the last three years: Pinecone, Weaviate, Qdrant, Milvus, Chroma, LanceDB, Vespa, Marqo, and dozens more. Each one claims to be the fastest, most scalable, or most developer-friendly. Each one raised venture funding. Each one is burning cash.
Most of them will not survive. Not because they are bad products, but because the market cannot support 30 specialized vector databases. The same way the Age of Exploration could not support 300 competing trade routes.
Why Most Will Fail
The core problem - embeddings are becoming a commodity feature. PostgreSQL has pgvector. SQLite has sqlite-vss. Every major cloud database added vector search. When the incumbent databases offer "good enough" vector search as a checkbox feature, the standalone vector database needs to be dramatically better to justify a separate dependency.
What Survives
The expeditions that survived the Age of Exploration shared common traits:
- They found a genuinely valuable route that incumbents could not replicate
- They built infrastructure (ports, maps, relationships) that created switching costs
- They adapted to conditions rather than following a rigid plan
Vector databases that survive will similarly need to own a niche that general-purpose databases cannot serve - real-time similarity search at massive scale, specialized indexing for specific embedding types, or deep integration with specific AI frameworks.
The Lesson for Builders
If you are building on a vector database today, choose the way you would choose a shipping route in 1500 - favor the one with the strongest backing, the most established ports of call, and the highest likelihood of still existing in five years. Or use pgvector and avoid the question entirely.
Fazm is an open source macOS AI agent. Open source on GitHub.