Vector DB and ANN vs PHE conflict, is there a practical workaround? [D]

Reddit r/MachineLearning / 4/30/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The post discusses a core tension between using vector databases with ANN indexing (e.g., HNSW/IVF) and applying Partially Homomorphic Encryption (PHE), since encrypted embeddings often prevent efficient ANN search and force slower exact or linear scanning.
The author considers an alternative architecture: store encrypted embeddings as BLOBs in a regular database and use metadata-based filtering (e.g., RFID/tag-like narrowing) to reduce the candidate set before running similarity on a smaller subset.
Key concerns include whether this metadata-first filtering approach can scale to around one million+ embeddings and whether database retrieval/filtering would outperform ANN in real deployments.
The author asks the community for practical methods to combine ANN with encrypted embeddings, including potential hybrid approaches such as secure enclaves, partial decryption, tiered search, and examples of real-world privacy-preserving vector search systems at scale.
Overall, the goal is to balance privacy (secure storage/processing of embeddings) with performance (fast retrieval) while avoiding simply reinventing a less efficient vector database.

Hey everyone,

I have been digging into vector databases, ANN search, and privacy preserving techniques (specifically PHE), and I have hit a design roadblock that I would love some input on.

The problem:

Using a vector DB with ANN (HNSW, IVF, etc.) is great for fast similarity search at scale.

But if we introduce Partially Homomorphic Encryption (PHE), we lose the ability to efficiently use ANN.

This happens because encrypted embeddings force us into linear scan or exact computation, which makes ANN useless.

What I am considering:

One workaround I thought of is to drop the vector DB entirely, store embeddings in a standard database as BLOBs, and use something like RFID or tag based filtering to narrow down candidates before computing similarity.

The idea is to reduce the search space first using metadata, then run similarity on a much smaller subset.

Concerns:

Will this scale to millions of embeddings?

Is database retrieval and filtering actually faster than ANN in practice?

Am I just reinventing a worse version of a vector database?

Questions for the community:

Is there a practical way to combine ANN with encrypted embeddings?
Are there hybrid approaches like secure enclaves, partial decryption, or tiered search that actually work in production?
Would a metadata first filtering pipeline (RFID or tags to subset to similarity) scale better than I think?
Are there any real world systems doing privacy preserving vector search at scale?

Context:

Potential scale is around 1 million plus embeddings.

Priority is balancing privacy and performance.

Use case is fast retrieval with secure storage of embeddings.

Would really appreciate any insights, papers, or architecture suggestions.

submitted by /u/XPERT_GAMING
[link] [comments]