HNSW vs IVF: Choosing and Tuning an ANN Index

You're indexing 40 million product-description embeddings for semantic search. The infrastructure team is memory-constrained and pushes back on any index that "keeps a big graph in RAM." Meanwhile product wants recall to stay high even under that constraint.

1. Explain, at the level of how each algorithm actually searches, why HNSW uses more memory than IVF.
2. Given the memory constraint, which would you propose, and how would you recover recall using its tunable parameter?
3. A colleague reports "recall dropped after we lowered nprobe to speed up queries, and it's worse for some categories of products than others." Explain why an uneven drop, not just a uniform one, is expected.