Sizing and Sharding a Large Embedding Table

An ads CTR model needs embeddings for 800 M user IDs, 50 M ad IDs and one user×ad-category cross with 2 B possible values. The plan is 32 dimensions in fp32 for every table.

1. Compute the naive memory footprint of the three tables.
2. Propose a concrete plan (hashing bucket counts, dimension choices, sharding) to bring the footprint under 20 GB while limiting damage to accuracy. Show the arithmetic.
3. Why is a parameter-server style architecture (or row-wise sharding across GPUs) more appropriate here than plain data-parallel all-reduce?