Arguing Against End-to-End Scene Regeneration

1. Why the proposal fails on fidelity, regardless of output quality

A diffusion model conditioned on a prompt and a reference photo is optimizing for plausibility, not identity — nothing in its training objective or sampling process guarantees that the specific product in the output is pixel-for-pixel the same object as the input, only that the output is a plausible, prompt-consistent image. Even a diffusion model that reproduces the product extremely well most of the time will, on some fraction of requests, subtly alter shape, color, proportions, or printed text — because it has no explicit mechanism forcing exact reproduction of a specific region, only a general tendency (from strong conditioning) to stay close to the reference. Given the Step 1 assumption that product fidelity is a hard, non-negotiable constraint (not a quality target to optimize), "usually good" is not an acceptable failure rate — a shopper who buys based on an image where the product was regenerated slightly differently has a legitimate complaint that has nothing to do with how aesthetically pleasing the image is.

2. Can architecture selection alone fix this?

No. The comparison table's five axes (latency, quality, diversity, training stability, controllability) all describe properties of how well a generative model produces plausible images, not whether it can guarantee exact preservation of a specific region. Swapping to a different generative architecture — a GAN, an autoregressive model, a different diffusion variant — doesn't change the fundamental issue: any model whose job is "generate a plausible image conditioned on this" is optimizing the wrong objective for a hard identity constraint. This is a structural mismatch between the task (exact-preservation-plus-generation) and what generative models as a class are built to guarantee, not a quality gap any specific architecture choice on that table closes.

3. The minimal fix, and what it does/doesn't guarantee

Add an explicit segmentation/matting stage before generation, and condition the generative model on the resulting mask so that the product region is either (a) hard-composited back in after generation using the original, untouched cutout pixels, or (b) clamped in the diffusion latent at every denoising step so the model is never actually free to modify that region's content. This keeps "one diffusion pass for the background" while adding a discriminative, non-generative stage specifically for the part of the problem that needs an exact guarantee rather than a plausible one. What it guarantees: the pixels inside the mask are provably unchanged (or, with latent clamping, effectively unchanged up to decode-time artifacts), since they are either copied directly or held fixed through generation. What it does not guarantee: that the mask itself is perfectly accurate — a matting error (a slightly wrong boundary, a missed thin edge like a strap or a wire) can still let a sliver of the product region be affected by the surrounding generation, which is exactly why matting IoU and boundary-focused losses (Step 5) and product-region invariance as a post-hoc gate (Step 6) are both necessary — the architecture fix reduces the problem to "get the mask right," it doesn't eliminate the need to verify the mask and the composite were actually correct.