PixSDS: Why Latent SDS Makes Noisy Pixels

Vsevolod Skorokhodov

EPFL

ECCV 2026 Workshop on Low-Level Vision Frontiers (LoViF)

Paper Code ArXiv coming soon Workshop

PixSDS repairs latent score distillation gradients to reduce structured artifacts while preserving semantic content.

Abstract

Score Distillation Sampling (SDS) enables text-to-3D generation by optimizing rendered images with a pretrained diffusion prior, but latent SDS often produces structured color artifacts and high-frequency texture noise.

We identify a failure mode caused by VAE-induced pixel drift: an optimized image can move along pixel-space directions that are weakly constrained by the VAE encoder. Its latent representation remains clean and semantically meaningful while the image itself accumulates visible artifacts.

Motivated by this observation, we propose PixSDS, a lightweight VAE-consistent gradient repair method. PixSDS decodes a latent SDS look-ahead step and uses the decoded image as a clean direction for pixel-space optimization. It requires no diffusion-model retraining, renderer changes, or replacement of the SDS objective. Experiments in controlled 2D optimization and text-to-3D generation show substantially fewer structured artifacts while preserving semantic content.

Observation

During latent SDS optimization, we observe that even though the images are noisy, their VAE representations remain clean and semantically meaningful.

Clean Latent SDS representation despite noisy pixels

Method

Latent SDS can keep an image's VAE representation clean while allowing its pixels to drift into structured noise. PixSDS adds a clean, VAE-consistent direction to the original SDS update.

1. Look ahead

Apply the latent-space SDS step to obtain the next latent target.

2. Decode

Decode that target through the pretrained VAE into a clean image direction.

3. Repair

Combine the normalized clean direction with the original semantic SDS update.

2D Generation Results

In controlled text-to-2D optimization, PixSDS reduces structured color artifacts and high-frequency noise across prompts. It achieves the strongest FID, BRISQUE, and CLIP-IQA noisiness scores among the evaluated SDS-style optimization methods while remaining competitive in semantic alignment and perceived quality.

Each row should show one prompt, with methods arranged in columns for direct comparison.

3D Generation Results

PixSDS integrates into existing text-to-3D pipelines. In DreamGaussian it produces cleaner textures with fewer structured patterns; in LucidDreamer it reduces floating noisy Gaussians and artifacts around the generated asset.

DreamGaussian

Cleaner surface textures and fewer structured artifacts than the SDS baseline.

LucidDreamer

Fewer floating Gaussians and reduced noise around the generated objects.

Ablation Study

The latent look-ahead target and per-pixel normalization are both important for balancing clean guidance with semantic placement.

BibTeX

@inproceedings{skorokhodov2026pixsds,
  author    = {Skorokhodov, Vsevolod},
  title     = {PixSDS: Why Latent SDS Makes Noisy Pixels},
  booktitle = {ECCV Workshop on Low-Level Vision Frontiers (LoViF)},
  year      = {2026}
}