Floater-Free-Gaussian-Splatting

This repo uses the absolute value of the gradient of pixel pairs for GS to accumulate the gradient of each GS. Since the preprint paper absGS did the same thing, therefore, you can also consider this repo as an unofficial implementation of absGS.

Compared to Pixel-GS, our project can achieve the removal of floaters without significantly increasing the number of GS. In some scenarios where the point cloud distribution is good, it can reduce the number of point clouds. Compared to Radsplat, our method does not require training zipnerf, and the training time on a 3090 is approximately 30 minutes.

Dataset

In this project, you can use:

• synthetic dataset from NeRF, and NSVF
• real-world dataset from Mip-NeRF 360 and tandt_db.

And the data structure should be organized as follows:

data/
├── NeRF
│   ├── Chair/
│   ├── Drums/
│   ├── ...
├── NSVF
│   ├── Bike/
│   ├── Lifestyle/
│   ├── ...
├── Mip-360
│   ├── bicycle/
│   ├── bonsai/
│   ├── ...
├── tandt_db
│   ├── db/
│   │   ├── drjohnson/
│   │   ├── playroom/
│   ├── tandt/
│   │   ├── train/
│   │   ├── truck/

Run

Environment

git clone https://github.com/ingra14m/floater-free-gaussian-splatting --recursive
cd floater-free-gaussian-splatting

conda create -n abs-gaussian-env python=3.8
conda activate abs-gaussian-env

# install pytorch
pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116

# install dependencies
pip install -r requirements.txt

Train

python train.py -s your/path/to/the/dataset -m your/path/to/save --eval

# Mip-360
python train.py -s your/path/to/the/dataset -m your/path/to/save --eval -r [2/4]

# Others
python train.py -s your/path/to/the/dataset -m your/path/to/save --eval

It should be noted that we adopted the same approach as ZipNeRF, Pixel-GS, and RadSplat, downsampling outdoor scenes (bicycle, garden, stump, flower, treehill) by 4 times and indoor scenes (bonsai, counter, kitchen, room) by 2 times.

Render

python render.py -m your/path/to/save --eval --skip_train

Render Video

python render.py -m your/path/to/save --eval --skip_train --skip_test --render_video

Metrics

python metrics.py -m your/path/to/save

Results

Mip360

https://github.com/ingra14m/robust-gaussian-splatting/assets/63096187/e0f34e5f-ee83-442b-86fc-422c57c40a6b

https://github.com/ingra14m/floater-free-gaussian-splatting/assets/63096187/52d99636-cc9c-4b57-aaf1-651759e7e4c7

https://github.com/ingra14m/floater-free-gaussian-splatting/assets/63096187/0b1b9687-d75e-4c69-bc98-34c37c56e86c

NeRF

NSVF

Methods

Don't forget to install the new diff-gaussian-rasterization in diff-gaussian-rasterization-extentions. This pipeline supports pre-filter, depth visualization, and uses additional variables to store the contribution of each pixel to the GS gradient (the contribution should obviously be positive).

// vanilla gradients for densification
atomicAdd(&dL_dmean2D[global_id].x, dL_dG * dG_ddelx * ddelx_dx);
atomicAdd(&dL_dmean2D[global_id].y, dL_dG * dG_ddely * ddely_dy);

// abs gradients for densification
atomicAdd(&dL_dmean2D_densify[global_id].x, fabsf(dL_dG * dG_ddelx * ddelx_dx));
atomicAdd(&dL_dmean2D_densify[global_id].y, fabsf(dL_dG * dG_ddely * ddely_dy));

BibTex

This idea is the same as absGS and Gaussian Opacity Fields. The difference is that we have set the densify_grad_threshold to 0.0005, and all other parameters are used as in vanilla 3D-GS. If you find this project useful, please don't forget to cite these two awesome papers.

@article{ye2024absgs,
  title={AbsGS: Recovering Fine Details for 3D Gaussian Splatting},
  author={Ye, Zongxin and Li, Wenyu and Liu, Sidun and Qiao, Peng and Dou, Yong},
  journal={arXiv preprint arXiv:2404.10484},
  year={2024}
}

@article{Yu2024GOF,
  author    = {Yu, Zehao and Sattler, Torsten and Geiger, Andreas},
  title     = {Gaussian Opacity Fields: Efficient High-quality Compact Surface Reconstruction in Unbounded Scenes},
  journal   = {arXiv:2404.10772},
  year      = {2024},
}