README
TriplaneTurbo: Instant Text-to-3D Generator from Multiple 2D Diffusion Priors
Paper | Project Page
⚙️ Dependencies and Installation
Follow threestudio to set up the conda environment, or use our provided instructions as below.
- Create a virtual environment:
conda create -n scaledreamer python=3.10
conda activate scaledreamer
- Add plugin
mkdir custom
cd custom
git clone https://github.com/87003697/amortized
cd ..
- Install PyTorch
# Prefer using the latest version of CUDA and PyTorch
conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=12.1 -c pytorch -c nvidia
- (Optional, Recommended) Install xFormers for attention acceleration.
conda install xformers -c xformers
- (Optional, Recommended) Install ninja to speed up the compilation of CUDA extensions:
pip install ninja
- Install major dependencies:
pip install -r requirements.txt
export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
pip install git+https://github.com/KAIR-BAIR/nerfacc.git@v0.5.2
If you encounter errors while installing iNGP, it is recommended to check your gcc version. Follow these instructions to change the gcc version within your conda environment. Then return to the repository directory to install iNGP and NerfAcc ⬆️ again.
conda install -c conda-forge gxx=9.5.0
cd $CONDA_PREFIX/lib
ln -s /usr/lib/x86_64-linux-gnu/libcuda.so ./
cd <your repo directory>
- Install CUDA KNN extension for point-based rendering:
cd custom/primiturbo/extern/knn
python build_ext.py # Compile the CUDA KNN extension
python setup.py install # Install the extension
cd ../../../../
Download 2D Diffusion Priors.
- Save SD-v2.1-base and MVDream to the local directory
pretrained.
python scripts/download_pretrained_models.py
🕹️ Experiments
Git update
Use the following code to update the code to the latest version
git pull
cd custom/amortized
git pull
cd ../..
Group-0 08.08.2024
sh scripts/group_0/exp1_3DTopia_step_scheduler_4_triple_16_vanilla_16_bias_lora_prompt_60k.sh
sh scripts/group_0/exp2_3DTopia_step_4_triple_16_vanilla_16_bias_lora_prompt_60k.sh
Each experiment lasts for around 5 days.
Results are saved in outputs_group_0 directory.
Group-1 09.05.2024
Update with the following step:
-
Please git update first. Run
git pullin two directories, as explained ⬆️. -
Please prepare sd-turbo and sd-v1.5 into
pretraineddirectory. -
Run the following script to download another diffusion prior
wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/RichDreamer/nd_mv_ema.ckpt -O ./pretrained/nd_mv_ema.ckpt
Now the files in pretrained directory is as follows. A little bit messy but will be refined in the future.
pretrained
├── sd-turbo
├── stable-diffusion-v1-5
├── stable-diffusion-2-1-base
├── nd_mv_ema.ckpt
├── sd-v2.1-base-4view
Two experiments are as follows.
sh scripts/group_1/exp1_3DTopia__base_step_4__asd_mv+rd+sd_volsdf+cpumc__triple_16_vanilla_16_bias_lora_prompt.sh
sh scripts/group_1/exp2_3DTopia__turbo_step_4__asd_mv+rd+sd_volsdf+cpumc__triple_16_vanilla_16_bias_lora_prompt.sh
Each experiment lasts for around 7 days.
Results are saved in outputs_group_1 directory.
Contributors
Similar repos
Languages
Python
68.9%
Shell
29.6%
Jupyter Notebook
1.3%
README
TriplaneTurbo: Instant Text-to-3D Generator from Multiple 2D Diffusion Priors
Paper | Project Page
⚙️ Dependencies and Installation
Follow threestudio to set up the conda environment, or use our provided instructions as below.
- Create a virtual environment:
conda create -n scaledreamer python=3.10
conda activate scaledreamer
- Add plugin
mkdir custom
cd custom
git clone https://github.com/87003697/amortized
cd ..
- Install PyTorch
# Prefer using the latest version of CUDA and PyTorch
conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=12.1 -c pytorch -c nvidia
- (Optional, Recommended) Install xFormers for attention acceleration.
conda install xformers -c xformers
- (Optional, Recommended) Install ninja to speed up the compilation of CUDA extensions:
pip install ninja
- Install major dependencies:
pip install -r requirements.txt
export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
pip install git+https://github.com/KAIR-BAIR/nerfacc.git@v0.5.2
If you encounter errors while installing iNGP, it is recommended to check your gcc version. Follow these instructions to change the gcc version within your conda environment. Then return to the repository directory to install iNGP and NerfAcc ⬆️ again.
conda install -c conda-forge gxx=9.5.0
cd $CONDA_PREFIX/lib
ln -s /usr/lib/x86_64-linux-gnu/libcuda.so ./
cd <your repo directory>
- Install CUDA KNN extension for point-based rendering:
cd custom/primiturbo/extern/knn
python build_ext.py # Compile the CUDA KNN extension
python setup.py install # Install the extension
cd ../../../../
Download 2D Diffusion Priors.
- Save SD-v2.1-base and MVDream to the local directory
pretrained.
python scripts/download_pretrained_models.py
🕹️ Experiments
Git update
Use the following code to update the code to the latest version
git pull
cd custom/amortized
git pull
cd ../..
Group-0 08.08.2024
sh scripts/group_0/exp1_3DTopia_step_scheduler_4_triple_16_vanilla_16_bias_lora_prompt_60k.sh
sh scripts/group_0/exp2_3DTopia_step_4_triple_16_vanilla_16_bias_lora_prompt_60k.sh
Each experiment lasts for around 5 days.
Results are saved in outputs_group_0 directory.
Group-1 09.05.2024
Update with the following step:
-
Please git update first. Run
git pullin two directories, as explained ⬆️. -
Please prepare sd-turbo and sd-v1.5 into
pretraineddirectory. -
Run the following script to download another diffusion prior
wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/RichDreamer/nd_mv_ema.ckpt -O ./pretrained/nd_mv_ema.ckpt
Now the files in pretrained directory is as follows. A little bit messy but will be refined in the future.
pretrained
├── sd-turbo
├── stable-diffusion-v1-5
├── stable-diffusion-2-1-base
├── nd_mv_ema.ckpt
├── sd-v2.1-base-4view
Two experiments are as follows.
sh scripts/group_1/exp1_3DTopia__base_step_4__asd_mv+rd+sd_volsdf+cpumc__triple_16_vanilla_16_bias_lora_prompt.sh
sh scripts/group_1/exp2_3DTopia__turbo_step_4__asd_mv+rd+sd_volsdf+cpumc__triple_16_vanilla_16_bias_lora_prompt.sh
Each experiment lasts for around 7 days.
Results are saved in outputs_group_1 directory.
Similar repos
Contributors
Languages
Python
68.9%
Shell
29.6%
Jupyter Notebook
1.3%