Universal 3D Shape Matching via Coarse-to-Fine Language Guidance

under development...

Preprocessing

Perform segmentation, part name inference and post-processing.

1. run_partfield_features.py

   • Extract partfield features from a set of .ply files.
   • The features are saved in a .npz file.

2. run_partfield_segmentation.py

   • Segment a .ply file into parts using partfield features.
   • The segmentation result is saved in a .ply file.

3. process_segmentation_results.py

   • Process the segmentation results.

4. run_gpt_inference.py

   • Infer the part labels of a .ply file using GPT.

5. process_gpt_results.py

   • Back-project GPT results for 2D images to 3D

Semantic Feature Extraction

• Extract visual features using DINO
• The visual features are upscaled using JAFAR
• Extract language embeddings using FGClip