NTU Smart Construction Industry-Academia Collaboration (ICM)

This project is a collaboration between National Taiwan University and the industry on smart construction, focusing on the development of an image annotation and analysis system for building safety inspections.

Project Introduction

This project integrates multiple deep learning models and technologies, including:

• Image Annotation System: Image captioning based on the CLIP model.
• Violation Detection: Automatic identification system for construction safety violations.
• Regulation Matching: Smart matching and analysis of legal regulations.
• Attention Mechanism Visualization: Visualization and analysis of the model's decision-making process.

Project Structure

├── models/                # Trained models
│   ├── attention_model/   # Attention mechanism model
│   ├── regulation_model/  # Regulation matching model
│   └── llm_models/        # Large language models
├── scripts/               # Training and prediction scripts
│   ├── CLIP/              # CLIP related scripts
│   └── CLIP_prefix_caption/ # CLIP prefix captioning
├── docs/                  # Documentation
├── notebooks/             # Jupyter notebooks
├── experiments/           # Experiment results
├── logs/                  # Execution logs
└── ntu_scon_final_report  # Project introduction and final report

Key Features

1. Image Annotation System

• Multimodal image understanding based on the CLIP model.
• Supports Chinese image caption generation.
• Integrated with a Traditional Chinese language model for Taiwan.

2. Violation Detection System

• Automatic identification of construction safety violations.
• Supports classification of multiple violation types.
• Provides visualization of detection results.

3. Regulation Matching System

• Smart matching of legal regulations.
• Supports querying a regulation database.
• Provides analysis of violations and corresponding regulations.

License

N/A

Contact Information

For questions or suggestions, please contact the project team.