KnowledgeXLab/ComAct

Official Implementation of "ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm".

13

stars

6

commits

Python

primary language

Jul 1, 2026

updated

KnowledgeXLab.github.io/ComAct

README

ComAct: Reframing Professional Software Manipulation through COM-as-Action Paradigm

arXiv Project Page License Hugging Face Model Hugging Face Benchmark Hugging Face Dataset

The official implementation of ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm. We identify the Component Object Model (COM) as a unified executable action space across professional software, reframing GUI-fragile, API-fragmented automation as deterministic program synthesis. See the project page for demos and full results.

📰 Updates

  • 2026-06-20: 🎉 Paper and project page are publicly available.

🎯 Getting Started

🚧 Coming soon: a detailed guide for preparing dataset, setting up VM environment and running agent training & evaluation end-to-end.

Here's a quick orientation:

ComponentDescriptionLink
Benchmark1,000 tasks across SolidWorks, Inventor, AutoCAD; 7 engineering activities🤗 ComCADBench
EnvironmentParallel Dockerized Windows VMs for training & evaluationcomforge/
Agent9B self-correcting agent via 3-stage training: text-to-code SFT → agentic SFT → GRPO🤗 ComActor

🎓 Acknowledgements

We acknowledge the outstanding open-source contributions from Qwen3.5, ms-swift, Text2CAD, Fusion 360 Gallery, and SketchGraphs.

📬 Contact

For any questions or feedback, please:

📜 Citation

If you find our paper and code useful, please kindly cite us.

@misc{ai2026comactreframingprofessionalsoftware,
      title={ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm},
      author={Jiaxin Ai and Tao Hu and Xuemeng Yang and Shu Zou and Hairong Zhang and Daocheng Fu and Yu Yang and Hongbin Zhou and Nianchen Deng and Pinlong Cai and Zhongyuan Wang and Botian Shi and Kaipeng Zhang and Licheng Wen},
      year={2026},
      eprint={2606.13239},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2606.13239},
}

Contributors

jiaxin-ai

4 commits

projudge

2 commits

KnowledgeXLab/ComAct

Official Implementation of "ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm".

13

stars

6

commits

Python

primary language

Jul 1, 2026

updated

KnowledgeXLab.github.io/ComAct

README

ComAct: Reframing Professional Software Manipulation through COM-as-Action Paradigm

arXiv Project Page License Hugging Face Model Hugging Face Benchmark Hugging Face Dataset

The official implementation of ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm. We identify the Component Object Model (COM) as a unified executable action space across professional software, reframing GUI-fragile, API-fragmented automation as deterministic program synthesis. See the project page for demos and full results.

📰 Updates

  • 2026-06-20: 🎉 Paper and project page are publicly available.

🎯 Getting Started

🚧 Coming soon: a detailed guide for preparing dataset, setting up VM environment and running agent training & evaluation end-to-end.

Here's a quick orientation:

ComponentDescriptionLink
Benchmark1,000 tasks across SolidWorks, Inventor, AutoCAD; 7 engineering activities🤗 ComCADBench
EnvironmentParallel Dockerized Windows VMs for training & evaluationcomforge/
Agent9B self-correcting agent via 3-stage training: text-to-code SFT → agentic SFT → GRPO🤗 ComActor

🎓 Acknowledgements

We acknowledge the outstanding open-source contributions from Qwen3.5, ms-swift, Text2CAD, Fusion 360 Gallery, and SketchGraphs.

📬 Contact

For any questions or feedback, please:

📜 Citation

If you find our paper and code useful, please kindly cite us.

@misc{ai2026comactreframingprofessionalsoftware,
      title={ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm},
      author={Jiaxin Ai and Tao Hu and Xuemeng Yang and Shu Zou and Hairong Zhang and Daocheng Fu and Yu Yang and Hongbin Zhou and Nianchen Deng and Pinlong Cai and Zhongyuan Wang and Botian Shi and Kaipeng Zhang and Licheng Wen},
      year={2026},
      eprint={2606.13239},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2606.13239},
}

Contributors

jiaxin-ai

4 commits

projudge

2 commits

Languages

Python

92.5%

Shell

6.8%