trajectories.zip contains all trajectories evaluated in our paper, including

• WebArena
  • GPT-4 CoT baseline agent extracted from WebArena official release
• Android
  • Auto-UI base/large
  • CogAgent
  • Human demonstrations from Android-in-the-Wild
• iOS
  • CogAgent basline
  • CogAgent + Self-training
  • CogAgent + Filtered BC

ios-offline-trajectories.zip contains CogAgent's trajectories on iOS emulator, which is then used to refine the CogAgent model through Filtered BC

webarena-reflexion-<evaluator-type).zip contains webarena trajectories on refining the GPT-4 CoT baseline with a model-based evaluator (Figure 3 of our paper)

• GPT-4V
• Captioner + GPT-4
• Captioner + Mixtral
• Oracle Evaluator