MT-Mind2Web Dataset

MT-Mind2Web is constructed by using the single-turn interactions from Mind2Web, an expert-annotated web navigation dataset, as the guidance to construct conversation sessions.

Statistics

Dataset Structure

• "task_id" (str): unique id for each task
• "website" (str): website name
• "domain" (str): website domain
• "subdomain" (str): website subdomain
• "turns" (list[dict]): list of subtasks
  • "annotation_id" (str): unique id for each subtask
  • "confirmed_task" (str): subtask description
  • "action_reprs" (list[str]): human readable string representation of the action sequence
  • "actions" (list[dict]): list of actions (steps) to complete the subtask
    • "action_uid" (str): unique id for each action (step)
    • "raw_html" (str): raw html of the page before the action is performed
    • "cleaned_html" (str): cleaned html of the page before the action is performed
    • "operation" (dict): operation to perform
      • "op" (str): operation type, one of CLICK, TYPE, SELECT
      • "original_op" (str): original operation type, contain additional HOVER and ENTER that are mapped to CLICK, not used
      • "value" (str): optional value for the operation, e.g., text to type, option to select
    • "pos_candidates" (list[dict]): ground truth elements. Here we only include positive elements that exist in "cleaned_html" after our preprocessing, so "pos_candidates" might be empty. The original labeled element can always be found in the "raw_html".
      • "tag" (str): tag of the element
      • "is_original_target" (bool): whether the element is the original target labeled by the annotator
      • "is_top_level_target" (bool): whether the element is a top level target find by our algorithm. please see the paper for more details.
      • "backend_node_id" (str): unique id for the element
      • "attributes" (str): serialized attributes of the element, use json.loads to convert back to dict
    • "neg_candidates" (list[dict]): other candidate elements in the page after preprocessing, has similar structure as "pos_candidates"