Dataset Card for CAMERA📷:

Table of Contents:

• Dataset Card for Camera
  • Table of Contents
  • Dataset Details
    • Dataset Description
    • Dataset Sources
  • Uses
    • Direct Use
    • Dataset Information
    • Data Example
    • Dataset Structure
  • Citation

Dataset Details

Dataset Description

CAMERA (CyberAgent Multimodal Evaluation for Ad Text GeneRAtion) is the Japanese ad text generation dataset, which comprises actual data sourced from Japanese search ads and incorporates annotations encompassing multi-modal information such as the LP images.

Dataset Sources

• Homepage: Github
• Paper: Striking Gold in Advertising: Standardization and Exploration of Ad Text Generation
  • [NEW!] Our paper has been accepted to ACL2024, and we will update the paper information as soon as the proceedings are published.

Uses

Direct Use

• Dataset with lp images (with-lp-images)

import datasets
dataset = datasets.load_dataset("cyberagent/camera", name="with-lp-images")

• Dataset without lp images (without-lp-images)

import datasets
dataset = datasets.load_dataset("cyberagent/camera", name="without-lp-images")

Dataset Information

• with-lp-images

DatasetDict({
    train: Dataset({
        features: ['asset_id', 'kw', 'lp_meta_description', 'title_org', 'title_ne1', 'title_ne2', 'title_ne3', 'domain', 'parsed_full_text_annotation', 'lp_image'],
        num_rows: 12395
    })
    dev: Dataset({
        features: ['asset_id', 'kw', 'lp_meta_description', 'title_org', 'title_ne1', 'title_ne2', 'title_ne3', 'domain', 'parsed_full_text_annotation', 'lp_image'],
        num_rows: 3098
    })
    test: Dataset({
        features: ['asset_id', 'kw', 'lp_meta_description', 'title_org', 'title_ne1', 'title_ne2', 'title_ne3', 'domain', 'parsed_full_text_annotation', 'lp_image'],
        num_rows: 872
    })
})

• without-lp-images

DatasetDict({
    train: Dataset({
        features: ['asset_id', 'kw', 'lp_meta_description', 'title_org', 'title_ne1', 'title_ne2', 'title_ne3', 'domain', 'parsed_full_text_annotation'],
        num_rows: 12395
    })
    dev: Dataset({
        features: ['asset_id', 'kw', 'lp_meta_description', 'title_org', 'title_ne1', 'title_ne2', 'title_ne3', 'domain', 'parsed_full_text_annotation'],
        num_rows: 3098
    })
    test: Dataset({
        features: ['asset_id', 'kw', 'lp_meta_description', 'title_org', 'title_ne1', 'title_ne2', 'title_ne3', 'domain', 'parsed_full_text_annotation'],
        num_rows: 872
    })
})

Data Example

{'asset_id': 6041,
 'kw': 'GLLARE MARUYAMA',
 'lp_meta_description': '美容サロン ブルーヘアー 札幌市 西区 琴似 創業34年 かゆみ、かぶれを防ぎ、美しい髪へ',
 'title_org': '北海道、水の教会で結婚式',
 'title_ne1': '',
 'title_ne2': '',
 'title_ne3': '',
 'domain': '',
 'parsed_full_text_annotation': {
  'text': ['表参道',
   '名古屋',
   '梅田',
    ...
   '成約者様専用ページ',
   '個人情報保護方針',
   '星野リゾートトマム'],
  'xmax': [163,
   162,
   157,
    ...
   1047,
   1035,
   1138],
  'xmin': [125,
   125,
   129,
    ...
   937,
   936,
   1027],
  'ymax': [9652,
   9791,
   9928,
    ...
   17119,
   17154,
   17515],
  'ymin': [9642,
   9781,
   9918,
    ...
   17110,
   17143,
   17458]},
 'lp_image': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=1200x17596>}

Dataset Structure

Citation

@inproceedings{mita-etal-2024-striking,
    title = "Striking Gold in Advertising: Standardization and Exploration of Ad Text Generation",
    author = "Mita, Masato  and
      Murakami, Soichiro  and
      Kato, Akihiko  and
      Zhang, Peinan",
    editor = "Ku, Lun-Wei  and
      Martins, Andre  and
      Srikumar, Vivek",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand and virtual meeting",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.acl-long.54",
    pages = "955--972",
    abstract = "In response to the limitations of manual ad creation, significant research has been conducted in the field of automatic ad text generation (ATG). However, the lack of comprehensive benchmarks and well-defined problem sets has made comparing different methods challenging. To tackle these challenges, we standardize the task of ATG and propose a first benchmark dataset, CAMERA, carefully designed and enabling the utilization of multi-modal information and facilitating industry-wise evaluations. Our extensive experiments with a variety of nine baselines, from classical methods to state-of-the-art models including large language models (LLMs), show the current state and the remaining challenges. We also explore how existing metrics in ATG and an LLM-based evaluator align with human evaluations.",
}