This Is The World (Recommender Systems Made It For You): Capturing Risk Pathways for Children through a Multi-Risk, Pluralistic Assessment of Recommenders

This repository contains the code to reproduce the experiments from the paper: "This Is The World (Recommender Systems Made It For You): Capturing Risk Pathways for Children through a Multi-Risk, Pluralistic Assessment of Recommenders" by Robin Ungruh and Maria Soledad Pera, accepted as a Full Paper at UMAP 2026, which can be found here.

Abstract

Recommender systems increasingly shape children's online experiences, yet the risks they pose to children remain poorly understood. Formal risk assessments are rare, overlook children's experiences, treat them as a homogeneous group, and typically consider broader compliance-oriented risks on a population-wide level. To provide a more nuanced and child-centered perspective, we conduct a comprehensive risk assessment that examines multiple risks simultaneously, and captures individual-level exposure rather than relying solely on aggregated metrics. Further, we explore the behavioral characteristics that make some children more susceptible to such exposure to grasp how individual differences shape the manifestation of risks in practice. Our results highlight the need for risk assessments that recognize heterogeneity among child users and account for the conditional nature of online harm. Further, findings spark the discussion about the potential impact that diverse risks present in recommendation environments can have on children.

Citation

Robin Ungruh and Maria Soledad Pera. 2025. This Is The World (Recommender Systems Made It For You): Capturing Risk Pathways for Children through a Multi-Risk, Pluralistic Assessment of Recommenders. In Proceedings of the 30th ACM conference on user modeling, adaptation and personalization (UMAP '26), June 8 - 11, 2026, Gothenburg, Sweden. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3774935.3806148

Reproducibility Code

Set up environment

conda env create -f environment.yml -n 2026_UMAP_CRA

conda activate 2026_UMAP_bs

pip install -r requirements.txt

pip install -e .

Data

Please create a data directory which includes the relevant data. This should include the Goodreads dataset, including relevant subsets (as seen below). Add the raw data as seen below.

├── data
│   ├── raw
│   │   ├── Goodreads
│   │   │   ├── goodreads_books_children.json.gz
│   │   │   ├── goodreads_books_comics_graphic.json.gz
│   │   │   ├── ...
│   │   │   ├── goodreads_books_young_adult.json.gz
│   │   │   ├── goodreads_books.json.gz
│   │   │   ├── goodreads_book_works.json.gz
│   │   │   ├── goodreads_interactions.csv
│   │   │   ├── book_id_map.csv
│   ├── processed
│   │   ├── Goodreads
│   │   │   ├── rec
│   │   │   │   ├── child_eval_test.tsv
│   │   │   │   ├── child_eval_UPs.tsv
│   │   │   │   ├── train.tsv
│   │   │   │   ├── val.tsv
│   │   │   ├── ...
│   ├── StereoBias
│   │   ├── stereotype
│   │   │   ├── train.tsv
│   │   │   ├── test.tsv
│   │   │   ├── val.tsv
├── ...

Add the absolute path of data to config.env.

Preprocessing

Goodreads

GR_create_genre_dict.py
GR_process_works.py
GR_preprocess.py
create_user_profiles --dataset gr -all_interactions
analyze_child_proportions.ipynb
stratified_sampling.ipynb
GR_splitting.py

Recommendation Experiment

Hyperparameter Validation

As the validation of Popularity and Random is very expensive (RecPack implementation creates huge matrices), we recommend not running the hyperparameter tuning on these models. This is not necessary as there are no tunable parameters. However, the prediction script requires information about these in the validation-results file. The tuning for these models is currently commented out. We recommend to keep it this way and add the respective lines from hyperparameter_tuning as we have done.

Best Hyperparameters: The explored ranges can be seen in hyperopt_configs. Further, the results from the hyperparameter validation can be seen in hyperparameter_tuning. The model with the highest nDCG was selected to train the final recommender and create recommendations.

To tune the remaining parameters, see the configuration file hyperopt_configs and run recommender_experiment/hyperparameter_validation.py --dataset goodreads with varying arguments for --algorithm. We include ItemKNN, SLIM, and MultVAE.

Aggregate tuned hyperparameters in hyperparameter_tuning.

To create recommendations, we can run recommender_experiment/create_recommendations.py or recommender_experiment/create_recommendations_in_batches.py. Both have essentially the same functionality and create recommendations based on the model with the best hyperparameters. However, the latter computes the recommendations in smaller batches due to the high number of test users in Goodreads (use the --batch_size argument to do so). Provide both scripts with arguments such as --n 25 --algorithms ItemKNN SLIM MultVAE --dataset Goodreads. We use the generation in batches for Popularity and Random with a batch size of 500 each. For Random and MultVAE, we also set --seeds 42 43 44 45 46 to repeat the recommender experiment with different random seeds.

Stereotype detection

• Download the stereotype subset of StereoBias and add it to the data directory as shown above
• Run STL/stereotype_detection_train.py to create the stereotype prediction model.

Other detection methods

For the other risks, we use established pretrained models. For validation of the text moderation model, see analysis in test_text_moderation_bert.ipynb.

All models can be accessed through risk_predictors.py. See predict_risks.ipynb for a simple example.

Creating risk profiles

To identify risks in recommended items, first run python Risk_Detection_Models/recommendation_risk_computation.py --algorithms Random Popularity SLIM ItemKNN RecVAE --n 25.

Analyzing Results

Compute performance scores with recommender_experiment/compute_performance_scores.py. As we compute average scores across runs with different seeds, we set -use_seeds.

Compute item statistics, particularly popularity score, using result_analysis/compute_item_stats.py and compute the user behavior profiles with result_analysis/generate_behavior_profiles.py

For detailed analysis and reporting of results, see the following scripts:

• performance_analysis.ipynb: Analyze the performance metrics.

• overall_risk_exposure.ipynb: Predict overall risk scores. Visualize them using boxplots, tables, and conduct tests between algorithms. Visualize them using spider plots. Compute and visualize other statistics such as Coefficient of Variance, IQR, Percentiles, Gini, Skewness and correlations between risks.

• exposure_consistency_across_algos.ipynb: Analyze to what degree exposure to risks in one algorithm correlates to exposure in another.

• behavior_risk_correlations.ipynb: Analyze to what degree behavior correlates with risk outcomes.

• high_risk_exposure.ipynb: Analyze users with high-exposure to certain risks. Get the number of users as well as their proportion among all users.

• high_risk_exposure_behavior.ipynb: Analyze to what degree behavior of users is connected to them having high risk exposure. Conduct tests to identify significant differences between behavior.

• illustrate_example.ipynb: To play around with individual users, their behavior, their recommendations and risk scores.