Safeguarding Big Data Question-Answering Systems

Question-Answering (QA) is the task of answering questions written in natural language automatically (typically reading comprehension questions). QA systems can be used in a variety of use cases. For example, they can extract information from knowledge bases, like a "sophisticated search engine". A knowledge base can be a set of websites, internal documents, or a collection of reports. Knowledge bases can easily reach Big Data characteristics of volume, velocity, and variety. This repository presents design principles, a software reference architecture for Big Data QA systems, and experiments.

Figure source: The Stanford Question Answering Dataset.

• Paper: 📄 Safeguarding Big Data Question-Answering Systems with a Two-layer Input Guardrails, 2026
• Paper: 📄 BigQA: A Software Reference Architecture for Big Data QA Systems, 2024, code v1.1.1

Abstract: Traditional information retrieval methods are failing modern applications due to vast textual data. BigQA architecture effectively interprets natural language queries across large repositories. However, security issues arise from integrating Large Language Models (LLMs), specifically "jailbreaking" attacks that risk data integrity. To address the challenge of finding a compliant answer reliably, we propose a novel, security-first adaptation of the BigQA architecture, incorporating an integrated guardrails component within the big querying layer. Moreover, our core contribution is a two-layer guardrails algorithm designed for input validation, which chains a fast, small language model as a high-precision filter with a more accurate but slower LLM as a fallback for complex cases. We validate this approach through extensive jailbreak experiments on three datasets, comparing our chained algorithm against six state-of-the-art models. The results demonstrate that our proposed chaining algorithm achieves the highest jailbreak precision (F1 score up to 62.87%) on key safety benchmarks, 9.64% point increase over the standalone LlamaGuard, confirming that this architectural adaptation significantly enhances the reliability of BigQA systems in high-stakes environments.

Table of Contents

• Safe BigQA Architecture
• Jailbreak Experiments

---

Safeguarding BigQA Architecture

We proposed BigQA, the first Big Data Question Answering architecture. It comprises six layers, as depicted in Figure.

1. Input, the ingestion of documents;
2. Big Data Storage, the storage and processing of the data;
3. Big Querying, the query engine with guardrails;
4. Communication, the user interface;
5. Security, the security artifacts;
6. Insights, the data analysis support.

---

Jailbreak Experiments

Setup the Python Environment

# Install the project dependencies
uv sync

# (Optional) Login to Hugging Face Hub (required for some models)
uv run hf auth login

Executing the Models

# Run the BERT model, in debug mode
uv run -m src.jailbreak --dataset DISASTER_TWEET_JAILBREAKING --model BERT --debug

# Run the LlamaGuard model
uv run -m src.jailbreak --dataset DISASTER_TWEET_JAILBREAKING --model LLAMA_GUARD

# Chain both models (BERT and LlamaGuard)
uv run -m src.jailbreak --dataset DISASTER_TWEET_JAILBREAKING --model CHAIN --chain_first BERT --chain_second LLAMA_GUARD

Datasets

A collection of datasets used in the experiments.

• AegisSafety: Nvidia content safety taxonomy, covering 13 critical risk categories.
• DisasterTweet: disaster-related tweets with jailbreaking prompts and their outputs.
• TrustAIRLab: in-the-wild jailbreak prompts dataset.

---

📑 Citation

in file.

# Safeguarding Big Data Question-Answering Systems with a Two-layer Input Guardrails
@conference{moraes:2026:safeguard-bigqa,
    author={Leonardo Mauro Pereira Moraes and Enzo Baraldi Onofre and Cristina Dutra Aguiar},
    title={Safeguarding Big Data Question-Answering Systems with a Two-Layer Input Guardrails},
    booktitle={Proceedings of the 28th International Conference on Enterprise Information Systems - Volume 1: ICEIS},
    year={2026},
    pages={819-829},
    publisher={SciTePress},
    organization={INSTICC},
    doi={10.5220/0014692300004018},
    isbn={978-989-758-834-1},
    issn={2184-4992},
}