Skip to main content

CCC.GenAI.C01: Model Input Filtering and Sanitisation

Control ID:CCC.GenAI.C01
Title:Model Input Filtering and Sanitisation
Objective:Inspect and validate input before it is passed to a GenAI model in order to filter or sanitise adversarial queries and prevent sensitive data leakage.
Control Family:
Data

Related Threats

IDTitleDescriptionExternal MappingsCapability MappingsControl Mappings
CCC.GenAI.TH01Prompt InjectionPrompt injection may occur when crafted input is used to manipulate the GenAI model's behaviour, resulting in the generation of harmful or unintended outputs. Prompt injection can be either direct (performed via direct interaction with the model) or indirect (performed via external sources ingested by the model). Both text-based and multi-modal prompt injection is possible.
4
1
0
CCC.GenAI.TH03Sensitive Information DisclosureSensitive data can be memorised by the model from user interaction or training and may then be leaked to unintended and unauthorised parties by querying the model, for example through crafted prompts.
4
1
0

Related Capabilities

IDTitleDescription
CCC.Core.F14API AccessThe service exposes a port enabling external actors to interact programmatically with the service and its resources using HTTP protocol methods such as GET, POST, PUT, and DELETE.
CCC.GenAI.F15Text-Based PromptsAbility to input prompts in plain text.
CCC.GenAI.F16Structured PromptsAbility to provide structured input such as JSON as prompts.
CCC.GenAI.F17Contextual PromptsAbility to provide context or background information within the prompt to guide the response.
CCC.GenAI.F18Interactive PromptsAbility to use conversational prompts to create interactive dialogues.
CCC.GenAI.F19Image-Based PromptsAbility to input an image as a prompt to generate a response.
CCC.GenAI.F20Custom Template PromptsAbility to define custom templates or structures for prompts to standardize interactions with the models.
CCC.GenAI.F21Generate ContentAbility to generate a response given a foundation model, parameter values, and a prompt.
CCC.GenAI.F24Content ModerationEnsure the service detects and filters abusive, harmful, and sensitive information to ensure responsible and safe use of the service.

Guideline Mappings

Reference IDEntry IDStrengthRemarks
FINOS-AIGF
AIR-PREV-003
0
User/App/Model Firewalling/Filtering
FINOS-AIGF
AIR-PREV-017
0
AI Firewall Implementation and Management
FINOS-AIGF
AIR-PREV-002
0
Data Filtering From External Knowledge Bases
FINOS-AIGF
AIR-DET-001
0
AI Data Leakage Prevention and Detection
SAIF
Input Validation and Sanitization
0
-
MITRE-ATLAS
AML.M0020
0
Generative AI Guardrails
MITRE-ATLAS
AML.M0021
0
Generative AI Guidelines
MITRE-ATLAS
AML.M0015
0
Adversarial Input Detection

Assessment Requirements

IDDescriptionApplicability
CCC.GenAI.C01.TR01Untrusted input such as user queries, RAG data or tool output MUST be validated before it is passed to a GenAI model.
tlp-clear
tlp-green
tlp-amber
tlp-red
CCC.GenAI.C01.TR02If malicious patterns such as prompt injection or sensitive data are detected during input validation, the input MUST be blocked or sanitised.
tlp-clear
tlp-green
tlp-amber
tlp-red