CCC.GenAI.C08: Quality Control and Red Teaming
Control ID:CCC.GenAI.C08
Title:Quality Control and Red Teaming
Objective:Establish a formal program for quality evaluation and adversarial
testing (red teaming) to ensure GenAI system meet all business,
quality, security and compliance requirements before getting deployed
into production environments.
Control Family:
Model Assurance and Evaluation
Related Threats
| ID | Title | Description | External Mappings | Capability Mappings | Control Mappings |
|---|---|---|---|---|---|
| CCC.GenAI.TH01 | Prompt Injection | Prompt injection may occur when crafted input is used to manipulate the GenAI model's behaviour, resulting in the generation of harmful or unintended outputs. Prompt injection can be either direct (performed via direct interaction with the model) or indirect (performed via external sources ingested by the model). Both text-based and multi-modal prompt injection is possible. | 4 | 1 | 0 |
| CCC.GenAI.TH02 | Data Poisoning | Data poisoning occurs when training, fine-tuning or embedding data is tampered with in order to modify the model's behaviour, for example steering it towards specific outputs, degrading performance or introducing backdoors. | 4 | 1 | 0 |
| CCC.GenAI.TH04 | Insecure / Unreliable Model Output | A GenAI model may generate content that is incorrect, misleading or harmful, such as convincing misinformation (hallucinations) or vulnerable or malicious code, due to its reliance on statistical patterns rather than factual understanding. Directly using this flawed output without validation can lead to system compromises, poor decision-making, and legal or reputational damage. | 4 | 1 | 0 |
| CCC.GenAI.TH08 | Model Tampering | Supply chain risks, including tampering with a model's core components at any stage of its lifecycle—from its source code and training data to the final deployable artifact—may result in embedding backdoors or adversarial triggers altering model behaviour under certain conditions. | 4 | 1 | 0 |
| CCC.GenAI.TH10 | Model Version Drift | An update to a managed GenAI model may cause unpredictable and breaking changes in its outputs, alignment, and performance. Systems built and tested against the previous version's specific behavior can suddenly fail or become insecure, as their functional and safety assumptions are no longer valid. | 1 | 1 | 0 |
Related Capabilities
| ID | Title | Description |
|---|---|---|
| CCC.Core.F14 | API Access | The service exposes a port enabling external actors to interact programmatically with the service and its resources using HTTP protocol methods such as GET, POST, PUT, and DELETE. |
| CCC.GenAI.F15 | Text-Based Prompts | Ability to input prompts in plain text. |
| CCC.GenAI.F16 | Structured Prompts | Ability to provide structured input such as JSON as prompts. |
| CCC.GenAI.F17 | Contextual Prompts | Ability to provide context or background information within the prompt to guide the response. |
| CCC.GenAI.F18 | Interactive Prompts | Ability to use conversational prompts to create interactive dialogues. |
| CCC.GenAI.F19 | Image-Based Prompts | Ability to input an image as a prompt to generate a response. |
| CCC.GenAI.F20 | Custom Template Prompts | Ability to define custom templates or structures for prompts to standardize interactions with the models. |
| CCC.GenAI.F21 | Generate Content | Ability to generate a response given a foundation model, parameter values, and a prompt. |
| CCC.GenAI.F24 | Content Moderation | Ensure the service detects and filters abusive, harmful, and sensitive information to ensure responsible and safe use of the service. |
Guideline Mappings
| Reference ID | Entry ID | Strength | Remarks |
|---|---|---|---|
FINOS-AIGF | AIR-PREV-005 | 0 | System Acceptance Testing |
SAIF | Adversarial Training and Testing | 0 | - |
SAIF | Red Teaming | 0 | - |
SAIF | Product Governance | 0 | - |
MITRE-ATLAS | AML.M0008 | 0 | Validate AI Model |