CCC.GenAI.F24: Content Moderation
Capability ID:CCC.GenAI.F24
Title:Content Moderation
Description:Ensure the service detects and filters abusive, harmful, and sensitive
information to ensure responsible and safe use of the service.
Mapped Threats
| ID | Title | Description | External Mappings | Capability Mappings | Control Mappings |
|---|---|---|---|---|---|
| CCC.GenAI.TH01 | Prompt Injection | Prompt injection may occur when crafted input is used to manipulate the GenAI model's behaviour, resulting in the generation of harmful or unintended outputs. Prompt injection can be either direct (performed via direct interaction with the model) or indirect (performed via external sources ingested by the model). Both text-based and multi-modal prompt injection is possible. | 4 | 1 | 0 |
| CCC.GenAI.TH02 | Data Poisoning | Data poisoning occurs when training, fine-tuning or embedding data is tampered with in order to modify the model's behaviour, for example steering it towards specific outputs, degrading performance or introducing backdoors. | 4 | 1 | 0 |