AI/ML / Gen AI

Threats

Version:

ID	Title	Description
CCC.GenAI.TH01	Prompt Injection	Prompt injection may occur when crafted input is used to manipulate the GenAI model's behaviour, resulting in the generation of harmful or unintended outputs. Prompt injection can be either direct (performed via direct interaction with the model) or indirect (performed via external sources ingested by the model). Both text-based and multi-modal prompt injection is possible.
CCC.GenAI.TH02	Data Poisoning	Data poisoning occurs when training, fine-tuning or embedding data is tampered with in order to modify the model's behaviour, for example steering it towards specific outputs, degrading performance or introducing backdoors.
CCC.GenAI.TH03	Sensitive Information Disclosure	Sensitive data can be memorised by the model from user interaction or training and may then be leaked to unintended and unauthorised parties by querying the model, for example through crafted prompts.
CCC.GenAI.TH04	Insecure / Unreliable Model Output	A GenAI model may generate content that is incorrect, misleading or harmful, such as convincing misinformation (hallucinations) or vulnerable or malicious code, due to its reliance on statistical patterns rather than factual understanding. Directly using this flawed output without validation can lead to system compromises, poor decision-making, and legal or reputational damage.
CCC.GenAI.TH05	Model Overreliance	Model overreliance and misplaced implicit trust in the output of a GenAI model may lead to the acceptance of inaccurate, biased or insecure outputs without proper validation or oversight, potentially resulting in operational failueres, compliance breaches and flawed decision making.
CCC.GenAI.TH06	Unintended Action by a Model-Based Agent	A model-based agent, given the authority to execute tools or interact with APIs, may perform an action that is harmful, incorrect, or not aligned with the user's true intent in response to a prompt. This can be caused by the model misinterpreting an ambiguous prompt or being manipulated by an adversary into misusing its delegated authority.
CCC.GenAI.TH07	Insecure Plugin	A plugin integrated with a GenAI model may contain vulnerabilities such as poor input validation or improper access control. An adversary may exploit these flaws by crafting a prompt that causes the model to pass a malicious payload to the plugin, potentially leading to system compromise, data exfiltration or privilege escalation.
CCC.GenAI.TH08	Model Tampering	Supply chain risks, including tampering with a model's core components at any stage of its lifecycle—from its source code and training data to the final deployable artifact—may result in embedding backdoors or adversarial triggers altering model behaviour under certain conditions.
CCC.GenAI.TH09	Lack of Explainability	The "black box" nature of GenAI models makes it difficult or impossible to understand the specific reasoning behind a given output. This opacity makes it challenging to diagnose failures, detect hidden biases, and meet regulatory requirements for decision transparency.
CCC.GenAI.TH10	Model Version Drift	An update to a managed GenAI model may cause unpredictable and breaking changes in its outputs, alignment, and performance. Systems built and tested against the previous version's specific behavior can suddenly fail or become insecure, as their functional and safety assumptions are no longer valid.