Skip to main content

AI/ML / Multi Agent Refarch / Controls / DEV

Data Quality and Classification

CCC.MARefArc.CN04 · PREV

Assess the quality of, and assign classification and sensitivity labels to, all data used for grounding, training, and fine-tuning, and enforce handling rules derived from those labels throughout the Knowledge and LLM layers.

Related Capabilities

IDTitleDescription
CCC.MARefArc.CP14Approved-model registry and lifecycleCatalog of approved models with metadata, version information, configuration parameters, and usage constraints, ensuring agents access only models meeting organizational, regulatory, and security standards.
CCC.MARefArc.CP11Adaptive learningGenerates learning signals based on execution outcomes to refine prompts, adjust agent configurations, or improve tool-selection strategies.
CCC.MARefArc.CP12Authoritative knowledge source basesInternal and external repositories of structured data, unstructured documents, and graph-based representations that provide authoritative information for grounding.
CCC.MARefArc.CP13Vector-based semantic retrievalVector databases providing semantic search and grounding so agents can find relevant information from large text corpora.
CCC.MARefArc.CP20Feedback engineCollects and aggregates structured and unstructured feedback from users, evaluators, and automated systems, including correctness assessments, preference signals, and quality ratings, to inform system improvement.
CCC.MARefArc.CP21Human supervision and oversightMechanisms for human reviewers to inspect, approve, correct, or override agent outputs, supporting human-in-the-loop and human-over-the-loop workflows for sensitive or high-impact tasks.
CCC.MARefArc.CP05Agent-ingress zero-trust guardrailsTreats all inputs as untrusted and enforces authentication, authorization, input validation, content filtering, access control, rate limits, and dynamic policy before any request reaches an agent.
CCC.MARefArc.CP02Human-in-the-loop output reviewApplication-embedded controls that allow users to review, approve, or modify agent outputs before they are executed or shared.
CCC.MARefArc.CP01User-facing application surfacePresentation and orchestration surface (web, mobile, chatbot, workflow tool, or integrated enterprise system) that captures user intent, forwards requests to the agent layer, and returns agent outputs.
CCC.MARefArc.CP16Model-interaction zero-trust guardrailsEnforces authentication and authorization for every inference request and applies input validation against prompt injection, output filtering and redaction, access control, rate limits, and cost management before and after model execution.
CCC.MARefArc.CP22Runtime protectionMonitors agent actions and model outputs during execution to detect unsafe, non-compliant, or anomalous behavior, enforcing constraints, blocking disallowed actions, or triggering escalation.

Related Threats

IDTitleDescription
CCC.MARefArc.TH06Foundation-model training and fine-tuning data poisoningAdversaries tamper with training, fine-tuning, or third-party data feeds behind the approved models, mislabeling data or embedding backdoor triggers and biases that corrupt downstream decisions without visible symptoms until a major failure.
CCC.MARefArc.TH07Adaptive-learning and continuous-learning exploitationThe adaptive-learning capability that refines prompts and configurations from execution outcomes can be steered by an adversary who systematically feeds misleading signals, gradually skewing agent behaviour when validation of learning inputs is inadequate.
CCC.MARefArc.TH22Poor-quality, drifting, and bias-amplifying dataInaccurate, incomplete, outdated, or biased grounding and training data lead to unreliable outputs, while data and concept drift erodes predictive power over time and amplifies historical errors at scale.
CCC.MARefArc.TH23Discriminatory outputs from biasBiased training data, architectural and feature choices, proxy variables such as postal codes, and uncorrected feedback loops cause systematically discriminatory outcomes against protected groups, with legal and reputational exposure.
CCC.MARefArc.TH25Non-compliant outputs and model-risk-management gapsAI-generated advice, marketing, or communications that fail KYC, suitability, disclosure, record-keeping, or model-risk-management expectations create regulatory exposure; weak supervision and accountability lines turn this into direct non-compliance.
CCC.MARefArc.TH26Intellectual-property leakage and licensing violationsOutputs may replicate copyrighted training material, employees may leak trade secrets into AI tools, and improper platform licensing or terms-of-service violations create contractual and legal liability.
CCC.MARefArc.TH01Model memorization leaks sensitive data across sessionsThe hosted models accessed through the LLM layer may memorize sensitive inputs or training data and later disclose customer PII, proprietary algorithms, or trading strategies, including cross-user leakage into unrelated sessions.
CCC.MARefArc.TH02Hosted-provider data-handling exposureSensitive data submitted through the LLM gateway to third-party hosted models is exposed when the provider lacks transparent encryption, retention limits, or secure-deletion guarantees, leaving the institution without control over data it no longer holds.
CCC.MARefArc.TH03Embedding inversion and membership inference on the vector storeVectors stored for semantic retrieval can be inverted to reconstruct original source text, or probed to infer whether specific confidential information is present, exposing PII or proprietary content held in the knowledge layer.
CCC.MARefArc.TH04Embedding-store poisoning degrades retrieved contextAn actor with write access injects malicious or misleading embeddings into the vector store, degrading the accuracy of retrieved grounding context; the dense numerical representation makes the tampering hard to detect.
CCC.MARefArc.TH05Vector-store access-control, encryption, and audit gapsMissing role-based access control, encryption at rest, or audit logging on the vector store allows unauthorized retrieval, modification, or undetected exfiltration of embeddings derived from sensitive internal data.
CCC.MARefArc.TH16Confident hallucination and fabricated factsLacking ground truth and faced with ambiguous prompts or helpfulness-biased tuning, the model fabricates plausible but false facts, figures, or citations, presented with high fluency that makes errors hard to catch and likely to be acted upon.

Assessment Requirements

IDTextApplicability
CCC.MARefArc.CN04.AR01All data sources feeding the Knowledge Layer or used for fine-tuning MUST carry a classification and sensitivity label.tlp-clear, tlp-green, tlp-amber, tlp-red
CCC.MARefArc.CN04.AR02Data quality checks for accuracy, completeness, freshness, and bias MUST be applied to grounding and training data, and data failing thresholds MUST be quarantined from production use.tlp-clear, tlp-green, tlp-amber, tlp-red

Guideline Mappings

FrameworkIDRemarks
finos-airAIR-PREV-006