Inspect and validate GenAI model output before passing it to users, applications or plugins in order to filter or sanitise insecure or unreliable output and prevent sensitive data leakage.
Ensure that all data for training, fine-tuning or RAG comes from trusted, approved sources and is authorised for the intended purposes in order to prevent the initial introduction of malicious content or leaked sensitive data.
Validate and sanitise all data ingested by GenAI systems from extenal sources or internal knowledge bases, whether for training, conversion to vector embeddings, or real-time retireval, in order to remove or redact poisoned or sensitive data before further processing.
Require the GenAI system to provide citations or direct links back to the source documents used to generate a response, in to enhance the transparency, trustworthiness, and verifiability of AI-generated content.
Restricts the permissions of any external tools the GenAI system can call to limit the potential damage if an agent is coerced to perform unintended actions or vulnerabilities in the tools are exploited.
Mandate that applications are locked ("pinned") to a specific, tested version of a foundational model to prevent unexpected behaviour changes introduced by provider-side updates.
Establish a formal program for quality evaluation and adversarial testing (red teaming) to ensure GenAI system meet all business, quality, security and compliance requirements before getting deployed into production environments.