What is it?
This assessment examines the security of AI and machine learning systems, focusing on data integrity, model robustness, and the protection of intellectual property. It identifies risks such as adversarial examples, data poisoning, model inversion, or unauthorised model access.
What could happen?
If attackers can manipulate training data or query models in malicious ways, AI-driven decisions become unreliable. Fraudulent transactions might slip through, spam filters could fail, or private data inferred from models might leak—ultimately eroding trust and causing financial and reputational harm.
What to do about it?
Foundational: Ensure proper data sanitisation before training models and limit public exposure of model endpoints.
Outcome: Immediate reduction in opportunities for data poisoning or unauthorised queries.
Enhanced: Implement adversarial detection tools, watermarking or fingerprinting models to prevent unauthorised use, and apply strict access controls and authentication for model APIs.
Outcome: More robust defence against common AI manipulation techniques.
Comprehensive: Deploy continuous monitoring of model performance and anomaly detection systems that identify suspicious input patterns. Regularly retrain models with verified datasets and integrate privacy-preserving techniques like differential privacy.
Outcome: A resilient, trustworthy AI environment that can adapt to evolving threats.
