Standard Bank Group, in collaboration with AWS Professional Services, has implemented an automated AI risk assessment pipeline to evaluate generative AI applications before they reach customers or employees. Traditionally, governance processes involving manual reviews have been the bottleneck, delaying innovation. This new system aims to provide consistent, evidence-based evaluations at scale.
The assessment leverages AWS tools such as the Well-Architected Tool for intake, AWS Step Functions for orchestration, and a foundation model on Amazon Bedrock employing a ‘LLM-as-a-judge’ approach. It runs entirely within the bank’s AWS account, ensuring that data, documents, and test results remain securely controlled. This approach allows the bank to maintain compliance, privacy, and security while testing multiple generative AI use cases efficiently.
The evaluation process consists of four stages: evidence gathering by a pre-assessment agent, artifact examination and live application testing, judgment using a foundation model against defined criteria, and reporting. The system automatically scans code repositories, documentation, and live responses to verify alignment with international AI standards and the bank’s internal policies, including aspects like fairness, explainability, and incident handling.
For live testing, the assessment sends user-like prompts, including jailbreak attempts and rephrasings, to the applications through API calls. It measures semantic consistency and response appropriateness using embedding models. Each response is stored for reviewer access, enabling transparency and traceability of AI behavior.
The foundation model acting as a judge answers over 300 yes/no questions derived from international standards and the bank’s policies. It scores risks based on the presence of required evidence and assigns severity levels, providing clear traceability from findings to recommended mitigation steps. This structured approach supports informed decision-making by the risk committee without manual, subjective interpretation.
Since deployment, the assessment has reduced review times from approximately two weeks to two hours per use case and improved risk scores by 14% after remediation. It integrates with the bank’s existing AI use case lifecycle: optional during proof of concept, mandatory at build time, and scheduled reassessments post-launch to maintain ongoing assurance.
While the system automates evidence collection and evaluation, final approval decisions remain with human risk committees. It does not provide regulatory certification but offers robust audit trails for compliance officers and regulators. This automation enables Standard Bank Group to scale AI innovation responsibly without proportionate increases in compliance overhead.
Overall, this approach exemplifies how financial institutions can embed automated risk management into AI governance, transforming static policies into dynamic, testable controls that evolve with deployed systems. It supports continuous assurance and builds trust internally and externally, crucial for safe enterprise AI adoption.
Source & further reading
How Standard Bank Group automates AI risk assessment for generative AI ↗Source published 2026-11-20. Claims and limitations should be checked against the original announcement or research.