When procuring third-party AI systems, companies are faced with blackboxes that offer limited insight into the potential risk they pose or the behavior they exhibit. But there are ways to gain more confidence and control during the procurement process. QuantPi offers technical assessments to enable standardized comparisons and streamlined ways to assess providers compliance readiness.
During our Digital Week on Trustworthy AI, we wanted to dive even deeper into this complex topic. So we invited Dr. Cari Miller, a leading expert in AI risk management and organizational governance to share her insights into the challenges and best practices for procuring AI solutions.
The below is a glimpse into the conversation we had with Dr. Cari Miller. To learn more, we invite you to watch the on-demand session.
Risk Appetite matters: Each AI procurement needs a tailored risk appetite to ensure suitable governance.
Contracts as Compliance Tools: AI contracts should go beyond standard legalities to include safeguards for data rights, monitoring, and risk responses.
Vendor Transparency: Always assess vendor governance maturity to align with standards like the EU AI Act.
Continuous Monitoring: Post-procurement monitoring ensures the AI system stays compliant and trustworthy over time.
Dr. Miller: Procurement can be considered as the cornerstone of AI governance. For that you need to have your people ready and you have to have your organization ready. People Readiness means AI literacy and domain-specific knowledge for individuals throughout the organization. Organizational Readiness is about policies, practices in place and understanding your organizational capacity. And then we finally get to Value Capture where procurement sits and never forget Change Management.
Dr. Miller dives into each component of the five step Risk Management Framework, which is preceded by an assessment whether there actually is a Legitimate Business Need:
Step 1: Establish a Risk Appetite for each procurement (the heartbeat of the framework)
Step 2: Set up Risk-Aware Solicitation Requirements
Step 3: Conduct a Risk Assessment to interrogate whether the vendors are compliant to identify risks and map mitigations
Step 4: Specify Risk Controls in the contract so everyone is on the same playing field
Step 5: Note Risk Monitoring agreements like for risk tolerance in the contract
Note: The on-demand webinar offers more detailed insights into each component of the framework.
Dr. Miller: Alternative to the Risk Management Framework for Procuring AI Systems 1.0 (RMF PAIS 1.0) there also is the IEEE P3119 Standard for AI Procurement. The difference between those two huge components of procuring AI is that the RMF PAIS 1.0 is rather high-level for strategic guidance whereas the IEEE P3119 provides procedural guidance. The RMF PAIS 1.0 augments the organizational risk management practices that you already have, adding an extra layer to use procurement as a big tool. The IEEE P3119 however assists existing procurement practices pragmatically with guides, tools, rubrics and templates to facilitate analysis and implementation of risk management.
Dr. Cari Miller is a subject matter expert in AI risk management and organizational governance practices, a certified change manager, and an experienced corporate strategist. She is a co-founder and Board Chair at the AI Procurement Lab and a researcher at the Center for Inclusive Change. Dr. Miller delivers AI training programs, procurement guidance, readiness assessments, and policy analysis. She was named as one of 100 Brilliant Women in AI Ethics globally in 2023.
QuantPi pioneers trustworthy AI with its holistic platform for rigorously testing AI systems for bias, robustness, compliance and other critical performance metrics, ensuring transparency and actionable insights for successful AI transformations. Backed by the EU and emerging from CISPA, QuantPi advances responsible AI globally, building confidence in intelligent systems.
For more information, email contact@quantpi.com