NIST Cybersecurity White Paper (CSWP) 31, Proxy Validation and Verification for Critical AI Systems: A Proxy Design Process, is now available. The goal of this work is to increase trust in critical AI systems (CAIS) by developing proxy systems to verify and validate a CAIS.
This document presents a five-phase process for identifying and/or building non-critical proxy systems that have high similarity to critical artificial intelligence (AI) or machine learning (ML) systems. The non-critical proxy system is used to indirectly validate and verify the critical system by enabling the creation of difficult or dangerous test cases in a way that imbues confidence in the scenarios. The results of the test cases presented to the non-critical proxy can then be imputed to the critical system. The process involves a way to demonstrate and measure “similarity” between the two systems.