Can You Rely on Your AI? Applying the AIR Tool to Improve Classifier Performance

Software Engineering Institute (SEI) Webcast Series

Modern analytic methods, including artificial intelligence (AI) and machine learning (ML) classifiers, depend on correlations; however, such approaches fail to account for confounding in the data, which prevents accurate modeling of cause and effect and often leads to prediction bias. The Software Engineering Institute (SEI) has developed a new AI Robustness (AIR) tool that allows users to gauge AI and ML classifier performance with unprecedented confidence. This project is sponsored by the Office of the Under Secretary of Defense for Research and Engineering to transition use of our AIR tool to AI users across the Department of Defense. During the webcast, the research team will hold a panel discussion on the AIR tool and discuss opportunities for collaboration. Our team efforts focus strongly on transition and provide guidance, training, and software that put our transition collaborators on a path to successful adoption of this technology to meet their AI/ML evaluation needs.

What Attendees Will Learn:

• How AIR adds analytical capability that didn't previously exist, enabling an analysis to characterize and measure the overall accuracy of the AI as the underlying environment changes

• Examples of the AIR process and results from causal discovery to causal identification to causal inference • Opportunities for partnership and collaboration