Skip to content

Systemic Controls for Managing Risk in AI/ML Systems

By Dean Leffingwell, Jeff Gallimore, John Rzeszotarski, Michael Edenzon, Peter Ajemba

 

This paper, published in the Fall 2024 Enterprise Technology Leadership Journal, addresses the challenges of managing risks associated with artificial intelligence (AI) and machine learning (ML) systems. The authors aim to educate auditors, internal risk directors, and software leaders on the differences between existing risk management controls and those needed for new AI/ML systems, including generative AI. The paper provides an overview of the control environment for AI/ML systems, focusing on six unique assets and suggesting controls for managing associated risks throughout the AI/ML development life cycle.

  • Format PDF
  • Pages 22
  • Publication Date

Features

  • New Asset Types

    AI/ML systems introduce new types of software assets, such as model weights and datasets, which require specific controls and management practices.

  • Life Cycle Approach

    Effective risk management for AI/ML systems requires controls at each stage of the development lifecycle, from data preparation to continuous monitoring.

  • Specialized Controls

    Traditional software development controls are insufficient for AI/ML systems; new controls are needed to address unique risks such as model drift, data quality issues, and hardware dependencies.

  • Continuous Monitoring

    Ongoing surveillance of AI/ML systems in production is crucial for detecting issues like model drift, data anomalies, or outputs beyond expected thresholds.

About the Resource

This paper, published in the Fall 2024 Enterprise Technology Leadership Journal, addresses the challenges of managing risks associated with artificial intelligence (AI) and machine learning (ML) systems. The authors aim to educate auditors, internal risk directors, and software leaders on the differences between existing risk management controls and those needed for new AI/ML systems, including generative AI. The paper provides an overview of the control environment for AI/ML systems, focusing on six unique assets and suggesting controls for managing associated risks throughout the AI/ML development life cycle.

The authors discuss the various stages of AI/ML system development, including data preparation, data management, model development, model evaluation, model deployment, and continuous monitoring. For each stage, they provide sample controls to mitigate specific risks. The paper emphasizes the importance of understanding the new types of software assets introduced by AI/ML systems and the need for organizations to update their practices to accommodate these new components. The authors also highlight real-world examples of problems that can occur when AI/ML assets are not properly managed, underscoring the importance of implementing effective controls.

Dean Leffingwell
Jeff Gallimore
John Rzeszotarski
Michael Edenzon
Peter Ajemba
Dean Leffingwell

Dean Leffingwell

Cofounder and Chief Methodologist at Scaled Agile, Inc.

To Author Archive
Jeff Gallimore

Jeff Gallimore

Chief Technology and Innovation Officer, Co-Founder at Excella

To Author Archive
John Rzeszotarski

John Rzeszotarski

John Rzeszotarski assists organizations with strategic planning and leadership in the solution and infrastructure focus areas; moreover, John provides thought leadership to large enterprises that need to focus on reliability, scalability, regulatory, and other business considerations. His expertise spans many verticals with a focus on digital, payments, security, development, and his primary passion is solving business and IT problems thru technology, process, and culture transformations.

To Author Archive
Michael Edenzon

Michael Edenzon

Michael Edenzon is a senior IT leader and engineer that modernizes and disrupts the technical landscape for highly-regulated organizations. Michael provides technical design, decisioning, and solutioning across complex verticals and leverages continuous learning practices to drive organizational change. He is a fervent advocate for the developer experience and believes that enablement-focused automation is the key to building compliant software at scale.

To Author Archive
Peter Ajemba

Peter Ajemba

Peter Ajemba is an expert in machine learning and artificial intelligence systems. He works with organizations to apply AI/ML to optimize business processes and systems and to develop data and insights products. A former chief technology officer for a drug discovery venture that pioneered AI/ML use in bio-signal amplification, Peter provides technology leadership in product design, process improvement, regulatory strategy, and team development across consumer products, medical devices, and drug discovery verticals.

To Author Archive