How to Effectively Apply Third-Party Risk Management Principles to Generative AI

It's difficult to find a financial services organization that's not engaging (or actively debating) the use of generative artificial intelligence (AI). With masters of the universe making visionary proclamations, such as Jamie Dimon, CEO at JPMorgan Chase, stating that "AI could be as transformative as electricity or the internet", pressure continues to fill the boardrooms and senior leadership ranks with questions — and demands — on how to effectively use AI to transform operations.

On the technology front, there's a consolidation of early winners utilizing their infrastructure foothold and mass exposure of consumer products to get ahead. Google, Amazon Web Services (AWS), and Microsoft (in partnership with OpenAI) have already established a presence in financial services through their cloud integrations. Meanwhile, Meta and Apple are leveraging their widespread use of personal devices and consumer apps to make inroads into the financial services sector.

This leaves financial services organizations with a narrow vision as to how and where they should focus their risk management efforts.  

Navigating regulatory and risk management challenges

In June 2023, joint guidance from the Board of Governors of the Federal Reserve System (FRB), the Federal Deposit Insurance Corporation (FDIC), and the Office of the Comptroller of the Currency (OCC) published interagency guidance on risk management for third-party relationships. Although agencies acknowledged requests for more specific AI guidance, they chose a broad, principle-based approach.

As a result, risk managers are closely examining and dissecting this guidance as they work to implement AI across their organization's internal, and eventually, customer-facing use cases. Specific components being examined cover unique vendor landscape, existing entrenchment, and impact on financial services strategic risks:

• Addressing the need for 'independent testing and objective reporting of results and findings', especially in a field where the required machine learning expertise is scarce, both internally and across vendor partners
• Choosing an acceptable 'conformity assessment or certification by independent third parties' in a space where financial services regulators have published little specific guidance, and broader government and trade associates are only recently beginning to publish and battle-test principles and risk management control recommendations
• Effectively implementing contract provisions that allow for 'periodic, independent audits of the third party and its relevant subcontractors, consistent with the risk and complexity of the third-party relationship', given that the large technology partners hold vast internal and political power, and the black box nature of AI naturally inhibits efforts for explainability, demonstrated evidence, and traceability
• Proactively inserting 'ongoing monitoring and independent reviews' into the technology lifecycle to address changing risk or material issues, specifically as these risks are changing as AI models learn and mature
• Establishing a process to provide evidence of risks to the vendor to facilitate remediation of identified issues or course correct the operational processes

Emerging guidance on AI-specific cybersecurity risks

More recently in March of 2024, the U.S. Department of Treasury released guidance to the financial services industry focused on cybersecurity risks, a priority for U.S. regulators, senior management, and board members of these organizations.

'Managing Artificial Intelligence-Specific Cybersecurity Risks in the Financial Services Sector’ tackles a number of third-party risk management concerns, in particular how to decipher 'explainability for black box AI solutions'. The Treasury admits the concern of lack of explainability, specifically around safety, privacy, and consumer protection concerns, and points to the research and development community as providing an avenue for answers.

And while the Treasury alludes to frameworks to support longer-term assessment, it's clear that a mix of experience, talent, research, independence, and evidence will be part of the solution. This mix of experience, talent, research, independence, and evidence is producing breakthrough AI research each day that's highly useful for financial services organizations.

AI researchers at Dynamo AI divide explainability into four assessable components, with risk-mitigating controls alongside each:  

1. Transparency: Making the development and training process of AI systems clear and understandable to users, developers, and stakeholders
   • Control example: Complete end-to-end data tracing that tracks the exact data sources that were used for training and enables ability to remove noncompliant data and retrain
2. Interpretability: Enabling people to comprehend how an AI system arrives at its decisions or predictions, and what factors influence its outputs
   • Control example: Incorporate ongoing monitoring that pinpoints why models are hallucinating and giving false statements
3. Accountability: Ensuring the processes used to train and deploy AI systems can be monitored and held accountable for their decisions and actions, especially under the lens of user privacy, fairness, and bias for critical domains, such as healthcare, finance, and law
   • Control example: Implement data leakage and Personal Identifiable Information (PII) extraction algorithms assess models for data memorization vulnerabilities
4. Trust: Building trust between users and the outputs of AI systems in terms of safety, truthfulness, and helpfulness
   • Control example: Build customizable guardrails that ensure AI outputs are compliant with regulations, truthful, and high quality that drive business value

Overcoming barriers to effective AI oversight

Each of these four components are crucial in deploying AI safely and effectively. However, there are significant roadblocks to achieving each in practice. Companies can be increasingly secretive of their model development processes, infamously going down the path of closed source black box models in 2021. This not only makes AI transparency extremely challenging, if not impossible, to achieve, but calls into question interpretability, accountability, and trust.  

Where does that leave financial services in terms of effective independent oversight of AI? Dynamo AI identifies several key strategies emerging as organizations seek to balance competition, innovation, and risk management:

• Continuous evaluation of your AI technology stack and partners, assessing for areas of risk. This is particularly relevant in evaluating whether the technology vendor is also providing risk management metrics or controls on the AI being deployed.  
• Incorporate methods to intake and assess new research on AI, its risks, and control methods. This may involve in-house initiatives within risk and audit divisions and ensuring vendors have dedicated research teams or capabilities to review and adapt to the latest AI deployment and risk management strategies.  
• Establish effective controls for AI as part of the pre- and post-deployment and ongoing monitoring strategy. Controls implemented pre-deployment may need to provide input back to any AI organizational governance function established, as the risks identified (or mitigated) during this phase may inform the overall risk profile of the use case and its impact. Ongoing monitoring should be constant, and embedded in the AI stack, with clear outputs across a variety of technical and risk management skills.  
• Require specific control reports and expectations from vendors to help inform stakeholders about expectable AI testing and validation. Aligning this within your process, risk, and control self-assessment processes is a recommended way to bring together a collective set of stakeholders who all require knowledge about risk management of AI.  

Dynamo AI provides financial services organizations with the tools needed to assess and demonstrate AI compliance. Schedule your free demo.

‍

‍

‍