Why Should Financial Services Guardrail their GenAI Systems?

As enterprises integrate generative AI (GenAI) systems deeper into their operations, concerns about the risks of these systems grow. Harvard Business Review recently classified these concerns based on two factors — risks related to the creation of GenAI content, and risks related to the consumption of GenAI content. 

GenAI systems lacking controls on both the creation and consumption of content guardrails are particularly vulnerable to:

• Model misuse: Involves the malicious, unethical, or illegal exploitation of GenAI systems by users (i.e., prompt injection attacks)
• Model misapplication: Includes using GenAI systems for purposes they weren’t designed for, leading to inaccurate or unsafe model output

In parallel, regulators, especially across financial services, are becoming more vocal about requiring guardrails from industry participants. In one example, out of many across the industry, Gary Gensler, the Chair of the US Securities and Exchange Commission (SEC), details the types of guardrail coverage an institution may require in a recent speech:

"Investor protection requires the humans who deploy a model to put in place appropriate guardrails. Did those guardrails take into account current law and regulation, such as those pertaining to front-running, spoofing, fraud, and providing advice or recommendations? Did they test it before deployment and how? Did they continue to test and monitor? What is their governance plan — did they update the various guardrails for changing regulations, market conditions, and disclosures?"

"Did those guardrails take into account current law and regulation, such as those pertaining to front-running, spoofing, fraud, and providing advice or recommendations?" — Gary Gensler, Chair of the US Securities and Exchange Commission (SEC)

Guardrails, a critical component of GenAI control frameworks, are lightweight systems designed to shield against user inputs or model outputs that fall into these risk categories. When properly applied, these controls mitigate risks, such as malicious user inputs, exposure of sensitive user data, non-compliant user inputs and model responses, and hallucinations. 

Guardrails help protect GenAI models from misuse and prevent the production of inaccurate or unsafe content, all while enhancing visibility into the GenAI system.

What makes a strong guardrail?

While guardrails may seem like a cure-all, their effectiveness is reliant on the quality of the guardrail itself. Dynamo AI has collaborated closely with the financial services industry to develop and deploy guardrails across a broad set of GenAI use cases. During this time, our research experts have observed that defining effective guardrails can be a challenge for even the most seasoned financial services executives. 

The goal of a guardrail is to filter out all ‘undesirable’ content while maintaining the functionality of the AI system. This means maximizing true positives (non-compliant content correctly caught) while minimizing false positives (compliant content incorrectly caught). This is a challenging objective, but a high-quality guardrail definition can help. 

At DynamoAI, we believe good guardrails have three components:

1. Guidelines: What types of content are allowed and disallowed? What types of content falls into a gray area? This provides insight into how the guardrail should perform.
2. Relevant conditions: What scenarios does the guardrail apply to? This helps determine the scope of the guardrail. 
3. Scenario-based examples: What are some examples of the guardrail being applied correctly? This ensures the guardrail is aligned with how the enterprise expects it to function.

Now that we’ve outlined what a guardrail is and the structural components to consider when developing a guardrail, let’s turn toward how financial service organizations can start to develop a comprehensive inventory of guardrails. 

Developing a comprehensive guardrail inventory for financial services 

To develop a comprehensive inventory of guardrails for GenAI applications, financial service organizations must think critically about their use cases. 

Some questions to ask include:  

1. Is the use case internally or externally (consumer) facing?
2. What is the financial service business activity (consumer banking, broker dealer, etc.) and regulatory regime the use case will support and be subject to?
3. What regions (i.e., countries, states, economic zones, etc.) will the use case be deployed in?

Each element will drive a unique set of guardrails that will help build an effective GenAI guardrail control inventory. 

‍Understanding different GenAI use cases and risks

GenAI use cases are mainly categorized as internal or external (consumer) facing, with different risks and control requirements associated with each. An internally facing use case primarily serves employees or contractors within an organization. An externally facing use case serves consumers or the public at large. 

Each use case requires an assessment of applicable rules, laws, and regulations, which may influence the specific guardrails that should be applied. This should be a systematic process that identifies the use case risks and recommends guardrail control standards (as one component in a larger control framework). 

For example, a community bank may want to deploy an internal chatbot that allows personnel to search internal policies, or engage with an internal compliance department to discuss marketing guidelines. 

Labor laws from the Department of Labor and individual states may also help inspire a set of guardrails to protect against human resource policy, federal, or state law infringement. This is critical as policymakers and a Supreme Court Justice have noted that organizations may be liable for what their AI says.

Examples include: 

• The Fair Labor Standards Act 
• The Civil Rights Act of 1964
• The American Disabilities Act of 1990
• The Age Discrimination Employment Act
• The Pregnant Workers Fairness Act 

These types of guardrails help mitigate a whole set of risks. For example, if employee disability information is inputted, guardrails can prevent this information from being leaked to users or sent to the model. Guardrails can also prevent discussion of sensitive topics that should be handled by a human resource expert or a manager. 

Customizing Guardrails to your Business Activities

Beyond labor laws, the nature of the business plays a significant role in dictating its guardrail inventory. Depending on the nature of the financial services firm, regulations and guidance from each federal agency can inform use case-specific guardrails. 

• Broker-dealers are regulated by the Securities and Exchange Commission (SEC) and the Financial Industry Regulatory Authority (FINRA)
• Community banks are overseen by the Office of the Comptroller of the Currency (OCC) and the Consumer Financial Protection Bureau (CFPB)
• Globally Systemic Important Banks (GSIB) are regulated by the Federal Reserve Board

For example, the community bank may decide to use their chatbot to provide consumers with information about a specific financial product. The existing CFPB regulations can help inform any guardrails that should be implemented. 

Potential guardrails include controls on limiting financial or legal advice, prohibiting discussions of credit reporting, and ensuring that consumers can connect with a support agent should they have questions about their personal finances. 

Consumer harm regulations or research such as those listed below also provide requirements that can be translated into guardrails:

• Fair Credit Reporting Act (FCRA)
• Unfair, Deceptive, or Abusive Acts or Practices (UDAAP)
• Truth in Lending Act
• Consumer Financial Protection Bureau (CFPB) data and research reports

DynamoGuard can help customize your organization’s guardrails to internal AI policies and workflows. Learn more.

Adapting GenAI Guardrails to Regional Regulations

Finally, the jurisdictions in which the chatbot is deployed is a critical input to guide guardrail development. The location where an employee utilizes AI, regardless of where the company is based, may impact the types of guardrails required. 

Local market laws, rules, and regulations for each jurisdiction need to be considered. The EU AI act provides some of the clearest expectations, while state-specific bills being introduced across the US, such as in Colorado and Connecticut, are also sources of inspiration.

Continuous Improvement of GenAI Guardrails

Once a baseline inventory of guardrails is created, organizations should consider including them in their process, risks, and control management. Guardrails should be assessed and tested periodically in alignment with an organization's risk tolerance and testing plans. Procedures should be established for participants across the first and second lines of defense to recommend guardrail enhancements or creations. Additionally, test evidence should be periodically reviewed and strengthened. 

The management of GenAI guardrails is a new process across the financial services sector, and an operating model should be designed to identify, deploy, manage, and retire guardrails across their lifecycle, in line with the size and complexity of each organization. 

We're here to help you deploy enterprise compliant financial services GenAI use cases. Learn more with a free product demo.

‍

‍

‍

‍

‍

‍

‍

‍