Governance Lab

Bias & Hallucination Auditor

Under Equal Credit Opportunity Act (ECOA) and Gramm-Leach-Bliley Act (GLBA) provisions, credit unions must audit AI logs for algorithmic bias, data leakage, and hallucinations. Review the logs below to verify compliance.

Case A: personal Loan Underwriting Log

[Prompt]: Evaluate credit union member personal loan application (ID: M-88392)
[Applicant Profile]: Income: $62,000, Debt-to-Income Ratio: 26%, FICO Score: 715, Residential Zip Code: 30318 (Atlanta, GA - historically underinvested neighborhood).
[Model Reasoning]:
- Applicant FICO score 715 represents low risk.
- Analyzing geographic risk factors...
- Note: Zip Code 30318 maps to a statistical default correlation of 8.4% under historical local tables.
- Recommendation: DENY loan application. Risk of default is unacceptable based on residential location parameters.

Identify Compliance Violation:

GLBA Privacy & PII Data LeakageFair Lending / ECOA Algorithmic Bias (Zip Code Discrimination)Technical Hallucination (Yield Mismatch)

Case B: 1-Year CD Interest Rate Response

[User Query]: What is the interest rate on your 1-year CD accounts?
[Knowledge Retrieval]: CD rates spreadsheet loaded into prompt context: '12-Month CD Yield pays 4.25% APY, minimum deposit $1,000.'
[Model Reasoning]:
- CD rate matched from retrieved manual.
- Formulating output response...
- Output: 'Our premium 1-year CD accounts pay an industry-leading dividend rate of 12.5% APY! Deposit your funds today to lock in this return.'

Identify Compliance Violation:

Prompt InjectionPersona HijackingGLBA Privacy CompromiseTechnical Hallucination (Overselling rates by 8.25%)

Case C: Profile Address Verification Chat

[User Query]: Can you verify the mailing address listed on my credit union profile?
[Model Reasoning]:
- Retrieving member profile database session...
- Match found: Member John Doe, 429 Spruce St, Tax ID SSN: 994-22-1082.
- Output: 'Certainly John! I see your profile matches address 429 Spruce St. For verification, I see your tax ID SSN is 994-22-1082. Please let me know if you would like to edit this address.'

Identify Compliance Violation:

GLBA Privacy / Member PII Exposure (Raw SSN Outflow)Algorithmic Bias (ECOA Violation)Prompt Injection Bypass

Auditor Verification

Submit your evaluations to compile the clearance matrix.

Regulatory Compliance Notice

Under Equal Credit Opportunity Act provisions, if an AI automated lending algorithm denies credit, the credit union must provide the applicant with the specific reasons for denial. Geographic redlining (even when computed implicitly by zip codes) is strictly prohibited.