What is this course about?
Financial institutions have the clearest case for private deployment and the least clarity on how to do it. This course covers what client data actually leaves with a hosted model, open-weight selection and the capability trade, inference economics for a bank workload, GLBA obligations applied to AI infrastructure, and operating a private model.
Who is this course for?
- Infrastructure and platform engineering teams
- Information security architects
- Technology risk staff
- CTOs and heads of technology
What do I need before starting?
- Infrastructure and systems background
- Familiarity with your institution's security architecture
What will I be able to do afterwards?
- Trace exactly what client data leaves with a hosted model
- Select open-weight models appropriate to financial workloads
- Size hardware and model inference economics for a bank workload
- Apply GLBA Safeguards obligations to AI infrastructure
- Operate a private model including patching and incident response
What does each module cover?
What client data actually leaves?
45 minTracing a request through a hosted deployment to see what crosses the boundary.
Objectives
- Trace data flow through a hosted deployment
- Identify everything that crosses the boundary
- Assess the exposure against institutional policy
Topics
Activity. Trace a real workflow's data flow through a hosted provider and document every crossing.
Which open-weight models fit financial workloads?
55 minModel selection, licensing, and the capability assessment for financial tasks specifically.
Objectives
- Evaluate open-weight models for financial tasks
- Assess licence terms for commercial financial use
- Benchmark against your real workloads
Topics
Activity. Benchmark two open-weight models against real financial workloads.
How do you size and cost inference?
55 minHardware sizing and inference economics for a bank's concurrency and latency requirements.
Objectives
- Size hardware for concurrency and latency requirements
- Model inference cost against hosted alternatives
- Plan for peak load
Topics
Activity. Size and cost a deployment for a stated concurrency and latency requirement.
What does GLBA require of AI infrastructure?
50 minSafeguards Rule obligations applied to the systems running the model.
Objectives
- Apply Safeguards Rule requirements to AI infrastructure
- Design controls meeting the obligations
- Document for examination
Topics
Activity. Map Safeguards requirements onto your AI infrastructure design.
What about cross-border and foreign providers?
45 minData transfer, foreign legal process, and jurisdictional exposure in provider selection.
Objectives
- Assess cross-border transfer exposure
- Evaluate foreign legal process risk
- Set provider jurisdiction policy
Topics
Activity. Assess jurisdictional exposure across your current AI providers.
How do you fine-tune on proprietary data safely?
55 minAdaptation on institutional data that stays institutional.
Objectives
- Fine-tune on proprietary data within the boundary
- Prevent training data extraction from the resulting model
- Version and control adapted models
Topics
Activity. Fine-tune within the boundary and test the result for training data extraction.
How do you operate a private model?
50 minPatching, monitoring, and incident response for infrastructure you now own.
Objectives
- Design the operational runbook
- Plan patching and model updates
- Build incident response without vendor support
Topics
Activity. Write the operational runbook and run an incident tabletop.
Building the private deployment architecture
60 minThe workshop module: a complete architecture with a verified data-flow diagram.
Objectives
- Produce a complete deployment architecture
- Verify no client data leaves the boundary
- Document for technology risk review
Topics
Activity. Build the architecture and verify with network testing that no client data egresses.
What is the capstone project?
Private deployment architecture with verified data containment
Design a complete private deployment: traced hosted-model exposure, benchmarked open-weight selection, sized and costed inference, GLBA-mapped controls, jurisdictional assessment, in-boundary fine-tuning, and an operational runbook — with network-verified containment.
Deliverable: A deployment architecture with packet-level evidence that client data does not egress.
How are learners assessed?
- Containment verified with network monitoring, not configuration review
- Benchmark run against real financial workloads
- Runbook tested by someone who did not write it
What ships with the course?
Facilitator guide
Session-by-session running order, discussion prompts, and the questions that reliably derail a room.
Learner workbook
Exercises, checklists, and the templates each module's activity produces.
Hands-on lab environment
A sandboxed ibl.ai deployment so exercises run against real agents, not screenshots.
Assessment bank
Scenario questions and rubric criteria mapped to each stated learning outcome.
Source bibliography
Every primary regulation and standard cited on this page, linked and dated.
Which AI agents does this course use?
The hands-on modules run against agents already deployable on the ibl.ai platform for financial services.
Where does the course material come from?
Every module is grounded in primary sources — the regulation, standard, or research itself, not a summary of it. Each was resolved at authoring time.
- Gramm-Leach-Bliley Act guidance
Federal Trade Commission
Safeguards Rule obligations applied to AI infrastructure in Module 4.
- FFIEC
Federal Financial Institutions Examination Council
Technology risk and examination expectations for infrastructure.
- Transformers documentation
Hugging Face
Technical reference for open-weight deployment and fine-tuning.
- NIST SP 800-53 Rev. 5
NIST
Control catalog the infrastructure design maps to.
Delivery notes
Binding guidance for anyone preparing and delivering this course.
- Module 8's containment verification must be empirical, at packet level. Technology risk reviewers ask for evidence and a configuration screenshot will not satisfy them.
- Module 6's extraction testing is under-taught. Fine-tuned models can leak training data and a financial institution's fine-tuning corpus is client data.
- Module 2 must benchmark on real financial tasks. General benchmark performance does not predict performance on regulatory text or transaction narrative work.
- Module 3 should be honest about when hosted is cheaper. At low volume it usually is, and the case for private deployment here is containment, not cost.
- Re-verify the open-weight landscape at every revision. Licences and capabilities change and a stale recommendation is actively harmful.
Why run AI training on a platform you own?
You own the course, not a licence to it
Course content, learner data, and the platform run inside your perimeter — you own all the code and the data.
Model-agnostic delivery
Run the course's AI components on any LLM — Claude, GPT, Llama, Gemini, Command — and switch anytime.
No per-seat training licences
Usage-based or self-hosted, so cost tracks actual use rather than headcount.
Deploy anywhere
Cloud, private VPC, on-premise, or fully air-gapped — including for cohorts that cannot use public AI tools.
Frequently asked questions
What does the Private LLMs in Finance: Keeping Client Data In-House course cover?
Financial institutions have the clearest case for private deployment and the least clarity on how to do it. This course covers what client data actually leaves with a hosted model, open-weight selection and the capability trade, inference economics for a bank workload, GLBA obligations applied to AI infrastructure, and operating a private model. It runs 6.5 hours across 8 modules across 8 modules, at advanced level, and closes with a capstone: Private deployment architecture with verified data containment.
Who should take Private LLMs in Finance: Keeping Client Data In-House?
It is written for Infrastructure and platform engineering teams, Information security architects, Technology risk staff, CTOs and heads of technology. Prerequisites: Infrastructure and systems background; Familiarity with your institution's security architecture.
Can we run this course on our own infrastructure?
Yes. ibl.ai is model-agnostic and deploy-anywhere — cloud, private VPC, on-premise, or fully air-gapped — and you own all the code and the data. Cohort data, submissions, and any material learners upload stay inside your perimeter, which matters for financial services teams that cannot send work to a public AI tool.
How do we get access to Private LLMs in Finance: Keeping Client Data In-House?
Request access and we will set it up for your cohort — hosted by ibl.ai, or running against your own deployment. Tell us the group size and timing you need, and whether it should run inside your own perimeter.
How much does AI training for financial services cost on ibl.ai?
There is no per-seat pricing — you pay for usage or self-host and pay only for the infrastructure, so a 5,000-person rollout does not cost 5,000 licences. 1.6M+ users across 400+ organizations run the platform this way, including NVIDIA, MIT, and Syracuse University.