Back to Blog
Projects & Builds

Scaling QA for 1500+ APIs at CIBC

How I built predictability into one of the largest middleware API ecosystems in Canadian banking.

12 min read
Jul 22, 2025

When I took ownership of the Shared Services middleware QA function at CIBC, I inherited something most people don't fully appreciate the weight of:

1500+ SOAP APIs powering every non-financial transaction across a Tier-1 bank.

Account updates. Profile changes. Authentication flows. Settings. Alerts. Help desk workflows. Every one of them routed through this middleware layer. Every one of them needing quality.

The problem wasn't a lack of effort. The problem was no architecture behind the quality.

1500+

APIs Managed

22

Team Members

50+

Programs Delivered

Millions

Daily API Calls

1

The First Problem: Nobody Knew What Existed

The first thing I discovered: there was no single authoritative map of the API ecosystem. No inventory of what owned what. No record of dependencies. No classification of risk.

What we were missing:

  • Which APIs existed and who owned them
  • What systems each API touched upstream and downstream
  • What data each API required and in what format
  • The dependency chains — what broke when an API changed
  • The real business impact if an API failed

A bank is like an iceberg. The APIs you see are a fraction of what's underneath. So I started with a full API census.

We built a centralized inventory: functional description, request/response schema, downstream services, upstream triggers, authentication type, risk classification, business usage frequency, environment availability, logging readiness, test coverage status, responsible team, and LOB owner.

This wasn't documentation. This was architecture-level visibility. For the first time, the bank had a single source of truth for every API dependency in the middleware layer.

2

Categorizing APIs by What Actually Matters

Most QA teams categorize APIs by business domain. At middleware scale, that's not enough. I introduced a framework based on how APIs behave, not just what they do.

High-risk transactional

Examples: Login validation, session refresh, profile updates, customer lookup

Failure impact: Critical — impacts customer trust immediately

High-dependency orchestration

Examples: Account linking, address validations, notification triggers, security challenges

Failure impact: Cascading — one failure causes multi-system outages

Static or reference-data

Examples: Config lookups, branch metadata, product catalog

Failure impact: Localized — contained failures

Vendor-integrated

Examples: Credit agency lookups, external identity checks

Failure impact: Unpredictable — depends on external SLA

What this categorization changed:

  • We stopped treating all APIs the same
  • Regression became risk-based instead of brute force
  • Coverage dashboards actually reflected business risk
  • Automation investments aligned with where failures hurt most
  • Architecture teams used the data for dependency planning
3

Building the Organization Around Outcomes

A team of 22 engineers was reshaped — not by title, but by purpose.

Middleware Integration Squad

Owns API-level testing across all LOBs

Regression Squad

Maintains and evolves the risk-based regression strategy

Release Readiness Squad

Gates releases with confidence indicators, not guesses

Automation Squad

Builds and maintains modular, environment-agnostic test infrastructure

Each squad aligned to an outcome. Each outcome tied to a business metric. No ambiguity about what success looked like.

4

Regression That Actually Prevents Outages

The biggest mindset shift: regression at scale is not a giant test suite. It's dependency management.

What our regression strategy was actually built on:

Contract Validation

Automated schema diffing detected when a team made an unannounced breaking change before it reached any downstream consumer.

Blast Radius Prediction

Before any API version change, we produced a consumer impact list — every team, every system affected.

Data Orchestration

Automated data seeding, cleanup, and environment resets eliminated 30–40% of false failures that were masking real ones.

Multi-Hop Chain Testing

API A → API B → API C → downstream service. Most outages come from chains, not endpoints. We tested the chains.

5

Metrics That Executives Actually Use

Leadership doesn't want test counts. They want answers to questions they're actually asking.

1

Release Readiness Radar

"What's blocking the release?"

2

System Stability Index

"Which APIs are breaking the ecosystem?"

3

Automation Maturity Heatmap

"Where should we invest next?"

4

Dependency Risk Score

"Which chains are fragile?"

5

SLA Reliability

"How stable are vendor APIs?"

6

Production Insight Loop

"What incidents happened in prod? How do we back-propagate them into test plans?"

This transformed QA from reactive to a source of real-time truth for decision-makers.

6

The Outcome: A Middleware Platform the Bank Can Trust

What we delivered:

  • ✔ Regression time reduced significantly across all programs
  • ✔ Predictability improved — releases became calm, not chaotic
  • ✔ Defect clusters in production dropped
  • ✔ LOB confidence in middleware quality increased measurably
  • ✔ Automation-driven releases became the new standard
  • ✔ Middleware QA recognized as a strategic pillar, not a testing checkbox

Today, this ecosystem serves millions of Canadians — silently and reliably.

What This Really Was

Owning QA for 1500+ APIs isn't about testing. It's about understanding architecture, governing quality for an entire ecosystem, leading people across organizations, and building systems that prevent failures before they happen.

Most people test systems.

I help architect reliability into them.

Enjoyed This Article?

I write about building reliable systems, leading teams, and shipping products that matter.