Black Knight, Inc. was bringing the power of AI to the mortgage industry. AIVA is an AI that understands business grammar, learns skills, and acts as an alternative workforce for mortgage lenders — capable of learning new skills in days as it trains. One of its skills is Compile Loan: it organizes borrower, vendor, lender, and closing documents into a complete, indexed file, reading, classifying, and entering key data directly into the loan origination system. AIVA takes the first pass.
But in a regulated workflow, nothing the AI does can stand on its own. AIVA processes thousands of documents an hour, and every transaction lands in an operator’s queue to be verified and approved before the loan can move forward. The operator is the bottleneck by design — and at fifteen minutes per loan, that bottleneck was expensive enough to put the economics of the whole approach in question.
The screen operators had gave them none of AIVA’s work to build on. No queue depth, no visible confidence, no distinction between what had been checked and what hadn’t. Operators were re-deriving conclusions the AI had already reached.
The AI had already done the work. The interface made operators do it again.
The Approach
My hypothesis was that fifteen minutes was an interface problem, not a model problem. If the screen surfaced AIVA’s confidence, highlighted exactly what it had already done, and made correction a single gesture, operators would stop re-deriving and start verifying — and the review could collapse to two or three minutes without touching the model.
I designed two operator surfaces against that hypothesis — Classify Documents and Data Extract — on one shared principle: the operator’s job is to correct AIVA’s labels, never to start from scratch. Behind them sits a supervisor’s second-round review, where every correction becomes both a compliance record and a training signal. I did this work as the sole designer, and carried it through the Black Knight acquisition.
AIVA’s accuracy and classification are rendered on every document set, scannable at a glance. An operator decides where to spend attention before opening anything — the low-confidence work announces itself instead of hiding in a queue that looks uniform.
Correction as a single gesture
Feedback controls sit inline with the thing being corrected. Fixing a misclassification is one action, not a detour — which is what makes correcting cheaper than re-deriving, and what keeps the training loop fed with real operator judgement.
Queue state visible without opening anything
Operators can see how many files are waiting, and tagged documents are visually distinct from untagged ones. Progress is a property of the screen rather than something the operator has to hold in their head across a shift.
Designed for the physical act of scanning
Sizable thumbnails, rapid scrolling, zoom in and out, and partial zoom on a single region of a page. At thousands of documents an hour, the ergonomics of reading are the product — a half-second of squinting per page is the difference between the target and missing it.
One audit trail serving compliance and training
Every transaction records whether AIVA was right and whether an operator intervened. That single record satisfies the regulated workflow and improves the model — rather than building a compliance log and a feedback pipeline as two separate systems.
Classify Documents
Classification is the first pass over a loan packet — AIVA sorts pages into document types, and an operator confirms or corrects the sort. It is the highest-volume screen in the workflow, so it set the ceiling on the whole review time.
Old design
the interface operators started with
What the redesign had to fix
Operators can see at a glance how many files are in the queue.
Highlight AIVA’s accuracy and classification on each set of documents for quick scans.
Quick controls for operator feedback.
Operators can go back through their work to verify or correct their classifications.
Tagged documents are visually distinct from untagged ones — labeled or moved to a "done" state.
Rapid scrolling through pages with sizable thumbnails, plus zoom in and out for a readable view.
Partial zoom so operators can expand the important region of a page.
New designs
1 / 7Classify Documents — redesigned
demo — Classify Documents in action
Data Extract
Once pages are classified, AIVA pulls the values that matter — borrower income, insurance terms, closing figures — and writes them into the origination system. Here an operator isn’t judging a category but a number, and a wrong number carries further than a wrong label.
Old design
data extraction, before the redesign
What the redesign had to fix
Operators can see how many files are in the queue and approve without hunting for data points.
Highlight AIVA’s accuracy and extracted data points on each page for quick scans.
Quick controls for feedback that help train AIVA by correcting errors.
Rapid scrolling through pages, with alternate data options surfaced when applicable.
New designs
1 / 4Data Extract — redesigned
demo — Data Extract in action
Supervisor Review & Audit Trail
After operators correct AIVA and give feedback, a supervisor performs a second-round verification. The audit trail of every transaction shows how often AIVA was correct and how often an operator had to give manual feedback — serving both the regulated workflow and the training loop that makes AIVA better over time.
1 / 8Supervisor review & audit trail
Outcome
15 → 2–3minutes per loan — the target review-time collapse
2operator surfaces redesigned — Classify Documents and Data Extract
100%auditable transactions — compliance record and training loop in one
2019the year this trust layer shipped — before the vocabulary existed
The redesign reframed AI from a tool the operator used to a partner the operator verified. The result is a UI vocabulary I’ve reused in every AI product since — Clinical Study Master’s source attribution, KAM Field Advisor’s structured responses, the Trinity Design System’s AI module. Different surfaces, same trust layer.
