A financial data office answers "do we have a licence for that" in seconds

Challenge

• Sales priced deals without seeing data costs. Some deals turned out unprofitable.
• The internal dependency tracker did not cover every asset class. Bulk usage queries took a full day of engineering time each.
• Every vendor had its own report template, channel, and cadence. Some reporting obligations were going unfulfilled.
• Three different data office leads in one year. Institutional knowledge kept walking out the door.

Approach

• Extracted nine canonical fields per contract (permitted usage, fees, term, notice deadline, reporting obligations, delivery method, price increase mechanics, termination rights, contact and parties).
• Vectorless retrieval with PageIndex keeps clause structure intact. Every answer cites the contract, section, and page.
• Two CrewAI agents share one knowledge base, one on the legacy PostgreSQL dependency tracker, one on the new ClickHouse and Iceberg platform.
• Per-field confidence scoring routes low-confidence extractions to a human review queue.
• Automated quarterly reports for the three highest-effort vendors. AUM reports stay semi-automated where client-side data is required.

What shipped

• Document repository ingestion and OCR pipeline for scanned PDFs.
• Nine-field extraction with confidence scoring and amendment versioning.
• Knowledge base Q&A with source citations.
• Deal pricing tools, license check, cost calculator, coverage check, vendor comparison.
• Compliance dashboard, utilisation vs entitlement, overage alerts, notice calendar.
• Two platform agents (old and new) and a data lineage explorer.
• Auto-generated reports for top exchange and data-provider vendors with a submission tracker.
• SOP generation from extracted terms.

Outcomes

• Bulk vendor usage queries, from one engineering day to under 5 seconds at p95.
• Reporting cycle time, from 1+ day per report to a reviewed draft generated automatically.
• Notice misses, zero by construction via the calendar and alerting layer.
• Deal pricing visibility, live for sales and product teams from day one of pilot.

Stack

• NestJS
• Next.js
• CrewAI agents (Postgres and ClickHouse and Iceberg)
• PostgreSQL via Prisma
• BullMQ on Redis
• PageIndex with LiteLLM
• AWS S3. Integrations include cloud document repositories
• internal wikis
• the client's dependency tracker
• and ClickHouse with Iceberg on S3

What's next

Expand the vendor-agent automation to lower-effort reports, layer AI-assisted negotiation prep on top of the knowledge base, and extend SOP generation as the system of record for procurement governance.