bytelyst-devops-tools/docs/INTERVIEW/02-competency-deepdives.md

# 02 · Competency Deep-Dives

One section per competency-matrix row. Each gives you: **the concept** (so you sound
fluent), **our anchor** (so it's credible), **say-this** talking points, and the **honest
edge** (where to pivot to roadmap rather than overclaim).

---

## A. Agentic Frameworks — LangGraph / LangChain (must) · ADK / A2A / AutoGen (nice)

**Concept.** LangGraph models an agent as a **state graph**: typed shared state, nodes
(LLM calls / tools), and conditional + cyclic edges, with a checkpointer for durability
and human-in-the-loop. LangChain supplies the primitives (tool binding, structured output,
retrievers). ADK is Google's agent SDK; **A2A** is an open agent-to-agent interop protocol
(agent cards, task lifecycle, message/artifact exchange). AutoGen favors conversational
multi-agent loops.

**Our anchor.** `agent-queue/` is a *real* multi-engine orchestrator (claude·codex·devin)
with an explicit `inbox→doing→done/failed` state machine and requeue cycle — see
`01-ecosystem-rag-fabric.md §3`. `packages/mcp-client` + `mcp-server` provide tool binding;
`packages/llm-router` is the model-selection layer a node would call.

**Say this.**
- "LangGraph's value over a raw loop is **typed state + conditional/cyclic edges + checkpointing** — which is exactly the trio I'd want for a CRAG/Self-RAG loop that re-retrieves."
- "I treat **routing** as a first-class node: a cheap classifier decides single-shot vs. multi-hop vs. tool-only, so I'm not paying for a full agent loop on a FAQ."
- "For A2A I'd expose each of our product agents with an **agent card** (capabilities, auth, cost) so a supervisor agent can delegate — that's the natural evolution of agent-queue's engine flag."

**Honest edge.** "My shipped orchestration is framework-light by choice (bash-portable).
The LangGraph port is scoped (`04 §A`); conceptually it's a 1:1 mapping, not new ground."

---

## B. RAG Architecture — hybrid retrieval, reranking, HyDE, Self-RAG (must) · RAPTOR, multimodal (nice)

**Concept.**
- **Hybrid retrieval** = dense (vector) ⊕ sparse (BM25) ⊕ optionally graph, fused (RRF — reciprocal rank fusion) then **reranked** by a cross-encoder (scores the (query, passage) pair jointly; far more precise than bi-encoder similarity). **ColBERT** = late-interaction reranking (token-level MaxSim) — accurate and scalable.
- **Context compression** = drop/condense retrieved spans to fit the budget and reduce distraction.
- **HyDE** = embed a *hypothetical answer* the LLM drafts, not the raw question — closes the query/document vocabulary gap.
- **CRAG** = grade retrieved docs; if weak, correct (web/alt-source or rewrite) before generating.
- **Self-RAG** = the model emits reflection tokens deciding *whether to retrieve* and *whether its draft is supported*, looping if not.
- **RAPTOR** = recursively cluster + summarize chunks into a tree; retrieve at the abstraction level the query needs (great for "summarize this 200-page filing").

**Our anchor.** `packages/extraction` + `extraction-service` (:4005) already turn URLs/docs
into retrievable units. `invt_trdg`'s AI chat is a retrieve-then-reason loop over structured
data today. The hybrid index + rerank + HyDE/CRAG/Self-RAG loop is the headline enhancement
(`04 §B`).

**Say this.**
- "I default to **hybrid + rerank** because pure-vector misses exact-match terms (a regulatory clause number, a ticker, an account ID) that BM25 nails."
- "HyDE and reranking attack **different** failures — HyDE fixes *recall* (you retrieved the wrong thing), reranking fixes *precision* (you retrieved too much). I tune them independently against context-recall vs. context-precision."
- "RAPTOR earns its cost on **long regulatory corpora** where the answer spans sections; for transactional Q&A it's overkill."

**Honest edge.** Lead with the *reasoning about when to use each*, which is the architect
signal; the implementations are well-trodden and scoped on our roadmap.

---

## C. Structured Retrieval — Text-to-SQL, schema-aware retrieval (must) · Snowflake Cortex, BigQuery ML (nice)

**Concept.** Text-to-SQL turns NL into SQL against a known schema. The enterprise risks are
**wrong joins, full-table scans, and data leakage**. Mitigations: restrict to **read-only
semantic views**, inject **schema + few-shot exemplars** (schema-aware retrieval over table/
column descriptions), validate/parse the SQL before execution, enforce **row-level security**,
and cap cost. Warehouse-native options (Snowflake Cortex, BigQuery ML) push inference next to
governed data.

**Our anchor.** `invt_trdg`'s AI chat is **schema-aware tool-calling** — NL maps to typed
operations (get quote, create trade plan, manage watchlist/alerts) over a known domain. That's
the *safe* form of Text-to-SQL: the model picks a vetted, parameterized tool rather than
emitting arbitrary SQL.

**Say this.**
- "I prefer **typed tool-calling over free Text-to-SQL** wherever the query space is bounded — it's auditable and injection-resistant. I reserve generative SQL for genuine ad-hoc analytics, behind read-only views with RLS."
- "Schema-aware retrieval is itself a RAG problem: I embed table/column docs and retrieve the *relevant* schema slice into the prompt, so the model isn't drowning in a 400-table catalog."

**Honest edge.** "Free Text-to-SQL against a warehouse I've prototyped more than shipped;
in production I've leaned on the tool-calling pattern because it's safer in regulated data."

---

## D. Unstructured Retrieval — PDF parsing, layout-aware chunking (must) · multimodal (nice)

**Concept.** Naïve "split every 1000 chars" chunking destroys meaning. **Layout-aware**
parsing (PyMuPDF, Unstructured.io) preserves headings, tables, lists, reading order; **OCR**
(Tesseract/Azure Doc Intelligence) handles scans. **Semantic chunking** splits on topic
boundaries, not byte counts. Tables and figures need special handling (serialize tables to
markdown; caption images). Each chunk carries **provenance metadata** (doc id, page, section)
— mandatory for citations.

**Our anchor.** `packages/extraction` + `extraction-service` already perform URL/task/doc
extraction; `notelett` models structured notes. Layout-aware PDF + OCR is the additive piece
(`04 §B`).

**Say this.**
- "Chunking is where most RAG quality is won or lost. I chunk on **document structure first, size second**, and I always attach page/section provenance so the answer can cite a clause, not 'a document.'"
- "For regulatory filings I keep **tables intact** and store a text serialization alongside — losing a covenant table to a mid-row split is a correctness bug, not a formatting one."

---

## E. Graph RAG — KG + vector hybrid (must) · SPARQL / ontology design (nice)

**Concept.** Vector RAG finds *similar* text; it can't answer *"which entities are connected
to X within 2 hops."* Graph RAG retrieves a **subgraph** (entities + relationships) and feeds
it as structured context — ideal for "show the ownership chain," "trace this transaction
counterparty network," KYC/AML link analysis. Patterns: extract entities/relations at ingest,
build a knowledge graph, retrieve by **graph traversal (Gremlin/Cypher) + vector seed**, then
generate over the fused context. SPARQL/ontologies add formal semantics (RDF, classes).

**Our anchor.** We run **Azure Cosmos DB** (`packages/cosmos`), which exposes the **Gremlin**
graph API — the JD lists "Azure Cosmos Gremlin" explicitly. `event-store`/`events` already
model entity relationships over time. Standing up a Gremlin KG + graph-augmented retrieval is
`04 §D`.

**Say this.**
- "I use the graph for the **'connected-to' questions** vector search can't answer, and seed traversals from vector hits — vector finds the entry node, the graph supplies the neighborhood."
- "In banking this is the **AML/KYC** sweet spot: link analysis across counterparties is inherently graph-shaped."

**Honest edge.** "Cosmos Gremlin is in our stack; the KG is a clean build on infrastructure
we already operate, not a new platform bet."

---

## F. Vector Databases — Pinecone / Weaviate / Azure AI Search (must) · Qdrant, pgvector, multi-tenancy (nice)

**Concept.** Trade-offs: **Pinecone** (managed, serverless, fast to ship), **Weaviate/Qdrant**
(open, hybrid + filtering, self-host control), **Azure AI Search** (managed hybrid: vector +
BM25 + semantic rerank in one service — the natural Azure pick), **pgvector** (lives in your
Postgres → transactional consistency, one backup story, lowest ops). **Multi-tenancy** =
namespaces (Pinecone) / index-per-tenant (Azure) / collection or payload filter (Qdrant) /
schema or row filter (pgvector). Metadata filtering is as important as the ANN index.

**Our anchor.** Postgres is in the stack → **pgvector** is our lowest-friction path and gives
transactional consistency with the source rows. Multi-tenancy is already first-class
(`productId`, two-instance Hermes) — see `01 §6`.

**Say this.**
- "On Azure I'd default to **Azure AI Search** because it gives me hybrid + semantic reranking as a managed service — fewer moving parts in a regulated audit boundary."
- "For tight transactional coupling (the vectors must stay consistent with the source-of-truth rows) I reach for **pgvector** — one database, one backup, one ACL model."
- "I pick the vector DB **last**, after I know tenant count, filter cardinality, recall target, and the audit boundary — it's a consequence of requirements, not a religion."

---

## G. Grounding & Eval — RAGAS, TruLens, faithfulness SLAs (must) · LangSmith, LLM-as-judge (nice)

**Concept.** The core metrics:
- **Faithfulness / groundedness** — is every claim supported by retrieved context? (anti-hallucination)
- **Answer relevancy** — does the answer address the question?
- **Context precision** — are the *top* retrieved chunks the relevant ones? (reranker quality)
- **Context recall** — did we retrieve *all* needed evidence? (retriever quality)
- **Answer correctness** — vs. ground truth.

**RAGAS** computes these (often LLM-as-judge). **TruLens** adds the "RAG triad" + feedback
functions + tracing. **DeepEval** is pytest-style assertions for CI. **LangSmith** = tracing/
eval ops. An **SLA** turns metrics into gates ("faithfulness ≥ 0.9 or abstain").

**Our anchor.** `flowmonk` is a *grounding pattern made flesh*: the deterministic scheduler is
the source of truth, and **the AI layer is constrained to explanation / safe recommendation** —
it cannot invent a plan, only narrate one. That's hallucination mitigation by architecture.
`diagnostics-client` / `telemetry-client` / `monitoring` + Hermes dashboards are the
ready-made home for an eval harness + drift monitor (`04 §E`).

**Say this.**
- "I don't ship grounding as a vibe — I ship it as **gates**: a faithfulness threshold that triggers abstain-and-escalate, evaluated continuously, with regressions blocking deploy in CI (DeepEval)."
- "flowmonk taught me the cheapest hallucination fix: **don't let the LLM be the source of truth.** Make a deterministic engine authoritative and scope the model to explanation. In banking that's the difference between a tool and a liability."
- "Eval is a **two-loop** system: offline (golden set in CI) catches regressions pre-deploy; online (sampled production traces + LLM-judge) catches **factual drift** as corpora and models change."

---

## H. Cloud Platform — Azure (AI Foundry / OpenAI / Search) (must) · AWS Bedrock, GCP Vertex (nice)

**Concept.** Azure AI Foundry = the build/eval/deploy hub; Azure OpenAI = governed model
endpoints with content filtering and data-residency; Azure AI Search = managed hybrid index;
Cosmos DB = docs + Gremlin graph. The architect skill is **provider-portability**: abstract the
model behind a router so Azure OpenAI / Bedrock / Vertex are swappable, and keep data in the
customer's tenancy.

**Our anchor.** Azure Cosmos DB in production (`_AZURE/`, `packages/cosmos`).
`packages/llm-router` is the provider-abstraction layer — Azure OpenAI today, Bedrock/Vertex
swap-in, no app rewrite. `packages/ollama-client` covers on-prem/air-gapped inference.

**Say this.**
- "I keep a **router seam** so model choice is a config decision, not an architecture decision — and so a bank can pin everything to its own Azure tenant for residency."
- "On-prem / air-gapped is real in banking; `ollama-client` in our stack means I've thought about the **no-egress** deployment, not just the SaaS path."

---

## I. AI Governance — access-controlled RAG, Zero Trust (must) · SR 11-7, EU AI Act (nice)

**Concept.** Governance is **structural**, applied at every hop (see `01 §4–5`):
access-controlled retrieval (you can only retrieve what your identity entitles you to),
row/column masking, role-aware context injection, immutable audit, instant kill-switch,
model cards, RACI. Zero Trust for agents = never trust the agent's request implicitly; verify
identity + scope at the tool boundary every call.

**Our anchor.** `packages/auth` + `fastify-auth` (identity/scope), `field-encrypt` /
`client-encrypt` (column/field masking), `feature-flag-client` + `kill-switch-client` (instant
constraint), `event-store` (immutable audit), MCP server (explicit tool boundary). This is the
**deepest, most defensible** part of the story.

**Say this.**
- "I design retrieval so that **masking and audit are inescapable** — they live at the platform/MCP layer, so no product surface can route around them. That's what 'governance by architecture' means."
- "A kill-switch that disables a model or single tool **without a redeploy** is a hard SR 11-7 requirement: supervisors must be able to constrain a model in production immediately."

(Full regulatory mapping in `05-banking-blueprints.md`.)

---

## J. Domain: Banking — support / compliance automation (must) · model risk, KYC/AML (nice)

**Concept.** The flagship use cases: **customer-support automation** (grounded answers from
policy/product docs with citations + escalation), **compliance-document retrieval** (find the
controlling clause across filings/policies), **regulatory reporting**, **model risk management**
(SR 11-7), **KYC/AML** (entity/network link analysis → graph RAG).

**Our anchor.** `invt_trdg` is our **regulated-industry analog**: market data, trade plans,
alerts, profiles — a domain where wrong answers have consequences and auditability matters. The
patterns (typed tool-calling, audit, abstain-on-uncertainty) port directly to a bank.

**Say this.**
- "My trading product is the closest non-bank analog to a banking workload: it taught me to **default to abstain over guess** when money or compliance is on the line, and to make every answer traceable to a source."
- "KYC/AML is where my graph-RAG and my governance stories converge — link analysis on a knowledge graph, behind access-controlled, audited retrieval."

**Honest edge.** "I haven't shipped inside a chartered bank; my regulated-domain reps come from
trading and from designing for SR 11-7 / EU AI Act controls — which I can walk through concretely."