Active Research
Production AI systems require verifiable behavior, governed data handling, and auditable agent decisions. The engineering discipline for this does not yet exist. This research builds toward it.
Behavioral Topology
Trajectory-level verification for multi-turn AI systems. Conversations evaluated as paths through semantic space, not as sequences of independent turns.
Runtime AI verification requires conversation-level geometry. The unit of analysis is the trajectory, not the turn. A path can degrade while every individual point on it passes inspection.
Falsified if — point-in-time signals prove sufficient to detect multi-turn failure modes — intent drift, sycophantic convergence, context erosion — without access to trajectory history.
The path a conversation takes through semantic space is a risk signal. I built a method that learns to score it from labeled outcomes alone — no domain expertise, no rules to write — and validated it on a held-out benchmark. The signal is real.
- Do trajectory baselines built in one domain generalize across models, task types, and deployment contexts?
- Region path scores predict outcome as a leading indicator in one domain — which other geometric signals extend this, and how early can reliable warning be established?
- How do trajectory signals compose with point-in-time signals into a unified verification framework?
- What is the right visualization layer for operators who need to read a conversation's health at a glance?
- Can trajectory fingerprints classify the kind of failure — not just that something is wrong, but what type of wrong?
Harness Engineering
Current agent protocols carry capabilities but not governance. This track develops a communication standard that embeds authorization, provenance, and policy constraints in every interaction between an LLM and its periphery — tools, memory, sessions, external APIs.
Runtime Data Governance
Agentic systems handle PII and sensitive data in ways current privacy frameworks weren't designed for. This track explores runtime architectural approaches to anonymization, data minimization, and sensitive data handling.
Agent failures are sequence properties, not step properties — per-action checks cannot catch them. A method that scores a conversation's trajectory in real time, learning what a failing path looks like from outcome labels alone, with no domain expertise and no hand-written rules.
The argument for trajectory-level verification: four failure modes point-in-time metrics miss, a geometric toolkit for detecting them, and why a conversation must be evaluated as a path, not a sequence of turns.
If you embed every turn in a conversation and treat the resulting sequence of points as a path through semantic space, do geometric properties of that path discriminate successful from failed conversations? An empirical study using UltraInteract trajectory pairs to move the framework from theoretical argument to measured ground truth.
A practitioner companion to the main article. For each of the seven trajectory signals: the precise formula, what to embed, cost at scale, and implementation notes. Everything needed to instrument a production system without reading the underlying research literature.
I'm looking for practitioners with production data on multi-turn agent failures and researchers applying dynamical systems or geometric thinking to LLM reliability. If any of this maps to what you're working on, I'd welcome the conversation.
prasadbhamidi@gmail.com →