Four doors into one system. Same rules behind every door.
The Trust Contract
One action. Four surfaces. One safety model.
The hard part of agentic UX is not the chat. It is deciding when the system may act, and proving what happened after it did.
Everything on this page shipped
Pick a depth.
The big type stands on its own. The fold-out panels under each scene hold the machinery.
Four doors. One promise.
Think of a bank: a branch, an ATM, a phone line, an app. Four ways in, one account, one set of rules. The ATM can't hand you money the teller would refuse. Trackly works the same way.
System · what stays the same, what differs on purpose
Same everywhere: what an action means, who is asking, what they may touch, when a human must say yes, and what gets written down afterward.
Different on purpose: the UI previews and confirms visually. The CLI is terse and scriptable. The API is strict and stable. MCP describes every tool in plain text, because text is all an agent gets.
Story · what this looks like in production
Trackly runs an iOS and macOS app, a web app, a versioned API, an npm CLI, a local MCP server, and a hosted OAuth MCP server, across 170K+ jobs at 3,800+ companies, plus a packaged plugin in marketplace review. I built the parts that keep those doors honest with each other: shared schemas, one approval model, one auth story, one telemetry boundary. The rest of this page is the times that was hardest.
Intent → guard → action → proof.
Every door tells this same story, including the ending where the system is not sure.
Express intent
A person or an agent asks, through whichever door fits.
Translate once
The door turns the request into one standard description of the action.
Gate the risk
Who is asking. What would change. Is the state fresh. Is there a way back.
Act + prove
Do only what was allowed. Check it happened. Write it down. Not sure? Stop.
Approval is a signed check, not a blank one.
The human approves one exact plan, amount filled in. If the plan or the world changes, the signature dies and the system asks again.
System · the standard action description
What every action carries: what to do, who asked, which records, the state it was planned against, the exact proposed change, how risky and how reversible it is, whether a human must approve, a key that makes retries safe, and the receipt written after execution.
When things go wrong: unknown or stale state never executes. A changed plan voids its approval. A timeout means "outcome unknown," so inspect state before retrying. A verification mismatch stops, escalates, or undoes. It never shrugs.
For an agent, schema is the screen.
A person picks from a dropdown. An agent reads a block of text and picks for you. That text is the agent's screen. Write it vague, and the agent picks wrong with full confidence.
Visa sponsorship ( ) All ( ) Exclude "No" ( ) Only "Yes" A job that says nothing shows "Not mentioned."
sponsorship: all | exclude_no | only_yes "exclude_no hides jobs that explicitly say they do not sponsor (~13%). only_yes keeps only jobs that explicitly say they do (~2%; most postings never state a policy either way). Default: all."
Those percentages are printed on the tool itself because they are the safety rail. Out of 1,000 fresh US jobs, ~20 say "we sponsor," ~130 say "we do not," and ~850 say nothing at all. Now watch what the schema text does to the same request:
Vague schema
- The user asks: "jobs that have not ruled me out"
- The agent sees three bare values and guesses
only_yessounds safest - ~20 jobs come back
- The user concludes "the market is closed to me." False.
Schema as shipped
- The description warns that "yes" is ~2% and silence is normal
- The agent picks
exclude_no - ~870 jobs come back: only the 130 explicit "no" doors are removed
- Every silent job stays visible, labeled "Not mentioned"
System · a filter that refuses to silently do nothing
One catch. The backend applies this filter only to the fresh job feed, never to jobs you already saved. It refuses to hide a saved job behind your back: a saved card keeps showing the text you saved it with, while verdicts keep getting re-checked in the background, and the two can disagree.
So when a caller asks for the filter without naming a view, the CLI and MCP add status=new ("fresh feed only") on their own. Without that, --sponsorship exclude_no by itself would come back completely unfiltered while you believe it worked. You asked for a filter, so the client either applies it for real or narrows the request until it can.
The same trap exists in one SQL operator: NOT (status = 'no') also throws away every job with no answer, because comparing against a missing value returns "unknown," not false. That is "silence becomes a No" in a single line of code. The shipped predicate guards it with CASE ... ELSE FALSE and a test that locks it in.
Story · the first time the schema lied
Early on, the MCP schema accepted locationFilter: ["us","non_us"]. Perfectly valid to the schema. The backend silently dropped what it did not understand and returned US-only jobs: the exact opposite of the ask, delivered with HTTP 200 and a plausible-looking list. Four review bots flagged it in the same week. The fix rejects the combination at the schema, so the agent gets a visible error instead of a confident wrong answer (cli #14). The sponsorship model above went through the same discipline in cli #65 and #112: make the schema tell the truth about the backend, including the uncomfortable parts.
The guard shipped. 24 minutes later it failed.
One boolean, "stop storing my sensitive answers," deletes every stored sensitive answer: address, demographics, visa status. We shipped a two-step confirmation for it. It was bypassed the same morning. By me.
The setup that matters: Trackly has two MCP servers. One is hosted, which we run and can update instantly. One ships in the npm package, so it is installed software on the user's machine, like an app. To the backend, both are just callers. The confirmation was written into both of them. The backend itself had no check: it trusted its callers to have confirmed.
The guard ships
A real challenge-and-confirm flow, written into both MCP servers: the hosted one and the new npm release.
24 minutes later
My laptop is still running last week's npm install. That copy has no guard. It forwards the raw call.
One call
The backend obeys, because it never checked. 33 encrypted answers hard-deleted. The recovery archive did not exist yet. Nothing came back.
7 hours later
The check moves into the backend, inside the delete's own transaction. Both MCP servers delete their guard code.
Before: guard in the callers
- Confirmation lived in the two MCP servers, not the backend
- The npm one is installed software: every user runs whatever version they last installed
- The backend accepted the raw delete from anyone
- One boolean. No archive. No undo.
After: check where the data dies
- The server recomputes what would be deleted, inside the delete's own transaction
- It answers with a one-time challenge tied to the exact profile version
- The caller must echo the challenge back. Any change voids it.
- Archive first, delete second. If archiving fails, nothing deletes.
- Old clients get a refusal they cannot bypass
System · the two-step protocol, as a conversation
- Agent: "Delete the stored sensitive answers."
- Server: "Not yet, and nothing was deleted. Here is exactly what would be erased: address, gender, visa status, every stored sensitive answer, listed by name. And here is a code. It only works while the profile still looks exactly like it does right now."
- The agent shows the person that list. The person says yes, for real.
- Agent: the same request again, this time carrying the code.
- Server: "Did anything change since I issued the code? One edited answer kills it, and we start over. Still identical? Then I copy everything into a 30-day recovery archive first and delete second. The two happen together or not at all: if the copy fails, nothing is deleted."
Story · the fix was deleting code
The counterintuitive part: the durable fix removed code. Both MCP servers deleted their guard logic entirely. They now just pass the request through and relay the server's answer. The rule exists in exactly one place, where the data lives, so no installed copy can ever fall behind it again.
The iOS and web apps still confirm with their own "are you sure?" dialog and delete in one step; the dialog is their confirmation. But even those deletes now copy everything into a 30-day recovery archive first, and an admin can restore from it (close-ai #1292 → #1296, cli #89 → #90).
Measure behavior. Refuse content.
An MCP server is a black box. A web app sees every click; an MCP server ships 55 tools into someone else's agent and sees nothing back. I wanted the usage signal without reading anyone's job search. So the analytics keep the shape of what happened and throw away the words.
Kept: enough to run a product
Refused: anything a person typed
"keywords": { "length": 23 } ← the search's length, never its words
"$mcp_intent": "job_discovery" ← classified from prose, prose discarded
"distinctId": "mcp-anon-4fbe81a9…" ← random per process; the backend owns the real identity
System · PostHog, minus the parts that leak
It is built on PostHog's MCP SDK with the transport swapped out. The npm package holds no project key (a write key in a public package is an unrotatable credential in everyone's node_modules) and no user ID; each process mints a random anonymous ID. Events go to Trackly's backend, which treats its own shipped client as untrusted: it re-sanitizes everything from scratch, so an old or forked CLI cannot widen the privacy boundary. The backend attaches identity server-side, signs every event, and forwards it to PostHog. Signing matters because PostHog's collection endpoint is public: unsigned usage numbers are vibes, signed ones are evidence. And only 6 of the 55 tools may carry payloads at all, every one a public-data read. A new tool is private by default.
Story · the first production reading
First verified reading: 114 signed events from 7 authenticated users, cleanly split local vs hosted. My favorite part: the first automated recommendation refused to fire. It had 9 latency samples against its own 20-call minimum, so it stayed monitor-only. The measurement system holds itself to the same evidence bar the agents get held to (close-ai #1440, #1461, #1523, #1535 · cli #103, #110).
The agent orders from the menu. It never enters the kitchen.
The chat agent answers questions over 170K jobs with no database access of its own. Four separate walls keep that true, and none of them live in the prompt.
The menu
Reads go through "safe views": descriptions cut at 1,000 chars so hiring managers' emails and phones never reach the model. Raw data and embeddings are not on the menu. 14 views like this.
The bouncer
A small fast model screens every message for injection before the main model runs. The route enforces it, not the agent, so a compromised agent cannot skip it. 5 denials in an hour locks the account.
The cashier keeps the pen
The model cannot write. A write tool only files a pending action with a server-minted ID; the change happens when the real UI confirms it. An injection saying "set confirm to true" points at nothing.
The door check
Output is scanned before it streams: invented citations stripped, emails and phones redacted, secrets blocked. Behind it all, per-user daily caps and a fleet-wide spend brake.
System · nobody gets to type SQL at production, not even me
When we need a production diagnostic, there is no box to type SQL into. Anywhere. There are four pre-approved questions, ordered by number, like dishes on a fixed menu. Adding a fifth means a code review, not a text field.
Ordering one is deliberately heavy: you type a confirmation phrase, and the question runs in a throwaway machine that first double-checks it is connected to the right database as the read-only account. The answer format only has room for numbers and a few fixed labels, so actual data rows physically cannot come back through it. And every run leaves a record: who asked, which question, on exactly which version of the code (close-ai #1372).
Normalize duplicates. Preserve conflicts.
An order form arrives with the delivery address written twice. Both lines identical? A stutter. Ship it. Two different addresses? You do not pick one and hope. You stop, because that is exactly what a scam looks like.
System · the real version, with OAuth words
During login, an MCP client sends a "resource" value: the address its access key is allowed to work at. Pinning a key to one address is what stops a stolen key from being replayed somewhere else. That value is the delivery address of the story above, and the key is the package.
Some real clients, due to bugs, sent that value twice in one request. Reject them all, strictly by the spec, and those users cannot log in at all. Quietly take the first copy, the friendly fix, and you have also decided to "take the first copy" on the day the two copies differ, which is the attack. The shipped rule sits in between: if every copy is non-empty and byte-for-byte identical, treat it as one. If they differ, or any is empty or malformed, refuse the login (close-ai #1598, cli #113).
The line: fix the stutter, never the disagreement. Compatibility code may remove noise; it may never remove a warning sign.
Trust is not an approval modal.
It is knowing which claim is proven, by which record, at which moment. Trackly Apply fills job applications with a person's own data, so every phase earns its own proof. And Submit stays human. Always.
Story · proof-bearing execution
Execution and recovery live in a server-owned state machine (cli #93), so a crashed browser or a lost tab recovers from authoritative state instead of a guess. Each phase files a value-free receipt: which jobs were approved, which form was reached, what was filled, what the human reviewed, how Trackly's tracker was reconciled, and that leftover tabs were actually closed (cli #116). The hosted agent surface gets least privilege, with Submit deliberately absent (close-ai #1442).
Trust is earned per action class.
Never granted globally to "the agent." Every rung needs evidence, a bounded scope, and a way back down.
Higher consequence → fresher state, stronger approval, better recovery.
System · promotion and demotion
An action class climbs a rung when its numbers stay boring: task success, guardrail compliance, escalation accuracy, verification quality, and recovery performance, all stable for that scope. It climbs back down when drift, overrides, or near misses cross a threshold. Demotion is not a failure of the program. Demotion is the program.
Preview the fleet change, not just the command.
For a rebalance, the thing a human approves should be a decision-ready plan: fresh state, exact impact, rollback, and the receipt it will leave behind.
Current fleet state
One array is hot. Capacity and performance are uneven.
Move 3 volumes from B → C
- Fresh state: read 14s ago
- Blast radius: 3 of 184 volumes
- Headroom: B 91% → 67%
- Replication: stays compliant
- Window: 02:00–03:00
- Rollback: exact reverse plan
Why this fits Everpure's public direction
Everpure's public material frames Pure1 as the intelligence and Fusion as fleet-wide action. Fusion MCP starts observability-first, with preview-first plans and explicit approval described for later phases. SafeMode already trained their customers to expect multi-party authorization and recoverability. The grammar on this page lands on that roadmap almost word for word. This is a candidate sketch, not a claim about their internal plans.
System · what the approval object carries
Current and proposed placement. Affected arrays and volumes. Capacity and performance headroom after the move. Replication, RPO, and RTO impact. The change window. Reversibility and the recovery point. Who must approve. A retry-safe key. The checks that verify success. The immutable receipt.
Measure the decision, not the demo.
A polished answer means nothing if the system acts outside scope, escalates poorly, or cannot prove the result.
The nouns change.
The trust grammar doesn't.
I have not managed storage arrays. I have built the trust mechanics for automation that touches what people cannot afford to lose: their data, their money, their job search.
Four doors in production
UI, versioned API, CLI, local and hosted MCP. Shared meaning, deliberate differences.
From a real failure
A one-version-old client bypassed a fresh guard and deleted real data. The check moved into the transaction, recovery became part of the design, and the clients deleted their copies so nothing can drift again.
To fleet-scale actions
Fresh state, exact plans, scoped approval, verification, recovery, and autonomy earned per action class.
Short answers, with somewhere to point.
What did you actually build?
Schemas agents cannot misread, a destructive-action check inside the database transaction, per-tool OAuth scopes, telemetry that refuses content, and deletes that archive first.
Evidence →How do four surfaces stay coherent?
The backend owns what every action means. Each door presents it its own way, on purpose. No door gets to change the meaning.
Design →What should an agent never do?
Act on stale state, exceed its scope, assume approval, destroy the way back, or claim success without checking.
Safety story →How do you measure trust?
Task success, blocked out-of-scope attempts, escalation accuracy, time to human intervention, verification mismatches, recovery success.
Eval loop →How does this transfer to storage?
Same grammar, bigger blast radius. Approve exact plans against fresh fleet state, preview the impact, keep the undo, write the receipt.
Design sketch →What is your gap?
The storage nouns are new. The trust mechanics are not: scope, reversibility, approval, audit, failure behavior.
Honest close →Stories first. Numbers attached.
Three stories carry the conversation. The rest is depth for technical follow-ups.
The three hero stories
| Story | What happened | Where it lives |
|---|---|---|
| The filter that refused to lie | One three-state sponsorship meaning across UI, API, CLI, and MCP. Silence never becomes a "No," and a requested filter never silently does nothing. | cli #112, with #14 and #65 |
| The 24-minute bypass | A stale client skipped a fresh confirmation guard and deleted 33 real answers. The guard moved into the delete's own transaction, archive-first, and old clients now fail closed. | close-ai #1296 · cli #90 |
| Analytics that cannot read your search | PostHog events carry shapes, lengths, enums, and outcomes. Never words, never keys, never identity. Signed, so the numbers are evidence. | close-ai #1440 → #1535 · cli #103, #110 |
API, OAuth, and the hosted MCP
| Story | What shipped | Where |
|---|---|---|
| Least privilege as UX | Granular OAuth scopes per tool, grant binding, narrowed refresh, a closed tool registry. | close-ai #1189 |
| A hosted agent surface with limits | Hosted MCP and plugin runtime, exact-resource OAuth, resumable Apply, Submit deliberately absent. | close-ai #1442 |
| Operability without leakage | Fail-closed writes, bounded diagnostics, response-size and content limits. | close-ai #1478, #1488 |
| Compatibility that keeps the rule | Collapse identical duplicate OAuth values only; conflicts still fail. | close-ai #1598 · cli #113 |
| The deterministic door | OpenAPI 3.1, API-key auth, stable routes over shared domain services. | API v1 |
CLI, local MCP, and distribution
| Story | What shipped | Where |
|---|---|---|
| Schemas that tell the truth | Tool schemas aligned to real backend behavior; typed filters; visible rejections instead of silent drops. | cli #14, #65 |
| Trust includes distribution | OIDC trusted publishing with provenance; token, redirect, and config hardening. | cli #36, #45 |
| Consistency is not sameness | Kept nested agent chat hosted-only instead of mirroring it blindly into the CLI. | cli #49 |
| Docs parity is not runtime parity | Local and hosted schemas locked together with contract versions and deployed source lineage. | cli #86, #106 |
| Human-in-the-loop as a state machine | Server-owned execution and recovery with per-phase receipts; Submit stays human. | cli #93, #116 |
Determinism and approval protocols
| Story | What shipped | Where |
|---|---|---|
| Bounded agent access | Audited read-only query runner: 4 compiled query IDs, no SQL input exists. | close-ai #1372 |
| Policy outranks confidence | Deterministic outside evidence vetoes conflicting model writes. | close-ai #1412 |
| Approval as a transaction | Plan and review split, exact digest, signed approval, recovery lock, atomic receipt. | close-ai #1281 · web #446 |
| Uncertainty as a first-class value | One uncertainty-aware sponsorship predicate plus eval provenance gates. | close-ai #1578 |
Exact source snapshots
trackly-cli main: fbb9ededf9999c7abe65315f9226799dd9515bfc
close-ai backend main: 3150c87d361d6399c43d1bb92f79d126f55d8200
TracklyWeb main: 5c85c8cd3d48cfc42b31ddf975f79eb75f4b1bc2
TracklyMac main: 5738eb4da724a61f8d8573528acfb18ab1577e93
TracklyApp main: a1c018b12c4445f4a14dd32f11bf71e78e88228d
Anything I cannot prove fresh (marketplace status, user counts, provider health) stays off this page.