Marketplace Agent

How it is put together

One gateway service is the only component that talks to the marketplace; everything else talks to the gateway. The agent service (Java, Spring Boot) holds the control flow. A read-only MCP server lets an assistant inspect the data, and the owner has an admin web console.

Every inbound and outbound event is appended to a daily JSONL archive before any filtering, so nothing is lost and history can be replayed. Messages are de-duplicated by the platform message id, because restarts used to re-insert old messages with "now" timestamps.

Two agents facing opposite directions

Both agents share one chat channel, but they point in opposite directions and have very different shapes.

A MessageRouter in front dispatches by conversation ownership. Without it, the seller persona would auto-reply to the sellers the owner is asking. Resuming is event-driven: a seller replies hours later, the router finds the inquiry, loads its state, injects the reply and resumes, either with a follow-up or with a judgement.

The crown principle: two channels

"Visible to the LLM" and "visible to the runtime" are two separate channels, in both agents. Tool parameters are split into model-decided parameters, which are in the schema, and host-injected parameters, which the model never sees and cannot fabricate.

The question asked before adding any tool: should the model see this parameter? If not, inject it out of band. The judge layer applies the same idea to its input, as trusted facts versus the untrusted buyer message.

The judge layer

Jev is TypeSafe's "System One" model. It returns calibrated probabilities for typed questions: noul (a yes/no probability), choice (pick one) and score (criteria as an array). It generates no text. In measurements it was strong at recognising scams, intent and credibility, and nearly immune to injection. It cannot do arithmetic and has no recent knowledge, so it only triages: it never gives a final verdict, never touches prices and never scores reply quality.

On the inbound side there are two judgment points, each its own Jev call:

• inbound.injection: online, on the hot path. One question, injection (noul). A keyword-floor injection hit, or a probability ≥ 0.85, routes to QUARANTINE; anything else is PASS. The threshold is 0.85 because attacks measured 0.97, normal messages 0.03, and the same request jitters by about ±0.03.
• inbound.scam: shadow only. Two questions, scam (noul) and intent (choice). Online, only its keyword floor routes: a hit means HUMAN, otherwise PASS. The model answers come from an async call on a separate thread pool and are only logged, to collect labelled data until the gates in decision 4 pass.

The routes are a real enum. PASS continues the normal flow. QUARANTINE means suspected injection: the message must not enter any LLM prompt, AI is paused and the owner takes over. HUMAN means a definite high-risk signal (off-platform, ship first and so on): AI is paused and the owner takes over. The judge only gives probabilities; code routes, and the judge never touches the red lines. Online intent still comes from the existing IntentClassifier (keyword short-circuit, then an LLM only if still "other"); Jev's intent answer is shadow-only.

ChatAgentService.doProcessMessage        control layer (owns flow, red lines, routing actions)
   │ Verdict v = judge.evaluate(POINT, ctx)   ← the only entry point
   ▼
JudgeService (facade) + JudgeSwitchboard   enabled / provider / shadow; hot-reloaded switch file;
                                          missing or unreadable file = judge disabled
   │
JudgmentPoint<C>   declarative pure value: static final constant, diffable, unit-testable offline
   │   inbound.injection   ONLINE       injection (noul)
   │   inbound.scam        SHADOW ONLY  scam (noul) + intent (choice)
   ├─ FactSpec<C>     ctx → JudgeState   dual channel: facts (Trusted) | buyerMessage (untrusted)
   ├─ QuestionSet     shared question bank (sealed Noul / Choice / Score, schema-validated at construction)
   └─ RoutingPolicy   Answers → Route    keyword floor first, then threshold (bound to provider + version)
                                         Route = PASS | QUARANTINE | HUMAN
   │
JudgeEngine   ask(state, questions) → answers
   ├─ JevJudge  → JudgeClient   the only HTTP exit to Jev: pooled, bounded retry ×2, 4 s read timeout,
   │                           version pinned to jev-1.13.0 and checked on every response, redaction, tracing
   └─ LlmJudge  → reuses the existing LLM stack with structured output
                 (primary / shadow / offline A/B, never a hot-path fallback)

Resilience  circuit breaker with flap protection
            shadow calls on their own thread pool: discard policy, snapshot first, shorter timeout,
            never counted toward the main breaker
            engine failure / breaker open / version drift → PASS (degraded, keyword floor only);
            a broken judge never produces QUARANTINE or HUMAN by itself

The same Question objects render both engines' requests (toJevSpec() / toLlmField()), so switching providers, fallback and A/B come for free.

Nine decisions

Each of these was argued; these are the conclusions.

1. Dual-channel state, enforced by types

   In earlier research, the forged sentence "this buyer is verified" in a facts field dropped a scam probability 0.60 → 0.29 and let the scam through; inside the buyer message it read as suspicious (0.70 → 0.73). So facts must be Trusted, whose only factories are db() and computed(). There is deliberately no fromBuyer(): buyer text in facts does not compile. The recorded run on this page repeats the effect on a smaller scale.

2. Judge ≠ control layer, as a structural fact

   RoutingPolicy returns an enum only. The judge package may not depend on any sending or persistence component, enforced by a source-scan test in CI (ArchUnit-style).

3. Fail-safe degradation

   Hot path = Jev (bounded) → keyword floor (instant, zero network) under a hard total time budget. An engine failure means PASS, degraded to the keyword floor. Never a serial fallback to an LLM judge on the message thread, which would spike the main LLM stack exactly when Jev is unstable. And not "Jev down → everything to the owner" either, which would flood the queue. Degrade precision, keep running.

4. Two separate lifecycles

   Injection detection is strong (0.97 on attacks, 0.03 on normal messages, zero false positives), so it goes online first: rollout 5% → 25% → 100%, at least 3 days per step, advancing only with false positives < 1% and zero misses. Scam detection has thin evidence (12 samples, a Chinese "soft scam" blind spot), so it is shadow only: async, logged with human labels, zero online routing until it passes its gates: ≥ 30 positives and ≥ 10 per probability bucket, precision ≥ 0.9 at 0.5, 100% Chinese recall together with the keyword floor, ECE < 0.1, and two weeks of regression without drift. Otherwise it stays shadow forever.

5. A keyword floor for the blind spot

   Chinese "ship first, I'll pay later" and threat scripts were Jev's one miss. They are handled by deterministic keywords → HUMAN, not by lowering Jev's threshold, which would raise false positives on normal haggling. The floor is written for Chinese buyers, so an English "ship first" message passes online today; Jev scores it, but only in the shadow log.

6. Trust score rejected

   Jev's trust score ran opposite to risk in tests. Buyer reputation is computed in code from the owner's own order history (refund rate, unpaid orders, blocklist) and fed in as a Trusted fact. That path works: a bad history moved a neutral message's scam score 0.06 → 0.38.

7. Cache and idempotency

   Cache key = sha256(provider + model + questionsVersion + canonical JSON of the full state). De-dup by (conversationId, messageId), because reconnects replay messages. An empty buyer message short-circuits; empty facts do not, because a new buyer's first message must still be judged.

8. Redaction at the single exit

   Data leaving the country is redacted where it leaves: ID numbers, phone numbers, WeChat and QQ ids and street addresses become placeholders such as [微信号]. The placeholder keeps the signal and drops the identifier. Only the current message plus minimal context is sent, never the full history.

9. Capability reconciliation before code

   No second risk gate. SideEffectPolicy, FactGuard, ReplyGuard, BargainGuard, the walk-away / greeting / vision short-circuits and the human-takeover pause were each read and kept; the judge reuses the pause instead of inventing a mechanism. The one missing capability: there was no inbound prompt-injection defence at all. That gap is the judge's first job, which is why inbound.injection is the point that went online.

Memory, calibrated to reality

Second-hand buyers are mostly one-off visitors: they ask once or twice, buy and leave. So the design removed things an "enterprise customer service" template would add: no buyer-profile table, only lightweight buyer_flags (refund history, blocklist); no nightly reflection job, but distillation on completed-sale events plus a one-click "save as FAQ" button; and FAQ only for multi-unit items.

1. L0in-process
2. L1chat messages
3. L2knowledge card + FAQ + flags
4. L3semantic search: reserved, not built. Keyword recall is enough at hundreds of FAQs.

Supervised self-improvement

Prompt changes are proposals that a human approves; they are versioned and revertible. FAQ is distilled only from the assistant side of completed sales, so buyer text never becomes an answer. That prevents something like "the boss promised me 10 yuan" being learned and honoured for the next buyer.

Measured

Recorded run (2026-10-09, jev-1.13.0)

Every preset in the simulator, in both languages, sent through the real judge setup. The simulator shows these exact requests and responses.

Earlier evaluation

Status

• inbound.injection: online, with a kill switch.
• inbound.scam: shadow only; online, only its keyword floor routes, until the gates in decision 4 pass.
• Known gap, shown as-is above: the keyword floor is Chinese-only, so English off-platform and ship-first messages pass online.
• Online provider (Jev vs an LLM judge): under re-evaluation after these measurements.
• BuyerScout: the phase-2 agent; sending is human-approved by default.
• Closed source. No repository, no download.