Open Studio
system breakdown

Inside OpenWiki: Resumable Page-Job Lifecycle, Grounded Claims Reconciliation & Confined Backend Architecture

A production teardown of LangChain's OpenWiki documentation engine—deconstructing the resumable .run.json page-job state machine, repo-lines-v1 evidence relocation, SHA-256 sidecar durability proofs, and defense-in-depth shell confinement.

FullStack AI Hub Editorial24 min readPublished 29 Sep 20262 primary sources
Inside OpenWiki: Resumable Page-Job Lifecycle, Grounded Claims Reconciliation & Confined Backend Architecture

Evidence boundary

This system breakdown analyzes the OpenWiki lifecycle engine (src/generation), grounded claims subsystem (src/claims and src/okf), confined agent runner (src/agent), and MCP host integration (src/integrations/mcp) at commit 27d835cc617019795065ea0b30d9eb23e62b9789 (release v0.4.0).


1. Resumable .run.json Lifecycle & Source-Drift Invalidation

Documentation agents that generate dozens of wiki pages in a single monolithic conversation inevitably suffer from context exhaustion, partial writes on crash, and silent staleness when code changes mid-run. OpenWiki replaces monolithic prompting with a deterministic, crash-resumable state machine persisted in openwiki/.run.json (src/generation/repository-run.ts):

code(29 lines)
1+-----------------------------------------------------------------------------+
2| Phase 0: beginRepositoryRun() |
3| 1. prepareClaimsRuntime() -> runClaimsPreflight() (precedes no-op check!) |
4| 2. Compute SHA-256 sourceFingerprint across non-ignored repository tree |
5| 3. Write durable openwiki/.run.json (phase = "planning") |
6+-----------------------------------------------------------------------------+
7 |
8 v
9+-----------------------------------------------------------------------------+
10| Phase 1: submitRepositoryPlan() |
11| Normalize proposed pages + inject mandatory Claim-Issue repair jobs |
12| Sort /openwiki/quickstart.md LAST -> persist .run.json (phase="generating") |
13+-----------------------------------------------------------------------------+
14 |
15 v
16+-----------------------------------------------------------------------------+
17| Phase 2: Sequential PageJob Loop (nextRepositoryPage -> submitRepositoryPage)|
18| For each pending PageJob: |
19| - Write /openwiki/<page>.md & validate OKF YAML frontmatter |
20| - Resolve repo:// claims -> persist .claims/<page>.json -> prove SHA-256 |
21+-----------------------------------------------------------------------------+
22 |
23 v
24+-----------------------------------------------------------------------------+
25| Phase 3: finishRepositoryRun() |
26| Require stable sourceFingerprint -> Mermaid/Index/Link/OKF finalizers |
27| Prove repository-wide Claim durability -> unlink openwiki/.run.json LAST |
28+-----------------------------------------------------------------------------+
9 lines hidden
Missing evidence claim: openwiki-durable-page-job-lifecycle

Two ordering invariants govern repository-run.ts:

  1. Claims Preflight Precedes Update No-Op Detection:
Missing evidence claim: openwiki-claims-preflight-precedence

During openwiki update, prepareClaimsRuntime() audits every existing .claims/*.json sidecar before getUpdateNoopStatus() checks git rev-parse HEAD. Even if no Git commits occurred since .last-update.json, any unresolved or stale repo:// claim forces an active run and automatically injects repair PageJob entries via addRequiredClaimIssueJobs(). 2. Whole-Plan Invalidation on Source Drift: Both resumeRepositoryRun() and finishRepositoryRun() recompute createRepositorySourceFingerprint(). If any non-ignored repository file changed while the agent was authoring pages, requireStableSourceFingerprint() deletes state.plan, reverts phase to "planning", persists the updated .run.json, and throws a conflict RepositoryRunError. And .run.json is unlinked strictly as the final statement of finishRepositoryRun()—any earlier failure leaves the checkpoint intact for deterministic resumption.


2. Per-Page Worker Isolation & Non-Delegating Middleware

Missing evidence claim: openwiki-per-page-worker-isolation

Whether driven natively via runNativeRepositoryGeneration() (src/agent/repository-runner.ts) or externally over MCP (src/integrations/mcp/server.ts), OpenWiki never reuses an agent thread across pages:

  • Read-Only Planner Agent: Receives PLANNER_FILESYSTEM_TOOLS (read_file, ls, glob, grep) and submit_plan. It cannot mutate any files.
  • Single-Page Scoped Writer Agents: For each pending PageJob, runNativeRepositoryGeneration() constructs a fresh DeepAgents instance whose OpenWikiLocalShellBackend restricts writableWikiPages to [job.path].
  • Stripping Subagent Delegation: Because deepagents mounts a general-purpose task subagent tool by default, repository-runner.ts injects NO_DELEGATION_MIDDLEWARE to strip task from every model call:
typescript(9 lines)
1const NO_DELEGATION_MIDDLEWARE = createMiddleware({
2 name: "OpenWikiRepositoryWorkerNoDelegation",
3 wrapModelCall: (request, handler) =>
4 handler({
5 ...request,
6 tools: request.tools?.filter(({ name }) => name !== "task"),
7 }),
8});

If a worker submits invalid YAML frontmatter or unresolvable repo:// evidence to submit_page, createSubmissionRejection() converts the RepositoryRunError into an error-status ToolMessage with concrete retry instructions, keeping the single-page worker loop alive until assertPageClaimsDurable() passes.


3. Grounded Claims Engine, repo-lines-v1 Relocation & OKF Projection

Missing evidence claim: openwiki-line-range-evidence-relocation

Every factual page under /openwiki/ is paired with a machine-owned sidecar under openwiki/.claims/<page>.json. When a worker submits claims (ProposedPageClaim[]), RepositoryEvidenceResolver (src/claims/evidence/repository/resolver.ts) resolves repo://path#Lstart-Lend citations without trusting the LLM:

  1. 7-Field Context-Hash Relocation (repo-lines-v1:sha256:): Line numbers shift whenever upstream files are edited above a cited function. When establishing a line-range citation, RepositoryEvidenceResolver computes the SHA-256 digest of the selected lines plus 7 relocation metadata fields (LineRangeVersionMetadata):
    • selectedLineCount, firstSelectedLineHash, and lastSelectedLineHash
    • precedingContextLineCount and precedingContextHash (up to 3 lines before the span)
    • followingContextLineCount and followingContextHash (up to 3 lines after the span) On subsequent runs, if the original line numbers no longer match contentHash, the resolver scans the file using the first/last line hashes and 3-line boundary context hashes to deterministically relocate the span or classify the claim as stale / unresolved.
  2. Dual-Digest Synchronization & Transactional Rollback: During finalize(), synchronizeClaimsVerification() (src/okf/claims-verification.ts) projects { by: "openwiki/0.4.0", at: ... } into each Markdown file's YAML frontmatter verified list while retaining any human or third-party verifier stamps. Because modifying frontmatter changes the Markdown file's bytes, finalizeVerificationProjection() immediately calls session.refreshPageVersions() to update pageVersion (sha256:<64-hex>) in openwiki/.claims/<page>.json. If any sidecar hash refresh fails, rollbackClaimsVerification() restores the exact pre-projection Markdown bytes.

4. Defense-in-Depth Confinement: OpenWikiLocalShellBackend

Missing evidence claim: openwiki-docs-only-shell-confinement

Because OpenWiki documents arbitrary repositories, all repository source files and commit messages are treated as untrusted evidence that may contain indirect prompt injection payloads. OpenWikiLocalShellBackend (src/agent/docs-only-backend.ts) wraps the DeepAgents LocalShellBackend with three non-bypassable security gates:

  1. .openwikiignore Read & Discovery Gate: read, write, edit, uploadFiles, and downloadFiles hard-deny any path matching .openwikiignore. Discovery tools (ls, glob, grep) silently filter out ignored paths and reject unbounded root globs (**/*) or .git metadata scans.
  2. docsOnly & writableWikiPages Write Confinement: In repository mode, writes and edits outside /openwiki/ (or outside the active job's single writableWikiPages entry) are rejected before touching the OS filesystem.
  3. Hidden .claims Sidecars & Anchored Shell Allowlist: The openwiki/.claims directory is completely hidden from agent filesystem tools and blocked via referencesClaimsState() regex checks. Furthermore, whenever .openwikiignore rules are active, arbitrary shell execution is disabled and restricted to a strict 2-command anchored allowlist (/^pwd$/u and /^git\s+(?:--no-pager\s+)?rev-parse\s+HEAD$/u), preventing shell-based exfiltration of ignored secrets.

Continue Reading

All Blog Posts