--- title: Audit an application's decision topology description: A read-only audit method for finding hidden AI agent authority, mapping decision points and atom contracts, proving trigger reachability, and ranking control gaps. eyebrow: DECISION TOPOLOGY AUDIT keywords: AI agent decision topology audit, hidden automation authority, agent control audit last_updated: 2026-09-30 --- # Audit an application's decision topology A decision topology audit maps where actions are proposed, authorized, and executed, and which inputs influence them. Use it to find ungoverned paths and prioritize where to add agent control before changing runtime behavior. Run discovery read-only. Do not move logic, register EMUs, or add runtime calls until the report is reviewable and its scope is approved. ## What the audit must answer For each consequential outcome: 1. Where can it be proposed? 2. Where is it authorized today? 3. Which state, model outputs, and historical events influence it? 4. Which code executes it? 5. Is there a human review path? 6. What happens when context is missing or a dependency fails? 7. Can one named Memrail decision point cover every path to that outcome? 8. Would every candidate EMU be reachable and every action connected? The result is not a count of `if` statements. It is a graph of authority and execution. ## Audit scope Include: - agent loops, planners, routers, and tool dispatchers; - API handlers, middleware, webhooks, queues, and scheduled jobs; - prompts and system instructions that change behavior; - classifier thresholds and model confidence checks; - feature flags, experiments, configuration policy, and database-driven rules; - human approval screens and escalation queues; - action executors and external side effects; - existing Memrail calls, ATOM builders, EMU JSONL, tools, and events. Exclude generated code, vendored dependencies, fixtures, and tests during initial discovery, then use tests to confirm intended behavior after the topology is mapped. ## 1. Establish project boundaries Identify services, deployable units, and existing Memrail projects. Use project coherence as an application contract: verify each EMU's tool schema, version, and actual executor together. A validator schema fallback is not proof of project-scoped runtime connectivity. ```bash rg --files -g 'package.json' -g 'pyproject.toml' -g 'requirements*.txt' \ -g 'Dockerfile*' -g 'docker-compose*.yml' -g 'Chart.yaml' -g 'kustomization.yaml' rg -n --glob '*.{py,ts,tsx,js,jsx,json,jsonl,yaml,yml}' \ 'AMI_PROJECT|project=|project:|projects=|emus\.jsonl|ToolRegistry' ``` Record ownership and deployment boundaries even when Memrail is not yet installed. ## 2. Find existing control integration ```bash rg -n --glob '*.{py,ts,tsx,js,jsx}' \ 'AsyncAMIClient|AMIClient|@memrail/sdk|from memrail|\.decide\(|decision_point|decisionPoint' rg -n --glob '*.{py,ts,tsx,js,jsx}' \ 'state\(|tag\(|atoms_from_dict|atomsFromDict|ingest_event|emit_event|emit_events|emitEvent|emitEvents' ``` For each `decide` call, capture its stable name, file and symbol, target project, atoms supplied, callers, action handlers, and failure behavior. Compare each call with its stored EMU bindings: `decision_point` must match exactly. Verify source, policy, and lifecycle metadata survive application adapters. ## 3. Discover consequential decisions Search broad patterns, then inspect each result in context: ```bash rg -n --glob '*.{py,ts,tsx,js,jsx}' \ 'if .*\.(tier|priority|status|role)|switch .*status|match .*status' rg -n --glob '*.{py,ts,tsx,js,jsx}' \ 'threshold|confidence|score.*[><=]|feature.?flag|experiment|variant|ab.?test' rg -n --glob '*.{py,ts,tsx,js,jsx}' \ 'execute_tool|tool_call|function_call|send|create|update|delete|refund|approve|deny|escalat|route' rg -n --glob '*.{py,ts,tsx,js,jsx,md,yaml,yml}' \ 'system.?prompt|instructions|must not|never |always |guardrail|moderation' ``` Mark a finding consequential when it can change money, access, state, user-visible communication, routing, legal/compliance posture, or an irreversible workflow transition. Do not automatically migrate every conditional. Input validation, local invariants, low-level error checks, and executor safety checks usually remain in code. ## 4. Trace authority from proposal to effect For each material action, work backward from the side effect: ```text payment_client.refund() ← refund tool executor ← agent tool dispatcher ← model tool proposal ← billing agent prompt + refund classifier ← API request + customer record ``` Label each edge: - **observation**: provides facts; - **inference**: produces a prediction or bounded tag; - **authority**: chooses whether or what should happen; - **execution**: performs the state change; - **recording**: captures the result. An inference-to-execution edge with no explicit authority check is the highest-value candidate for agent control. ## 5. Map data flows into ATOM candidates For every proposed decision point, inventory the context that is available before execution: ```yaml decision_point: billing.refund location: src/agents/billing.py:RefundAgent.execute callers: - POST /v1/refunds - retry_refund_job state_candidates: customer.id: source: authenticated request type: string presence: always refund.amount: source: validated request type: number presence: always tag_candidates: refund_intent: source: bounded classifier values: [duplicate_charge, service_failure, fraud, other, unknown] presence: always event_candidates: customer.received.refund: producer: payment success callback attributes: [customer_id, refund_id, request_id] ``` Capture conditional omissions and sentinel behavior. “Builder exists” is insufficient if it is never called on the path where the EMU must match. ## 6. Inventory action connectivity ```yaml actions: tool_call.refund_payment: schema_registered: true executor: src/tools/payments.py:refund_payment version: 1.0.0 projects: [billing] retry_policy: bounded idempotency: refund_request_id outcome_event: payment.completed.refund decision_prompt.refund_review: handler: operations refund queue timeout: 24h timeout_outcome: decline route.fraud_review: handler: missing ``` Inspect real handlers, not only registrations. A stub, unconfigured credential, nonexistent destination, or swallowed error is a partial connection. Trace authorization separately from selection. Test every dispatcher and adapter against the [integration invariants](/concepts/#integration-invariants), including consent, evaluation, lifecycle, and metadata preservation. ## 7. Build the reachability matrix For every current or proposed EMU, cross-reference dependencies and actions: | EMU | Point | Required context | Trigger reachable | Action connected | Project coherent | |---|---|---|---:|---:|---:| | `billing.small_refund` | `billing.refund` | amount, request ID, reason tag | yes | yes | yes | | `billing.repeat_refund` | `billing.refund` | customer ID, refund event | partial | yes | yes | | `billing.fraud_route` | `billing.refund` | fraud tag | yes | no route handler | yes | Flag four distinct failures: - **invoke gap:** a consequential path never reaches the decision point; - **atom gap:** required context is absent, stale, conditionally omitted, or wrongly typed; - **action gap:** the selected action has no complete handler; - **coherence gap:** the EMU and tool belong to different projects. Test required-atom presence on every caller. Positive predicates on absent atoms return false, but negation can make them true. The atom schema registry (ASR)'s observed schemas may span team workspaces; a clean validation report is evidence, not proof of per-point coverage. ## 8. Inspect event correctness Events deserve a separate pass because absence can invert policy behavior. For every `event.*` clause: - confirm a producer runs after the material outcome; - confirm the emitted name exactly follows the trigger’s `subject.verb.object` order; - confirm timestamps are timezone-aware; - confirm attributes include entity IDs referenced by `WHERE`; - confirm query windows fit the organization's configured retention (90-day default), not an assumed universal cap; - guard required IDs with `EXISTS` before negated event filters; unresolved placeholders can leave a negative predicate true; - confirm a negative event clause is not a ghost guard that is always true. ## 9. Rank findings by authority and consequence Use a simple qualitative priority: | Priority | Condition | Example | |---|---|---| | Critical | model or heuristic can directly cause irreversible/high-stakes effect | payment, deletion, account access | | High | control can be bypassed or required event/action connection is missing | alternate tool dispatcher path | | Medium | policy is reachable but difficult to audit or safely evolve | thresholds duplicated in services | | Low | naming, documentation, or non-consequential topology debt | unnamed read-only routing hint | Do not collapse the audit into one percentage. Report counts by gap type and preserve the evidence path for each finding. ## Deliverable template Create `TOPOLOGY_STATE.yaml` or an equivalent review artifact: ```yaml metadata: generated_at: 2026-09-06 scope: repository root mode: read-only summary: projects: 2 decision_points: 6 consequential_paths: 11 invoke_gaps: 2 atom_gaps: 4 action_gaps: 1 coherence_gaps: 0 hidden_authority_sites: 3 decision_points: [] hidden_authority: [] atom_contracts: [] event_producers: [] action_connectivity: [] reachability_matrix: [] prioritized_findings: - id: AUTH-001 severity: critical evidence: file: src/agents/billing.py symbol: RefundAgent.execute behavior: model-selected refund tool executes directly recommendation: insert billing.refund control point before dispatcher migration_phase: 1 unknowns: [] out_of_scope: [] ``` Use exact file and symbol evidence. Line numbers are helpful during one revision but should not be the only locator because they drift. ### Visualize the topology for inspection After creating or updating `TOPOLOGY_STATE.yaml`, try to generate `TOPOLOGY_STATE.html` beside it and open the page for visual inspection. The YAML remains the canonical review artifact; the HTML is a derived view and must not add facts, relationships, or conclusions that are absent from the YAML. Make the visualization useful for inspecting the system, not just reading serialized YAML. Show project and deployment boundaries, proposal → authority → execution → outcome paths, decision points, atom/event/action connections, hidden-authority sites, bypasses, gaps, prioritized findings, evidence locations, and unknowns when those sections exist. Label inferred relationships and render missing information as unknown rather than inventing edges. Prefer one self-contained HTML file with embedded CSS and, only when useful, embedded JavaScript. It should work locally without a framework, CDN, network request, or newly installed dependency. Escape repository-derived text before inserting it into HTML. If a browser or HTML preview is available, inspect the result for clipping, unreadable labels, disconnected edges, and misleading grouping. If the environment cannot render HTML, still produce the best-effort file when possible and report that it was not visually verified. Do not widen a read-only audit merely to create the derived view. ## Agent-ready audit prompt ```text Use the memrail skill to perform a read-only decision topology audit of this repository. Do not edit code, register EMUs, change external state, or expose secrets. Map: 1. project and deployment boundaries; 2. existing Memrail SDK usage, decision points, atoms, EMUs, tools, and events; 3. consequential decisions in code, prompts, flags, model thresholds, and human workflows; 4. every proposal → authority → execution → outcome path; 5. candidate atom contracts and event producers; 6. trigger reachability, action connectivity, and project coherence gaps; 7. unsafe fallbacks and paths that bypass explicit control. Produce TOPOLOGY_STATE.yaml using exact file and symbol evidence. Rank findings by consequence and authority. Separate verified facts from inferences and unknowns. Recommend phased seams for migration, but do not implement them. After the YAML is complete, try to generate a self-contained TOPOLOGY_STATE.html visualization beside it and open it for visual inspection. Treat the YAML as canonical. Show boundaries, decision paths, connections, gaps, findings, evidence, and unknowns without inventing missing relationships. If HTML rendering is unavailable, report that the derived file was not visually verified. ``` Install the skill first from the [agent installation page](/agents/) if it is unavailable. ## Audit completion criteria - Every in-scope material executor has a traced proposal and authority path. - Every consequential path is mapped to a current or candidate decision point. - Every atom candidate has a source, type, presence rule, and value domain. - Every event clause has an exact producer and required attributes. - Every action has a real handler and project assignment. - Bypass paths, unsafe fallbacks, and unknowns are explicit. - Findings cite repository evidence and distinguish fact from inference. - `TOPOLOGY_STATE.html` was generated from the canonical YAML when possible, opened for visual inspection when the environment supports it, and its verification status was reported. - No runtime, policy, credential, or external-system changes occurred during discovery.