--- name: neqsim-autonomous-investigation version: "1.1.0" description: "Autonomous observe-hypothesize-predict-test-discriminate loop for operational anomalies. USE WHEN: solving a PEPR action, root-cause or operational study where the symptom, driver or important relationships are NOT given up front. Uses neqsim.process.diagnostics.RelationshipGraph for lead-lag relationship discovery across historian tags, then hands off to neqsim-root-cause-analysis." last_verified: "2026-07-19" requires: java_packages: [neqsim.process.diagnostics, neqsim.process.automation] --- # NeqSim Autonomous Investigation Skill Make agents *investigate* instead of *follow a checklist*. Use this skill when a task (PEPR action, root-cause, operational study, digital-twin deviation) does **not** tell you the symptom, the driver, or which relationships matter. The goal is that the agent discovers the important relationships from the data and the flowsheet on its own, forms competing hypotheses, and tests them — reaching findings that were not spelled out in the task. ## When to Use - A PEPR action or work order describes a *problem* but not a *cause*. - Historian data is available but no one has said "look at tag X vs tag Y". - A model-vs-plant deviation appears and the responsible variable is unknown. - Any "why is this happening?" question where spoon-feeding relations is wrong. Do **not** use this to replace a known, well-scoped calculation — if the symptom and mechanism are already given, go straight to `neqsim-root-cause-analysis` or the relevant discipline skill. ## The Investigation Loop (mandatory ordering) Run this loop *before* fixing a scope. Never assume the task's stated classification is correct — treat it as a hypothesis to challenge. 1. **Observe (no assumptions).** Pull *all* available tags, not just the ones the task names. Scan each tag against its own baseline and (when available) STID design envelope for what is abnormal. Discover cross-tag relationships with `RelationshipGraph` — including *lead-lag direction*, which distinguishes a driver from a follower. 2. **Hypothesize (compete).** Generate **at least three** competing causal hypotheses, always including a "not a real problem / instrument or data artifact" hypothesis. Seed them from the discovered leaders (candidate causes), not from intuition alone. 3. **Predict (differentiate).** For each hypothesis, state what it implies for *other* tags/streams. Two hypotheses that predict the same thing cannot be distinguished — find a prediction where they disagree (the discriminating test). 4. **Test.** Use NeqSim (`runProcess`/`runFlowAssurance`/simulation verification via `RootCauseAnalyzer`) plus historian evidence (`EvidenceCollector`) to check each prediction. 5. **Discriminate & iterate.** Keep the hypothesis that best explains the *pattern across relationships*, not a single number. Loop until one dominates or the data is exhausted; report residual ambiguity honestly. **Report the relationships you discovered, not just the answer.** A finding without its supporting lead-lag relationships and discriminating test is not complete. ## Unsupervised relationship discovery — `RelationshipGraph` `RelationshipGraph` (in `neqsim.process.diagnostics`) scans every tag pair in a historian data set with **no symptom and no hypothesis supplied**, and reports which tags move together and, crucially, **which moves first**. Lead-lag directionality is the signal an agent uses to pick candidate causes on its own. ### Java ```java import java.util.Map; import neqsim.process.diagnostics.RelationshipGraph; RelationshipGraph graph = new RelationshipGraph(); graph.setTimestamps(timestamps); // optional: enables lag in seconds graph.setMaxLagSamples(10); // search +/- 10 samples graph.setMinAbsCorrelation(0.5); // only report |r| >= 0.5 List edges = graph.analyze(historianData); String relationshipReport = graph.toTextReport(edges); for (RelationshipGraph.Relationship r : edges) { // r.getSource() leads r.getTarget() (candidate cause -> candidate effect) // r.getDirection(): LEADS or SYNCHRONOUS // r.getLagSamples() / r.getLagSeconds(): how far ahead the driver moves // r.getCorrelation(): strength & sign at the best lag } ``` ### Python (task notebook / runner) ```python RelationshipGraph = ns.JClass("neqsim.process.diagnostics.RelationshipGraph") graph = RelationshipGraph() graph.setTimestamps(timestamps) # java double[]; omit if unavailable graph.setMaxLagSamples(10) graph.setMinAbsCorrelation(0.5) edges = graph.analyze(historian_map) # Map for r in edges: print(r.getSource(), "->", r.getTarget(), "r=", round(r.getCorrelation(), 2), "lag_s=", r.getLagSeconds()) ``` ### Reading the output - `A -> B (r=+0.85, leads by 300 s)` — A moves first; A is a **candidate cause** of B. Prioritise hypotheses about A. - `A <-> B (r=+0.90, synchronous)` — tightly coupled with no detectable lag; may share a common driver — look for a third tag that leads both. - A strong statistical edge that does **not** follow a physical process path (upstream -> downstream in the flowsheet) is a candidate common-cause or instrument artifact, not a direct cause. ## External (non-historian) candidate signals: weather `RelationshipGraph` only sees what is in the historian data set you hand it. For outdoor/topside equipment, ambient temperature, wind, or sea state can be a driver that is not tagged anywhere in the plant historian. Before concluding "no cause found" for an anomaly on weather-exposed equipment, pull the historical weather for the site and event window with the community `neqsim-weather-data` skill (`WeatherDataService.get_historical`) and add it as an extra column to the data handed to `RelationshipGraph.analyze(...)`, exactly like any other tag. A lead-lag edge from an ambient-temperature or wind-speed series into the anomalous tag is then a discovered candidate cause, to be hypothesised and tested like any other `RelationshipGraph` finding — not asserted from correlation alone. ## Auto-detect the symptom — `AnomalyScanner` You should not have to be told the symptom either. `AnomalyScanner` (in `neqsim.process.diagnostics`) scans every tag against its own robust baseline (median / MAD) and, when supplied, its STID design envelope, and reports abnormal tags plus a **candidate symptom** inferred from the tag name. Detection kinds: `THRESHOLD_HIGH/LOW` (crosses a design limit), `SPIKE_HIGH/LOW` (robust-z outlier), `TREND_UP/DOWN` (sustained drift). ```java AnomalyScanner scanner = new AnomalyScanner(); scanner.setDesignLimit("Compressor-1.vibration", Double.NaN, 7.1); // optional List anomalies = scanner.scan(historianData); Symptom candidate = scanner.suggestSymptom(anomalies); // e.g. HIGH_VIBRATION ``` ## Promote statistics to causes — `CausalTopologyModel` A statistical lead-lag edge is not proof of causation. `CausalTopologyModel` overlays the flowsheet connectivity (which equipment feeds which) on the `RelationshipGraph` edges and classifies each: `CAUSAL_CANDIDATE` (leader is upstream of follower and moves first), `LOCAL` (same equipment), `COUNTER_FLOW` (lead-lag opposes process flow — feedback), or `COMMON_CAUSE_OR_ARTIFACT` (no process path — a shared hidden driver or an instrument artifact). ```java Map> adjacency = CausalTopologyModel.buildDownstreamAdjacency(processSystem); CausalTopologyModel model = new CausalTopologyModel(adjacency, tagToEquipment); List edges = model.classify(relationships); ``` ## One call — `RootCauseAnalyzer.analyzeAutonomous()` The three steps above plus the Bayesian scoring are chained in a single entry point. **No symptom is required** — the analyzer scans anomalies, infers the symptom, discovers relationships, and classifies them against topology, then converts hypothesis-matched anomaly and physically consistent topology findings into weighted evidence used in ranking. ```java RootCauseAnalyzer rca = new RootCauseAnalyzer(processSystem, "Compressor-1"); rca.setHistorianData(historianData, timestamps); rca.setDesignLimit("Compressor-1.vibration", Double.NaN, 7.1); // Autonomous: no setSymptom() call needed. RootCauseReport report = rca.analyzeAutonomous(); // anomalies + relationships + RCA // or, to also get causal-vs-artifact classification: RootCauseReport report2 = rca.analyzeAutonomous(tagToEquipment); // + CausalTopologyModel rca.getLastAnomalies(); // what looked abnormal rca.getLastRelationships(); // who leads whom rca.getLastCausalEdges(); // causal candidate vs common-cause/artifact ``` Only `LOCAL` and `CAUSAL_CANDIDATE` edges matching a hypothesis fingerprint affect ranking. `COUNTER_FLOW`, `COMMON_CAUSE_OR_ARTIFACT`, and `UNKNOWN` findings remain reportable but are not treated as causal support. This conservative admission rule prevents correlation alone from inflating confidence. Python: get the classes with `ns.JClass("neqsim.process.diagnostics.AnomalyScanner")`, `...RelationshipGraph`, `...CausalTopologyModel`, `...RootCauseAnalyzer`. ## Cooperation (chain both directions) ``` plant-data / enterprise-plant-data (historian tags) alarm-events / maintenance-api / STID (events, work orders, design limits) v neqsim-autonomous-investigation (this skill: discover relations + hypotheses) v neqsim-root-cause-analysis (Bayesian scoring + simulation verification) v neqsim-process-safety / discipline (consequence, if the cause is a hazard) ``` - **Gather ALL data first (do not spoon-feed one tag):** pull the full historian tag map via `neqsim-plant-data` (community) or `enterprise-plant-data` (historian/Seeq); add alarm & event history (`enterprise-alarm-events`), maintenance work orders / notifications (`enterprise-maintenance-api`), and STID design limits when available. Feed the whole tag map (plus design limits) into `AnomalyScanner` / `RelationshipGraph` — the more context, the better the discovery. - **Downstream:** hand the discovered leaders and their lags to `neqsim-root-cause-analysis` as the candidate causes / expected signals, so the Bayesian scorer verifies them with a NeqSim simulation instead of relying on a fixed symptom. - The one-call path `RootCauseAnalyzer.analyzeAutonomous(tagToEquipment)` runs the whole chain (anomaly scan -> symptom inference -> relationship discovery -> topology classification -> Bayesian scoring) so the agent only supplies data + flowsheet. ## Limitations - Correlation and lead-lag are **screening** signals, not proof of causation. Always confirm with the flowsheet topology and a NeqSim simulation. - Default correlation is linear (Pearson); enable rank/Spearman mode (`RelationshipGraph.setUseRankCorrelation(true)`) to catch strong monotonic non-linear couplings. Strongly non-monotonic couplings may still be under-reported — consider transforming variables (log, rate-of-change) first. - Lag resolution is limited by the sampling interval; sub-sample lags round to the nearest sample. - Common-cause structure (one hidden driver behind many tags) is flagged only indirectly — look for a tag that leads several others, or a `COMMON_CAUSE_OR_ARTIFACT` verdict from `CausalTopologyModel`.