AI Evidence Trail: Audit Graphne's Model Calls, Tool Calls and Decisions

Overview

Every Graphne answer and AI task leaves a record. The record shows which model answered, the exact instructions (system prompt) it received, each tool call it made, including each graph query, what each call returned, and what the model saw. Use it to check an answer, to explain a result to an auditor, or to find out why a run failed.

The Evidence tab on the AI Provider page lists these records. Graphne's Reasoning panel and the investigation report show the same trail in context. When someone acts on a recommendation, GraphnAI also records whether they took it as given, changed it or closed it.

Records are kept for 90 days. The decisions are kept for 7 years in the AI governance audit.

GraphnAI seals each record when it writes it, so a changed or deleted record shows up in an integrity check. Events an Admin should see, such as a failed check or a kill switch change, are written as incidents and can page your escalation channel.

---

Prerequisites

• Role: Admin, for the Evidence tab and Open evidence.
• AI configured: At least one AI provider. See Multi-Provider AI Configuration.
• Data: Runs made after this feature was installed. Earlier answers have no evidence.
• Paging (optional): An escalation channel, webhook or email, set up on the Autonomy tab.

---

Step-by-step

Find a run in the Evidence tab

1. Open AI Provider from the admin menu and select the Evidence tab.
2. Filter by run type, outcome, model or date. The list loads more rows as you scroll.
3. Select a row to open its flyout.

The tab covers these run types: Chat answer, Explanation, Fix summary, Fix analysis, Investigation, Alert triage, Risk summary and Background analysis.

Outcomes are Answered, From cache, Partial answer, Declined, Blocked, Stopped, Failed and Completed. A stopped run names who or what stopped it.

To see refused queries and fix analysis rows, switch Runs to Other records. Those rows show the refusal rule or the analysis result instead of a run.

To share a view, select Export CSV. It downloads the rows loaded so far, so scroll to load more first.

To link someone to one run, copy the page address after you open its flyout. It ends in ?tab=evidence&run=<id>.

Read a run's chain

The flyout lists the run's steps in order.

• Model call: The provider, the model, the input and output tokens, and how the call ended. A Fallback badge means the first provider failed and the next one answered. Select View instructions to read the exact system prompt sent.
• Tool call: Asked with lists what Graphne passed to the tool. Each graph query sits under a label for who wrote it, such as "Graphne's query" or "Built-in query". Returned summarizes the result. What Graphne saw shows the exact text the model received.
• Fingerprints: A fingerprint is a hash of the data, so it changes if the data changes. Compare them to tell whether two reads returned the same data.
• Tasks this started: Links to runs this one launched, such as an investigation started from a chat answer.

If a result contained text that looked like instructions, the step says how many passages Graphne treated as data.

Check an answer in Graphne chat

Any signed-in user can do this for their own answers.

1. Ask Graphne a question.
2. Open the Reasoning line above the answer.
3. Read each step. Asked with shows what Graphne passed to the tool, and the step shows what it returned.
4. Read Data from below the steps for the sources Graphne used.

Chat answers show no confidence rating yet.

To read the model's own notes, select Show model reasoning. The notes load on demand. You see the button on your own questions, and Admins see it on any question. If Graphne served your answer from the cache, it was produced by someone else's question. The panel then says the model reasoning isn't shown here.

Graphne never sends these notes back to any model. A model that doesn't share its reasoning shows "This model didn't share its reasoning."

Check how an investigation reached its findings

1. Open a finished report from the Investigations page.
2. Scroll to How Graphne reached this, below the findings.
3. Read the phase steps, the data sources and the confidence meter.
4. Admins can select Open evidence to open the run in the Evidence tab and read the model's own notes.

See what people decided

When you run or close a Graphne or fix analysis recommendation in Select Fire, GraphnAI records one of three outcomes:

The record names you, the subject and the target. It sits in the AI governance audit for 7 years. A run's evidence flyout lists the decisions made on it under What people did.

Check that AI records haven't been changed

GraphnAI seals every AI audit record when it writes it. A seal is a keyed signature (HMAC) over the record. A changed record, a deleted record or a gap in the sequence breaks the check. This covers chat and AI tasks, decisions, alert triage, incidents and prompt versions.

To see the last check, open the Evidence tab and read the Record integrity card. GraphnAI checks every record type once a day.

To check now, select Check integrity now. The Record integrity check dialog shows one entry per record type, such as "Chat and AI tasks: intact (18,233 records)". It can take a few minutes. You can run it about 6 times a minute before it asks you to wait.

A failed entry says what went wrong in plain terms:

• "N records failed their integrity seal": the records were changed after they were written.
• "N records are missing from the sequence": records were deleted.
• "Recent records were removed": the newest records are gone.
• "N records weren't saved because the write failed": the write failed, so these aren't tampering.
• "N records weren't saved when the server stopped unexpectedly": the server stopped before it saved them.

Other lines explain records that are gone for an ordinary reason, such as a privacy request or the retention period. They don't fail the check. If a check runs out of time, the badge reads Incomplete. Select Check again to cover the rest.

Records written before sealing started can't be checked. The card says "N earlier records were written before integrity sealing started and can't be checked." Nothing is reset on upgrade. These records stay readable, exportable and erasable, and GraphnAI never reports them as intact or failed. The seal marks where sealing began. It doesn't prove the earlier records were never changed.

In the Evidence flyout, a record whose seal fails shows a red Changed after it was recorded badge.

Review AI incidents

An incident is an AI event an Admin should see. Open the Incidents tab on the AI Provider page. A red badge on the tab counts open high incidents.

Severities are Info, Low, Medium and High.

A Verification mismatch is one of these cases:

• "An investigation's risk rating was below the findings it cited." Graphne raised the risk to the findings' level. The severity is High when the most severe finding is Critical or the level rose two or more levels. Otherwise it's Medium.
• "Another vendor's model disagreed with an investigation's summary." or "Another vendor's model disagreed with a remediation analysis." The severity is Medium.
• "An investigation's summary couldn't be checked by a model from another vendor." or "A remediation analysis couldn't be checked by a model from another vendor." The Verification model is from the same vendor as the author, or its vendor is unknown, so the check doesn't count. The severity is Low.
• "A provider's answer didn't match the format Graphne asked for." The severity is Low. Graphne records at most one per AI task, provider entry and format in 10 minutes.

An Admin turns the cross-model checks on in the Verification section of the Providers tab. They run only then. With the default paging level (High only), only a High risk raise pages. Disagreements and same-vendor checks are recorded but don't page. Each check in a run is its own incident, so a disagreement after a risk raise in the same investigation isn't counted as a repeat.

To find an incident:

1. Open the Incidents tab.
2. Filter by Type, Severity (Low and above, Medium and above, High only), Status (Open or Acknowledged), and the From date and To date.
3. Select a row to open its flyout.

The list shows Time, Severity, What happened, Who, Status and Sent. Sent tells you what happened to the page:

• "Sent to webhook" or "Sent to email".
• "Couldn't send to webhook" or "Couldn't send to email".
• "Not sent: below the paging level".
• "Not sent: the same incident was sent in the last 10 minutes".
• "Not sent: no escalation channel".

The flyout adds the details as labelled rows, who acknowledged it and their note. Open evidence appears when the incident belongs to a run.

For a Verification mismatch, the flyout's details include Checked by, the model that ran the check. AI's risk level is what the investigation said, Findings' risk level is the highest level among the findings it cited, and Levels raised is the gap Graphne closed.

To acknowledge an incident:

1. Select it, then select Acknowledge.
2. Optionally answer "What did you find?". The note can be 500 characters.
3. Select Acknowledge.

An incident can be acknowledged once. If another Admin got there first, the dialog says "Someone already acknowledged this incident." Select Close.

Incidents are kept for 2 years. Every Admin also gets an in-app notification titled "AI incident" that opens the Incidents tab.

Choose which incidents page you

1. Open AI Provider and select the Autonomy tab.
2. Set Page for AI incidents at to High only, Medium and high, or All except information. The default is High only.

Incidents at or above that level go to your armed escalation channel, a webhook or email. Lower levels are still recorded on the Incidents tab. The top line of that tab shows the current level and channel. Without an escalation channel it reads "Incidents are recorded, but no one is paged. Set up escalation on the Autonomy tab."

The same incident is sent only once in 10 minutes. A failed page never halts Full-Auto.

---

Configuration

Ask whoever runs your server to turn off the model reasoning setting to stop storing the model's own notes. They're the largest optional part of a record. With the setting off, Show model reasoning says reasoning isn't recorded on this server. Other evidence is unaffected.

---

Troubleshooting

The Evidence tab says "Couldn't load the evidence"

Select Retry. If it keeps failing, open Platform Health.

The tab shows "No Graphne activity in this range"

Widen the date range or clear the filters. Runs from before this feature was installed have no record, and records older than 90 days are removed.

A run link says it isn't in the record

The run passed its 90-day retention period, or it never existed. Check the run ID.

Show model reasoning says reasoning isn't recorded on this server

The server setting that stores model reasoning is off. Ask whoever runs the server to turn it on. It applies to new answers only.

Show model reasoning doesn't appear

The answer came from the cache for someone else's question, or it has no evidence record. Ask the question again, or open the run in the Evidence tab as an Admin.

The card says "Integrity check failed"

Open the Incidents tab and read the High audit integrity incident. Run Check integrity now to see which record type failed and why. Records that weren't saved because a write failed or the server stopped unexpectedly aren't tampering. Changed or missing records are. Keep the evidence and escalate to your security team.

The check says "Too many checks in a short time"

Wait a minute and select Check again.

The card says N earlier records can't be checked

They were written before sealing started. They stay readable, but GraphnAI can't prove they're unchanged. The count doesn't clear.

An incident wasn't sent because there is no escalation channel

Set up a webhook or email escalation channel on the Autonomy tab. The incident stays recorded, and Admins still get the in-app notification.

The dialog says "Someone already acknowledged this incident."

Another Admin acknowledged it first. Select Close and read their note in the flyout.

Open evidence doesn't appear in a report

Only Admins see it.