Your AI Connector Has an Audit Log. Have You Read It?

Every AI connector worth buying advertises an audit log. It is usually the answer to the compliance question, and it is usually the shortest paragraph in the documentation. Almost nobody asks the follow-up, which is what the log actually contains.

It is worth asking, because the audit log is the one part of a connector that writes to disk and stays there. Whatever it records is a second copy of your data, living outside the practice management system that has the access controls, the retention policy and the audit trail you already trust.

This post is a diagnostic. It works on any connector, ours included, and it takes about ten minutes.

What a log is for, and what it drifts into

The purpose is narrow: answer who asked this system about which matter, and when. That is what a regulator, an insurer or a client asks, and it is satisfied entirely by references. A timestamp, a tool name, a caller, a matter id, an outcome.

The drift happens because the easiest way to write an audit entry is to record the arguments the tool was called with, and the arguments are where the content is. A search tool is called with a search term. A note tool is called with the note. Nobody sets out to log a client name. It arrives as a parameter.

The three that leak most often

1. Search queries

Any tool that searches contacts, matters or documents takes a query string. On a practice management system that string is almost always a person: a client, an opposing party, a witness. A log that records the query has recorded who the firm was looking up and when, which is a more sensitive dataset than it first appears. Nobody notices, because the parameter is called query and looks like metadata.

2. Note and case text

Notes are where the narrative lives: intake interviews, call summaries, case history, sometimes negotiation positions. A create-note tool takes a subject and a body. Logging the subject alone is often enough to reconstruct the matter. This one also shows up in a partial form that reads as a compromise and is not: truncating a note to its first hundred characters still stores the opening of every note the firm wrote.

3. Custom field values

This is the one that surprises people, because it depends on how the firm uses the platform. On a generalist system, custom fields are where practice-specific facts end up: the incident date, the amount in dispute, the date of a written demand, a name. A log that records a field write records the fact. A log that records which field was written records nothing sensitive at all, and answers the audit question just as well.

// A copy of the record
{ "tool": "update_matter", "matter_id": 4821,
  "args": { "custom_field_values": [
    { "custom_field": { "id": 55001 }, "value": "12,400.00" },
    { "custom_field": { "id": 55004 }, "value": "2026-03-14" }
  ] } }

// A reference to it
{ "tool": "update_matter", "matter_id": 4821,
  "args": { "custom_field_ids": [55001, 55004] } }

Run the check yourself

Locally installed connectors almost always write to a dotfile in the user home directory. Find it, then look.

ls -la ~ | grep -i 'mcp\|connector\|'"$VENDOR"
tail -5 ~/.<connector>/audit.log

The shape tells you most of it in five lines. Then run the sharper test. Do a normal piece of work through the connector using a distinctive string you can search for: put a nonsense word in a search box, a note, a custom field. Then look for it.

grep -c 'ZZQQ-CANARY' ~/.<connector>/audit.log

Any count above zero means the log is storing content. Repeat for each surface: a search, a note, a field write, a document upload. They fail independently, and a connector that handles notes carefully often still logs the search query.

While you are in there

Check three more things. Whether the file rotates or grows forever, because an unbounded log is an unbounded exposure. Whether it is readable by anyone other than its owner. And whether it is inside a folder that syncs to a cloud drive or gets swept into a machine backup, which quietly moves the copy somewhere you did not evaluate.

Denylist versus allowlist, and why it decides the outcome

Under the hood there are two ways to sanitise an audit entry, and the choice determines whether the problem recurs.

A denylist names the fields to hide, typically access tokens, secrets and passwords, and copies everything else through. It is the common default because it is written once and looks thorough. It is a guess about what is sensitive, made by a developer who cannot know that a firm keeps loss amounts in field 55001. It also fails open: the next parameter anyone adds is logged, because it was not on a list nobody remembered to update.

An allowlist names the fields that may be recorded, per tool, and replaces everything else with a placeholder. It is more work up front and it fails closed. An argument nobody classified is redacted rather than published, and adding a tool forces a decision instead of allowing an omission.

The stronger version pairs the allowlist with a test that pushes a canary value through every tool the connector exposes and fails if it turns up in the log. That is what makes the property survive contact with a codebase that keeps growing, because it catches the tool written next year by someone who never read this.

What to ask a vendor

The last one separates the vendors who have thought about this from the ones who have not. An intention is not a control. A failing test is.

The point

None of this is an argument against audit logging. A firm putting AI anywhere near client data should be able to account for what happened, and a log of tool calls against matter identifiers is a reasonable way to do it under ABA Formal Opinion 512 and its equivalents.

It is an argument for reading the file. The feature and the liability are the same artifact, and which one you have depends entirely on what is written in it.

Frequently Asked Questions

Next step

If you are evaluating connectors for a firm, run the canary test on each one before you compare feature lists. If you are deploying AI inside a practice and want the logging, retention and access questions settled properly rather than assumed, that is what our firm deployment work covers. More on the surrounding decisions in our legal tech practice.