If you've scoped an AI search rollout for a firm, you already know the demo always looks great. You type a plain-English question, results come back in a blink, an AI summary sits on top. Then someone in the room asks the question that ends the demo: "Will this show every user only the documents they're allowed to see? And can we prove who searched for what?"
That's where the project either gets real or gets shelved. Search speed is a commodity now. Access control and a defensible query log are not. They're the difference between a tool your firm can actually deploy and a confidentiality incident waiting to be discovered.
This is a vendor-agnostic walk through the options. We build custom search, so we have a horse in the race, and we'll say where custom is the wrong answer. The goal here is to give you a checklist you can hold every option against, including ours.
Why permissions are the part that breaks
What does "document-level permissions" actually mean for a law firm?
It means three overlapping things, and a search tool has to honor all three. Matter-level confidentiality: a paralegal staffed on one matter shouldn't surface documents from a matter they're not on. Ethical walls: a lawyer screened off a client because of a conflict must never see that client's files, even in a snippet. Need-to-know: some documents (HR, partner compensation, sealed settlements) are restricted regardless of matter.
Your document management system already encodes most of this. The mistake firms make is assuming a new AI search layer inherits it. It usually doesn't. The AI tool builds its own index, and unless that index carries the same access rules, you've just built a fast way to leak privileged documents.
Why can't you just filter results after the search runs?
Because the leak happens before the filter. If the engine searches everything and you strip out forbidden documents in the application layer, you've already exposed their existence. Result counts give it away ("847 matches" when the user should see 12). Autocomplete suggests a walled-off client's name. An AI summary quietly reads a document it shouldn't and paraphrases it. The user never opens the file, but the confidential information already reached their screen.
The rule that decides everything: the permission filter has to run inside the search query, not after it. The index must be physically incapable of returning a document the current user isn't cleared for. Anything else is a post-hoc cleanup that's already too late.
The four approaches, and how each handles permissions and audit
Most "AI-powered search" options for legal teams fall into one of four buckets. None is wrong. They fit different firms. Here's the honest comparison against the two requirements you named: permissions and a query audit trail.
| Approach | Examples | Permission model | Query audit |
|---|---|---|---|
| DMS-native AI | iManage Insight+, NetDocuments | Inherits your existing DMS ACLs and ethical walls | Varies by product and tier |
| Litigation / eDiscovery AI | Relativity aiR, Everlaw | Workspace + matter-level security, strong | Strong, review-grade logging |
| General legal AI assistant | Harvey, CoCounsel | App-level only, does not read your DMS ACLs by default | Vendor-controlled, limited export |
| Custom search | Elasticsearch / OpenSearch build | Document-level security filters mapped to your ACL source of truth | Append-only log you own and export |
DMS-native AI (iManage, NetDocuments)
If your documents already live in iManage or NetDocuments and your ethical walls are configured there, the native AI layer is the path of least resistance. It reads the access model you've already built, so a walled-off user stays walled off. The catch is scope and audit depth. These tools search the repository they own, not the rest of your stack (shared drives, email archives, a separate billing system), and the granularity of query-level audit logging differs by product and license tier. Confirm in writing whether you get per-query logs or just per-document access logs. Those aren't the same thing.
Litigation platforms (Relativity, Everlaw)
Relativity and Everlaw were built around exactly the requirement you have: per-matter security and defensible logging, because eDiscovery lives or dies on chain of custody. If your use case is review inside a specific matter, this is mature, audited, and hard to beat. It's less suited to firm-wide knowledge search across every practice area and every old matter, and the cost model is built for active litigation, not casual lookup. Right tool, specific job.
General legal AI assistants (Harvey, CoCounsel)
Harvey and CoCounsel are excellent at drafting, summarizing, and research over the documents you hand them. Where teams get surprised is repository-wide search. By default these assistants don't read your DMS access controls, so pointing one at your entire document store without a permission layer in front of it is the leak scenario from earlier, with an AI on top. They can be made safe, but that requires an integration that resolves each user's rights and filters retrieval before the model ever sees a document. That integration is a build, not a setting.
Custom search (Elasticsearch / OpenSearch)
When documents are spread across multiple systems, or your confidentiality and audit requirements are stricter than an off-the-shelf product will commit to, custom search becomes the honest answer. You define one access-control source of truth, map it into document-level security filters in the index, and write every query to an append-only log you control. You own the audit data, the retention policy, and the export format. The trade-off is real: it's a project, not a purchase, and it only pays off above a certain scale and sensitivity. Below that, buy the off-the-shelf tool.
The query audit trail nobody specs until it's needed
What should the audit trail actually record?
Teams usually ask for "an audit trail" and stop there. The detail matters. A query log that defends you in a privilege dispute or a conflicts review captures: who ran the query, the exact query text, the timestamp, which documents the engine returned, which ones the user opened, and the access decision the system applied to that user. That last field is the one people forget. Recording that a screened lawyer's search returned zero conflicted documents is how you prove the wall held.
The log has to be append-only. If an entry can be edited or deleted after the fact, it isn't evidence, it's a note. This is the same append-only discipline we apply to compliance logging across regulated platforms. We wrote up the patterns in our guide to audit trails that pass compliance inspections, and the legal version is the same shape with privilege instead of PHI.
Why does ABA Opinion 512 make this more urgent?
ABA Formal Opinion 512 put supervision of generative AI squarely on lawyers. If an AI search tool is reading client files and producing answers, someone has a duty to supervise what it touched and how. A query audit trail is how that supervision becomes something you can demonstrate rather than assert. It's also your record if a client ever asks whether their confidential matter was exposed to a tool, or to a screened individual, during the period in question.
How permission-aware search gets built (and stays fast)
Doesn't filtering by permission slow everything down?
It's the most common worry, and the answer is no, when the filter is part of the query rather than a step after it. Each document gets indexed with the security groups, matters, and clients it belongs to. At search time the system resolves the current user's identity and group membership, then attaches an access filter to the query so the engine only ever scores and returns documents that user is cleared for. The filter and the search are one request.
On a custom Elasticsearch build for a legal client, we ran this on more than 1 million documents and returned results in about 100ms with document-level security applied on every query. The firm recovered an estimated $220,000 a year in time their lawyers had been losing to slow, permission-blind document hunting. Fast and walled-off aren't in tension. Bolting permissions on after the fact is what makes search slow, because then you over-fetch and clean up.
Four things to verify on any AI legal search option before you sign:
- Does the permission filter run inside the query, so the index physically can't return forbidden documents (including in counts, autocomplete, and AI summaries)?
- Where is the single source of truth for access rights, and how does the tool stay in sync when matters and walls change?
- Is the query log append-only, and can you export it on your own schedule?
- Does coverage include every repository your lawyers actually search, or just one system?
When is custom search the wrong call?
When your documents already live in one DMS, your walls are configured there, and that vendor's AI gives you exportable query logs. Buy it. Custom only wins when you have multiple repositories to unify, requirements an off-the-shelf product won't contractually meet, or a scale where licensing per-seat AI gets more expensive than owning the search layer. We've laid out that build-vs-buy math with the actual cost crossover. If the numbers say buy, we'll tell you to buy.
A quick decision guide
Hold your firm against these and the answer usually falls out:
- Everything's in iManage or NetDocuments, walls configured there? Start with the native AI layer. Confirm query-level audit export before you commit.
- The need is matter-scoped review and chain of custody? Relativity or Everlaw. They were built for it.
- You want drafting and research, on documents you control, not repository-wide search? Harvey or CoCounsel, with a permission layer if you point them at anything firm-wide.
- Documents are scattered across systems, or your audit and confidentiality bar is higher than any vendor will sign up to? Custom search, with document-level security and an audit log you own.
The thread through all four: the permission model and the audit trail are the spec, not the search box. Decide those first. For more on what custom legal search covers end to end, see our work on AI legal document search architecture and ROI and why firms move off generic DMS search.
Frequently asked questions
What AI-powered search options respect document-level permissions and provide an audit trail of queries?
Four approaches. DMS-native AI (iManage Insight+, NetDocuments) inherits the access controls and ethical walls already in your document system, though query-level audit logging varies by product. Litigation platforms (Relativity aiR, Everlaw) enforce workspace and matter-level security with strong audit logging, but are scoped to review workspaces rather than your whole repository. General legal AI assistants (Harvey, CoCounsel) apply their own app-level permissions and don't automatically read your document ACLs. Custom search on Elasticsearch enforces document-level security filters mapped to your own access-control source of truth and writes every query to an append-only log you own. Fit depends on where documents live and how strict your confidentiality and privilege requirements are.
How do you enforce ethical walls in AI legal document search?
At query time, not after retrieval. Each document is tagged with the matters, clients, and security groups it belongs to. When a user searches, the system resolves their identity and group membership first, then applies a filter so the index only ever returns documents that user is permitted to see. The wall lives in the search layer. Filtering results in the application after the engine returns them risks leaking titles, snippets, and the very existence of walled-off documents through counts, autocomplete, or AI summaries.
What should a legal search query audit trail capture?
Who ran the query, the exact query text, the timestamp, which documents were returned, which were opened, and the access decision applied to that user. Keep it append-only so entries can't be edited or deleted later. This supports privilege defense, ABA Opinion 512 supervision duties, conflicts review, and malpractice defense, and it lets you prove after an incident that a screened user never saw a conflicted matter.
Can general AI assistants like Harvey or CoCounsel search our whole repository with permissions intact?
Not by default. They're strong at drafting and research over the content you give them, but they apply their own app-level permissions rather than reading your DMS access controls and ethical walls. To make them enforce your firm's permissions across the full repository, you need an integration layer that resolves each user's rights and filters what the assistant can retrieve. That integration is custom work, not a checkbox.
How fast can permission-aware search run on a large document set?
Permission filtering doesn't have to slow search down. On a custom Elasticsearch deployment for a legal client, we returned results in about 100ms across more than 1 million documents with document-level security applied on every query. The access filter is part of the query, so it runs inside the same sub-second request rather than as a separate step afterward.
Scoping AI search for a firm?
We build permission-aware legal document search with ethical-wall enforcement and an audit log you own. 100ms on 1M+ documents, document-level security on every query. We'll also tell you when buying off-the-shelf is the better call.
See Our Elasticsearch Service →Product capabilities (iManage, NetDocuments, Relativity, Everlaw, Harvey, CoCounsel) change over time and vary by license tier. Verify audit logging and permission behavior against the vendor's current documentation and your contract before relying on it. The 100ms and $220k/year figures are from a custom Elasticsearch legal search platform we built. If a detail here is out of date, email office@oktopeak.com and we'll correct it.