crucial AI use instances of an enterprise right now, doc comparability ranks alongside conversational chatbots. Organizations spend an enormous variety of person-hours evaluating contracts, insurance policies, technical specs, authorized petitions, analysis papers and plenty of extra to establish variations, dangers, revisions and semantic inconsistencies.
Nonetheless, doc comparability is much extra advanced than conventional textual content distinction. For one, these instruments are supposed to be efficient assistants to authorized and industrial professionals, scientists and others who count on the evaluation to be on the stage of depth and language as could be anticipated from a junior skilled within the area.
A good tougher downside is that which means in enterprise paperwork often isn’t contained in remoted chunks. It’s embedded inside sections, hierarchies, clause groupings and relationships. And these sections could also be scattered throughout a number of pages of a doc spanning over a 100 pages. For instance, a credit score settlement could outline collateral limitations in a single part, exceptions to these a number of pages later, and describe enforcement rights below a totally totally different article. If one other settlement is being in contrast in opposition to this with the factors reminiscent of “collateral construction, safety pursuits, and lien necessities,” the system should establish, retrieve, and synthesize all of those structurally scattered sections collectively earlier than any significant comparability can happen.
Proxy-Pointer architecture, with its structure-aware, but low-cost retrieval pipeline that preserves doc hierarchy throughout retrieval and comparability, is ideally suited to this activity. Utilizing a mix of hierarchical breadcrumb embeddings and light-weight LLM re-ranker, it is ready to exactly extract semantically aligned areas throughout paperwork earlier than comparative reasoning begins.
On this article, I’m sharing the design and real-world outcomes of a flexible doc comparator able to analyzing each extremely advanced monetary Credit score Agreements and educational analysis papers. As you’ll discover within the structure described within the subsequent part, the core comparability engine is separated from the upstream doc processing and downstream report formatting and technology, enabling the system to be simply tailored to any new doc area (reminiscent of insurance coverage insurance policies, medical tips, or tax codes). All that’s required is an upstream extraction pipeline to construction the enter for hierarchical tree technology, and a downstream replace to the LLM’s analytical persona and report formatter—leaving the core multi-stage retrieval and comparability pipeline completely untouched.
Additionally, I’m including the total code, to my present open-source Proxy-Pointer github repository, together with a 5 minute quickstart.
Doc Comparator Structure
Right here is an outline of the logical structure. The LLM used is gemini-3-flash together with gemini-embedding-001 (dimension: 1536) for vector embeddings.
Architectural Tiers
Upstream Extraction Layer
Converts any incoming uncooked doc construction right into a standardized, machine-readable hierarchy.
Packages Concerned
-
extract_pdf_to_md.py: Handles upstream ingestion, changing PDFs into clear, hierarchically formatted Markdown. build_doc_index.py: Parses Markdown headers, filters administrative noise, and builds the hierarchical JSON construction map (_structure.json).
Core Comparability Engine
Coordinates semantic search over hierarchical doc nodes.
Packages Concerned
criteria_validator.py: Dynamically detects thedoc_type(e.g., Educational vs. Authorized) and performs an preliminary feasibility verify on the consumer’s comparability standards, to determine if the factors is related for the recognized doc kind.section_selector.py: Implements Stage 1 Proxy-Pointer retrieval. It identifies and extracts essentially the most related sections of Doc 1 based mostly on consumer standards utilizing FAISS semantic search and an LLM re-ranker.cross_retriever.py: Implements Stage 2 Proxy-Pointer retrieval. It performs a focused semantic search inside Doc 2’s vector house utilizing the context of the chosen Doc 1 sections (pairing the Doc 1 part content material with the consumer’s standards because the question). The Proxy-Pointer pipeline is extraordinarily correct in figuring out the proper semantically analogous sections for comparability.section_comparator.py: Coordinates pairwise evaluations of matching sections, passing them to the LLM to research alignments and discrepancies.
Downstream Presentation Layer
Tailors the analytical output to the audience and codecs the ultimate visualization.
Packages Concerned
build_comparison_prompt(incriteria_validator.py): The immediate assigns the suitable persona (e.g., Skilled Educational Researcher or Senior Authorized Counsel) based mostly on the detecteddoc_type.report_builder.py: Renders the ultimate comparability report side-by-side utilizing skilled CSS colours and extremely readable format formatting. The report may also be downloaded as a markdown file.
Dataset Used
For the prototype, publicly available Credit score Agreements, Emerson (136 pages) and Texas Roadhouse (190 pages) are used. These have been intentionally chosen as they’ve totally different constructions and belong to totally different industries. Emerson is a utility supplier, and its settlement reads like a sovereign company treasury doc based mostly on credit score company rankings, whereas Texas Roadhouse’s settlement is very custom-made, constructed particularly round restaurant leases, multi-entity subsidiary constructions, and dynamic leverage ratios.
As well as, I added the function to check analysis papers for which I chosen VectorFusion and VectorPainter, which have been utilized in my article on Multimodal Answers RAG. They’re each papers within the extremely specialised subject of text-to-vector graphics technology. Whereas each share an equivalent technical basis—utilizing differentiable rendering (reminiscent of DiffVG) to optimize Scalable Vector Graphics (SVG) paths by way of diffusion fashions—they differ considerably of their methodological execution. This slender, shared-domain relationship is a tough take a look at case for our comparability engine, of its capacity to bypass surface-level similarities and as an alternative consider delicate architectural variations, which we will see within the subsequent part.
Comparability of Credit score Agreements
I ran a number of totally different queries with a various set of standards; the detailed stories are absolutely included within the repository, and a snapshot is shared under. The Streamlit UI accepts two paperwork (both in .pdf or .md format) as enter, with the comparability carried out strictly from the angle of Doc 1. For instance, if Doc 1 is Emerson and Doc 2 is Texas Roadhouse, the ultimate comparability is framed round Emerson.
There are three steps to the method. First, it selects all sections from the Emerson settlement which can be related to the consumer’s standards. For every chosen part, it finds as much as three comparative sections in Texas Roadhouse, after which performs a side-by-side evaluation. Together with the detailed evaluation, the system offers a practical Function, a Discrepancy Ranking, and a Danger Route (or Methodological Tradeoff for educational papers)
Within the following 4 instances, Doc 1 is Emerson, Doc 2 is Texas Roadhouse.
Standards 1: collateral construction, safety pursuits, ensures, and lien necessities
Standards 2: occasions of default, lender cures, acceleration rights, and treatment intervals
Standards 3: monetary covenants, leverage ratio necessities, and borrower compliance obligations
Standards 4a: representations and warranties, materials antagonistic impact clauses, and disclosure obligations
For edge case testing, right here is the above “warranties” standards with the paperwork switched. Within the following, Doc 1 is Texas Roadhouse and Doc 2 is Emerson.
Standards 4b: representations and warranties, materials antagonistic impact clauses, and disclosure obligations
Evaluation of Credit score Settlement comparability
What the above outcomes present is that Proxy-Pointer isn’t just matching clauses by key phrases or incomplete chunks, it’s taking a look at them from the persona of a authorized analyst, somebody who understands how credit score works, throughout these extremely numerous industries. One being an investment-grade utility, and the opposite a midsize restaurant chain. For example, it identifies the financial and authorized penalties hidden beneath superficially comparable language — like structural subordination danger inside a adverse pledge, enterprise-value preservation inside disposition covenants or litigation publicity inside disclosure representations.
One other statement is that the evaluation remained directionally constant when the paperwork have been flipped. It didn’t anchor itself to Emerson as Doc 1, however as an alternative re-evaluated the agreements from the Texas Roadhouse perspective. It appropriately recognized which settlement positioned extra restrictions on the borrower, which gave lenders higher management throughout defaults, which was extra weak to belongings being moved out of attain, and which required the corporate to reveal extra info. None of those are explicitly written in both agreements. They change into evident to a authorized analyst when a number of clauses, exceptions, thresholds, and definitions are learn collectively. The consequence feels much less like a easy clause comparability and extra like understanding how danger and management are shared between the borrower and the lender.
Analysis Paper Comparability
For the VectorFusion and VectorPainter papers, I in contrast utilizing the next standards: Evaluate how every paper approaches model management and primitive initialization in vector graphics synthesis. Particularly, analyze how VectorFusion makes use of path reinitialization and raster pattern initialization versus how VectorPainter extracts and rearranges vectorized strokes from a reference picture utilizing stroke imitation studying and style-preserving losses
Right here is one comparability:
The evaluation reveals a deep domain-intensive comparability, a device {that a} researcher can use to check each papers with out studying them of their entirety. Proxy-Pointer strikes past surface-level structure matching and identifies the deeper design philosophy behind each papers. As well as, it appropriately acknowledges that VectorFusion treats SVG technology as a dynamic optimization downside with steady path reinitialization, whereas VectorPainter approaches it as a style-guided synthesis downside targeted on inventive consistency and realized stroke historical past. What was additionally fairly attention-grabbing was that it may join concepts unfold throughout utterly totally different sections of the papers and steadiness the underlying limitations. This demonstrates a fine-grained evaluation of two techniques in the identical slender area however that work in a different way.
Open-Supply Repository
Proxy-Pointer is absolutely open-source (MIT License) and could be accessed at Proxy-Pointer Github repository. The Doc Comparator is being added to the repo along with the present Textual content-Solely and Multimodal Answering bots.
A 5-minute quickstart will allow you to check rapidly with accessible information.
DocComparator/
├── src/
│ ├── comparability/
│ │ ├── cross_retriever.py # Stage 2 PP Retrieval (Doc 2)
│ │ ├── section_comparator.py # Pairwise LLM analysis engine
│ │ └── section_selector.py # Stage 1 PP Retrieval (Doc 1)
│ ├── extraction/
│ │ └── extract_pdf_to_md.py # LlamaParse PDF ingestion & formatting
│ ├── indexing/
│ │ └── build_doc_index.py # Skeleton tree & FAISS vector builder
│ ├── report/
│ │ └── report_builder.py # Markdown report technology logic
│ ├── validation/
│ │ └── criteria_validator.py # Persona injection & standards feasibility
│ └── config.py # Core configurations and mannequin definitions
├── information/ # Unified Information Hub
│ └── uploads/ # Uncooked PDFs and take a look at paperwork
├── outcomes/ # Artifact stories for the take a look at instances tried
└── app.py # Streamlit Comparator UIConclusion
Doc comparability utilizing a Chunk-Embed-Match strategy isn’t probably to offer good outcomes. In a posh enterprise doc reminiscent of Contract Phrases and Situations, semantic which means is encapsulated into sections and subsections containing dense textual content. Every of those sections could possibly be pages in size and a part of a really lengthy doc. For efficient comparability and evaluation – sections, definitions, exceptions, and structural relationships must be extracted collectively to make sense when learn collectively.
Proxy-Pointer with its correct two-step retrieval pipeline is right for this activity. Because the outcomes above present, even with a price range LLM reminiscent of gemini-flash, one can evaluate agreements or analysis papers such that it may protect the underlying intent and trade-offs hidden throughout structurally disparate sections.
The three-tier structure of the Doc Comparator can scale to different domains with no change to the comparability engine itself. This permits structure-aware retrieval to generalize higher than a custom-built device that works just for a particular kind of doc. Organizations can adapt this to their particular industries and use instances, with minimal incremental engineering effort.
Clone the repo. Attempt your individual paperwork. Let me know your ideas.
Join with me and share your feedback at www.linkedin.com/in/partha-sarkar-lets-talk-AI
All analysis papers used on this article can be found at VectorFusion and VectorPainter with CC-BY license. The credit score agreements are publicly accessible at SEC.gov. Code and benchmark outcomes are open-source below the MIT License. Photos used on this article are generated utilizing Google Gemini.
