GraphRAG in Practice: How to Build Cost-Efficient, High-Recall Retrieval Systems

article, Do You Really Need GraphRAG? A Practitioner’s Guide Beyond the Hype, I outlined the core rules of GraphRAG design and launched an augmented retrieval-and-generation pipeline that mixes graph search with vector search. I additionally mentioned why constructing a wonderfully full graph—one which captures each entity and relation within the corpus—could be prohibitively complicated, particularly at scale.

On this article, I increase on these concepts with concrete examples and code, demonstrating the sensible constraints encountered when constructing and querying actual GraphRAG programs. I additionally illustrate that the retrieval pipeline helps stability value and implementation complexity with out sacrificing accuracy. Particularly, we’ll cowl:

1. Constructing the graph: Ought to entity extraction occur on chunks or full paperwork—and the way a lot does this alternative really matter?
2. Querying relations and not using a dense graph: Can we infer significant relations utilizing iterative search-space optimisation as an alternative of encoding each relationship within the graph explicitly?
3. Dealing with weak embeddings: Why alphanumeric entities break vector search and the way graph context fixes it.

GraphRAG pipeline

To recall from the earlier article, the GraphRAG embedding pipeline used is as follows. The Graph node and relations and their embeddings are saved in a Graph database. Additionally, the doc chunks and their embeddings are saved within the database.

The proposed retrieval and response technology pipeline is as follows:

As could be seen, the graph consequence is just not straight used to answer person question. As a substitute it’s used within the following methods:

1. Node metadata (notably doc_id) acts as a powerful classifier, serving to establish the related paperwork earlier than vector search. That is essential for giant corpora the place naive vector similarity can be noisy.
2. Context enrichment of the person question to retrieve essentially the most related chunks. That is essential for sure sorts of question with weak vector semantics similar to IDs, automobile numbers, dates, and numeric strings.
3. Iterative search area optimisation, first by choosing essentially the most related paperwork, and inside these, essentially the most related chunks (utilizing context enrichment). This allows us to maintain the graph easy, whereby all relations between the entities needn’t be essentially extracted into the graph for queries about them to be answered precisely.

To show these concepts, we’ll use a dataset of 10 synthetically generated police stories, GPT-4o because the LLM, and Neo4j because the graph database.

Constructing the Graph

We might be constructing a easy star graph with the Report Id because the central node and entities linked to the central node.  The immediate to construct that might be as follows:

custom_prompt = ChatPromptTemplate.from_template("""
You might be an data extraction assistant.
Learn the textual content beneath and establish necessary entities.

**Extraction guidelines:**
- At all times extract the **Report Id** (that is the central node).
- Extract **folks**, **establishments**, **locations**, **dates**, **financial quantities**, and **automobile registration numbers** (e.g., MH12AB1234, PK-02-4567, KA05MG2020).
- Don't ignore any folks names; extract all talked about within the doc, even when they appear minor or position not clear.
  Deal with all of sorts of autos (eg; vehicles, bikes and so forth) as the identical sort of entity referred to as "Car".

**Output format:**
1. Listing all nodes (distinctive entities).
2. Determine the central node (Report Id).
3. Create relationships of the shape:
   (Report Id)-[HAS_ENTITY]->(Entity),
4. Don't create every other sorts of relationships.                                            

Textual content:
{enter}

Return solely structured knowledge like:
Nodes:
- Report SYN-REP-2024
- Honda bike ABCD1234
- XYZ School, Chennai
- NNN School, Mumbai
- 1434800
- Mr. John

Relationships:
- (Report SYN-REP-2024)-[HAS_ENTITY]->(Honda bike ABCD1234)
- (Report SYN-REP-2024)-[HAS_ENTITY]->(XYZ faculty, Chennai)
- ...
""")

Be aware that on this immediate, we aren’t extracting any relations similar to accused, witness and so forth. within the graph. All nodes could have a uniform “HAS_ENTITY” relation with the central node which is the Report Id. I’ve designed this as an excessive case, as an example the truth that we will reply queries about relations between entities even with this minimal graph, based mostly on the retrieval pipeline depicted within the earlier part. For those who want to embrace a number of necessary relations, the immediate could be modified to incorporate clauses similar to the next:

3. For particular person entities, the relation ought to be based mostly on their position within the Report (e.g., complainant, accused, witness, investigator and so forth).
    eg: (Report Id) -[Accused]-> (Individual Title)
4. For all others, create relationships of the shape:
   (Report Id)-[HAS_ENTITY]->(Entity),

llm_transformer = LLMGraphTransformer(
    llm=llm,
    # allowed_relationships=["HAS_ENTITY"],
    immediate= custom_prompt,
)

Subsequent we’ll create the graph for every doc by making a Langchain doc from the total textual content after which offering to Neo4j.

# Learn total file (no chunking)
with open(file_path, "r", encoding="utf-8") as f:
    text_content = f.learn()

# Create LangChain Doc
doc = Doc(
    page_content=text_content,
    metadata={
        "doc_id": doc_id,
        "supply": filename,
        "file_path": file_path
    },
)
attempt:
    # Convert to graph (total doc)
    graph_docs = llm_transformer.convert_to_graph_documents([document])
    print(f"✅ Extracted {len(graph_docs[0].nodes)} nodes and {len(graph_docs[0].relationships)} relationships.")

    for gdoc in graph_docs:
        for node in gdoc.nodes:
            node.properties["doc_id"] = doc_id

            original_id = node.properties.get("id") or getattr(node, "id", None)
            if original_id:
                node.properties["entity_id"] = original_id

    # Add to Neo4j
    graph.add_graph_documents(
        graph_docs,
        baseEntityLabel=True,
        include_source=False
    )
besides:
...

This creates a graph comprising 10 clusters as follows:

Key Observations

1. The variety of nodes extracted varies with LLM used and even for various runs of the identical LLM. With gpt-4o, every execution extracts between 15 to 30 nodes (relying upon the dimensions of the doc) for every of the paperwork for a complete of 200 to 250 nodes. Since every is a star graph, the variety of relations is one lower than the variety of nodes for every doc.
2. Prolonged paperwork lead to consideration dilution of the LLMs, whereby, they don’t recall and extract all the required entities (particular person, locations and so forth) current within the doc.

To see how extreme this impact is, lets see the graph of one of many paperwork (SYN-REPORT-0008). The doc has about 4000 phrases. And the ensuing graph has 22 nodes and appears like the next:

Now, lets attempt producing the graph for this doc by chunking it, then extracting entities from every chunk and merging them utilizing the next logic:

1. The entities extraction immediate stays identical as earlier than, besides we ask to extract entities apart from the Report Id.
2. First extract the Report Id from the doc utilizing this immediate.

report_id_prompt = ChatPromptTemplate.from_template("""
Extract ONLY the Report Id from the textual content.

Report Ids usually appear like:
- SYN-REP-2024

Return strictly one line:
Report: 

Textual content:
{enter}
""")

Then, extract entities from every chunk utilizing the entities immediate.

def extract_entities_by_chunk(llm, textual content, chunk_size=2000, overlap=200):
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=overlap
    )

    chunks = splitter.split_text(textual content)
    all_entities = []

    for i, chunk in enumerate(chunks):
        print(f"🔍 Processing chunk {i+1}/{len(chunks)}")
        uncooked = run_prompt(llm, entities_prompt, chunk)

        pairs = re.findall(r"- (.*?)s*|s*(w+)", uncooked)
        all_entities.lengthen([(e.strip(), t.strip()) for e, t in pairs])

    return all_entities

c. De-duplicate the entities

d. Construct the graph by connecting all of the entities to the central Report Id node.

The impact is sort of exceptional. The graph of SYN-REPORT-0008 now appears like the next. It has 78 nodes, 3X occasions the depend earlier than. The trade-off in constructing this dense graph are the time and utilization incurred for the iterations for chunk extraction.

What are the implications?

The impression of the variation in graph density is within the capacity to reply questions associated to the entities straight and precisely; i.e if an entity or relation is just not current within the graph, a question associated to it can’t be answered from the graph.

An strategy to minimise this impact with our sparse star graph can be to create a question such that there’s a reference to a distinguished associated entity more likely to be current within the graph.

As an illustration, the investigating officer is talked about comparatively fewer occasions than town in a police report, and there’s a greater likelihood of town to be current within the graph relatively than the officer. Due to this fact, to seek out out the investigating officer, as an alternative of claiming “Which stories have investigating officer as Ravi Sharma?”, one can say “Among the many Mumbai stories, which of them have investigating officer as Ravi Sharma?”, whether it is identified that this officer is from Mumbai workplace. Our retrieval pipeline will then extract the stories associated to Mumbai from the graph, and inside these paperwork, find the chunks having the officer identify precisely. That is demonstrated within the following sections.

Dealing with weak embeddings

Take into account the next related queries which might be more likely to be steadily requested of this knowledge.

“Inform me in regards to the incident involving Person_3”

“Inform me in regards to the incident in report SYN-REPORT-0008”

The main points in regards to the incident within the report can’t be discovered within the graph as that holds the entities and relations solely, and subsequently, the response must be derived from the vector similarity search. 

So, can the graph be ignored on this case?

For those who run these, the primary question is more likely to return an accurate reply for a comparatively small corpus like our take a look at dataset right here, whereas the second is not going to. And the reason being that the LLMs have an inherent understanding of particular person names and phrases resulting from their coaching, however discover arduous to connect any semantic which means to alphanumeric strings similar to report_id, automobile numbers, quantities, dates and so forth. And subsequently, the embedding of an individual’s identify is way stronger than that of alphanumeric strings.  So the chunks retrieved within the case of alphanumeric strings utilizing vector similarity have a weak correlation to the person question, leading to an incorrect reply.

That is the place the context enrichment utilizing Graph helps. For a question like “Inform me in regards to the incident in SYN-REPORT-0008”, we get all the small print from the star graph of the central node SYN-REPORT-0008 utilizing a generated cypher, then have the LLM use this to generate a context (interpret the JSON response in pure language). The context additionally accommodates the sources for the nodes, which on this case returns 2 paperwork, certainly one of which is the proper doc SYN-REPORT-0008. The opposite one SYN-REPORT-00010 is because of the truth that one of many connected nodes –metropolis is frequent (Mumbai) for each the stories.

Now that the search area is refined to solely 2 paperwork, chunks are extracted from each utilizing this context together with the person question. And since the context from the graph mentions individuals, locations, quantities and different particulars current within the first report however not within the second, it allows the LLM to simply perceive within the response synthesis step that the proper chunks are those extracted from SYN-REPORT-0008 and never from 0010. And the reply is shaped precisely. Right here is the log of the graph question, JSON response and the pure language context depicting this.

Generated Cypher:
cypher
MATCH (r:`__Entity__`:Report)
WHERE toLower(r.id) CONTAINS toLower("SYN-REPORT-0008")
OPTIONAL MATCH (r)-[]-(e)
RETURN DISTINCT 
    r.id AS report_id, 
    r.doc_id AS report_doc_id,
    labels(e) AS entity_labels,
    e.id AS entity_id, 
    e.doc_id AS entity_doc_id

JSON Response:
[{'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Person'], 'entity_id': 'Mr. Person_12', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Place'], 'entity_id': 'New Delhi', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Place'], 'entity_id': 'Kottayam', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels': ['__Entity__', 'Person'], 'entity_id': 'Person_4', 'entity_doc_id': 'SYN-REPORT-0008'}, {'report_id': 'Syn-Report-0008', 'report_doc_id': 'SYN-REPORT-0008', 'entity_labels':… truncated 

Pure language context:
The context describes an incident involving a number of entities, together with people, locations, financial quantities, and dates. The next particulars are extracted:

1. **Individuals Concerned**: A number of people are talked about, together with "Mr. Person_12," "Person_4," "Person_11," "Person_8," "Person_5," "Person_6," "Person_3," "Person_7," "Person_10," and "Person_9."

2. **Locations Referenced**: The locations talked about embrace "New Delhi," "Kottayam," "Delhi," and "Mumbai."

3. **Financial Quantities**: Two financial quantities are famous: "0.5 Million" and "43 1000's."

4. **Dates**: Two particular dates are talked about: "07/11/2024" and "04/02/2025."

Sources: [SYN-REPORT-0008, SYN-REPORT-00010]

Can relations be efficiently discovered?

What about discovering relations between entities? Now we have ignored all particular relations in our graph and simplified it such that there’s just one relation “HAS_ENTITY” between the central report_id node and remainder of the entities. This could indicate that querying for entities not current within the graph and relations between entities shouldn’t be attainable. Let’s take a look at our iterative search optimisation pipeline in opposition to quite a lot of such queries. We’ll contemplate two stories from Kolkata, and the next queries for this take a look at.

• The place the referred relation is just not current within the graph. Eg; “Who’s the investigating officer in SYN-REPORT-0006?” Or “Who’re the accused in SYN-REPORT-0006?”
• Relation between two entities current within the graph.  Eg; “Is there a relation between Ravi Verma and Rakesh Prasad Verma?”
• Relation between any entities associated to a 3rd entity. Eg; “Are there brothers in stories from Kolkata?”
• Multi-hop relations: “Who’s the investigating officer within the stories the place brothers from Kolkata are accused?”

Utilizing our pipeline, all of the above queries yield correct outcomes. Lets have a look at the method for the final multi-hop question which is essentially the most complicated one. Right here the cypher doesn’t yield any consequence, so the circulation falls again to semantic matching of nodes. The entities are extracted (Place: Kolkata) from the person question, then matched to get references to all of the stories linked to Kolkata, that are SYN-REPORT-0005 and SYN-REPORT-0006 on this case. Primarily based on the context that the person question is inquiring about brothers and investigating officers, essentially the most related chunks are extracted from each the paperwork. The resultant reply efficiently retrieves investigating officers for each stories.

Right here is the response:

“The investigating officer within the stories the place the brothers from Kolkata (Mr. Rakesh Prasad Verma, Mr. Ravi Prasad Verma, and Mr. Vijoy Kumar Varma) are accused is Ajay Kumar Tripathi, Inspector of Police, CBI, ACB, Kolkata, as talked about in SYN-REPORT-0006. Moreover, Praveen Kumar, Deputy Superintendent of Police, EOB Kolkata, is famous because the investigating officer in SYN-REPORT-0005.

Sources: [SYN-REPORT-0005, SYN-REPORT-0006]”

> Getting into new GraphCypherQAChain chain...
2025-12-05 17:08:27 - HTTP Request: ... LLM referred to as
Generated Cypher:
cypher
MATCH (p:`__Entity__`:Individual)-[:HAS_ENTITY]-(r:`__Entity__`:Report)-[:HAS_ENTITY]-(pl:`__Entity__`:Place)
WHERE toLower(pl.id) CONTAINS toLower("kolkata") AND toLower(p.id) CONTAINS toLower("brother")
OPTIONAL MATCH (r)-[:HAS_ENTITY]-(officer:`__Entity__`:Individual)
WHERE toLower(officer.id) CONTAINS toLower("investigating officer")
RETURN DISTINCT 
    r.id AS report_id, 
    r.doc_id AS report_doc_id, 
    officer.id AS officer_id, 
    officer.doc_id AS officer_doc_id

Cypher Response:
[]
2025-12-05 17:08:27 - HTTP Request: ...LLM referred to as

> Completed chain.
is_empty: True
❌ Cypher didn't produce a assured consequence.
🔎 Working semantic node search...
📋 Detected labels: ['Place', 'Person', 'Institution', 'Date', 'Vehicle', 'Monetary amount', 'Chunk', 'GraphNode', 'Report']
Person question for node search: investigating officer within the stories the place brothers from Kolkata are accused
2025-12-05 17:08:29 - HTTP Request: ...LLM referred to as
🔍 Extracted entities: ['Kolkata']
2025-12-05 17:08:30 - HTTP Request: ...LLM referred to as
📌 Hits for entity 'Kolkata': [Document(metadata={'labels': ['Place'], 'node_id': '4:5b11b2a8-045c-4499-9df0-7834359d3713:41'}, page_content='TYPE: PlacenCONTENT: KolkatanDOC: SYN-REPORT-0006')]
📚 Retrieved node hits: [Document(metadata={'labels': ['Place'], 'node_id': '4:5b11b2a8-045c-4499-9df0-7834359d3713:41'}, page_content='TYPE: PlacenCONTENT: KolkatanDOC: SYN-REPORT-0006')]
Expanded node context:
 [Node] It is a __Place__ node. It represents 'TYPE: Place
CONTENT: Kolkata
DOC: SYN-REPORT-0006' (doc_id=N/A).
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Establishment: Mrs.Sri Balaji Forest Product Non-public Restricted (doc_id=SYN-REPORT-0005)
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Date: 2014 (doc_id=SYN-REPORT-0005)
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Individual: Mr. Pallab Biswas (doc_id=SYN-REPORT-0005)
[Report Syn-Report-0005 (doc_id=SYN-REPORT-0005)] --(HAS_ENTITY)--> __Entity__, Date: 2005 (doc_id=SYN-REPORT-0005).. truncated
[Report Syn-Report-0006 (doc_id=SYN-REPORT-0006)] --(HAS_ENTITY)--> __Entity__, Establishment: M/S Jkjs & Co. (doc_id=SYN-REPORT-0006)
[Report Syn-Report-0006 (doc_id=SYN-REPORT-0006)] --(HAS_ENTITY)--> __Entity__, Individual: B Mishra (doc_id=SYN-REPORT-0006)
[Report Syn-Report-0006 (doc_id=SYN-REPORT-0006)] --(HAS_ENTITY)--> __Entity__, Establishment: Vishal Engineering Pvt. Ltd. (doc_id=SYN-REPORT-0006).. truncated

Key Takeaways

• You don’t want an ideal graph. A minimally structured graph—even a star graph—can nonetheless help complicated queries when mixed with iterative search-space refinement.
• Chunking boosts recall, however will increase value. Chunk-level extraction captures way more entities than whole-document extraction, however requires extra LLM calls. Use it selectively based mostly on doc size and significance.
• Graph context fixes weak embeddings. Entity sorts like IDs, dates, and numbers have poor semantic embeddings; enriching the vector search with graph-derived context is important for correct retrieval.
• Semantic node search is a strong fallback, to be exercised with warning. Even when Cypher queries fail (resulting from lacking relations), semantic matching can establish related nodes and shrink the search area reliably.
• Hybrid retrieval delivers correct response on relations, and not using a dense graph. Combining graph-based doc filtering with vector chunk retrieval permits correct solutions even when the graph lacks specific relations.

Conclusion

Constructing a GraphRAG system that’s each correct and cost-efficient requires acknowledging the sensible limitations of LLM-based graph development. Giant paperwork dilute consideration, entity extraction isn’t good, and encoding each relationship shortly turns into costly and brittle.

Nonetheless, as proven all through this text, we will obtain extremely correct retrieval and not using a totally detailed data graph. A easy graph construction—paired with iterative search-space optimization, semantic node search, and context-enriched vector retrieval—can outperform extra complicated and costly designs.

This strategy shifts the main focus from extracting every part upfront in a Graph to extracting what’s cost-effective, fast to extract and important, and let the retrieval pipeline fill the gaps. The pipeline balances performance, scalability and price, whereas nonetheless enabling subtle multi-hop queries throughout messy, real-world knowledge.

You possibly can learn extra in regards to the GraphRAG design rules underpinning the ideas demonstrated right here at Do You Really Need GraphRAG? A Practitioner’s Guide Beyond the Hype

Join with me and share your feedback at www.linkedin.com/in/partha-sarkar-lets-talk-AI

_{All pictures and knowledge used on this article are synthetically generated. Figures and code created by me}