GraphRAG – how agents can leverage structured knowledge and stay on track: #AI for dummies #6

Today we’ll deviate a bit from our applied AI and machine learning track to talk about agentic workloads. It’s on everyone’s minds and everyone has a different idea on how to do it. From Prompt Engineering, Loop Engineering, Harness engineering, to whatever the hype and buzzword is this week, we engineers get a feeling of always being behind. I know I do, at least.

But some of these “new” concepts are actually worth paying attention too. We’ve talked about embeddings, which are commonly used in RAG workloads here, but embeddings and RAG have a few limitations (although not useless at all), when compared to this “new” concept that is GraphRAG Let’s have a look at some Standard RAG vs. GraphRAG Trade-offs:

Topic	Standard RAG	GraphRAG
Primary Strength	Fast point-lookup of specific facts	Global synthesis & multi-hop reasoning
Ingestion Cost	Extremely low (vector embeddings)	High (LLM-driven entity/relation extraction)
Query Latency	Very fast (100–300 ms)	Slower (1–5+ seconds depending on graph depth)
Data Schema	Flat chunk collections	Complex hierarchical knowledge graph
Best Used For	Specific factual Q&A, localized search	Summarization, trend analysis, interconnected data

in short, use GraphRAG for Analytical, relational summary questions (“How do these 10 events connect?” or “What are the recurring issues across all customer support tickets?”).

And use Standard Vector RAG for: Short-form, direct lookups (“What is the refund policy?” or “What is the step-by-step setup guide?”).

But what is GraphRAG?

Let’s use a simple analogy that most developers can relate to:

• Traditional Vector RAG: Imagine having thousands of isolated isolated merge requests comments. When you ask, “What requirement lead to this architecture decision”, vector search finds folders that mention “architecture”, “decision”, or “requirement”, among other specific keywords. But if there are similar requirements and decision, vector search alone won’t connect those dots because no document may explicitly mention the link between requirement and decision.
• Knowledge Graph: This is similar to a classic detective board on the wall. Pins represent entities (Features, Decisions, Requirements, commits) and strings represent relationships (GENERATED_BY, LINKED_TO, COMMITED_BY).
• GraphRAG : uses vector similarity to find the most relevant “pins” on the board based on your question, and then instantly follows the red strings (graph edges) to extract the full connected context before handing it to the LLM.

So both are useful and can complement each other.

Let’s see this in practice

Let’s create a simple hands-on proof of concept. We’ll use PostgreSQL supported by two extensions: PGVector (which we saw earlier here for embedding support, and AGE, a “A Graph Extension”, which, as the name suggests, provides support for storing graphs within PostgreSQL.

Note: there are alternatives, namely Neo4J, which has great support for graph and vector semantic storage and retrieval. But we’ll stay Open Source for this article.

Our companion GitHub repo can be found here.

The structure of our project is simple:

graphrag/
├── Dockerfile
├── docker-compose.yml
├── init.sql
├── requirements.txt
└── demo.py

As usual, we’ll use docker and docker compose to provision our infrastructure. In this case we have a combination of AGE and PGVector which doesn’t have (at the time of writing this article), a public consolidated docker image. But fear not, we’ll create our own:

Custom PG docker file

# Dockerfile
FROM postgres:16

# Install build dependencies
RUN apt-get update && apt-get install -y --no-install-recommends     build-essential     postgresql-server-dev-16     git     bison     flex     ca-certificates     && rm -rf /var/lib/apt/lists/*

# Install pgvector (v0.7.4)
RUN git clone --branch v0.7.4 https://github.com/pgvector/pgvector.git /tmp/pgvector     && make -C /tmp/pgvector     && make -C /tmp/pgvector install     && rm -rf /tmp/pgvector

# Install Apache AGE (PG16 compatible release)
RUN git clone --branch PG16/v1.5.0-rc0 https://github.com/apache/age.git /tmp/age     && make -C /tmp/age PG_CONFIG=/usr/lib/postgresql/16/bin/pg_config install     && rm -rf /tmp/age

# Copy initialization script
COPY init.sql /docker-entrypoint-initdb.d/01-init.sql

Initialization SQL script

Then we need to inintialize our database, and since we’re not running migrations in our small PoC, we have an init.sql script that is executed on container creation:

-- init.sql
-- Enable required extensions
CREATE EXTENSION IF NOT EXISTS age;
CREATE EXTENSION IF NOT EXISTS vector;

-- Set database-level search_path so ag_catalog is available globally across client sessions
ALTER DATABASE graphrag_db SET search_path = ag_catalog, "$user", public;
SET search_path = ag_catalog, "$user", public;

-- Load Apache AGE module into initialization session
LOAD 'age';

-- Initialize Knowledge Graph
SELECT create_graph('tech_graph');

-- Table storing vector embeddings for entities (using 384 dimensions for all-MiniLM-L6-v2)
CREATE TABLE IF NOT EXISTS entity_embeddings (
    id SERIAL PRIMARY KEY,
    entity_name VARCHAR(255) UNIQUE NOT NULL,
    entity_type VARCHAR(100) NOT NULL,
    description TEXT NOT NULL,
    embedding vector(384)
);

-- Create HNSW Index for fast Cosine Distance vector queries
CREATE INDEX IF NOT EXISTS idx_entity_embeddings_cosine 
ON entity_embeddings USING hnsw (embedding vector_cosine_ops);

This SQL script does quite a few things:

1. Extension Activation

• CREATE EXTENSION IF NOT EXISTS age;: Installs Apache AGE, adding graph database capabilities and Cypher query support directly into PostgreSQL.
• CREATE EXTENSION IF NOT EXISTS vector;: Installs pgvector, introducing vector data types and similarity search operators to standard SQL queries.

2. Schema Search Path & Module Loading

• ALTER DATABASE graphrag_db SET search_path = ag_catalog, "$user", public;: Updates the database defaults so any future database connection automatically looks in ag_catalog first. Apache AGE keeps all its internal tables, types, and Cypher functions inside ag_catalog.
• SET search_path = ag_catalog, "$user", public;: Applies the search path change immediately to the active initialization session.
• LOAD 'age';: Explicitly loads the Apache AGE shared library module into the current connection memory so its execution hooks and Cypher functions become active.

3. Knowledge Graph Setup

• SELECT create_graph('tech_graph');: Calls the Apache AGE management function to create an isolated graph namespace named tech_graph, which will hold all nodes (entities) and edges (relationships).

4. Entity Embedding Table Schema

• CREATE TABLE IF NOT EXISTS entity_embeddings (...): Creates a standard relational table to store metadata for system entities (like microservices, databases, or APIs).
• embedding vector(384): Defines a specialized pgvector column set to 384 dimensions, which matches the vector output size of lightweight models like all-MiniLM-L6-v2. (which we’ll be using)

5. Vector Indexing

• CREATE INDEX ... USING hnsw (embedding vector_cosine_ops);: Builds an HNSW (Hierarchical Navigable Small World) graph index on the vector column configured for Cosine Distance (vector_cosine_ops). This avoids slow full-table scans, enabling sub-millisecond Approximate Nearest Neighbor (ANN) searches during the vector retrieval stage.

The Docker Compose (docker-compose.yml)

Notice that we instruct PostgreSQL to load age in shared_preload_libraries. This ensures Apache AGE background components and execution hooks initialize properly on server startup.

# docker-compose.yml
services:
  graphrag-db:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: graphrag_postgres
    command: postgres -c shared_preload_libraries=age
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgrespassword
      POSTGRES_DB: graphrag_db
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d graphrag_db"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  pgdata:

Python Dependencies (requirements.txt)

psycopg2-binary>=2.9.10
pgvector==0.3.2
sentence-transformers==3.0.1
numpy==1.26.4

Let’s now create our infra:

docker compose up -d --build

The Python GraphRAG Pipeline

Now for the fun part!

We’ll write our python logic in thedemo.pyfile, which populates a microservice architecture knowledge graph into Apache AGE, computes vector embeddings for each entity using sentence-transformers, and executes a two-stage GraphRAG retrieval pipeline.

Wow, that’s a lot of buzzwords. But we’ll cover them one by one, and by the end of this article they’ll be common terminology.

Note: To keep Cypher executions injection-safe and prevent string-formatting issues inside PostgreSQL $$ dollar-quoted blocks, we use helper functions that safely escape parameters before running queries.

# demo.py
import psycopg2
from pgvector.psycopg2 import register_vector
from sentence_transformers import SentenceTransformer
import time

# 1. Initialize local embedding model (384 dimensions, fast & lightweight)
print("Loading embedding model (all-MiniLM-L6-v2)...")
embedder = SentenceTransformer("all-MiniLM-L6-v2") # use device="cpu" if like me no grahics card is available

def get_db_connection():
    max_retries = 10
    for i in range(max_retries):
        try:
            conn = psycopg2.connect(
                dbname="graphrag_db",
                user="postgres",
                password="postgrespassword",
                host="localhost",
                port="5432"
            )
            conn.autocommit = True
            register_vector(conn)
            return conn
        except Exception as e:
            if i == max_retries - 1:
                raise e
            print(f"Waiting for database connection... ({i+1}/{max_retries})")
            time.sleep(2)

def escape_cypher_str(val: str) -> str:
    # Escapes single quotes for insertion into Cypher string literals
    return val.replace("'", "\'")

def setup_graph_and_vector_data(conn):
    cur = conn.cursor()
    cur.execute("LOAD 'age';")
    cur.execute("SET search_path = ag_catalog, '$user', public;")
    
    print("\n--- 1. Populating Graph Data via Apache AGE (Cypher) ---")
    
    try:
        cur.execute("SELECT drop_graph('tech_graph', true);")
    except Exception:
        pass
    cur.execute("SELECT create_graph('tech_graph');")
    
    # Define Entities (Nodes)
    entities = [
        {"name": "AuthService", "type": "Microservice", "desc": "Handles JWT authentication, password verification, and token issuance."},
        {"name": "UserService", "type": "Microservice", "desc": "Manages user profiles, preferences, and account metadata."},
        {"name": "PaymentService", "type": "Microservice", "desc": "Processes customer credit card payments and subscription billing."},
        {"name": "UserDB", "type": "Database", "desc": "PostgreSQL database storing encrypted user credentials and account profiles."},
        {"name": "PaymentGateway", "type": "ExternalAPI", "desc": "Third-party Stripe REST API integration for credit card processing."},
        {"name": "KafkaBroker", "type": "EventBus", "desc": "Apache Kafka message broker for asynchronous event processing."}
    ]
    
    for entity in entities:
        name_esc = escape_cypher_str(entity['name'])
        type_esc = escape_cypher_str(entity['type'])
        desc_esc = escape_cypher_str(entity['desc'])
        
        cypher = f'''
        SELECT * FROM cypher('tech_graph', $$
            CREATE (n:{type_esc} {{name: '{name_esc}', description: '{desc_esc}'}})
            RETURN n
        $$) as (n agtype);
        '''
        cur.execute(cypher)
        print(f"Created Node: {entity['name']} ({entity['type']})")

    # Define Relationships (Edges)
    relationships = [
        ("UserService", "DEPENDS_ON", "UserDB"),
        ("AuthService", "READS_FROM", "UserDB"),
        ("AuthService", "COMMUNICATES_WITH", "UserService"),
        ("PaymentService", "CALLS", "PaymentGateway"),
        ("PaymentService", "PUBLISHES_TO", "KafkaBroker"),
        ("UserService", "SUBSCRIBES_TO", "KafkaBroker")
    ]
    
    for src, rel, target in relationships:
        src_esc = escape_cypher_str(src)
        rel_esc = escape_cypher_str(rel)
        target_esc = escape_cypher_str(target)
        
        cypher = f'''
        SELECT * FROM cypher('tech_graph', $$
            MATCH (a {{name: '{src_esc}'}}), (b {{name: '{target_esc}'}})
            CREATE (a)-[r:{rel_esc}]->(b)
            RETURN r
        $$) as (r agtype);
        '''
        cur.execute(cypher)
        print(f"Created Relationship: ({src}) -[:{rel}]-> ({target})")

    print("\n--- 2. Populating Vector Embeddings via pgvector ---")
    cur.execute("TRUNCATE TABLE entity_embeddings;")
    
    for entity in entities:
        vector = embedder.encode(entity['desc']).tolist()
        cur.execute(
            '''
            INSERT INTO entity_embeddings (entity_name, entity_type, description, embedding)
            VALUES (%s, %s, %s, %s)
            ON CONFLICT (entity_name) DO UPDATE SET embedding = EXCLUDED.embedding;
            ''',
            (entity['name'], entity['type'], entity['desc'], vector)
        )
        print(f"Stored Vector Embedding for: {entity['name']}")

def run_graphrag_pipeline(conn, user_query):
    cur = conn.cursor()
    cur.execute("LOAD 'age';")
    cur.execute("SET search_path = ag_catalog, '$user', public;")

    print(f"\n=======================================================")
    print(f"USER QUERY: \"{user_query}\"")
    print(f"=======================================================")

    # Step 1: Vector Search (Find Top-K Seed Entities via pgvector using Cosine Distance <=> operator)
    query_vector = embedder.encode(user_query).tolist()
    cur.execute(
        '''
        SELECT entity_name, entity_type, description, (embedding <=> %s::vector) AS cosine_distance
        FROM entity_embeddings
        ORDER BY cosine_distance ASC
        LIMIT 2;
        ''',
        (query_vector,)
    )
    seed_entities = cur.fetchall()

    print("\n[STEP 1: Vector Search via pgvector (Cosine Distance)]")
    for name, etype, desc, dist in seed_entities:
        print(f" -> Found Seed Entity: {name} (Type: {etype}) | Cosine Distance: {dist:.4f}")

    # Step 2: Graph Traversal (Retrieve Topological Connections via Apache AGE Cypher)
    print("\n[STEP 2: Graph Traversal via Apache AGE (Cypher)]")
    retrieved_graph_context = []

    for name, etype, desc, dist in seed_entities:
        name_esc = escape_cypher_str(name)
        # Query incoming and outgoing relationships to capture full topological neighborhood
        cypher_query = f'''
        SELECT * FROM cypher('tech_graph', $$
            MATCH (start {{name: '{name_esc}'}})-[r]-(connected)
            MATCH (source)-[r]->(target)
            RETURN source.name, type(r), target.name
        $$) as (source agtype, rel agtype, target agtype);
        '''
        cur.execute(cypher_query)
        edges = cur.fetchall()
        
        for src_node, edge_rel, tgt_node in edges:
            src_clean = str(src_node).replace('"', '')
            rel_clean = str(edge_rel).replace('"', '')
            tgt_clean = str(tgt_node).replace('"', '')
            edge_str = f"({src_clean}) -[{rel_clean}]-> ({tgt_clean})"
            if edge_str not in retrieved_graph_context:
                retrieved_graph_context.append(edge_str)
                print(f" -> Discovered Graph Connection: {edge_str}")

    # Step 3: Synthesize Augmented GraphRAG Context
    print("\n[STEP 3: Augmented GraphRAG Context for LLM Prompt]")
    prompt_context = "=== GRAPH-RAG AUGMENTED CONTEXT ===\n"
    prompt_context += "Identified Key Entities:\n"
    for name, etype, desc, dist in seed_entities:
        prompt_context += f"- {name} ({etype}): {desc}\n"
    
    prompt_context += "\nKnowledge Graph Topology & Relationships:\n"
    for edge in retrieved_graph_context:
        prompt_context += f"- {edge}\n"
    prompt_context += "==================================="

    print(prompt_context)

if __name__ == "__main__":
    connection = get_db_connection()
    setup_graph_and_vector_data(connection)
    
    # Execute query
    user_question = "Where are user credentials stored and which services access or manage user data?"
    run_graphrag_pipeline(connection, user_question)
    
    connection.close()

To run this, create and activate a virtual environment, then run the python file:

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

The code is commented and meant to be self explainatory, but here’s a summary of what is happening:

The script executes an end-to-end GraphRAG pipeline in PostgreSQL by combining vector similarity search with Cypher graph traversal.

1. Model Initialization & Connection Management

• Embedding Model Loading: Loads all-MiniLM-L6-v2 via sentence-transformers to generate 384-dimensional dense vector embeddings locally.
• Database Connection Retry Loop (get_db_connection): Handles connection logic to graphrag_db with a 10-try fallback loop, enabling auto-commit mode and calling register_vector(conn) so psycopg2 natively converts Python lists to PostgreSQL vector types.

2. Query Sanitization & Security

• Cypher Escaping (escape_cypher_str): Replaces single quotes (') with escaped quotes (\') to prevent string-formatting errors and injection vulnerabilities when interpolating dynamic Python strings inside PostgreSQL $$ dollar-quoted Cypher blocks.

3. Graph & Vector Data Population (setup_graph_and_vector_data)

• Graph Reset & Setup: Drops any existing tech_graph and re-initializes a clean graph using Apache AGE functions (drop_graph, create_graph).
• Node & Edge Ingestion: Runs dynamic Cypher queries to instantiate nodes (CREATE (n:Type ...)) representing microservices, databases, and APIs, as well as directed edges (CREATE (a)-[r:REL]->(b)) representing systemic relationships (DEPENDS_ON, READS_FROM, COMMUNICATES_WITH).
• Vector Embeddings Storage: Encodes each entity’s text description into a vector embedding and executes an INSERT ... ON CONFLICT DO UPDATE query against the entity_embeddings table in pgvector.

4. Hybrid GraphRAG Retrieval Pipeline (run_graphrag_pipeline)

• Step 1: Vector Search (pgvector): Embeds the user’s input query and executes a vector similarity query using the cosine distance operator (<=>). It extracts the top 2 closest matching “seed” entities (e.g., UserDB and UserService).
• Step 2: Graph Traversal (Apache AGE): Passes the identified seed entities into Apache AGE Cypher queries (MATCH (start)-[r]-(connected)). It traverses incoming and outgoing relationships to extract structural neighborhood connections that vector search alone would miss (e.g., discovering AuthService -[READS_FROM]-> UserDB).
• Step 3: Context Augmentation: Formats the extracted seed entities and graph edges into a structured text context payload (=== GRAPH-RAG AUGMENTED CONTEXT ===) ready to be passed directly into an LLM generation prompt.

The output shall be similar to:

-- $ python demo.py 

Loading embedding model (all-MiniLM-L6-v2)...

--- 1. Populating Graph Data via Apache AGE (Cypher) ---
Created Node: AuthService (Microservice)
Created Node: UserService (Microservice)
Created Node: PaymentService (Microservice)
Created Node: UserDB (Database)
Created Node: PaymentGateway (ExternalAPI)
Created Node: KafkaBroker (EventBus)
Created Relationship: (UserService) -[:DEPENDS_ON]-> (UserDB)
Created Relationship: (AuthService) -[:READS_FROM]-> (UserDB)
Created Relationship: (AuthService) -[:COMMUNICATES_WITH]-> (UserService)
Created Relationship: (PaymentService) -[:CALLS]-> (PaymentGateway)
Created Relationship: (PaymentService) -[:PUBLISHES_TO]-> (KafkaBroker)
Created Relationship: (UserService) -[:SUBSCRIBES_TO]-> (KafkaBroker)

--- 2. Populating Vector Embeddings via pgvector ---
Stored Vector Embedding for: AuthService
Stored Vector Embedding for: UserService
Stored Vector Embedding for: PaymentService
Stored Vector Embedding for: UserDB
Stored Vector Embedding for: PaymentGateway
Stored Vector Embedding for: KafkaBroker

=======================================================
USER QUERY: "Where are user credentials stored and which services access or manage user data?"
=======================================================

[STEP 1: Vector Search via pgvector]
 -> Found Seed Entity: UserService (Type: Microservice) | Distance: 0.9018
 -> Found Seed Entity: UserDB (Type: Database) | Distance: 0.9732

[STEP 2: Graph Traversal via Apache AGE (Cypher)]
 -> Discovered Graph Connection: (UserService) -[SUBSCRIBES_TO]- (KafkaBroker)
 -> Discovered Graph Connection: (UserService) -[COMMUNICATES_WITH]- (AuthService)
 -> Discovered Graph Connection: (UserService) -[DEPENDS_ON]- (UserDB)
 -> Discovered Graph Connection: (UserDB) -[READS_FROM]- (AuthService)
 -> Discovered Graph Connection: (UserDB) -[DEPENDS_ON]- (UserService)

[STEP 3: Augmented GraphRAG Context for LLM Prompt]
=== GRAPH-RAG AUGMENTED CONTEXT ===
Identified Key Entities:
- UserService (Microservice): Manages user profiles, preferences, and account metadata.
- UserDB (Database): PostgreSQL database storing encrypted user credentials and account profiles.

Knowledge Graph Topology & Relationships:
- (UserService) -[SUBSCRIBES_TO]- (KafkaBroker)
- (UserService) -[COMMUNICATES_WITH]- (AuthService)
- (UserService) -[DEPENDS_ON]- (UserDB)
- (UserDB) -[READS_FROM]- (AuthService)
- (UserDB) -[DEPENDS_ON]- (UserService)
===================================

Notice how pgvector correctly identified UserDB and UserService based on semantic cosine similarity. Then Apache AGE traversed the graph to discover that AuthService also reads from UserDB and communicates with UserService

A pure vector search would have completely missed AuthService because its text description didn’t match the query string as closely.

Under the Hood: Traps & Pitfalls to Watch Out For

(To be honest, when I first started combining Apache AGE with standard SQL queries, I ran into a few annoying gotchas. Here are three specific pitfalls you need to keep in mind…)

1. SQL Dollar-Quoting & Parameter Escaping in Cypher Queries

Apache AGE wraps Cypher queries inside PostgreSQL functions using dollar quotes ($$ ... $$). If your Cypher string contains unescaped single quotes or user inputs, string formatting can easily break or expose Cypher/SQL syntax errors.

TIP: Always sanitize and escape string variables (e.g. escaping single quotes ' to \') or use helper wrappers when building Cypher strings dynamically inside $$ blocks.

2. Handling agtype Strings in Python

Apache AGE returns Cypher query columns formatted as custom PostgreSQL agtype data types. When psycopg2 fetches agtype values, string primitives often include surrounding JSON quotes ("UserDB"). Be sure to strip or parse them into standard Python data types before constructing your prompt templates!

Always bound your hop counts explicitly in production: MATCH (a)-[*1..2]->(b).

3. Hop Depth Explosions (The Unbounded Traversal Anti-Pattern)

In Cypher, writing an open-ended path match like MATCH (a)-[*]->(b) will cause Apache AGE to traverse your entire database recursively. For large knowledge graphs, this will lock up your PostgreSQL CPU.

Production Reality Check & Security

Why use PostgreSQL for GraphRAG instead of running dedicated graph databases like Neo4j alongside vector databases like Pinecone or Qdrant?

1. Zero Multi-DB Synchronization Bugs: In dual-database architectures, keeping graph nodes in sync with vector collections when records are updated or deleted is notoriously prone to race conditions. With PostgreSQL, graph nodes, relational metadata, and vector embeddings reside in the same transaction.
2. Unified Backup & ACID Reliability: One pg_dump backs up your relational data, vector index, and knowledge graph simultaneously.
3. Simpler Infrastructure Footprint: You only manage, monitor, and scale a single database engine in production.

In short

GraphRAG bridges the gap between semantic relevance and structural truth. By pairing pgvector for vector search with Apache AGE for Cypher graph queries inside PostgreSQL, you get an incredibly versatile GraphRAG engine without adding complex multi-database infrastructure to your stack.

Hope you find this useful as I did and are able to apply some of it.

Cheers! 🙂