diniscruz.ai / writing / Knowledge Graphs

Using LLMs as Ephemeral Graph Databases: Empowering the Graph Thinkers in the Age of Generative AI

By Dinis Cruz and ChatGPT Deep Research · · 28 min read

PDF LinkedIn post

Contents · 5 sections
  1. Introduction
  2. LLMs as Ephemeral Graph Databases: The Concept
  3. Step-by-Step Tutorial: Building a Risk Management Graph with an LLM
  4. Benefits of the LLM-as-Graph Approach
  5. Conclusion

Introduction

Graph thinkers – those who naturally conceptualize information as networks of nodes and relationships – are poised to benefit immensely from advances in generative AI. Traditional graph databases represent data as nodes and edges (with edges denoting relationships between nodes), enabling rich semantic queries. However, using a graph database typically requires technical setup, data modeling, and query languages like Cypher or SPARQL. What if graph thinkers with no coding skills could leverage AI to build and explore graphs on the fly, using natural language alone? This paper introduces the concept of using Large Language Models (LLMs) as ephemeral graph databases – harnessing an LLM’s ability to understand and manipulate structured information in a conversational context, without a persistent database.

In an ephemeral graph database, the knowledge graph is created dynamically and exists only for the duration of an AI session, serving as a temporary reasoning structure. The graph is fresh and transient, constructed at query-time to suit the user’s context. This is in contrast to persistent knowledge graphs that are pre-built and stored long-term. Recent research and tools show the power of dynamic knowledge graphs in AI reasoning. For example, the Knowledge Graph of Thoughts (KGoT) architecture dynamically builds a task-specific knowledge graph to help an AI agent solve complex problems more effectively. Our approach similarly creates a graph on-the-fly, but with a twist: the LLM itself acts as the graph engine, interpreting user instructions to create nodes, edges, and even perform queries or visualizations, all in natural language.

This white paper, co-authored by Dinis Cruz and ChatGPT, demonstrates step-by-step how an LLM can be used as a graph database in a practical scenario. We will walk through a real-world example in a tutorial style – modeling a cybersecurity risk management scenario – to showcase how a non-programmer “graph thinker” can create and query an ephemeral graph using only an LLM. Along the way, we discuss the principles, benefits, and limitations of this approach, and how it empowers users to reason about complex relationships without traditional database tools.

LLMs as Ephemeral Graph Databases: The Concept

What does it mean to use an LLM as a graph database? In essence, we are asking the LLM to play the role of a database that natively understands nodes and edges. The LLM takes in natural language instructions (prompts) that describe data (entities and their connections) and returns structured representations or answers, effectively simulating the behavior of a graph database. The term “ephemeral” highlights that the graph only persists within the LLM’s conversational context – there is no external storage of the data structure. If the session ends or the context is cleared, the “database” resets. However, within a single continuous session, the LLM’s memory (its context window) serves as the storage for nodes and relationships that have been described.

Traditional databases have well-defined operations (often summarized as ACID properties – Atomicity, Consistency, Isolation, Durability) to reliably store and retrieve data. In our LLM-driven approach, we won’t get full ACID guarantees, but we will mimic the basic CRUD operations of a database in conversation: we can Create nodes and edges, Read (retrieve or query) data from the graph, Update (transform) parts of the graph, and even Delete nodes or relationships if needed – all via prompting the LLM. We treat the LLM as a black box graph engine: we feed it inputs (graph data or queries in natural language) and receive outputs (new graph information, query results, or visualizations). There is no specialized graph query language required; the “query language” is plain English augmented with a structured style that we define through instructions.

Several key principles underlie using LLMs in this way:

In the following sections, we put these principles into practice. We will build an example graph step by step, demonstrating how a user can instruct the LLM to populate and query the graph. Our example will focus on a cybersecurity risk scenario. This scenario is chosen for its richness: it involves technical entities (systems, vulnerabilities), human entities (users, owners), policies/controls, and cascading impacts – an ideal playground for graph thinking. As we progress from simple to more complex graph operations, we encourage you to imagine how similar techniques could apply to other domains (such as project planning, knowledge management, or storytelling) where relationships between entities are key.

Before diving in, it's worth noting that this approach aligns with a broader trend of dynamic knowledge graph reasoning. Others in the AI community have explored building temporary graphs for on-demand reasoning. For instance, one approach constructs a new knowledge graph at query time, using it as a thinking tool to derive answers, and then discards it – calling this “ephemeral reasoning”. Our use of an LLM as an ephemeral graph database taps into the same idea: instead of retrieving facts from a static knowledge base, we are building a relevant knowledge graph on the fly with the help of the LLM, and using it to reason about a problem in a transparent, structured way.

With the concept established, let’s move on to the hands-on example. In a workshop setting, you could follow these steps interactively with an LLM (such as ChatGPT or similar) to recreate the experience.

Step-by-Step Tutorial: Building a Risk Management Graph with an LLM

Scenario: You are a risk manager in a cybersecurity context, and you want to model the relationships between a potential risk, its causes, its impacts, and the stakeholders involved. You will use an LLM to construct a graph representing this information, as if the LLM were a graph database you can query. We will start simple and gradually build a complex graph.

For clarity, we will show example user prompts (in bold) and the kind of LLM responses (in italic) one might get when following this process. Think of this as an interactive lab.

Step 1: Initializing the Graph Engine

First, we need to instruct the LLM to behave like a graph database. This typically involves a prompt that sets the context. For example:

User: 

You are a graph database that I can interact with using natural language. 

I will give you commands to create nodes and edges representing a knowledge graph. 

You will maintain a list of all nodes and edges. 

Only respond with the updated state or confirmation. Let’s start with an empty graph.

By giving a prompt like the above, we prime the LLM to take on the role. The LLM might respond with a confirmation that the graph is initialized:

LLM:

"Graph initialized. No nodes or edges currently exist."

Now we have a “blank slate” graph in the conversation. We can begin adding data to it.

Step 2: Creating Nodes (Entities)

We will add our first piece of information: a cybersecurity risk statement. Suppose the risk we want to model is: “User accounts can be compromised.” This will be one of our primary nodes (let's call it a Risk node). We notice this statement has a couple of meaningful parts: the concept of User (a type of actor) and Account Compromise (an event or outcome). It would be useful to have those as separate nodes so we can link them and add further detail later.

Let's create three nodes:

  1. The risk itself (as a text or issue node).
  2. A node for the actor User (type: Persona).
  3. A node for Account Compromise (type: Event or Outcome).

We instruct the LLM accordingly:

User: 

Create a node for the risk 'User accounts can be compromised'. 
Also create a node 'User' (type Persona) and a node 'Account Compromise' (type Event)

If the LLM follows correctly, it will acknowledge these nodes. For instance:

LLM: 

Nodes created: 
  (1) User accounts can be compromised  [Risk]
  (2) User  [Persona]
  (3) Account Compromise  [Event]

Now, if we ask the LLM to list all nodes, it should show those three:

User: "List all current nodes."

LLM:: 

  1. User accounts can be compromised; 
  2. User; 
  3. Account Compromise

At this stage, we have nodes but no relationships yet. It’s like having isolated points. The power of a graph comes from linking these nodes.

Step 3: Establishing Basic Relationships

We will now create edges (links) between the nodes to reflect their relationships. From the risk statement, “User accounts can be compromised”, we can infer:

So, we add two edges:

We prompt the LLM to add these edges:

User:

Create an edge from 'User accounts can be compromised' to 'User' labeled 'impacts'. 

Also create an edge from 'User accounts can be compromised' to 
'Account Compromise' labeled 'involves'.

The LLM should update its internal representation and confirm:

LLM: 

"Edges created: 
  'User accounts can be compromised' --(impacts)--> 'User'; 
  'User accounts can be compromised' --(involves)--> 'Account Compromise'

Now our tiny graph has three nodes and two edges. We can ask for a summary:

User: "Show me all nodes and their connections." LLM:

User accounts can be compromised (Risk)
  – impacts → User (Persona)
  – involves → Account Compromise (Event)

User – (no further connections)
Account Compromise – (no further connections)*"

(The above is a textual representation; an LLM could also output this in a structured form or even a quick ASCII diagram.)

Already, by treating the LLM as a graph engine, we’ve structured a basic risk statement into a graph format. In a conventional graph database, we would have to define these entities and relationships with database commands or code. Here it’s done conversationally.

Step 4: Expanding the Graph – Adding Impacts and Consequences

Next, we enrich the graph by thinking through impacts of an account compromise. In risk management, a single technical risk often has multiple layers of consequences:

Let’s add nodes for each of these impacts:

Now, we link these to reflect causality:

We provide these instructions to the LLM:

User: 

Add nodes:
    'Sensitive Data Exposed' (Impact). 
    'Breaks GDPR Compliance' (Compliance Impact). 
    'Financial Impact (~~ $500K)'. 
    'Company Performance Risk'. 

Now link them: 

    'Account Compromise'           --(leads to      )--> 'Sensitive Data Exposed'; 
    'Sensitive Data Exposed'       --(causes        )--> 'Breaks GDPR Compliance'; 
    'Sensitive Data Exposed'       --(results in    )--> 'Financial Impact (~~ $500K)'; 
    'Breaks GDPR Compliance'       --(results in    )--> 'Financial Impact (~~ $500K)';
     'Financial Impact (~~ $500K)' --(contributes to)--> 'Company Performance Risk'."

This is a lot to add in one go, and a good LLM will handle it stepwise. The LLM might respond with confirmation or a summary of new nodes/edges:

LLM: 

Nodes created: 
  Sensitive Data Exposed; 
  Breaks GDPR Compliance; 
  Financial Impact (~ $500K); Company Performance Risk.

Edges created:

* Account Compromise         --(leads to      )--> Sensitive Data Exposed
* Sensitive Data Exposed     --(causes        )--> Breaks GDPR Compliance
* Sensitive Data Exposed     --(results in    )--> Financial Impact (~ $500K)
* Breaks GDPR Compliance     --(results in    )--> Financial Impact (~ $500K)
* Financial Impact (~ $500K) --(contributes to)--> Company Performance Risk

Now our graph has grown significantly. Let’s interpret what we have constructed. Starting from the original risk node, one can traverse a chain of reasoning:

Through this graph, a graph thinker can see a clear lineage from a technical risk at the IT level all the way to the strategic risk at the business level. This is powerful for explaining why cybersecurity issues matter in business terms. In a typical risk register (often just a spreadsheet), these relationships are not so explicit. But our graph, created with minimal effort via the LLM, makes the dependency chain transparent.

We can ask the LLM to visualize or summarize this subgraph. One simple way is to request an ASCII-art tree or outline:

User: "Show the risk 'User accounts can be compromised' and all downstream impacts as a hierarchy."

LLM: 

User accounts can be compromised:

  → Account Compromise (event)
      → Sensitive Data Exposed (impact)
          → Breaks GDPR Compliance (compliance impact)
          → Financial Impact (~ $500K) (business impact)
              → Company Performance Risk (strategic risk)

(In the above, indentation shows layers of consequence. Both “Breaks GDPR Compliance” and “Financial Impact” are shown under “Sensitive Data Exposed” as parallel results. “Financial Impact” then leads to “Company Performance Risk.”)

At this point, we have primarily built out the impact side of the risk graph (what happens if the risk materializes). Next, we will build out the cause and prevention side – what could lead to the risk and what controls exist or fail.

Step 5: Incorporating Causes and Preventive Controls

Understanding why a risk might materialize is as important as knowing its impact. For “User accounts can be compromised,” consider the typical causes:

  1. Credentials are compromised – e.g., a user's password was leaked or guessed.
  2. No Multi-Factor Authentication (MFA) – without a second factor, a leaked password is sufficient for an attacker to gain access.
  3. Security Monitoring Failure – the compromise wasn’t quickly detected by security systems (like a Security Information and Event Management system, SIEM), allowing the attacker to persist.

These can be modeled as contributing factor nodes. We will create:

We then link these to the core event Account Compromise as prerequisites or contributing factors. We might use a relationship like “allows” or “contributes to”:

In reality, all three cause factors might need to happen (in combination) for a full breach scenario. We won’t delve into logic gates (AND/OR) here, but the graph implicitly shows that if any of these causes are mitigated, the risk is reduced.

Let’s add these causes via the LLM:

User: 

Add causes: 
create nodes 'Credentials Leaked' (Cause), 
             'No MFA Enabled' (Cause),
             'Threat Not Detected' (Cause). 

Link each to 'Account Compromise':

    'Credentials Leaked'  --(contributes to)--> 'Account Compromise'; 
    'No MFA Enabled'      --(allows        )--> 'Account Compromise'; 
    'Threat Not Detected' --(aggravates    )--> 'Account Compromise'

After confirming nodes and edges, our graph now also has the upstream side of the risk:

Now think of controls or policies that correspond to each cause:

We will add control nodes and link them as mitigating factors for each cause:

Furthermore, these policies often come from standards or frameworks. For instance, an organization might be adhering to ISO 27001 or NIST standards, which have specific controls for passwords and MFA. We can represent a snippet of a standards hierarchy:

We might also add a high-level risk “Non-compliance with security standards”, which occurs if we fail to implement required controls. This could tie back into our compliance risk node or directly to Company Performance Risk (since failing audits has business impact too).

For brevity, let’s illustrate adding one standard and a couple of controls:

User: 

Create nodes:
    'Password Policy' (Control), 
    'MFA Policy' (Control), 
    'Security Monitoring Procedure' (Control). 

Link nodes:
    'Password Policy'               --(mitigates)--> 'Credentials Leaked'; 
    'MFA Policy'                    --(mitigates)--> 'No MFA Enabled'; 
    'Security Monitoring Procedure' --(mitigates)--> 'Threat Not Detected'. 

Now create node 'ISO 27001 Standard' and link it: 
    'ISO 27001 Standard' --(includes control)--> 'Password Policy'; 
    'ISO 27001 Standard' --(includes control)--> 'MFA Policy'; 
    'ISO 27001 Standard' --(includes control)--> 'Security Monitoring Procedure'

The LLM updates the graph accordingly. We won’t list every edge textually here, but conceptually, we’ve created a cause-and-control subgraph that mirrors the defense side of risk management. If we look at the whole picture now, Account Compromise sits in the middle:

This graph is now quite complex, but the LLM can handle it as long as it’s within its context memory. We may at this point ask for a summary or diagram to ensure everything is captured. For example, we could ask for a mermaid diagram (a popular text-based graph description that can be rendered visually):

User: "Provide a mermaid diagram code for the entire graph."

LLM:

"Here is the mermaid code:"

graph TD
    R["User accounts can be compromised"] --> U["User"]
    R --> E["Account Compromise"]
    E --> I1["Sensitive Data Exposed"]
    I1 --> C1["Breaks GDPR Compliance"]
    I1 --> B["Financial Impact (~ $500K)"]
    C1 --> B
    B --> TR["Company Performance Risk"]

    L["Credentials Leaked"] -- contributes --> E
    M["No MFA Enabled"] -- allows --> E
    D["Threat Not Detected"] -- aggravates --> E

    P["Password Policy"] -- mitigates --> L
    F["MFA Policy"] -- mitigates --> M
    X["Monitoring Procedure"] -- mitigates --> D

    S["ISO 27001 Standard"] --> P
    S --> F
    S --> X

"Which looks like this"

graph TD
    R["User accounts can be compromised"] --> U["User"]
    R --> E["Account Compromise"]
    E --> I1["Sensitive Data Exposed"]
    I1 --> C1["Breaks GDPR Compliance"]
    I1 --> B["Financial Impact (~ $500K)"]
    C1 --> B
    B --> TR["Company Performance Risk"]

    L["Credentials Leaked"] -- contributes --> E
    M["No MFA Enabled"] -- allows --> E
    D["Threat Not Detected"] -- aggravates --> E

    P["Password Policy"] -- mitigates --> L
    F["MFA Policy"] -- mitigates --> M
    X["Monitoring Procedure"] -- mitigates --> D

    S["ISO 27001 Standard"] --> P
    S --> F
    S --> X

(The above is an example of mermaid syntax that the LLM might produce. If rendered, it would show a full graph with nodes and directed edges as described.)

Such a diagram code can be copy-pasted into a mermaid renderer to visualize the graph. In a live workshop, one could actually have the LLM render this to an image if the environment allows, but the text itself is a clear specification of the graph.

Step 6: Adding Assets and Stakeholders

Thus far, our graph covers the abstract risk and controls. Now we ground it in the real world context: which systems and people are involved?

Imagine our organization has two systems relevant to this risk:

Both systems have user accounts that could be compromised, but they may have different security postures:

Also, consider who “owns” these systems:

We will add:

Now link assets to the controls they implement or lack:

Also link assets to the risk event if applicable:

Next, link roles to assets:

Link senior roles:

We won’t go further up to CEO for now.

We instruct the LLM to add these:

User: 

Add asset nodes 'HR System' and 'Marketing System'. 

Add role nodes 'HR Manager', 'Marketing Manager', 'Chief People Officer', 'Chief Marketing Officer'. 

Link assets to controls and risk: 
    'HR System'         --(implements)--> 'Password Policy'; 
    'HR System'         --(implements)--> 'MFA Policy'; 
    'Marketing System'  --(implements)--> 'Password Policy';  
        (Note: Marketing System lacks MFA). 

Link assets to risk: 
    'HR System'        --(at risk of)--> 'Account Compromise'; 
    'Marketing System' --(at risk of)--> 'Account Compromise'. 

Link ownership: 

    'HR Manager'        --(owns)--> 'HR System'; 
    'Marketing Manager' --(owns)--> 'Marketing System'. 

Link reporting: 
    'HR Manager'        --(reports to)--> 'Chief People Officer'; 
    'Marketing Manager' --(reports to)--> 'Chief Marketing Officer'."

The LLM adds these nodes and edges. Now our graph connects technology to people:

This sets the stage for a scenario where an incident occurs. We have all the pieces: the risk, causes, controls, assets, and stakeholders. We can now simulate an incident and see how the graph helps us respond.

Step 7: Simulating an Incident and Querying the Graph

The Incident: Let’s say a breach monitoring service alerts us that a large dump of usernames and passwords from our company has appeared on the internet. This is a credential leak event. We identify that among the leaked credentials, there are accounts belonging to users of both the HR System and the Marketing System. For example, five accounts from each system were found in the leak (10 accounts total).

We will add an Incident node to the graph:

Link it to the assets it affects:

Also link it to the risk or event:

And possibly link it to a response:

In practice, once this incident is in the graph, we can query the graph to answer critical questions:

Let’s add the incident to the graph via LLM:

User: 

"Add incident node 'Credential Leak (June 2025)' and link it: 
    'Credential Leak (June 2025)' --(affects 5 accounts in)--> 'HR System';
    'Credential Leak (June 2025)' --(affects 5 accounts in)--> 'Marketing System';
    'Credential Leak (June 2025)' --(triggers             )--> 'Account Compromise'

After updating, we can interrogate the graph by asking the LLM questions in plain language, since the graph is now embedded in its context. For example:

The above Q&A shows how the LLM can use the graph we built to generate useful outputs. It’s effectively performing graph queries under the hood by following the relationships we established, and then articulating the results in a human-friendly manner. This is a major advantage of using an LLM as the graph database: we can get narrative answers and contextual recommendations, not just data dumps.

Finally, the LLM could even produce a summary report of the whole scenario, combining everything: - a description of the risk, - which assets are affected, - what the causes and impacts are, - and what actions are needed.

Such a report is essentially a traversal of the graph formatted as a report – something a human analyst would normally do manually by looking at various data sources.

Benefits of the LLM-as-Graph Approach

Using LLMs as ephemeral graph databases offers several compelling benefits for graph thinkers and domain experts:

However, it’s important to also understand the limitations and considerations:

Despite these challenges, the demonstration shows a tantalizing possibility: empowering domain experts to directly build and query knowledge graphs through conversation. By using LLMs in this way, graph thinking can become a more widespread practice, applied in daily decision making, incident response, strategic planning, and beyond.

Conclusion

In the age of generative AI, we are witnessing new ways to interact with and organize knowledge. Using LLMs as ephemeral graph databases is one such innovation – it merges the structured clarity of graph models with the ease and intelligence of conversational AI. This approach effectively turns an LLM into a “graph brain” for the user, capable of not just remembering facts but also drawing connections and reasoning about them in context.

For graph thinkers, this is an empowering development. No longer constrained by needing technical tools or team support to set up a graph database, an individual can think out loud with an LLM and see their mental model take shape as a graph in real time. The white paper’s example walked through a cybersecurity risk scenario, but the template is general. Imagine using it to map out a medical diagnosis (symptoms→conditions→tests→treatments), a legal case (evidence→claims→laws→outcomes), or a novel’s plot and characters. The patterns of nodes and edges are everywhere, and an LLM can help weave them together.

In our example, Dinis Cruz provided the vision of empowering non-coders to harness graph thinking, and this collaboration with ChatGPT Deep Research has fleshed out how that vision can be implemented step-by-step. The result is a blueprint for a workshop or a self-guided exercise where anyone can try turning an LLM into their personal graph database for a while.

Moving forward, we anticipate more synergy between LLMs and graph technologies:

The key takeaway is that structured thinking and natural language need not be at odds. With the right prompts, an AI can be both a conversational partner and a structured data tool. Ephemeral graph databases via LLMs invite us to explore complex problems with the freedom to create, modify, and reason without upfront investment in software – the only requirement is our curiosity and clarity of thought (expressed in words).

As generative AI continues to evolve, those who think in graphs have a powerful ally at their disposal. By engaging with LLMs in this manner, we make our thinking visible and malleable. In doing so, we not only communicate with the machine, but also better explain to ourselves the intricate web of cause and effect, risk and reward, action and outcome that defines the problems we tackle. This, ultimately, is the promise of empowering the graph thinkers in the age of generative AI – turning ideas into interconnected knowledge, one prompt at a time.

Sources:

Released under CC BY 4.0. First published on docs.diniscruz.ai; this page as markdown.

← Empowering Workshops with Custom GPTs for GenAI TrainingFIST Meets the Semantic Knowledge Graph: Aligning Fast, Inexpensive, Simple, Tiny with Dinis Cruz's G³ Approach →