diniscruz.ai / writing / Development and GenAI

LLM Workflows/Stateflow Service - Technical Brief

By Dinis Cruz and ChatGPT Deep Research · · 35 min read

PDF

Contents · 7 sections
  1. Introduction
  2. Design Principles and Goals
  3. Service Architecture and Components
  4. Workflow Definition and Standards
  5. Example Workflow: Persona-Based Communication and Evaluation
  6. Additional Workflow Examples
  7. Implementation Considerations (Python & OSBot Framework)

Introduction

The LLM Workflows/Stateflow Service is a proposed stateless web service for executing AI-driven workflows with well-defined, deterministic steps. It acts as a state machine for orchestrating Large Language Model (LLM) calls and other actions in a controlled sequence. Unlike "agentic" AI systems that freely decide each action, this service uses a pre-defined flow (blueprint) to ensure predictability and provenance. The goal is to harness the power of LLMs (for analysis or content generation) within strict boundaries: the LLM can perform complex subtasks, but it never dictates the workflow's overall path[1]. This design follows the "Blueprint First, Model Second" principle, decoupling high-level logic from the probabilistic nature of LLMs[1]. The result is an execution environment where every step is explicit and auditable, yielding reliable behavior even in complex multi-step AI tasks.

Key collaborators: The creation of this service is a collaborative effort between human developers and AI assistants (LLMs). Developers will define the workflow blueprints, implement the engine, and integrate services, while advanced LLMs (such as ChatGPT) assist in brainstorming, research, and even code generation. For example, this very document is co-authored by Dinis Cruz and ChatGPT, reflecting the human-AI partnership in designing the system. By leveraging LLM support during development, the team can explore design options and documentation rapidly, then validate and refine them with human expertise. This collaboration ensures the final system design is both innovative and technically sound.

Design Principles and Goals

Service Architecture and Components

The LLM Workflows/Stateflow Service architecture is composed of several key components that together enable the definition and execution of these workflows:

The blueprint is analogous to a workflow script. The service includes a Blueprint Interpreter (or engine) which reads this definition and drives execution accordingly. Because the blueprint is declarative, we can inspect and validate it before running, and even visualize it as a flowchart.

Notably, the engine is agnostic to the overall workflow -- it doesn't keep a running memory of all previous steps beyond what is carried in the input context. This makes it easy to scale or even to pause and resume workflows by simply not calling the next step immediately. It is the responsibility of the Orchestrator (described next) to drive the engine through the steps.

The orchestrator could be implemented as part of the client using this service, or we could provide a utility in the service that takes a whole blueprint and automatically steps through it. However, having the orchestrator outside the core engine adds flexibility: workflows can be paused, inspected mid-run, or even modified between steps if needed (for advanced use cases).

In a serverless deployment (e.g., AWS Lambda via OSBot-FastAPI-Serverless), the orchestrator might be a state machine service (like AWS Step Functions itself or Azure Durable Functions) that triggers the next lambda invocation. Alternatively, a simple loop in an API endpoint (if using a container or persistent service) could also orchestrate the steps synchronously.

In summary, the architecture separates the declaration of what to do (blueprint) from the execution of how it's done (engine and orchestrator), with clear interfaces to external AI services and strict control logic enveloping any LLM calls. This design maximizes reliability and clarity, ensuring that even as we automate complex tasks with AI, the process remains transparent and governable.

Workflow Definition and Standards

Defining the workflow blueprint in a robust way is a critical aspect of this service. We want a format that is expressive, standard-compliant (when possible), and easy for developers to author and maintain. Given that our implementation language is Python (with OSBot-FastAPI and OSBot-FastAPI-Serverless frameworks), we have a couple of choices for how to represent workflows:

  1. Python Type-Safe Classes (Code as Blueprint): We can create Python classes (using OSBot's Type_Safe or Pydantic models) to represent the elements of a workflow -- e.g., a Flow class containing a list of Step objects, where each Step has fields like id, action, parameters, transitions, etc. Developers can then construct these classes in code or load them from a JSON. OSBot-FastAPI will automatically handle conversion between these classes and JSON schemas[3]. This means we get strong typing and validation for free. For example, if a step is missing a required field or has an invalid next step reference, the model validation can catch it before execution. This approach treats the blueprint almost like writing a program (in Python), which is then serialized to JSON when needed.

  2. JSON/YAML Workflow Schema (Data as Blueprint): Alternatively, we define a pure JSON (or YAML) schema for the workflow -- a text-based format that could be authored by hand or generated by tools. There are existing standards and schemas in the industry that we can draw inspiration from:

  3. BPMN 2.0: A long-standing standard for business process modeling (usually visual diagrams stored in XML)[5]. BPMN is very expressive (supports events, gateways, etc.), but it might be overly complex for our needs and not JSON-friendly by default.

  4. BPEL: An older XML-based language for web service orchestration[5]. Also quite heavy and tied to WS-* services.

  5. AWS Step Functions (Amazon States Language): A JSON-based state machine definition used in AWS Step Functions[6]. This is a practical and widely used format. It represents workflows as a set of states (Task, Choice, Parallel, etc.) with a StartAt and explicit Next transitions or Choice branches. It's quite suitable for our concept since it inherently models step-by-step execution with choices. The downside is that it's AWS-specific in some of its integration details, but we could adopt the structure. For example, we'd have \"Task\" states for calling LLM or Persona, and \"Choice\" states for branching on evaluation results, etc., all expressed in JSON.

  6. Azure Logic Apps: Similar to Step Functions, uses JSON (and often designed via a visual editor)[7]. Also could be a reference, though tied to Azure connectors.

  7. Workflow Description Languages (WDL/CWL): These are specialized languages for scientific and data workflows[8]. They emphasize reproducibility and usually model batch processing of data (like DNA sequencing pipelines). They might be too domain-specific for our interactive use-cases, but they show how to define steps, inputs, and outputs clearly.

  8. Argo Workflows / Tekton: Kubernetes-native workflow engines using YAML[9]. They often focus on CI/CD pipelines and container tasks. The concept of defining DAGs of steps is similar, though Argo's YAML could be more low-level (each step is basically a container spec).

  9. Custom DSLs: Many teams end up creating their own lightweight JSON/YAML schemas for workflows[10]. This is likely the approach we will take: design a JSON schema tailored to our AI workflow needs, while borrowing ideas from the above standards for structure and best practices. For instance, we might incorporate Step Function's idea of states and transitions, but simplify it, or include a field for natural language description like BPMN does, etc.

Given that we aim for a type-safe and easily maintainable solution, our plan is to define a JSON-based schema for the workflow and also represent it with Python classes for convenience. We can start from scratch or use a nascent standard like FlowSpec. FlowSpec is an open initiative to create a standardized JSON schema for AI automation workflows[11]. It recognizes that many workflow tools share a flow-chart backbone, and it attempts to unify this in a portable way. In FlowSpec, a workflow is defined by a title, description, a list of steps, and transitions between steps[11]. Each step has fields for what action to execute, its inputs, expected outputs, and what the next step(s) are depending on outcomes. It even allows global default transitions (like what to do on any failure)[12][13]. FlowSpec also enumerates existing workflow standards (as we did above) to validate the approach of a common schema[13].

After researching these options, our recommendation is to adopt a JSON state machine schema inspired by AWS Step Functions and FlowSpec. This gives us a known structure (states, next, choice, etc.) but we will customize it for our needs (for example, integrate the notion of budget and our specific action types). We will keep the schema human-readable and not too verbose. For instance, a simple flow with two steps might look like:

{
  "workflowName": "Simple Q&A",
  "startAt": "AskQuestion",
  "states": {
    "AskQuestion": {
       "type": "Task",
       "action": "call_LLM",
       "parameters": {
          "prompt": "Answer the user's question: {user_question}"
       },
       "resultVar": "answer",
       "next": "EvaluateAnswer"
    },
    "EvaluateAnswer": {
       "type": "Task",
       "action": "evaluator.rate_response",
       "parameters": {
          "response": "{answer}"
       },
       "resultVar": "score",
       "end": true
    }
  }
}

In this pseudo-JSON: - startAt specifies the entry step. - We have two states: one calls an LLM to get an answer, the next calls an evaluator to score that answer, then ends. - We use placeholders like {user_question} and {answer} to indicate passing data between steps (the engine would replace those at runtime with actual values from context). - This format is quite similar to Amazon States Language (each state has a Type and either a Next or End)[14][15], but with an action field that is our custom addition to specify what the Task does (since we are not tying directly into AWS Lambda ARNs as AWS does[16]).

We will formalize such a schema and provide a JSON Schema definition for it (so it can be validated). The use of Python with OSBot-FastAPI means we can also create corresponding classes. For example, a TaskState class and a ChoiceState class that inherit from a base State class, etc., enabling developers to construct workflows in Python fluidly. The OSBot-Fast-API toolkit will assist by ensuring these classes convert to Pydantic models easily, preserving the strong types[3]. This approach satisfies our need for a clear contract for workflows, while leveraging existing best practices from industry standards[13][17].

To summarize, the workflow definition will likely be expressed as JSON but with first-class support in Python. It will incorporate ideas from state machine standards (like having explicit states, transitions, start/end, etc.) and will be designed to be readable, easy to modify, and rigorous. By doing this, we make it easier for developers to create new workflows or adjust existing ones, and possibly even enable LLM-assisted workflow authoring in the future -- for instance, an LLM could take a high-level description and output a draft JSON workflow, which a developer then reviews and fine-tunes. The use of a standard schema also opens the door to visualization tools or workflow editors down the line.

Example Workflow: Persona-Based Communication and Evaluation

To illustrate how the stateflow service works, let\'s walk through a detailed example workflow. This scenario involves translating and conveying a critical message between two types of personas in an organization, and evaluating the communication's effectiveness. We will use the previously described actors: - Actor A: The originator of the message (could be a human user or an automated alert). In our scenario, the message is: \"A ransomware attack has hit Division X, which will impact the P&L (profit and loss) for this quarter.\" - Actor B: The Persona Service, which can assume different personas. We will use it in two modes: - Translator mode: to rephrase a message for a target persona's understanding. - Responder mode: to generate a reply as if coming from a persona. - Actor C: The Evaluator service, which will judge the quality of responses (e.g., does the response answer the question clearly, does the target persona understand the message, etc.).

The organizational context is that Board Members care about financial terms like P&L but might not understand technical cybersecurity jargon, whereas CISOs (Chief Information Security Officers) understand ransomware but might not grasp business impact jargon. Our message contains both technical (ransomware) and financial (P&L) terms, so it's challenging for either persona to fully understand without translation.

We will construct a workflow that explores different communication paths:

1. Direct Communication to CISO (No Translation):\ Actor A sends the original message directly to a CISO persona (via Actor B's responder mode acting as a CISO). The flow steps might be: - Step 1: persona.respond as CISO with input = \"Ransomware attack on Division X will impact P&L this quarter.\"\ → (Actor B generates a response as it thinks a CISO would reply. This CISO likely understands the ransomware part but may be confused or less concerned about P&L specifics. The response might say something focusing on cybersecurity mitigation but not address financial impact fully.) - Step 2: evaluator.rate_response on the CISO's reply, with criteria like completeness, clarity, appropriateness for the question.\ → (Actor C returns a score or feedback. We expect this might be a mediocre score if the CISO persona missed the financial aspect.) - Step 3: End. (We record the score and perhaps the content of the CISO's answer.)

Expected outcome: The CISO's answer might mention technical steps (e.g., "We are investigating the ransomware attack on Division X and working to contain it.") but not translate that into business terms. The evaluator might note that the board (who cares about P&L) would not get a full picture from this answer. The score could be low or moderate.

2. Direct Communication to Board Member (No Translation):\ Actor A sends the same message directly to a Board Member persona (Actor B acting as a board member): - Step 1: persona.respond as BoardMember with input = \"Ransomware attack on Division X will impact P&L this quarter.\"\ → (Actor B generates a response a board member might give. The board member persona might latch onto the P&L impact but be unsure about the technical details, possibly responding with something like "How severe is the ransomware attack and what are the projected losses?") - Step 2: evaluator.rate_response on the Board Member's reply.\ → (We expect the board member's answer might not be directly useful because the board persona might actually ask questions or express confusion about the ransomware aspect. The evaluator likely scores this low in terms of addressing the problem, since the board member persona didn't provide a solution or clear action.) - End.

This path shows how a mismatched communication (technical message to non-technical persona) might fail. The board member didn't provide a satisfying answer because they themselves didn't fully understand the technical side. The evaluator would likely flag that the communication was ineffective.

3. Translated Communication to CISO:\ Now we improve the communication. Actor A's original message will first be translated to the CISO's \"language\" (i.e., reframed in cybersecurity terms), then delivered to the CISO persona, and evaluated: - Step 1: persona.translate target=CISO, input = \"Ransomware attack on Division X will impact P&L...\"\ → (Actor B returns a translated message that a CISO would immediately grasp. For instance, it might elaborate the technical threat and downplay financial jargon: "Division X has been hit by ransomware, affecting operations; this could have a significant business impact this quarter.") - Step 2: persona.respond as CISO with input = translated message from Step 1.\ → (Now, receiving a message phrased in his context, the CISO persona can respond more appropriately. The answer might be like: "Understood. We have isolated the affected systems and are initiating incident response. We estimate recovery in 48 hours. Financial impact is being assessed in collaboration with finance." This is a more complete answer covering both tech and acknowledging financial impact, because the question was framed in terms the CISO cares about.) - Step 3: evaluator.rate_response on the CISO's new reply.\ → (Actor C would likely give a higher score here, since the response is clear, addresses the issue, and bridges to business impact. The evaluator might note that the communication was effective for the target audience.) - End.

We expect this translated workflow to yield a good outcome: the CISO persona understood the question after translation and responded in a way that likely satisfies a board or oversight evaluator.

4. Translated Communication to Board Member:\ Similarly, translate the message for a Board Member, then get a response: - Step 1: persona.translate target=BoardMember, input = original message.\ → (This might produce something like: "We estimate a hit to this quarter's profits due to a cyber incident (ransomware in Division X)." Essentially explaining ransomware impact in terms a board cares about, possibly avoiding jargon.) - Step 2: persona.respond as BoardMember with input = translated message.\ → (Now the board persona, fully aware of the financial framing, might respond appropriately, e.g.: "Understood. Ensure all necessary resources are allocated to IT to resolve this quickly. Let's prepare a statement for stakeholders about the financial impact.") - Step 3: evaluator.rate_response on this reply.\ → (Likely another high score -- the board member persona's answer is on point when the question was phrased in their terms.) - End.

This shows that with proper translation, even a non-technical persona can engage effectively.

5. Back-and-Forth Dialogue (CISO ⟷ Board, Mediated by Translations):\ We can extend the scenario to simulate an interactive dialogue between the CISO and Board Member personas. The idea is to have multiple turns: - First, the CISO receives a translated question (as in #3) and responds as CISO. - Then take the CISO's response, translate it for the Board, get a Board persona reply. - Then translate that reply back to CISO's terms, get CISO's next response. - Continue this exchange for a few iterations or until a budget limit is reached (to prevent infinite loops).

In the workflow blueprint, this could be represented by a loop or recursive transitions. For example: - Step 1: persona.translate to CISO (original message) -> output ciso_msg. - Step 2: persona.respond as CISO (ciso_msg) -> output ciso_reply. - Step 3: persona.translate to Board (ciso_reply) -> output board_msg. - Step 4: persona.respond as Board (board_msg) -> output board_reply. - Step 5: Loop condition: If board_reply or some context indicates conversation should continue AND budgets remain, go back to Step 1 (or a specific step) with board_reply now serving as the \"original message\" (Actor A's input) for the next round, targeting CISO again. - If loop ends (either a set number of rounds reached or budget exhausted), proceed to evaluation or finalization: - Step 6: evaluator.rate_response on the final response or on the overall dialogue quality. - End.

This looping construct is explicitly controlled. The blueprint would contain a Choice or condition check after Step 4 to decide whether to loop or exit. The budget for each persona ensures that, say, we don't allow more than N exchanges or Y tokens. For instance, we might give each persona service 3 calls budget. Each persona.respond call uses 1. So at most 3 rounds of responses per persona can happen (which is 3 CISO replies and 3 Board replies, for a total of 3 cycles) before the budget prevents further calls.

During this back-and-forth, each translation ensures both parties understand each other's messages in their own context. The Evaluator at the end might evaluate the overall success of the communication. Perhaps it looks at the final outcome: did they reach a mutual understanding or plan? We could even have the evaluator step after each reply, storing intermediate scores, but in practice it might suffice to evaluate at the end or only log the conversation.

This complex example demonstrates the power of the workflow approach: - We can coordinate multiple AI calls (translations, persona responses, evaluations) in a sequence that achieves a larger goal (effective communication). - Because it's all in a defined flow, we avoid chaos: e.g., the Board and CISO personas will not talk over each other or go off on tangents; they only respond when prompted by the workflow. - If something fails (say one of the steps returns an error or empty response), we could have failure paths defined. For example, if persona.respond fails due to no available LLM, the blueprint could go to a step that sends a default apology message or logs the failure. - The budget prevents infinite loops or runaway costs, which is something ad-hoc agent loops might suffer from.

In summary, this Persona Communication workflow shows a realistic use-case where deterministic orchestration of LLM-powered services adds significant value. It ensures that two different knowledge domains (technical vs business) can interact via AI intermediaries in a structured manner. The stateflow service makes it feasible to design such an interaction as a series of controlled steps, rather than leaving the entire conversation flow to an unpredictable AI agent. Each step's outcome is evaluated and can trigger specific next steps, which is exactly the kind of fine-grained control we need for enterprise applications.

Additional Workflow Examples

Beyond the persona translation scenario, the LLM Workflows service can support a wide range of other workflows. Here are a few example use-cases to demonstrate its versatility:

These examples scratch the surface. Essentially, any time we want an LLM or AI-driven process with multiple steps and we care about controlling those steps, this service can help. It provides the skeleton to plug in various AI and non-AI functions into a flowchart of actions.

By keeping the workflows declarative and using this service, organizations can codify complex procedures that involve AI into a form that's transparent, testable, and tunable. Need to change the persona or the prompt? Just update the blueprint. Want to add a step to log to a new database? Add it to the blueprint. Because the execution is isolated per step, these modifications won't affect other steps' correctness. This modularity and clarity is much harder to achieve if one tries to hard-code logic intermingled with LLM prompts in a single blob. Our service enforces good separation of concerns.

Implementation Considerations (Python & OSBot Framework)

The service will be implemented in Python 3.11+ (per OSBot-Fast-API requirements[18]) using the OSBot-Fast-API library and its serverless extension. Here we outline how we leverage these technologies and other implementation details:

In conclusion, the LLM Workflows/Stateflow Service is a cutting-edge approach to making LLM-based systems more robust, transparent, and controllable. By blending established workflow orchestration concepts with the latest AI capabilities, and implementing it with modern Python frameworks, we aim to create a service that developers and AI systems can collaboratively use and improve. It will empower the creation of AI-driven applications that have the creativity of LLMs and the reliability of traditional software -- a combination that is increasingly essential in high-stakes applications[19][20]. With this foundation, we anticipate a new class of solutions where humans specify the roadmap (workflow) and AI fills in the details (content), all under a structure that ensures safety and effectiveness.

[1] [19] [20] Blueprint First, Model Second: A Framework for Deterministic LLM Workflow

https://arxiv.org/html/2508.02721v1

[2] LangGraph vs AutoGen: How are These LLM Workflow Orchestration Platforms Different? - ZenML Blog

https://www.zenml.io/blog/langgraph-vs-autogen

[3] [4] [18] osbot-fast-api · PyPI

https://pypi.org/project/osbot-fast-api/

[5] [6] [7] [8] [9] [10] [11] [12] [13] [17] GitHub - woodyhayday/FlowSpec: FlowSpec: Automation Workflow Schema - A lightweight JSON schema for defining automations and multi-step workflows. Designed for AI Automation Workflows

https://github.com/woodyhayday/FlowSpec

[14] [15] [16] Amazon States Language

https://states-language.net/

Released under CC BY 4.0. First published on docs.diniscruz.ai; this page as markdown.

← Personas Service - Technical LLM BriefDinis Cruz's Research on API Security (2009-2025) →