diniscruz.ai / writing / Development and GenAI

Surrogate Dependencies: Simulating Backends for Offline-First Development

By Dinis Cruz and ChatGPT Deep Research · · 36 min read

PDF

Contents · 10 sections
  1. Introduction
  2. Background and Motivation
  3. What Are Surrogate Dependencies?
  4. How Surrogate Dependencies Work
  5. Benefits of Surrogate Dependencies
  6. Challenges and Considerations
  7. Implementation Strategies
  8. Surrogate Dependencies in TDD and LLM-Based Development
  9. Best Practices and Tips
  10. Conclusion

(converted to markdown by Claude Opus 4.1)

Introduction

Modern software development often relies on remote APIs and services, yet developers frequently need to work in environments where those backends are unavailable or unstable. Surrogate dependencies are a solution to this problem – they allow an application to run in a fully offline mode by simulating backend systems with prerecorded data. This concept was originally proposed as a way to align security and development needs, making developers more productive while still addressing AppSec concerns¹. In essence, a surrogate dependency acts as a stand-in for a live API, serving consistent responses from local JSON data instead of making network calls. This technical white paper introduces surrogate dependencies, compares them to traditional stubs and mocks, and provides implementation strategies and examples for frontend applications (particularly those that normally communicate with FastAPI or other web services). We also discuss how surrogate dependencies support offline-first workflows, test-driven development (TDD), and even AI-assisted (LLM-based) development environments.

Background and Motivation

Applications that depend on cloud services or microservice APIs face several challenges during development and testing:

The motivation behind surrogate dependencies is to resolve these issues by enabling applications to run with zero external dependencies. By capturing and simulating backend responses, developers get a fast, reliable "surrogate" for each dependency. Notably, this concept isn't purely about convenience – it also reflects a maturity in the development process. As Dinis Cruz notes, "the ability to run your apps offline also signifies that the application development environment has matured to a level where you have... mocked versions of your dependencies"⁴. In other words, building offline capability via surrogate dependencies forces teams to design clearer contracts and test against them, benefiting both development and security. Ultimately, an environment where one can run everything locally (or in a contained setup) tends to produce more robust software, as it encourages comprehensive integration testing and rapid iteration³ ⁵.

What Are Surrogate Dependencies?

A surrogate dependency is a stand-in for a backend service that an application can use during development, testing, or even in demos, without needing the real service. Practically, a surrogate dependency consists of static data files (e.g. JSON files) and logic in the application to load those files in place of making a network request. The surrogate mimics the API's outputs: when the app "calls" the surrogate, it returns a predefined response identical in structure (and often content) to what the real API would provide. This allows the rest of the application to behave as if it were connected to the actual backend. As summarized in an OWASP talk, surrogate dependencies "test the API and replay responses" – developers use integration tests or recording tools to lock in the API's behavior by saving responses in JSON format and then replay that data to the client, thereby allowing the client to run entirely offline⁶.

It's useful to compare surrogate dependencies to related concepts:

By using surrogate dependencies, developers essentially decouple their front-end application from the backend API implementation. The application's network layer can be pointed either at the real API or at the surrogate data source. When pointed at the surrogate, the experience is seamless: from the application's perspective, it receives JSON data shaped exactly as if it came from the live service – except it was loaded from a local store or CDN. This approach has been used in several projects to allow web apps to operate without any active backend, greatly enhancing testability and resilience in the development process.

How Surrogate Dependencies Work

1. Capturing Real API Responses: The first step is to gather real responses from the live backend. This is often done by writing integration tests or recorder scripts that exercise the real API endpoints. These tests invoke the API (e.g., making HTTP requests to the FastAPI service) and then save the resulting JSON responses to files. In Dinis Cruz's methodology, this process is described as using integration tests to "'lock' the API" – meaning you capture and lock-in the exact outputs of the API at a given time⁶. Each response is stored in a JSON file, typically organized in a structure mirroring the API routes (more on file organization in the next section). By committing these files to a repository (for example, a Git repo), the team creates a versioned, shareable dataset of what the API provided. An illustration of this capture process is shown below.

Figure 1: Integration tests call the live API and save its responses as JSON files in a surrogate data store. These files serve as the "surrogate dependencies" for offline use.

In this capture phase, the application itself is not yet involved – we are populating the surrogate data that the application will later consume. It's important to ensure the captured data covers the necessary use cases (all major endpoints, typical query parameters, etc.). Some teams capture not only normal responses but also error cases (e.g., a 404 or validation error response) so that the front-end can also handle those in offline mode. The capturing can be automated as part of a pipeline that regularly updates the surrogate data (for instance, after any API change or on a schedule) to avoid staleness.

2. Routing Application Requests to Surrogates: Once the JSON files are prepared, the application is configured to use them when in "surrogate mode." This typically involves a simple flag or mode switch in the application's configuration. The application's network layer (e.g., a fetch wrapper or API client module) checks this flag to decide whether to call the real API or to load a local file. In surrogate mode, what would normally be an HTTP GET to https://api.example.com/items/42 might instead become a fetch of surrogate_data/items/42.json from the local bundle or a CDN. The surrogate JSON contains the same fields the real API would return, so the rest of the application (UI logic, etc.) doesn't know the difference. The client effectively replays the previously recorded responses as if they were live⁶. The figure below illustrates an application using surrogate data in offline mode:

Figure 2: In offline mode, the application (client) fetches data from the surrogate store (pre-recorded JSON) instead of making calls to the live API. No network connection is required in this mode.

Crucially, all of this is achieved without monkey-patching or hacky overrides of low-level APIs at runtime. The ability to switch data sources is designed as part of the application architecture. For example, the app might have a central apiClient module or a function like fetchData(endpoint) that internally decides where to get the data. This design means that enabling or disabling surrogate mode is as simple as flipping a configuration, rather than modifying dozens of call sites or risking invasive patches to network libraries. The result is clean and maintainable: one code path for live mode, and one code path for surrogate mode, encapsulated in one place. In surrogate mode, developers can run the entire front-end application on their local machine (or even a static hosting environment) with no internet connection, and yet the app behaves almost exactly as it would online.

Benefits of Surrogate Dependencies

Surrogate dependencies bring a multitude of benefits to the development workflow:

In summary, surrogate dependencies support a development culture where one asks "Can you run your app offline?" – a question that is increasingly a measure of the quality of the development environment⁹. The teams that can confidently answer "yes" tend to have faster release cycles and fewer integration surprises, whereas those that cannot often struggle with integration issues late in the cycle¹⁰.

Challenges and Considerations

While surrogate dependencies bring many benefits, they also introduce certain challenges and trade-offs that teams should be aware of:

By being mindful of these challenges, teams can effectively use surrogate dependencies while mitigating risks. Many of these considerations (data syncing, security, toggling) can be addressed with automation and good discipline. In exchange for this effort, the payoffs – in productivity and software quality – are substantial.

Implementation Strategies

Implementing surrogate dependencies requires careful thought in the architecture of the application's front-end code, but it does not require heavy frameworks or external libraries. In fact, one of the goals is to achieve this with pure JavaScript/TypeScript, keeping things lightweight. Below, we outline key strategies and patterns for implementing surrogates cleanly:

Centralize and Abstract Network Communication

The foundation of a maintainable surrogate system is a centralized network layer. Instead of scattering fetch() calls (or Axios, etc.) throughout the codebase, the application should funnel all data retrieval through a small set of functions or a client class. For example, you might have an ApiClient object with methods like getUser(id), getOrders() etc., or a generic helper like fetchData(endpoint, params). By having this single chokepoint, you make it easy to insert the surrogate logic. All calls go through here, so this is where you implement:

const USE_SURROGATE = window.CONFIG?.surrogateMode ?? false; // a global flag set in config

async function fetchData(endpointPath) {
    if (USE_SURROGATE) {
        // Construct path to local JSON based on endpoint
        const surrogateUrl = `/surrogate_data/${endpointPath}.json`;
        const response = await fetch(surrogateUrl);
        return response.json();
    } else {
        const liveUrl = LIVE_API_BASE_URL + '/' + endpointPath;
        const response = await fetch(liveUrl, {/* credentials, headers, etc. as needed */});
        return response.json();
    }
}

In the code above, endpointPath might be something like "users/123" or "orders/list" depending on how you structure it. The key point is that the logic to decide surrogate vs. live is isolated in one place. If tomorrow you rename the surrogate data folder or change how you fetch it, you update it here only. Also, this makes it trivial to turn surrogate mode on or off via configuration. For example, you could have a build script that sets window.CONFIG.surrogateMode = true for a special "offline bundle," or toggle it via an environment variable in development. By avoiding direct network calls elsewhere, you also reduce the temptation for developers to accidentally bypass the surrogate (everyone goes through fetchData, no exceptions).

Another pattern is to use dependency injection or factory functions for the API client. For instance, you might have two implementations of an interface DataProvider: one that calls the network, one that reads local data. At startup, depending on mode, you supply one or the other to the app. This is more common in large applications or when using frameworks, but the idea is the same. The bottom line: architect your app such that swapping out the data source is a one-liner, not a refactoring nightmare. This also means avoiding deeply embedding URL strings for APIs in many components – keep them in a config or in the central client.

Organize Surrogate Data Mirrors

Your surrogate JSON files should be organized in a clear, predictable structure. Typically, the easiest approach is to mirror the API routes. For example, if your live API has endpoints like:

You can create a directory structure under a surrogate_data folder that corresponds to these. One scheme could be:

surrogate_data/
├── users
│   ├── list.json           ← data for GET /api/users
│   └── 42.json             ← data for GET /api/users/42
├── orders
│   ├── list.json           ← data for GET /api/orders (maybe general list)
│   └── date-2023-01-01.json ← data for GET /api/orders?date=2023-01-01
└── ... (more endpoints)

In this layout, we have a subfolder for each resource type, and within it, files for different queries or IDs. The naming convention should be intuitive. Here we used "list.json" for an index listing (no special query) and a combination like "date-2023-01-01.json" for a filtered query example. Some teams choose to incorporate the HTTP method in filenames (e.g., GET_users_list.json vs POST_users_create.json), or they create separate folders for GET/POST if needed. In most front-end uses, GET responses are the main concern, since POST/PUT requests in development are either less frequently tested offline or can reuse GET results for their outcomes.

It's often useful to include metadata in the surrogate files, especially if they were captured automatically. For example, a JSON file could include a comment (if using a format that supports it or a separate README) stating when and how it was obtained, or what endpoint and parameters it corresponds to. Keeping surrogate data well-documented helps avoid confusion. If multiple variants of data exist (like multiple user examples), ensure each file clearly indicates its purpose (for instance, user-42.json vs user-no-profile.json for a user with missing profile picture scenario).

When serving these files, if you are running a dev server (like webpack dev server or similar), you just need to ensure the surrogate_data folder is served as static content. In a plain setup, placing it in the public directory would suffice. If using a CDN approach, you might upload these JSON files to a storage bucket and configure fetchData to pull from that URL (which might even allow non-developers to update test data by just replacing JSON on the CDN).

Switching Modes Safely

How the application switches between live and surrogate mode can vary. Some common approaches:

Regardless of approach, it's a good practice to make the mode explicit. Logging which mode is active helps avoid confusion. Some teams color-code their UI (like a subtle background color change) or put an "[Offline]" badge when surrogate mode is on, purely to remind the user (developer) that "you're not hitting real servers now." This can prevent mistakes such as filing a bug that data isn't updating, when in fact you were looking at static surrogate data.

Example Scenario and Code Walkthrough

To cement these ideas, let's walk through a fictional scenario. Suppose we're building "TaskHub," a project management dashboard (to ground it in a realistic application). TaskHub's front-end is a single-page web application, and its backend is a FastAPI service with endpoints like /projects, /projects/{id}, /projects/{id}/tasks, etc., which return JSON data. We want to enable a surrogate mode for TaskHub so that front-end developers can work on the UI even if the FastAPI service is not running.

Surrogate Data Setup: We identify key endpoints and use an integration test script to capture data. For example, we GET /projects to retrieve a list of projects, and save that JSON as projects/list.json. We GET /projects/alpha (project with slug "alpha") to get a project detail, saving as projects/alpha.json. We also fetch sub-resources like /projects/alpha/tasks -> projects/alpha-tasks.json, and maybe a specific task /projects/alpha/tasks/42 -> projects/alpha-tasks-42.json. For variety, we might capture another project as well, say /projects/beta and its tasks. We ensure to include an example of an empty list (maybe one project that has zero tasks, to see how the UI behaves). All these JSON files are placed under a public/surrogate_data folder in the front-end repository. The structure might look like:

surrogate_data/
└── projects
    ├── list.json
    ├── alpha.json
    ├── alpha-tasks.json
    ├── alpha-tasks-42.json
    ├── beta.json
    └── beta-tasks.json

(For brevity we omit some potential files; in practice you'd add what's needed.)

Application Code Changes: In the TaskHub front-end code, we likely have a module responsible for API calls. Before surrogate support, it might have looked like:

// apiClient.js (before)
const API_BASE = "https://api.taskhub.example.com";

export async function getProjectList() {
    const res = await fetch(`${API_BASE}/projects`);
    return res.json();
}

export async function getProjectDetails(projectId) {
    const res = await fetch(`${API_BASE}/projects/${projectId}`);
    return res.json();
}

// ... more similar functions for tasks, etc.

To add surrogate capability, we modify this module:

// apiClient.js (after adding surrogate mode)
const API_BASE = "https://api.taskhub.example.com";
// Assume a global flag or an import that tells us surrogate mode
const SURROGATE_MODE = window.SURROGATE_MODE === true;

async function fetchJson(url) {
    const res = await fetch(url);
    if (!res.ok) {
        throw new Error(`Request failed with ${res.status}`);
    }
    return res.json();
}

export async function getProjectList() {
    if (SURROGATE_MODE) {
        return fetchJson("/surrogate_data/projects/list.json");
    }
    return fetchJson(`${API_BASE}/projects`);
}

export async function getProjectDetails(projectId) {
    if (SURROGATE_MODE) {
        return fetchJson(`/surrogate_data/projects/${projectId}.json`);
    }
    return fetchJson(`${API_BASE}/projects/${projectId}`);
}

export async function getProjectTasks(projectId) {
    if (SURROGATE_MODE) {
        return fetchJson(`/surrogate_data/projects/${projectId}-tasks.json`);
    }
    return fetchJson(`${API_BASE}/projects/${projectId}/tasks`);
}

// ... etc for other endpoints

We introduced a helper fetchJson for brevity, and wrapped each API call with a conditional check. This is a straightforward approach. We could refactor it further to remove repetition (for instance, a generic getter that constructs paths), but clarity is often more important, especially when maintaining the surrogate files. Notice how the surrogate file paths are constructed – they match the structure we decided on. In surrogate mode, getProjectDetails("alpha") will fetch /surrogate_data/projects/alpha.json (which the dev server will serve as a static file), whereas in live mode it goes to the real API.

Running in Surrogate Mode: Let's say we want to start the app in surrogate mode for development. We can include a small snippet in our HTML (or set window.SURROGATE_MODE via a script) for when we want offline mode. Alternatively, if using a build flag approach, we might have a separate HTML or toggle to set that global. In our scenario, we'll assume the developer manually sets a flag in a config file for simplicity. Once that's done, they run npm start for the front-end, and because the dev server serves the surrogate_data folder, the app loads data from those files. The UI comes up showing, for example, two projects "Alpha" and "Beta" (from list.json). If they click "Alpha", the app calls getProjectDetails("alpha"), which returns data from alpha.json. It also calls getProjectTasks("alpha"), getting data from alpha-tasks.json. All this happens quickly and without errors, because the data and structure are exactly what the app expects from the real API.

If the developer instead started the backend server and turned off the surrogate flag, the same code would call the live endpoints and (ideally) get the same data. This dual-mode has been achieved with minimal intrusion in the codebase and no external libraries – just careful design.

Integrating with CI/CD and Team Workflows

To ensure surrogate dependencies remain effective, integrate them into your team's workflow:

By following these implementation strategies and team practices, surrogate dependencies can be introduced in a robust, low-friction manner. The goal is to maximize the benefit (ease of use, productivity) while minimizing maintenance overhead and complexity.

Surrogate Dependencies in TDD and LLM-Based Development

Surrogate dependencies have a natural synergy with test-driven development (TDD) practices and are increasingly relevant in the context of LLM-based development environments (where AI assistants or agents are involved in coding and testing). Let's explore these connections:

Test-Driven Development: In TDD, developers write tests (often failing at first) and then implement code to make them pass, iterating rapidly. Surrogate dependencies can extend TDD from unit tests into the realm of integration tests. For example, a developer can write an integration test for the front-end: "When I load the project dashboard, it should show 2 projects and their details." Initially, without a backend, this test would fail because no data is coming in. But with surrogate approach, the developer can simultaneously create the surrogate JSON files representing the expected API responses (perhaps based on an API specification or using a dummy backend). They then run the app in surrogate mode to fulfill the test expectations. Essentially, the surrogate data serves as the expected outcome for the test – it is the "specification by example." Once the front-end code is implemented to parse and display that data, the test passes. This flips the usual need of having a live API available; instead the team can design API contracts and test against examples of those contracts even before the backend exists (or before it's accessible in the dev environment). It's similar to using mocks in testing, but much closer to real usage. Moreover, once the real API is available, the same tests can be executed against it to validate that reality matches the surrogate assumptions.

One can also incorporate surrogate data generation into TDD workflow. Consider that you have unit tests for the API (on the server side) that define what responses should be. Once those pass, you could automatically export those responses as JSON – which become the surrogate files for the front-end. Now the front-end can be developed and tested against exactly what the backend logic produced in tests. This creates a tighter feedback loop between front-end and back-end development. In scenarios where the backend is developed by another team or is an external service, surrogate dependencies allow your team to proceed with TDD for integration without waiting on the other side. You basically freeze a "contract" in the form of surrogate JSON and iterate on your side. Later, when integrating for real, any differences are quickly caught by tests (as discussed earlier).

LLM-Based Development Environments: The rise of large language model (LLM) assistants (like GitHub Copilot, ChatGPT, and others) in software development opens new possibilities and needs in the dev workflow. Surrogate dependencies can play an interesting role here:

In all these ways, surrogate dependencies prove to be a forward-looking practice. They not only solve immediate problems (offline dev, testing) but also pave the way for better human-AI collaboration in coding. By having a self-contained environment where all external interactions are encapsulated in data, we create a playground where both humans and AI can safely and efficiently co-create software.

Best Practices and Tips

To conclude the technical guidelines, here is a summary of best practices for implementing and using surrogate dependencies effectively:

By adhering to these best practices, teams can ensure that surrogate dependencies remain a help and not a hindrance. When done right, maintaining the surrogate mode becomes a normal part of development – a small upfront cost for a significant payoff.

Conclusion

Surrogate dependencies offer a powerful paradigm for modern software development: they empower developers to simulate and stand-in for complex backends with simple JSON files and toggled logic, unlocking offline development, faster testing, and greater resilience to change. We have discussed how surrogate dependencies work, how to implement them in a clean and maintainable way, and the many benefits they bring in productivity and software quality. By comparing them with traditional mocks, stubs, and service virtualization, we see that surrogates hit a sweet spot of realism and simplicity – they leverage real data to avoid the brittleness of hand-written mocks, yet remain straightforward to set up compared to full-blown service virtualization platforms.

This white paper, co-authored by Dinis Cruz (who has applied these concepts in numerous projects) and ChatGPT Deep Research, synthesizes both practical insights and forward-looking implications. Surrogate dependencies are not just a theoretical idea; they have been used in the field to great effect – for example, allowing apps to run entirely offline during security testing engagements, or enabling rapid prototyping of front-ends against evolving APIs. As Dinis Cruz highlighted in his OWASP presentation, such approaches help align what developers want (fast, flexible workflows) with what AppSec and QA teams want (reproducible, testable systems)¹. They essentially create a common layer – the surrogate data – that all stakeholders can refer to.

Looking ahead, the importance of surrogate dependencies is likely to grow. With more developers working remotely and asynchronously, having a self-contained development environment is invaluable. With microservices and third-party APIs proliferating, being able to decouple your development from external changes provides stability. And with AI increasingly in the mix, providing a controlled sandbox (rich with real examples) will be key for AI-assisted development and testing. Surrogate dependencies address all these needs.

In conclusion, adopting surrogate dependencies is a design decision that pays off by making your project more robust to external factors and more amenable to rapid, iterative improvement. It is a technique that encourages better understanding of one's own system (by explicitly capturing its inputs/outputs) and yields a safety net for development ventures large and small. We encourage teams to consider this approach, start small by capturing a few key endpoints, and experience the difference it can make. As the old adage goes, "Trust, but verify" – surrogate dependencies let you do exactly that with your integrations: trust that your app works offline with known good data, and verify against the real systems with confidence once you go online¹¹ ⁹. It is our hope that this paper has provided both the conceptual foundation and the practical guidance needed to leverage surrogate dependencies in your own projects, thereby fostering a more efficient and resilient development process.

Sources: The concept and practices described here draw from industry experience and prior art, including Dinis Cruz's presentation on Surrogate Dependencies⁶ ² and writings on DevSecOps practices¹² ¹³. These sources underline the critical role of offline-capable development and using real data for testing. Developers are encouraged to refer to those materials for a deeper understanding and for inspiration on extending surrogate dependency patterns to their specific tech stacks.

¹ ² ⁶ ⁸ Surrogate Dependecies (in NodeJS)- v1.0 https://owasp.org/www-chapter-london/assets/slides/OWASP20160929_NodeJS_Surrogate_Dependencies.pdf

³ ⁴ ⁵ ⁷ ⁹ ¹⁰ ¹¹ ¹² ¹³ Read SecDevOps Risk Workflow | Leanpub https://leanpub.com/secdevops/read

Released under CC BY 4.0. First published on docs.diniscruz.ai; this page as markdown.

← Iterative Flow Development (IFD) Methodology: JavaScript Web Application ImplementationPersonas Service - Technical LLM Brief →