Using backend factories during frontend testing

When constructing end-to-end tests for web applications there are many areas which can prove problematic and can discourage users from committing to writing them at all. One of those areas is setting up the system with test data. We have found that by sharing our backend fixture library with the frontend, we can side-step many of these problems and get better quality E2E tests.

When carrying out end-to-end tests we test the data that flows through the application in response to user actions; we test the network requests made and received by the client and the web server and we test the data created and destroyed during interactions of the various services. The areas touched during the execution prove that the application works as expected and in a consistent manner: correct and expected data is shown to the user in response to their actions.

(Our) End-to-end manifesto 

End-to-end testing is the closest approximation to how users interact with our application and a contrast to functional/component testing which tests the behaviour of isolated units. Such a test should be indistinguishable from a real user interacting with the system. Due to their repetitive nature, automating these tasks is essential to achieve the goal successfully.

Some of the challenges posed by the task of creating test data are traditionally solved by approaches of stubbed objects or snapshots of the current state of the application. Both approaches break the philosophy of what an end-to-end test is. In one, we fail to test the system-database interaction and in the other, we fail to create a real user-system interaction indistinguishable from a real user’s actions.

Data Factories and FactoryBot 

As the title suggests, the proposed solution to solve the challenge of creating appropriate and realistic data that aligns with our system’s expectations will use a fixture factory library https://github.com/thoughtbot/factory_bot, for a Ruby on Rails backend. Counterparts in other languages are available:

• java: https://github.com/topicusoverheid/java-factory-bot

• python: https://factoryboy.readthedocs.io/en/stable/

• go-lang: https://github.com/bluele/factory-go

• rust: https://github.com/mjkillough/factori

Over the years, the use of FactoryBot has become the de facto for backend testing in Rails, but projects which share data fixtures with the frontend are not a common sight. While it’s very standard to use a factory library to set up more backend focused tests, using this library in our end-to-end testing ensures that the data created in our E2E suite is always correct as it is driven by the backend fixtures and guarantees all the same validation rules are obeyed akin to production data created during real user interaction.

E2E testing in the context of a web app is usually driven by automated browser testing and a multitude of javascript frameworks are available in today’s market for this purpose. At Tines, the tool of choice is https://playwright.dev/. We use FactoryBot to create the data we want for our tests, without having to stub objects or take snapshots of the application. Instead, using FactoryBot, we make requests to our backend in the setup stage of the test, creating the objects we require for our test. The test then proceeds, exercising real APIs, creating database records, and so on… At the end of the tests, when the network connection with our system is torn down, the database is flushed of the objects created with FactoryBot.

// makeRecord helper used to create the data object in the test
function makeRecord(
    nameAndTraits: string | string[],
    args: Record<string, unknown> = {}
): any {
    return new Promise((resolve) =>
        fetchInPage(page, Paths.makeRecord(), {
            nameAndTraits,args,
        }).then(([statusCode, json]: [statusCode: number, json: unknown]) => {
            if (statusCode !== 201) {console.error(json);}
            resolve(json);
        })
    );
}

The above method is the heart and soul of the approach. It connects the frontend testing framework to the backend fixture library, allowing us to request fixture creation and reflect the changes in the page.

import { Page } from "playwright-chromium";

function fetchInPage(
    page: Page,
    url: string,
    data: Record<string, unknown> = {}
): any => makeRequest(page, url, postOptions(data));

const postOptions = (data = {}): RequestInit => ({
    headers: { Accept: "application/json", "Content-Type": "application/json" },
    method: "POST",
    body: JSON.stringify(data),
});

const makeRequest = (
  page: Page,
  ...fetchArgs: [string, RequestInit]
): Promise<DatabaseRecord> =>
  page.evaluate(
    (args: [string, RequestInit]) =>
      new Promise((resolve, reject) => {
        fetch(...args)
          .then((response) =>
            response.json().then((json) => resolve([response.status, json]))
          )
          .catch((error) => reject(error));
      }),
    fetchArgs
  );

// Sample Playwright test creating a user object

describe("Logged in user", () => {
    beforeEach(async () => {
        await page.goto(Paths.start()); 
    });

    afterEach(async () => {
        await page.goto(Paths.start());
    });

    it("Create logged in user", async () => {
        // The FactoryBot created user with all its proprties
        const user = await makeRecord(["user", "logged_in"]);

        // Interact with page using Playwright actions
    });
});

The reusable interface defined above is put into action in a Playwright test. It requests creation of a user object that can then be interacted with on the page via Playwright actions.

Advantages of Data Factories 

Using FactoryBot means that the only additional code on top of our testing framework that has to be written is the reusable interface to seed our data objects on the backend. The alternative is a custom object with properties required for every specific test adding up to a significantly higher maintenance cost when changes to the data structure have to be made in the backend and database.

class EndToEndTestsController < ActionController::Base
    def start
        # tests can start and end on this minimal, fast-loading page, 
        # and make same-origin requests to set up their initial state
        head :ok
    end

    def make_record
        name_and_traits = Array.wrap(transformed_params["name_and_traits"])
        arguments = transformed_params["args"]
        result = FactoryBot.create(*name_and_traits, arguments)
        render json: result.as_json, status: :created
    end

    private
    def transformed_params
        request.parameters.to_h
    end
end

The Rails wrapper needed to connect the network requests made by the testing framework to FactoryBot fixtures.

Some bonus advantages that also come with this approach include:

• We're more likely to catch bugs across the frontend and backend as we use real data created in the backend instead of stubs that may disguise incorrect behaviour.

• When used in Rails land, our maintenance costs are lower since we already have the factories created for RSpec testing of the backend code. If the data model or schema changes, the changes made will propagate to the E2E tests.

• Lower framework setup and test writing cost as we spend less time preparing data needed on a per test basis, instead using the reusable FactoryBot interface as well as reduced code duplication associated with the reusability.

Conclusion; the true value of our end-to-end strategy

After implementing the approach of using a backend data fixture library in frontend driven end-to-end testing, we believe that this is the best way for us to implement end-to-end testing.

The data created is not stubbed but is identical to the data created by a real user interaction allowing for real network requests and database queries. As a result, the tests more accurately represent a real user interaction following the philosophy of true end-to-end testing of the application’s workflow from beginning to end on its many interfaces.