Unit testing (Java|Type)Script with Sync vs. Async Promises

The No

Let’s start with a basic example I initially felt would have been helped by the ability to construct a perpetually pending promise (Peter Piper eat your heart out): In flight state for UI components… Disabling a form on submit is a common use case. Say we have a component (SubmitButton for our example) that should be in a disabled state when clicked, and is expected to remain so until the returned promise is resolved. A test to verify this could like something like:

it('should disable itself on submit (with async promise)', async () => {
    const onSubmit = jest.fn<Promise, []>().mockResolvedValue(true);

    render(<SubmitButton value="Hello!" onSubmit={onSubmit} />);

    const button = screen.getByText("Submit") as HTMLButtonElement;

    expect(button).not.toBeDisabled();

    act(() => button.click());
    expect(button).toBeDisabled();
    expect(onSubmit).toHaveBeenCalledTimes(1);
    expect(onSubmit).toHaveBeenCalledWith("Hello!");

    await waitFor(() => expect(button).not.toBeDisabled());
});

Initially I would have expected this test to fail as there’s no way to control when the promise resolved and indicate the button should now be enabled again. However it’s due to the async nature of promises and the utility method waitFor() that allows this test to work. If we were to use a synchronous resolved promise for this test we would never see the button in a disabled state. We need to return a pending promise mock and have an extra step for resolving it manually.

The equivalent test case using a synchronous promise mocking library (in this case the one I recently wrote, promitto):

it('should disable itself on submit (with sync promise)', async () => {
    const savePromise = promitto.pending(true);
    const onSubmit = jest.fn<Promise, []>().mockReturnValue(savePromise);

    render(<SubmitButton value="Hello!" onSubmit={onSubmit} />);

    const button = screen.getByText("Submit") as HTMLButtonElement;

    expect(button).not.toBeDisabled();

    act(() => button.click());
    expect(button).toBeDisabled();
    expect(onSubmit).toHaveBeenCalledTimes(1);
    expect(onSubmit).toHaveBeenCalledWith("Hello!");

    await act(() => savePromise.resolve().settled());
    expect(button).not.toBeDisabled();
});

Here we start off with a pending promise mock which will be returned when we click our submit button. Then we move it to a resolved state by calling resolve() inside an act wrapper. (If you’re curious about what settled() does here we’ll be discussing it in more detail later on.) After that we’re free to assert that our component has reverted back to an enabled state.

So in this case having a synchronous promise introduced more steps for no real benefit.

The Yes

When I finished the initial cut of promitto I started converting some of my test cases to use promise mocks in the modules I felt could benefit from it. To my horror it wasn’t a straight swap! I’d switch to the synchronous promises and tests that were passing suddenly failed. Here’s a contrived (for NDA reasons) example:

import getEvent from "api/getEvent";
jest.mock("api/getEvent");

describe("EventWatcher (with async Promises)", () => {
  let eventAction: EventAction;

  beforeEach(() => {
    getEvent.mockClear();
    eventAction = EventAction.CONTINUE;
    getEvent.mockImplementation(() => Promise.resolve(eventAction));
  });

  // ...

  it("should stop when done", () => {
    const watcher = new EventWatcher({
      eventAction: EventAction.DONE,
    });

    watcher.run();

    expect(watcher.isStopped()).toEqual(true);
  });
});

This is the original async version. When I changed beforeEach to use Promise.resolve(eventAction) to promitto.resolve(eventAction) the test case failed. If you’re particularly eagle-eyed you have already spotted it. Don’t worry if you don’t though because it took me a while to trace through and understand it as well. This test is failing with synchronous promises because there is a bug in our setup. We’re not setting the global eventAction value that our mock getEvent() call is returning. In the async version this bug was covered up because the fulfillment handlers did not run and override the initial eventAction state we tried to pass into our EventWatcher instance. To summarize the order of events with each:

Async:

1. beforeEach is called setting up our API call mocks.
2. We construct the watcher instance inside our test case with the DONE state.
3. We run the watcher instance which calls our getEvent API mock and returns an async promise. Our test case still has execution priority so the promise hasn’t had a chance to run it’s fulfillment handlers which updates the internal state of watcher.
4. Our expectations run and pass.

Sync:

1. Same as before, beforeEach is called setting our API call mocks.
2. We construct the watcher with our DONE state.
3. We run the watcher instance which calls getEvent and receives a synchronous promise. Since our promise mock started off in a fulfilled state the handlers attached via then are immediately executed, resetting our internal state to CONTINUE. (Interestingly enough if await is used in run() instead of then; the bug is not exposed.)
4. Our expectations run and fail because our global state overrode our local state.

This all happened because when I initially was building the test suite, I noticed this potential for inconsistency and ended up making a builder function to use instead of just constructing the instances directly.

Here’s the updated sync version with the builder call:

describe("EventWatcher (with PromiseMocks)", () => {
  let eventAction: EventAction;

  beforeEach(() => {
    getEvent.mockClear();
    eventAction = EventAction.CONTINUE;
    getEvent.mockImplementation(() => promitto.resolve(eventAction));
  });

  function buildWatcher(attributes: Pick<EventWatcher, "eventAction">) {
    eventAction = attributes.eventAction;
    return new EventWatcher(attributes);
  }

  // ...

  it("should stop when done", () => {
    const watcher = buildWatcher({
      eventAction: EventAction.DONE,
    });

    watcher.run();

    expect(watcher.isStopped()).toEqual(true);
  });
});

Notice the builder function has the same signature of the EventWatcher constructor but also sets the global variable to keep the state consistent. In retrospect I should have gone back to all my test cases and refitted them to use the build function rather than the constructor when I added it. Then it’s likely the sync promise would have been a drop in replacement with no issues. I do like that the sync implementation helped me catch this though!

The Kind Of

So we saw a scenario where there was some benefit to using synchronous promises in tests. It caught a bug in our tests. While I don’t think this particular case caused a bug in the application code, it got me thinking: How would we catch this setup bug with async promises? And what if these uncaught setup issues DID cause an application bug?

I’m not saying this is a good enough reason to always use sync promises for unit tests. The only reason sync promises caught this bug was because we were using then() to unwrap the relevant promises. If we had been using await it would have behaved the same as an async promise and the bug would have stayed hidden. So the question isn’t really about sync vs. async promises. It is more about: What kind of tools do we have in situations where we don’t have direct access to the promise we want to ensure is finished before we make our assertions?

Let’s pretend watcher.run() from our example above does some kind of setInterval() setup and calls getEvent() in each interval. I left this out as to not cloud our example with useFakeTimers() and advanceTimersByTime() calls. I just want to be clear that run cannot simply return the promise that updates the internal state.

If we tweak our example to capture the mocks coming from our function spy we can use settled() from promitto to ensure the call has been processed.

describe("EventWatcher (with async Promises)", () => {
  let eventAction: EventAction;
  let calls: PromiseMock<EventAction>[] = [];

  beforeEach(() => {
    getEvent.mockClear();
    eventAction = EventAction.CONTINUE;
    calls.length = 0;
    getEvent.mockImplementation(
      () => (calls[calls.length] = promitto.resolve(eventAction))
    );
  });

  // ...

  it("should stop when done", async () => {
    const watcher = new EventWatcher({
      eventAction: EventAction.DONE,
    });

    watcher.run();
    await calls[0].settled();

    expect(watcher.isStopped()).toEqual(true);
  });
});

This test correctly fails since we’re not setting global eventAction to the value we are trying to test with. To catch the original bug we’ve added the calls array and have our getEvent() mock implementation collect each mock promise generated into it. Finally in our test case we can wait for this promise mock to have settled() before making our assertions. settled() is a method that comes from promitto. I mentioned it in one of our earlier test cases and it seems like a good time to go into more detail about it.

promitto promises internally track the new promises created from attaching callbacks to it through then(), catch(), and finally(). These methods still return new promises like a good A+ promise should, but also adds them to an internal children collection. settled() returns a promise that will resolve once all of it’s children have been settled, which means all of it’s children’s children are settled… etc.

In our example simply using await calls[0]; or even just await Promise.resolve() achieves the desired outcome as well. For some reason this doesn’t sit right with me, nor is it 100% reliable in all cases. I like the semantic cleanliness of saying “wait for all work related to this promise to be finished” so we can be sure of our state before moving forward with expectations.

Conclusion

I started this post thinking I was going to be 50/50 on synchronous promises adding benefit to your test suite. However as I explored the use cases I picked out I realized there wasn’t much benefit to using synchronous test cases, especially when using async/await. A lot of libraries I’ve researched that have synchronous promises tout a performance benefit but I haven’t seen any benchmarks shared. I might benchmark this myself at some point, but for the time being I see little benefit to synchronous promises in test cases. Most of the tooling around testing (Java|Type)Script is built with async in mind.

While I built promitto to fulfill a specific need I thought was common place it turned out I had just been working on a lot of Weird Stuff (TM) in short succession. I set out to build a tool I could use to control resolution timing and added settled() on a whim. I feel like that might be where most of promitto’s value is now. The times it is needed is niche but I really like it for it’s semantic value of “I need to wait for related work to finish before moving on” – which is something we’ll always need to think about as (Java|Type)Script developers, regardless of whether or not we’re using synchronous or asynchronous promises.