Introduction

I would like to start a new series of posts called “Quick Tips”. In these articles, I will be sharing some tips and tricks I have gathered during my career as a software engineer. These articles will be shorter, focusing on a specific topic in each. I trust you’ll find them valuable!

The first topic I would like to cover is about a common anti-pattern I’ve seen in many tests. Let’s dive in!

Why Avoid Thread.sleep() in Your Tests?

Recently I’ve seen an integration test calling Thread.sleep() to wait for an async operation to complete. In my opinion, this approach is a red flag. Why?

Incorporating Thread.sleep() into your tests can introduce flakiness, leading to unreliable and inconsistent outcomes. This issue often manifests as the “it works on my machine” syndrome, where tests may pass in one environment but fail in another due to varying execution speeds or system conditions. To tackle this, developers tend to introduce constants - or even worse, magic numbers - for different environments: one for Alice’s machine, another for Bob’s, and another one for the CI-pipeline. This approach is a slippery slope that can lead to a maintenance nightmare.

An Illustration of the Problem

To shed light on the problem, let’s see a tangible, real-life example. In this integration test, we had a test case that verified that a given OpenSearch index was populated with data.

Here is the initial, naïve implementation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

@Test
@Order(4)
@DisplayName("OpenSearch Client should be able to list all persons")
void shouldListAllPersons() throws IOException {
    final var persons = new ArrayList<>(openSearchService.listAllPersons("persons"));
    assertEquals(1, persons.size());

    final var person = persons.get(0);
    
    assertNotNull(person);
    assertEquals("John", person.getFirstName());
    assertEquals("Doe", person.getLastName());
}

Let’s run the test:

1
2
3
4
5
6

mvn verify

...
[ERROR] com.github.tcsizmadia.opensearchsandbox.components.OpenSearchServiceIT.shouldListAllPersons -- Time elapsed: 0.097 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <1> but was: <0>
...

The test failed because the index was not yet populated by the time the assertion occurred. We were too fast!

To solve the problem we could call Thread.sleep() before the assertion:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

@Test
@Order(4)
@DisplayName("OpenSearch Client should be able to list all persons")
void shouldListAllPersons() throws IOException, InterruptedException {
    // Let's wait for 1 second to give the index time to populate
    Thread.sleep(1000L);

    final var persons = new ArrayList<>(openSearchService.listAllPersons("persons"));
    assertEquals(1, persons.size());

    final var person = persons.get(0);
    
    assertNotNull(person);
    assertEquals("John", person.getFirstName());
    assertEquals("Doe", person.getLastName());
}

1
2
3
4
5
6
7

mvn verify

...
[INFO] --- failsafe:3.2.1:verify (default) @ opensearch-sandbox ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Yay, the test passed! 🎉 Are we satisfied with this? Not exactly! I remain unconvinced that the index will consistently be populated within one second in every environment and during every test run. So, can we do it better?

A Better Solution: Awaitility

Meet Awaitility:

Awaitility is a DSL that allows you to express expectations of an asynchronous system in a concise and easy to read manner.

This is how we can use Awaitility to solve our problem:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

@Test
@Order(4)
@DisplayName("OpenSearch Client should be able to list all persons")
void shouldListAllPersons() {
    AtomicReference<ArrayList<Person>> persons = new AtomicReference<>(new ArrayList<>());

    await().atMost(5, SECONDS).untilAsserted(() -> {
        persons.set(new ArrayList<>(openSearchService.listAllPersons("persons")));
        assertEquals(1, persons.get().size());
    });

    final var person = persons.get().get(0);
    assertNotNull(person);
    assertEquals("John", person.getFirstName());
    assertEquals("Doe", person.getLastName());
}

Executing mvn verify resulted in a successful build — our tests passed as expected, which is positive. However, the most important achievement here is that we handled the async operation in a proper, reliable way.

Thanks to Awaitility’s semantic DSL, it’s pretty clear what happens here:

• we wait (await()) for the assertion to pass
• … but no more than 5 seconds (atMost(5, SECONDS)).
• the aforementioned assertion is defined inside of the lambda expression, which is passed to the untilAsserted() method.

If the index is not populated yet, the listAllPersons() method will return an empty list, and the assertion will fail. Awaitility will retry the assertion until it passes or the timeout is reached.

Conclusion

We’ve discovered a great tool we can use to write reliable tests when we have to cope with async operations. It’s a good practice to avoid Thread.sleep() in our tests and use a more sophisticated solution like Awaitility instead. In a future post, we will explore Awaitility in more detail.

I hope you learned something new today. 👋