The labs on this site need JavaScript to run. The same content is written out below.

Learn › Spring › Other features

Testing

When tests run, what goes into the Spring context, and how many times is it created?

There are several ways to test a single controller. You can create the objects with new and call them without Spring, load all the beans with @SpringBootTest, or load only the web layer with @WebMvcTest. Depending on the approach, different beans go into the context (the container that creates and holds the beans), and unless you fill in the missing beans with mocks, the test cannot even start. Run code in which OrderController → OrderService → OrderRepository depend on each other in that order with two test classes, and check the result of each test and the number of times a context was created.

Checked against real execution · 36 combinations were run, each in a new JVM, on Spring Boot 4.1.1, spring-test 7.0.9, JUnit Jupiter 6.0.3, and Mockito 5.23.0 and checked against the results

Concepts in this lab

Unit tests and integration tests

A unit test verifies the behavior of a single method or class in isolation from other parts. It runs fast and makes the cause of a failure easy to find. An integration test actually connects several components (such as the Spring container and a database) and verifies that they work together. It is closer to reality but slow. The usual approach is to have many unit tests and to keep integration tests focused on the core flows.

Test double (mock, stub)

A test-only object that stands in for a real object that the code under test depends on. A stub is made to return fixed values, and a mock can also verify which methods were called and how. They let you verify only the logic of the code under test, quickly and consistently, without a database or an external API.

@SpringBootTest and @WebMvcTest

These are two annotations with which Spring Boot creates the Spring context that a test uses. @SpringBootTest registers all the beans, just as when the application is started, so you can check that several layers work together, but many beans are registered and the context is large. @WebMvcTest registers only the beans of the web layer, such as controllers, and does not register classes annotated with @Service or @Repository. So unless you add the service that a controller depends on yourself with @MockitoBean, the context cannot load, and the test method fails before it even runs. In Spring Boot 4, @WebMvcTest is in the org.springframework.boot.webmvc.test.autoconfigure package of the spring-boot-webmvc-test module.

@MockitoBean

This is an annotation that replaces a bean in the Spring context of a test with a mock created by Mockito. When you put it on a field of a test class, the mock takes the place of the bean of that type if the context has one, and the mock is added as a new bean if it does not. A method of the mock returns a default value such as null unless you set the value to return with when(...).thenReturn(...). Spring Boot 4 does not have the earlier @MockBean and uses this annotation, which is part of Spring Framework. When the set of @MockitoBean fields differs from one test class to another, different contexts are needed, and the context is created that many more times.

Context cache (TestContext framework)

This is the store in which the testing support of Spring keeps a context that it has created once, so that it can hand the context out again to other test classes with the same configuration. Things such as the configuration classes, the active profiles, and the set of @MockitoBean fields must all be the same for a context to count as the same one, and if even one of them differs, a new context is created. The cache is inside the JVM that runs the tests, and @DirtiesContext closes the context in question and removes it from the cache. When a context with the same configuration fails to load, by default Spring does not try to create it again and fails the later tests immediately. When the configuration differs a little from one test class to another, the context is started several times and the whole test run becomes slower.

Try these

• Default settings — Only @WebMvcTest is added. The context does not load, so the test cannot even start.
• Filling in the service with a mock — With a @MockitoBean OrderService, the context loads and the test passes.
• Filling in only the repository with a mock — What the controller requires is the service, so the context still does not load.
• Setting no return value on the mock — The context loads, but the mock returns null, so the test fails on the line that checks the value.
• Loading everything with @SpringBootTest — All three beans are registered as real ones, so the test passes even without a mock.
• Different @MockitoBean fields — The second class cannot use the context in the cache and creates a new one.
• Without Spring, with new — The test calls the controller method directly without ever creating a context.

Running with the default settings

1. OrderControllerTest starts: JUnit runs the first test class. The class is annotated with @WebMvcTest(OrderController.class), so before the test method is called, Spring prepares the context that this class will use.
2. Context cache lookup: Once Spring creates a context for tests, it keeps the context in a cache and hands it out again when a test class with the same configuration comes along. This is the first class, so the cache is empty. Spring starts creating a new context (#1).
3. Bean registration (@WebMvcTest): Only what belongs to the web layer is registered. OrderController, which is annotated with @RestController, becomes a bean, and OrderService, which is annotated with @Service, and OrderRepository, which is annotated with @Repository, are not registered.
4. OrderController cannot be created: To create the controller, Spring needs the OrderService bean that the constructor requires, but the context does not have one. A NoSuchBeanDefinitionException is thrown and the loading of the context stops.
5. returnsOrderName is recorded as failed: There is no context, so the @Autowired MockMvc field cannot be filled. Not a single line of the test method body runs, and the failure is recorded as an IllegalStateException that says "Failed to load ApplicationContext".
6. OrderControllerSecondTest starts: The second test class runs next in the same JVM. Its annotations and fields are the same as those of the first class. This class needs a context too.
7. Context cache lookup: The configuration is the same as that of the first class, but the context for that configuration failed to load, so it is not in the cache. Spring does not try again to create a context that has failed once with the same configuration (the default threshold is 1). Without even starting to create it, Spring fails the test immediately with an IllegalStateException that says "failure threshold (1) exceeded".

Result of this run

• @WebMvcTest registers only the beans of the web layer, such as controllers. The OrderService that the controller depends on must be filled in by you with @MockitoBean for the context to load.
• Once a context fails to load with a given configuration, Spring does not try to create the context for that configuration again and fails the later tests immediately. Even when several tests appear to have failed, there may be a single cause, so look at the message of the very first failure.

Code

// First test class
@WebMvcTest(OrderController.class)
class OrderControllerTest {

    @Autowired
    MockMvc mockMvc;                      // a tool that sends requests without a server

    @Test
    void returnsOrderName() throws Exception {
        mockMvc.perform(get("/orders/1"))
               .andExpect(status().isOk())
               .andExpect(content().string("keyboard"));
    }
}

// Second test class, which runs next
@WebMvcTest(OrderController.class)
class OrderControllerSecondTest {
    // The fields and returnsOrderName() are the same as in OrderControllerTest
}

// ---- Code under test: controller → service → repository ----
@RestController
class OrderController {
    private final OrderService orderService;

    public OrderController(OrderService orderService) {
        this.orderService = orderService;
    }

    @GetMapping("/orders/{id}")
    public String name(@PathVariable Long id) {
        return orderService.findName(id);
    }
}

@Service
class OrderService {
    private final OrderRepository orderRepository;

    public OrderService(OrderRepository orderRepository) {
        this.orderRepository = orderRepository;
    }

    public String findName(Long id) {
        return orderRepository.findName(id);
    }
}

@Repository
class OrderRepository {
    // A map used in place of a database
    private final Map<Long, String> db = Map.of(1L, "keyboard");

    public String findName(Long id) {
        return db.get(id);
    }
}

Assumptions of this model

All 36 combinations were actually run on this PC and checked against the results (3 testing approaches × 4 mock choices × 4 second-class choices; for a test without Spring, the second-class setting has no meaning, so only 4 are counted). The versions are Spring Boot 4.1.1, Spring Framework and spring-test 7.0.9, JUnit Jupiter 6.0.3, Mockito 5.23.0, and JDK 21. The two test classes were run one after the other with the JUnit Platform Launcher, without Gradle or Maven, and because the context cache lives inside the JVM, a new Java process was started for each combination. What was compared: the results of the two classes (passed, assertion failed, context failed to load, failure that was not retried), whether the controller, service, and repository that each class used were real, Mockito mocks, or absent, the number of times the creation of a context was started (counted as the number of times the constructor of the configuration class was called, including the ones that failed to load), the number of times loading succeeded, and whether the two classes received the same context object. Spring Boot 4 has no @MockBean (it was confirmed that the spring-boot-test jar of 4.1.1 does not contain that class) and uses @MockitoBean of Spring Framework (org.springframework.test.context.bean.override.mockito). @WebMvcTest and @AutoConfigureMockMvc are in the org.springframework.boot.webmvc.test.autoconfigure package of the spring-boot-webmvc-test module. The package names differ from those of 3.x, so take care when you copy import statements. How auto-configuration was controlled: the classpath of this PC also contains Spring Security, H2 and HikariCP, Hibernate, Spring Data JPA, and Actuator. In the check runs, the exclude attribute of @SpringBootApplication left out five Spring Security auto-configurations and the DataSource, Hibernate JPA, and Spring Data JPA repositories auto-configurations. When the lab was run without excluding them, the request of @SpringBootTest was blocked by Spring Security and became 401, and @WebMvcTest passed as before. (This is the result of a separate run made while the lab was being built, and it is not in the saved check output. On this classpath, the list of auto-configurations that @WebMvcTest loads did not include Spring Security. The behavior when the spring-boot-security-test module is present was not verified.) The @SpringBootApplication class is omitted on the screen. Differences between the code on the screen and the code used for the check: in the check code, the test classes have different names, they inherit the same body from an abstract class, and there are additional lines that record whether an object is real or a mock and additional code that counts the contexts. "Has one more @MockitoBean" is the case that replaces one more thing that the first class did not replace (OrderRepository, or OrderService if that is already replaced). The database is replaced with an in-memory map both on the screen and in the runs. The threshold after which a context that has failed once with the same configuration is not created again is spring.test.context.failure.threshold, and its default is 1 (confirmed from the constant in spring-test 7.0.9). For @DirtiesContext, only the default behavior when it is put on a class (the context is closed after the tests of the class finish) is covered. On this PC, the number of bean definitions in the context was 303 for @SpringBootTest and 130 for @WebMvcTest (131 when a mock bean is added), and these numbers change with the classpath and the exclude above. The time taken differs on every run, so it is not in the result table. In one run on this PC, the first class took about 0.7s to 0.9s for the test without Spring (including Mockito setup), about 2.7s for @WebMvcTest, and about 3.4s to 4.4s for @SpringBootTest, and the second class, which reused the context, took 0.05s or less. On JDK 21, Mockito attaches itself as an agent at run time in order to create mocks and prints a warning, but this does not affect the behavior, so it was left as it is. The @Transactional rollback of test methods, other slices such as @DataJpaTest, @MockitoSpyBean, the maximum size of the cache, and the case where a build tool splits the tests across several processes (the cache is separate for each process) are not covered and were not verified by execution either.

Previous lab: Actuator and health checks · Next lab: Calling external APIs and timeouts