Spring tips #1: structuring configuration for integration tests
Spring comes with a lot of features which support integration testing. We have MockMvc, TestRestTemplate and many other useful utilities. Nevertheless, building a proper test context for our app rests in our hands, and there are some pitfalls along the road. In today’s article I would like to show you how to avoid one of them.
Our config grows
For the purposes of this article let’s imagine that we are about to build an app for managing some kind of orders. When it comes to the first step of organizing the configuration part of an application, we usually begin sparingly with just a couple of annotations and beans:
@Configuration
@EnableAutoConfiguration
@EnableJpaRepositories
@ComponentScan(basePackages = {"com.tratif.demo.web"})
@PropertySource("classpath:order-app.properties")
class OrderAppConfig extends WebMvcAutoConfiguration {
@Bean
public OrderService orderService(OrderRepository orderRepository) {
return new OrderService(orderRepository);
}
// maybe one or two other beans
}
Later on, it appears that the autoconfigured DataSource does not suit our needs and that the default ObjectMapper lacks some features, which we need. So we start to enhance the configuration class:
// all of the previous annotations
@PropertySource("classpath:order-app.properties")
class OrderAppConfig extends WebMvcAutoConfiguration {
@Value("${datasource.url}")
private String dbUrl;
@Value("${datasource.username}")
private String dbUser;
@Value("${datasource.password}")
private String dbPassword;
@Value("${datasource.driver-class-name}")
private String dbDriver;
// all of the previous beans
@Bean
public DataSource dataSource() {
BasicDataSource dataSource = new BasicDataSource();
// data source configuration
return dataSource;
}
@Bean
public ObjectMapper objectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
// object mapper configuration
return objectMapper;
}
}
As life goes on, we craft separate configuration classes to encompass our application with security layer, some metrics and so on:
// all of the previous annotations
@Import({SecurityConfig.class, MetricsConfig.class})
class OrderAppConfig extends WebMvcAutoConfiguration {
// as before
}
So, as you see, our config grows with the amount of code and features, which we add to the application. What about integration tests then?
Integration tests configuration
There is a common practice amongst Spring developers to implement a base class base class to be extended by concrete integration test classes. Not only can we add some useful methods to it, but we also easily can share the same cached application context between tests executions. Such a common test base can look as follows:
@AutoConfigureMockMvc
@WebAppConfiguration
@ContextConfiguration(classes = OrderAppConfig.class)
class IntegrationTestBase {
@Autowired
private MockMvc mockMvc;
// helper method to reduce amount of code in test cases
ResultActions placeNewOrder(String orderJsonPath) throws Exception {
return mockMvc.perform(post("/order")
.contentType(MediaType.APPLICATION_JSON)
.content(fileAsString(orderJsonPath)));
}
private String fileAsString(String filePath) {
// some implementation
}
}
Having this base class we can start writing actual integration tests such as the simple one presented below:
@RunWith(SpringJUnit4ClassRunner.class)
public class OrderPlacementIT extends IntegrationTestBase {
@Test
public void shouldPlaceNewOrderAndReturnProperJsonResponse() throws Exception {
placeNewOrder("validOrder.json")
.andExpect(status().isCreated())
.andExpect(jsonPath("$.description", equalTo("Order description")))
// some other expectations
;
}
// and some other tests
}
So far so good — we have the whole context set up for testing purposes and everything works like a charm.
Pitfall emerges
There comes a day, however, when we wish to persist some statistics about our order management service — percentage of cancelled orders, the origin of customers etc. We decide, that Elasticsearch will be best to store such information, and to feed it with data we implement a scheduled job. All these new services demand a new configuration class:
@EnableScheduling
@Configuration
public class StatisticsConfig {
@Value("${elasticsearch.port}")
private int port;
@Value("${elasticsearch.host}")
private String host;
@Value(("${elasticsearch.clustername}"))
private String clustername;
@Bean(destroyMethod = "close")
public TransportClient transportClient() throws UnknownHostException {
return new PreBuiltTransportClient(esSettings())
.addTransportAddress(esTransportAddress());
}
@Bean
public StatisticsCollector statisticsCollector(OrderRepository orderRepository,
TransportClient transportClient) {
return new StatisticsCollector(orderRepository, transportClient);
}
private Settings esSettings() {
return Settings.builder()
.put("cluster.name", clustername)
.build();
}
private TransportAddress esTransportAddress() throws UnknownHostException {
return new TransportAddress(InetAddress.getByName(host), port);
}
}
Of course, we import this newly crafted config into our OrderAppConfig. But because we probably do not have real connection to Elasticsearch while running tests, and because scheduler kicks in from time to time, our test logs are now cluttered with lots of exceptions like this:
SEVERE: Unexpected error occurred in scheduled task.
NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{39PAGvtwS266yR2zLjD9WA}{localhost}{127.0.0.1:9360}]]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:360)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:405)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:394)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:53)
at com.mfedkowicz.service.StatisticsCollector.collectStatistics(StatisticsCollector.java:23)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Although our tests should still pass (after all the scheduled job has its own thread), it may be possible that they slow down a bit. So the solution seems to be easy — we would like to disable beans depending on Elasticsearch in the test context. But here comes the question — how can we achieve this?
It is worth noting that our tests still pass beacause of two reasons:
- we send statistics in a scheduled job, which is executed in a separate thread
- Elasticsearch TransportClient is lazy — it does not check if it is able to actually connect to a node upon creation
Specifically the latter point is interesting. You could imagine a situation, when we would like to fail early if connection parameters are wrong. This would require making some test call during bean creation. Keep in mind that such approach would actually cause our integration tests to fail in the current configuration, because Spring would encounter an exception while constructing the context.
Workarounds
There are some possible workarounds which can come to our minds at first.
We can mock TransportClient bean like this:
@AutoConfigureMockMvc
@WebAppConfiguration
@ContextConfiguration(classes = { OrderAppConfig.class })
class IntegrationTestBase {
@Autowired
private MockMvc mockMvc;
@MockBean
TransportClient transportClient;
// as before
}
This does not seem like a good idea though. Depending on the actual implementation of StatisticsCollector, it could require a lot of code to ensure, that no NullPointerExceptions will fly around.
So maybe, instead of ordering Spring to inject @MockBean for TransportClient class, we can declare @MockBean for StatisticsCollector? This sounds a bit better. Spring will ensure to use our mock implementation and our test logs will be clean. But remember that all other beans from StatisticsConfig will still be processed by Spring just to sit useless in the context. For more complex configuration classes it could slow down the startup time of our tests.
Thinking further about the problem we may realize that it would be best to disable the whole statistics configuration class in our tests. One of the possibilities is to use spring profiles like this:
@EnableScheduling
@Configuration
@Profile("!tests")
public class StatisticsConfig {
// as before
}
@AutoConfigureMockMvc
@WebAppConfiguration
@ContextConfiguration(classes = { OrderAppConfig.class })
@ActiveProfiles("tests")
class IntegrationTestBase {
// as before
}
This is probably the best of the solutions presented so far. But in my opinion it is still another workaround, which feels a bit cheap. After all, we have just patched our tests by adding an annotation to the production code! Annotation that has "!tests" as its value! It surely does not feel right and we can do much better.
Solution
The ultimate solution is actually really simple. If you paid attention you may have noticed that the first paragraph in this article hides the nasty truth. When we started to develop our application, we gathered all of the beans in one configuration class. Only after some time we started to divide the config. This is the real reason why we have so little room for maneuver in our tests. We cannot import configuration classes selectively, because OrderAppConfig is responsible for two things — it gathers all other configs and it produces its own beans.
We can further split our configuration, though, and make OrderAppConfig just a container for all of the other configuration classes:
@Configuration
@Import({
MVCConfig.class,
DbConfig.class,
SecurityConfig.class,
MetricsConfig.class,
StatisticsConfig.class
})
@PropertySource("classpath:order-app.properties")
class OrderAppConfig {
// empty
}
@Configuration
@EnableAutoConfiguration
@ComponentScan(basePackages = {"com.tratif.demo.web"})
public class MVCConfig extends WebMvcAutoConfiguration {
@Bean
public OrderService orderService(OrderRepository orderRepository) {
return new OrderService(orderRepository);
}
@Bean
public ObjectMapper objectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
// object mapper configuration
return objectMapper;
}
}
@Configuration
@EnableJpaRepositories
public class DbConfig {
@Value("${datasource.url}")
private String dbUrl;
// other db properties
@Bean
public DataSource dataSource() {
BasicDataSource dataSource = new BasicDataSource();
// data source configuration
return dataSource;
}
}
Having such configuration layout makes it really easy to get rid of StatisticsConfig in our base test class. We can simply import only those configs, which we are interested in at the moment:
@AutoConfigureMockMvc
@WebAppConfiguration
@ContextConfiguration(classes = {
MVCConfig.class,
DbConfig.class,
SecurityConfig.class,
MetricsConfig.class
// no StatisticsConfig here
})
class IntegrationTestBase {
// as before
}
No bean is loaded unnecessarily anymore and we can sleep peacefully. What is more, we have divided the pile of the beans from original OrderAppConfig in a more logical way, which makes the production code much easier to navigate.
Conclusion
Writing integration tests for a complex Spring application is not an easy task. Sometimes we encounter problems with building a test context, which will be reasonably similar to the production config, but will not break or clutter our tests at the same time. In such situations it is tempting to use one of the many features provided by Spring in order to somehow patch the problem. Usually, though, taking a step back and rethinking the layout of our production configuration classes can help to implement much cleaner tests.
I hope you find this small tip useful. Happy coding!