Effective RESTful search API in Spring
Flexible RESTful search is very often a must-have for a web application. While the concept is easy and not new, it is very often implemented in a repetitive and tedious way. In this post I will demonstrate an effective way of implementing search API with Spring Data and Specification Argument Resolver.
Naive approach
With Spring Data it is very easy to create Repositories with custom search methods. For example:
public interface CustomerRepository extends JpaRepository<Customer, Long> {
List<Customer> findByFirstName(String firstName, Pageable pageable);
}
Then you can use this repository in your controller as follows:
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByFirstName(
@RequestParam("firstName") String firstName,
Pageable pageable) {
if (firstName == null) {
return customerRepo.findAll(pageable);
} else {
return customerRepo.findByFirstName(firstName, pageable);
}
}
}
Thanks to Spring Data support, we can easily map HTTP query parameters into a Pageable controller parameter. Unfortunately, we have to manually manage other request parameters to determine appropriate repository method. While it is not a big problem for a single request parameter, this approach becomes unacceptable when there are more variables.
Repository method explosion
Let’s assume that we want to filter customers by first and last name as well as their status. All filters are optional. This leads to repository method explosion and nasty ifology in the controller:
public interface CustomerRepository extends JpaRepository<Customer, Long> {
List<Customer> findByFirstName(String firstName, Pageable pageable);
List<Customer> findByLastName(String firstName, Pageable pageable);
List<Customer> findByFirstNameAndLastName(
String firstName, String lastName, Pageable pageable);
// other combinations omitted for sanity
}
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByFirstName(
@RequestParam("firstName") String firstName,
@RequestParam("lastName") String lastName,
@RequestParam("status") Status status, Pageable pageable) {
if (firstName != null) {
if (lastName != null) {
if (status != null) {
return customerRepo.findByFirstNameAndLastNameAndStatus(
firstName, lastName, status, pageable);
} else {
return customerRepo.findByFirstNameAndLastName(
firstName, lastName, pageable);
}
} else {
// other combinations omitted for sanity
}
} else {
// other combinations omitted for sanity
}
}
}
This is not an acceptable approach, but fortunately there is an existing solution to these problems.
Using Specifications
Spring Data introduces Specification abstraction, which can be used with a repository. Specification is a simple interface which provides toPredicate method which should return a JPA2 Criteria Predicate:
public interface Specification<T> {
Predicate toPredicate(Root<T> root, CriteriaQuery query, CriteriaBuilder cb);
}
A sample implementations for our Customer entity may have the form of:
class CustomerWithFirstName implements Specification<Customer> {
private String firstName;
// constructor omitted for brevity
public Predicate toPredicate(Root<T> root, CriteriaQuery query, CriteriaBuilder cb) {
if (firstName == null) {
return cb.isTrue(cb.literal(true)); // always true = no filtering
}
return cb.equal(root.get("firstName"), this.firstName);
}
}
class CustomerWithStatus implements Specification<Customer> {
private Status status;
// constructor omitted for brevity
public Predicate toPredicate(Root<T> root, CriteriaQuery query, CriteriaBuilder cb) {
if (status == null) {
return cb.;
}
return cb.equal(root.get("status"), this.firstName);
}
}
An important part is that our specification returns a dummy predicate if the filtering value is not available (see highlighted lines above). This will result in a query without any filtering, which is the desired behaviour for this API.
Any Specification can be passed to our repository as long as the repository implements JpaSpecificationExecutor. Multiple specifications can be combined with JPQL and or or. Spring Data provides Specifications helper class to achieve that:
Specification<Customer> activeSimpsons =
Specifications.where(new CustomerWithLastName("Simpson"))
.and(new CustomerWithStatus(ACTIVE));
The last part is resolving our Specification from HTTP query parameters in the controller:
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomers(
@RequestParam("firstName") String firstName,
@RequestParam("lastName") String lastName,
@RequestParam("status") Status status,
Pageable pageable) {
Specification<Customer> spec = Specifications.where(new CustomerWithFirstName(firstName))
.and(new CustomerWithLastName(lastName))
.and(new CustomerWithStatus(status));
return customerRepo.findAll(spec, pageable);
}
}
It is much better, but still there is a lot of tedious work to do (with writing custom specifications and then resolving them in the controller). Fortunately Specification Argument Resolver library can do all of that for us!
Automated Specification resolving
With Specification Argument Resolver you don’t have to write any implementations of Specification. It will be generated (at runtime) automatically, based on some annotations which you must add to the controller:
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomers(
@And({
@Spec(path = "firstName", spec = Equal.class),
@Spec(path = "lastName", spec = Equal.class),
@Spec(path = "status", spec = Equal.class)
}) Specification<Customer> customerSpec,
Pageable pageable) {
return customerRepo.findAll(customerSpec, pageable);
}
}
Additional benefit is that you can easily change the filtering logic (e.g. by switching equal to like or accepting multiple allowed statuses with in keyword). You can also extract annotations to a separate empty interface to keep your controllers simpler:
@And({
@Spec(path = "firstName", spec = Like.class),
@Spec(path = "lastName", spec = Like.class),
@Spec(path = "status", spec = In.class)
})
interface CustomerSpec extends Specification<Customer> {
}
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomers(CustomerSpec customerSpec, Pageable pageable) {
return customerRepo.findAll(customerSpec, pageable);
}
}
Much better, isn’t it? In the next sections we will explore other features of the library to further polish the API.
Flexible filtering
Let’s assume, that our Customer entity has multiple names mapped in an embedded Names class:
@Embeddable
class Names {
private String firstName;
private String lastName;
private String nickName;
// ...
}
@Entity
class Customer {
@Embedded
private Names names;
// ...
}
Let’s also assume, that we want to send a single name HTTP query parameter (/customers?name=Homer). All three names should be compared against the value of that parameter. We can easily achieve that by providing params value to @Spec annotation:
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByFirstName(
@Or({
@Spec(path = "names.firstName", params = "name", spec = Like.class),
@Spec(path = "names.lastName", params = "name", spec = Like.class),
@Spec(path = "names.nickName", params = "name", spec = Like.class)
})
Specification<Customer> customerSpec,
Pageable pageable) {
return customerRepo.findAll(customerSpec, pageable);
}
}
The above implementation would translate the following request:
GET /customers?name=Homer
into the following query:
select c from Customer c where c.names.firstName like '%Homer%'
or c.names.lastName like '%Homer%'
or c.names.nickName like '%Homer%'
Explicitly specifying HTTP parameters is especially useful for date range filtering. Let’s assume that we want to find all customers registered in the given period:
GET /customers?registeredAfter=2017-11-17®isteredBefore=2017-11-20
We can achieve that as follows:
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByFirstName(
@Spec(
path = "registrationDate",
params = { "registeredAfter", "registeredBefore" },
spec = DateBetween.class
)
Specification<Customer> customerSpec,
Pageable pageable) {
return customerRepo.findAll(customerSpec, pageable);
}
}
Filtering by Join attributes
What if we need to filter by attributes of joined entities? For example, our customers may have multiple addresses associated with them:
@Entity
class Address {
private String city;
@ManyToOne
private Customer customer;
// ...
}
@Entity
class Customer {
@OneToMany(mappedBy = "customer")
private Collection<Address> addresses;
// ...
}
We can use @Join annotation to specify the join and the related alias. Then we can just use the alias in @Spec:
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByCity(
@Join(path = "addresses", alias = "a")
@Spec(path = "a.city", params = "city", spec = Like.class)
Specification<Customer> customerSpec,
Pageable pageable) {
return customerRepo.findAll(customerSpec, pageable);
}
}
Of course you can use @Join with @And and @Or freely. You can also move annotations to a separate interface. A more complex example could have the form of:
@Join(path = "addresses", alias = "a")
@And({
@Spec(path = "a.city", param = "city", spec = Like.class),
@Spec(path = "status", spec = In.class);
})
interface CustomerSpec extends Specification<Customer> {
}
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByCity(CustomerSpec customerSpec,
Pageable pageable) {
return customerRepo.findAll(customerSpec, pageable);
}
}
Dealing with soft deletes
It is a common practice for web and enterprise applications to use soft delete, i.e. whenever an entity is supposed to be deleted, it is just marked with a flag (e.g. deleted = true ) instead of being physically removed from the database. This approach has many benefits, but makes querying more complex — we have to add where deleted = false to each search method. It is tedious and error-prone, so let’s see how we can make it easier.
For starters, we prepare the following specification interface:
@Spec(path = "deleted", constVal = "false", spec = Equal.class)
public interface NotDeletedEntity extends Specification<Customer> {}
Then we can use it as a foundation for our search methods. The first option is to use it as a controller method parameter and add additional annotations to it:
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByName(
@Spec(path = "name", spec = Like.class) // this specification ...
NotDeletedEntity spec, // ... will be combined with the one from NotDeletedEntity
// with 'and' keyword
Pageable pageable) {
return customerRepo.findAll(spec, pageable);
}
}
And this is it. We just have to use NotDeletedEntity instead of Specification as the controller method parameter type and the and deleted = false clause will be added to the query.
The second option is useful when we want to keep all the annotations on types instead of parameters. We can use interface inheritance as follows:
@Spec(path = "name", spec = Like.class)
interface NotDeletedCustomerByName extends NotDeletedEntity {}
@RestController
@RequestMapping("/customers")
public class CustomerController {
@Autowired
private CustomerRepository customerRepo;
@GetMapping
public Page<Customer> findCustomersByName(NotDeletedCustomerByName spec,
Pageable pageable) {
return customerRepo.findAll(spec, pageable);
}
}
All specification annotations from parent interfaces are combined with and keyword which is exactly what we need for the API.
Summary
This post presented implementation challenges related to RESTful search API with Spring Web and Spring Data. The naive approach resulted in the following problems:
- repository method explosion — each combination of HTTP query params needed a separate method in the repository
- tedious code in the controller — monstrous if-else logic to determine the parameter combination and select appropriate repository method
Spring Data Specification support simplified the implementation. There was no need to write multiple repository methods and the controller code was much simpler. Still, there was a need to write specification classes and combine them manually.
Finally, Specification Argument Resolver library was presented. It generates specifications on the fly, based on annotations. It also provides a simple way to combine different specifications (with or or and logic), resolves joins and handles soft deletes easily. The resulting implementation not only provides a flexible API but is also very concise. Of course there are other libraries which you can use (e.g. Querydsl), but I believe that the approach described here is a very lightweight option which you should consider for your project. Enjoy!
First of all, i would like to congratulate you for this post .
So i have a situation, in this type of approach with only Controller + Repo if i have a need to make some business calculation or some data manipulation, where can i put this? in the controller or i have to create another layer such as service?
thx for support.
Great! You eliminated bunch of redundancy code from this world)
Thank You.
Mr. Tomasz Kaczmarzyk, may God bless you !!
Mr. Tomasz Kaczmarzyk, the post is really admirable… Got high level confidence in writing complex search queries including joins..Hats off to you…
Hi Tomasz,
I got error below while hitting the request.
java.lang.IllegalArgumentException: Invoked method public abstract javax.persistence.criteria.PredicateCan you please have a look.
I am using Specification Argument Resolver
Hi Rawn, can you provide some test project / sample code to reproduce this issue?
Hi Tomasz,
Declared below in controller:
@And({ @Spec(path = "firstName", spec = LikeIgnoreCase.class), @Spec(path = "lastName", spec = LikeIgnoreCase.class), @Spec(path = "status", spec = LikeIgnoreCase.class) }) interface RoomSpec extends Specification { }and then the request handler method:
@GetMapping("/searchdetail") public List searchRooms(RoomSpec rmSpec){ return studentDAO.detailSearchRooms(rmSpec); }Initially i was getting below error:
{ "timestamp": 1561475862565, "status": 500, "error": "Internal Server Error", "exception": "org.springframework.beans.BeanInstantiationException", "message": "Failed to instantiate [org.springframework.data.jpa.domain.Specification]: Specified class is an interface", "path": "/hostel/searchdetail" }code was:
/* Detail Search rooms */ @GetMapping("/searchdetail") public List searchRooms( @And({ @Spec(path = "room_status",params = "room_status", spec = LikeIgnoreCase.class), @Spec(path = "room_catgry",params = "room_catgry", spec = LikeIgnoreCase.class), @Spec(path = "room_type",params = "room_type", spec = LikeIgnoreCase.class)}) Specification roomSpec) { return studentDAO.detailSearchRooms(roomSpec); }Then I changed.
Did you enable the resolver as described here: https://github.com/tkaczmarzyk/specification-arg-resolver#enabling-spec-annotations-in-your-spring-app ?
yes already enabled.
public class StudentApplication { public void addArgumentResolvers(List argumentResolvers) { argumentResolvers.add(new SpecificationArgumentResolver()); argumentResolvers.add(new PageableHandlerMethodArgumentResolver()); } public static void main(String[] args) throws Exception { SpringApplication.run(StudentApplication.class, args); } @Configuration @EnableJpaRepositories static class Config { public void addArgumentResolvers(List argumentResolvers) { argumentResolvers.add(new SpecificationArgumentResolver()); } } }I tried combining
@Join,@AndwithGreaterThan, ex.:@Join(path = "children", alias = "c") @And({ @Spec(path = "c.price", params = "priceFrom", spec = GreaterThan.class) })It does not work, keeps throwing
QuerySyntaxException– it tries to filter ‘price’ field but in parent class, not child. When I change spec toEqual, it does not throw an exception.Hey. I want to implement the methods in the search controller, but it does not work. I implement the method
getAll(), by default:on empty parameters on return:
/rest/ships@RestController @RequestMapping ("/ rest") public class ShipController { @GetMapping (value = "/ ships", produces = {MediaType.APPLICATION_JSON_UTF8_VALUE}) public Iterable getAll (@RequestParam (value = "pageNumber", defaultValue = "0") int page, @RequestParam (value = "pageSize", defaultValue = "3") int limit, @RequestParam (value = "order", defaultValue = "id") String orderBy) { Page list = service.getAll (page, limit, orderBy); // ... } // ... }This method works and the data is returned.
Further, when filling in the search form, you must optionally add parameters, they may be in different combinations.
For example:
/rest/ships?&minSpeed=0.5&maxCrewSize=5000&pageNumber=0&pageSize=3&order=IDI did it like this:
@GetMapping ( value = "/ ships", produces = {MediaType.APPLICATION_JSON_UTF8_VALUE}, params = {"pageNumber", "pageSize", "order"} ) public Iterable getAllWithFilter ( @And ({ @Spec (path = "pageNumber", defaultVal = "0", spec = Equal.class), @Spec (path = "pageSize", defaultVal = "3", spec = Equal.class), @Spec (path = "order", defaultVal = "id", spec = Equal.class), @Spec (path = "name", spec = Like.class), @Spec (path = "planet", spec = Like.class), @Spec (path = "after", spec = DateAfter.class), @Spec (path = "before", spec = DateBefore.class), @Spec (path = "minCrewSize", spec = GreaterThanOrEqual.class), @Spec (path = "maxCrewSize", spec = LessThanOrEqual.class), @Spec (path = "minSpeed", spec = GreaterThanOrEqual.class), @Spec (path = "maxSpeed", spec = LessThanOrEqual.class), @Spec (path = "minRating", spec = GreaterThanOrEqual.class), @Spec (path = "maxRating", spec = LessThanOrEqual.class), @Spec (path = "shipType", spec = Equal.class), @Spec (path = "isUsed", spec = Equal.class, defaultVal = "false") }) Specification customerSpec) { return service.getAllWithFilter (customerSpec); }This method does not work. I tried many different options, but, they do not work. Tell me how to properly implement the method.
Hi, I want to know is it possible that combine the specs dynamically? For example, I want to search users by username, but when the param is empty I want to return all users. It will be more than one conditions like that, so it’s hard to generate all methods we need, can I use the tools to achieve that?
I think this should be all possible with the current features. Please take a look on the README and especially on the https://github.com/tkaczmarzyk/specification-arg-resolver#handling-different-field-types
Great,
Thank you. Your post inspires me to create an efficient REST API.
Here you find my article “Spring Boot: How to design efficient REST API?”
https://medium.com/@makhlouf.raouf/spring-boot-how-to-design-efficient-rest-api-a0d90a6e5a27
By the way, you find That I referred to your article.
Please don’t hesitate to make feedbacks
You saved my day 😀
I update the post :
https://medium.com/quick-code/spring-boot-how-to-design-efficient-search-rest-api-c3a678b693a0
Thank you
Hi there. I am trying to write unit test for rest controllers which I used SPEC implementation on it. But I have an exception. Is there any simple example which shows writting the unit test for rest controllers with SPEC?
The exception I got is:
Request processing failed; nested exception is java.lang.IllegalStateException: No primary or default constructor found for interface org.springframework.data.jpa.domain.Specification
Any idea?
Did you add Specification Argument Resolver to your context configuration (using WebMvcConfigurer) as indicated in the readme? This looks to me like missing step in the test setup.
Does it support Spring Data Rest?
The library should have no conflicts with any other Spring projects IMHO.
Hey! Can I define in constVal reference to the function to calculate const value programmatically?
Example:
—-
MyClass calc value.. for example..
if current auth user has role USER then value=userID else value=null
—
If I understand your question correctly then yes, you can — please take a look on https://github.com/tkaczmarzyk/specification-arg-resolver#spel-support. I imagine you would have to declare a bean with the desired logic and access it from a SpEL expression.
The specification-arg-resolver uses multiple other libraries with vulnerabilities (indicated by maven).
This means, it is not usable for critical applications.
The only non-test and non-optional dependencies are: javax persistence and servlet api, spring-web, spring-webmvc and commons-lang3. All versions are managed by spring-boot-dependencies artifact. In a properly configured project it’s not even up to the library which versions of Spring-managed dependencies will be used (and if it’s really needed, versions may be easily overridden).