Smarter search with RSQL

RSQL is a query language for parametrized filtering of entries in RESTful APIs. It’s based on FIQL (Feed Item Query Language), that was originally specified by Mark Nottingham as a language for querying Atom feeds. However the simplicity of RSQL and its capability to express complex queries in a compact and HTTP URI-friendly way makes it a good candidate for becoming a generic query language for searching REST endpoints.

RSQL overview

RSQL introduces simple and composite operators which can be used to build basic and complex queries. The following table lists basic operators:

| Basic Operator | Description         |
|----------------|---------------------|
| ==             | Equal To            |
| !=             | Not Equal To        |
| =gt=           | Greater Than        |
| =ge=           | Greater Or Equal To |
| =lt=           | Less Than           |
| =le=           | Less Or Equal To    |
| =in=           | In                  |
|=out=           | Not in              |

These six operators can be used to do all sort of simple queries, for example:

• name==Fero: find all people whose name is Fero
• street!=Darna: find all people who do not live at Darna
• age=gt=10: find all people older than 10 (exclusive)
• age=ge=10: find all people older than 10 (inclusive)
• house=lt=3: find all people who have less than 3 houses
• house=le=3: find all people who have less than or 3 houses

The following table lists two joining operators:

| Composite Operator | Description         |
|--------------------|---------------------|
| ;                  | Logical AND         |
| ,                  | Logical OR          |

These two operators can be used to join the simple queries and build more involved queries which can be as complex as required. Here are some examples:

• age=gt=10;age=lt=20: find all people older than 10 and younger than 20
• age=lt=5,age=gt=30: find all people younger than 5 or older than 30
• age=gt=10;age=lt=20;(str=Fero,str=Hero): find all people older than 10 and younger than 20 and living either at Fero or Hero.

Note that while the complexity of the queries can grow, the complete expression still remains in a form which is easy to understand and quite compact. The latter property becomes very useful when considering how to embed RSQL queries into HTTP URIs.

Showtime

To illustrate building a REST API Query language, we're going to make use of a cool library: rsql-parser.
First, let’s add maven dependency to the library:

<dependency>
    <groupId>cz.jirutka.rsql</groupId>
    <artifactId>rsql-parser</artifactId>
    <version>2.1.0</version>
</dependency>

We need also to define the main entity we’re going to be working with :

public class Book {

    @Id
    @GeneratedValue
    private Long id;

    @NotNull
    private String title;

    @Column(length = 10000)
    @Size(min = 1, max = 10000)
    private String description;

    @NotNull
    private String isbn;

    @Column(name = "publication_date")
    @Temporal(TemporalType.DATE)
    @Past
    private Date publicationDate;

    @Column(name = "nb_of_pages")
    private Integer nbOfPages;

    @Enumerated
    private Language language;
...
}

The way RSQL expressions are represented internally is in the form of nodes and the visitor pattern is used to parse out the input. With that in mind, we're going to implement the provided RSQLVisitor interface (or the simplified NoArgRSQLVisitorAdapter).
But instead of doing this, we're going to use another nice library: rsql-jpa. This library provides converter of RSQL expression to JPA Criteria Query (object representation of JPQL), which is translated to SQL query. Also, the examples bellow are based on Spring, with both libraries, to add search REST endpoints.

Let's first dig into our BookService service, that contains simple methods converting given RSQL query into JPA query:

@Service
public class BookService {
    /**
     * Run the FIQL Query to SQL, execute it and return results
     * More detials about it: https://github.com/jirutka/rsql-parser
     * @param queryString
     * @return
     */
    public List<Book> searchByQuery(String queryString) {
        RSQLVisitor<CriteriaQuery<Book>, EntityManager> visitor = new JpaCriteriaQueryVisitor<>();
        CriteriaQuery<Book> query;
        query = getCriteriaQuery(queryString, visitor);
        List<Book> resultList = entityManager.createQuery(query).getResultList();
        if (resultList == null || resultList.isEmpty()){
            return Collections.emptyList();
        }
        return resultList;
    }

    /**
     * Run the FIQL Query to SQL, execute it and return the count of the query
     * More detials about it: https://github.com/jirutka/rsql-parser
     * @param queryString
     * @return
     */
    public Long countByQuery(String queryString) {
        RSQLVisitor<CriteriaQuery<Long>, EntityManager> visitor = new JpaCriteriaCountQueryVisitor<Book>();
        CriteriaQuery<Long> query;
        query = getCriteriaQuery(queryString, visitor);

        return entityManager.createQuery(query).getSingleResult();
    }

    private <T> CriteriaQuery<T> getCriteriaQuery(String queryString, RSQLVisitor<CriteriaQuery<T>, EntityManager> visitor) {
        Node rootNode;
        CriteriaQuery<T> query;
        try {
            rootNode = new RSQLParser().parse(queryString);
            query = rootNode.accept(visitor, entityManager);
        }catch (Exception e){
            log.error("An error happened while executing RSQL query", e);
            throw new IllegalArgumentException(e.getMessage());
        }
        return query;
    }

Easy peasy! Then, adding a corresponding REST endpoint is trivial:

@RestController
@RequestMapping("/books")
public class AttributeRessource {
    @Autowired
    BookService bookService;
/**
     *
     * @param query
     * @return the ResponseEntity with status 200 (OK) with a list of Books that matched your query,
     *  or status 400 (Bad request) if the attribute couldn't be returned
     */
    @GetMapping(path = "/", produces = { MediaType.APPLICATION_JSON_VALUE })
    public ResponseEntity<List<Book>> query(@RequestParam(value = "q") String query) {
        List<Book> result = null;
        try {
         result= bookService.searchByQuery(query);
        }catch (IllegalArgumentException iae){
            return  ResponseEntity.status(HttpStatus.BAD_REQUEST)
                    .body(result);
        }
        return  ResponseEntity.status(HttpStatus.OK)
                .body(result);
    }
...

Here’s a sample URL:

http://localhost:8080/books?search=title==feroHero*; nbOfPages<200

And the response:

[{
    "id":1,
    "title":"Fero Hero",
    "description":"Fero Sadi9 Hero",
    "isbn":"fero@hero.com",
    "publicationDate": "12/12/2016",
    "nbOfPages": 180,
    "language": "DARIJA"
}]