Laytoun' thoughts!

A look into Spring Cloud Gateway!

Mohammed Aboullaite -

If you're (or thinking of) using microservices, you should then be familiar, or at least bumped into the API gateway pattern. This pattern is described with great details by Chris Richardson at microservices.io. An API gateway acts as a single entry point into a system and allows to centralize many cross cutting concerns such as: routing mechanisms, cache management, security, monitoring/metrics and resiliency.

In this article, we'll explore a new project from the Spring cloud suite : Spring Cloud Gateway project, built on Spring 5, Spring Boot 2 and Project Reactor.

Spring Cloud gateway

When it comes to choosing an API gateway for your microservices, there are a variety of options: Zuul from netflix, Kong, Nginx, HAProxy, Traefik, cloud vendor's solutions such as Amazon API Gateway or Google Cloud Endpoints and (the new) Spring Cloud gateway from Pivotal.

Spring Cloud Gateway aims to provide a simple, yet effective way to route to APIs. When it receives request, Spring Cloud Gateway forwards it to a Gateway Handler Mapping, which determines what should be done with requests matching a specific route.

Instead of taking a traditional approach and listing all the features, I rather prefer to build a small , microservices based project, that showcase and describe it usage

Getting started

The minimalist project, openly available on [github],(https://github.com/aboullaite/spring-cloud-gateway) is composed of 2 services (BookStore and MovieStore), a gateway (based on spring cloud gateway obviously), Service Discovery (Eureka server) and the Hystrix dashboard. The project also requires a redis instance running and listening, to make use of the request.rate limit feature of Spring Cloud Gateway,

Routing

Spring Cloud Gateway supports 2 forms of to create routes: pure java (using RouteLocator) or config files. Our project make use of the second approach!

Let's take a look at a snippet from the application.yml from the gateway module:

spring:
  cloud:
    gateway:
      routes:
      - id: movie-store
        uri: lb://movie-store
        predicates:
        - Path=/api/movies/**
        filters:
        - name: RequestRateLimiter
          args:
            key-resolver: '#{@userKeyResolver}'
            redis-rate-limiter.replenishRate: 2
            redis-rate-limiter.burstCapacity: 2
        - RewritePath=/api/(?<movies>.*), /$\{movies}
        - AddResponseHeader=X-Some-Header, aboullaite.me

This configuration tells Spring Cloud Gateway to route all requests made to the gateway at /api/movies/ to the movie-store service. Notice the use of predicates and filters, so we can route handle based on certain conditions as well as alter the request/response as needed.
Worth nothing to mention that there are 3 main building blocks for Spring Cloud Gateway:

  • Route: the primary API of the gateway. It is defined by an ID, a destination URI, a collection of predicates and a collection of filters. A route is matched if aggregate predicate is true.
  • Predicate: Java 8 Function Predicate. it enables to match on anything from the HTTP request, such as headers or parameters.
  • Filter: Standard Spring Framework's WebFilter, where requests and responses can be modified before or after sending the downstream request.

As you might have already noticed, in the snippet above we used 3 filters:

  • AddResponseHeader filter to add X-Some-Header header containing the value aboullaite.me in the response.
  • RewritePath filter to rewrite the request path from /api/movies/** to /movies/**
  • RequestRateLimiter to implement request rate limiter.

To read more about Route filters, refer to this link.

Rate limiting

Rate limiting is mainly used to control the rate of traffic sent or received on the network. It helps to prevent decrease in the service quality or even outage due to high traffic and improve the reliability of the API. There are different types of rate limiting, each used to target a specific need.

Unlike zuul which doesn't have an integrated out of the box solution for rate limiting, Spring Cloud Gateway comes with a RequestRateLimiter filter to determine if the current request is allowed to proceed. The only supported implementation for now is redis, that requires the use of the spring-boot-starter-data-redis-reactive Spring Boot starter.

The RequestRateLimiter takes an optional keyResolver parameter and parameters specific to the rate limiter. Notice that the key-resolver takes a SpEL expression referencing the bean with the name userKeyResolver. This bean implements the KeyResolver interface that help to derive the key for limiting requests:

    @Bean
    KeyResolver userKeyResolver() {
        return exchange -> Mono.just("fero");
    }

We've also declared 2 additional params:

  • replenishRate: represents how many requests per second a user is allowed to do.
  • burstCapacity: defines the maximum number of requests a user is allowed to do in a single second.

Furthermore, this filter adds additional headers that indicate what is the rate limit for a client and how quickly it’s being used:

  • X-RateLimit-Replenish-Rate indicates the replenishRate value.
  • X-RateLimit-Burst-Capacity indicates the burstCapacity value.
  • X-RateLimit-Remaining indicates how many calls you have remaining.

When request submissions exceed the replenish rate and burst limits, the gateway fails the limit-exceeding requests and returns 429 Too Many Requests error responses to the client.

Using Hystrix

Circuit Breaker is an interesting pattern that improve your API reliability and prevent a network or service failure from cascading to other services and clients. Spring Cloud Gateway comes with Hystrix support, a Netflix library that implements the circuit breaker pattern.

hystrix.command.fallbackcmd.execution.isolation.thread.timeoutInMilliseconds: 2000
spring:
  cloud:
    gateway:
      routes:
      - id: book-store
        uri: lb://book-store
        predicates:
        - Path=/api/books/**
        filters:
        - name: RequestRateLimiter
          args:
            key-resolver: '#{@userKeyResolver}'
            redis-rate-limiter.replenishRate: 2
            redis-rate-limiter.burstCapacity: 2
        - RewritePath=/api/(?<books>.*), /$\{books}
        - name: Hystrix
          args:
            name: booksFallbackCommand
            fallbackUri: forward:/fallback/books

The /books/danger endpoint in the books-store module simulate an endpoint with poor performance that takes a long time to send its response. We additionally wrap the route that uses this API in a HystrixCommand: booksFallbackCommand . Now when the Hystrix wrapped route times out (i.e takes more than 2s) it will call /fallback/books endpoint in the Gateway app, which simply return an empty list:

@RestController
@RequestMapping("/fallback")
public class FallbackController {

    @GetMapping("/books")
    public ResponseEntity<List> booksFallback(){
        return ResponseEntity.ok(Arrays.asList());
    }
}

Note that this example also demonstrates Spring Cloud Netflix Ribbon load-balancing via the lb prefix on the destination URI.

That's it for this article! We explored some of the features and components that are part of Spring Cloud Gateway, mainly the rate limit and circuit breaker features! The project I've used can be found on github.