Microservices Level Up
Introduction to Microservices
August 17, 2020
Microservices Level Up
Service Discovery
August 17, 2020
Show all

The API Gateways

Microservices Level Up

Hello friends, how are you? We began the series of Microservice Architecture with two articles. This article is the third one from the same series. In this article, we will discuss the API Gateways. Like any other technology, the microservice architecture is constantly improving. The reason being that everyone is allowed to make the changes and come up with some improvement. It has a great future and API Gateways is also a part of it. So let us understand the what, the why, and the how regarding API Gateways!

What is an API Gateway and why do we need it?

Microservices are built in isolation. One should always understand this fact because this is one of the reasons why we need API Gateways. So what is an API Gateway? For any service-based architectures, there are a few common things or concerns. These concerns are shared among all the services. Microservice architecture is also a service-based architecture, right? Yes, it is! So there would be common concerns here as well. An API Gateway handles all these common cross-cutting concerns. One could consider it as a part of the upper layer of a software stack. In fact, API Gateway is one of those layers. 

Let us take a simple example. Consider an orchestra, the musicians or the members of the orchestra are microservices. They are playing their own instrument as per the requirement or guidelines. The director of this orchestra is the API Gateway. The director handles or manages the group of musicians and guides them to perform. Please note that the musicians are isolated and independent. They could have played anything they want, but they are working in a harmony under the guidance of the director. The soulful melody is an example of their precise team-work. In microservices, the API gateway ensures this harmony. If there isn’t an API Gateway, there are chances of no coordination among microservices. If ever, the coordination is good, then the same information has to be passed on to every microservice. Whereas with an API Gateway, all the common concerns are dealt by the API Gateway itself.

So what are the possible common concerns that the API gateway handles? While the concerns can vary as per the project requirement, we are trying to discuss the most common ones as below:

Number 1: Authentication

Number 2: Transport security

Number 3: Load-balancing

Number 4: Request dispatching 

Number 5: Dependency resolution

Number 6: Transport transformations

So now we know the importance of an API Gateway. Let us jump to the next section.

How to implement the API Gateway?

We have discussed the basic things related to API Gateway, but the most important task is to implement it. When it comes to implementation, API Gateway can be a bit tricky. So most of the time, these general considerations are included. Let us begin.

Design Considerations

Number 1: Authentication

Authentication is quite common. Most of the applications have authentication because even if there is no requirement for user authentication, there would be an authentication for the admin. But here, we are not talking about the usual authentication. We are discussing request/response authentication. It includes more than just id or password. So most of the gateways perform authentication for each request or for a series of requests. Each service is bounded by some rules and according to these rules, the gateway routes the requests to the related microservice. If the request can’t be processed, it returns an error code. There could be different reasons for unsuccessful requests/response.

Generally, most of the gateways include authentication information to the request while passing it to the microservices behind them. It helps the microservices to implement some user-specific logic whenever necessary. So the gateway should be reliable and precise.

Number 2: Performance and Scalability

Performance and scalability are the foundation characteristics of microservice architecture. Only a few companies like Netflix operate on an unimaginable scale. I specifically mentioned the word “unimaginable” because such companies handle billions of requests per day and their frequency is unpredictable. Sometimes, there could be thousands of requests per second and sometimes there could be tens of thousands. But whether it is a large company or a small company, performance and scalability are equally important. The responsibility of API Gateway is to ensure the same. The API Gateway should be built on such a platform that supports asynchronous non-blocking I/O.

For implementing a scalable API Gateway, numerous technologies can be used. For example, on the JVM, you can use NIO-based frameworks such as Netty, Vertx, Spring Reactor, or JBoss Undertow. If you wish to implement it in using a non-JVM option, then go for NodeJS. It is a platform built on Chrome’s JavaScript engine. Another option is NGINX Plus. It offers a mature, scalable, and high-performance server. It makes the deployment, configuration, and programming easy! NGINX Plus can be used to manage authentication, access control, load balancing requests, caching responses, and providing application-aware health checks and monitoring.

Number 3: Service Invocation 

One of the main reason for using an API to enable smooth inter-process communication among services. Any service oriented based application is a distributed system and inter-process communication becomes an important aspect. inter-process communication can be done in two different styles. One of them is the asynchronous style. It is a messaging-based mechanism. With the help of message broker services such as JMS or AMQP, it becomes possible to communicate. There are broker-less services as well, such as Zeromq. 

Here, the services are enabled to communicate directly. Another style of inter-process communication is the asynchronous communication. It can be done using HTTP, Thrift, etc. What do you think, a system uses which type of style from the above two? The answer is “both of them”. A system uses both types of styles. In fact, a system might use multiple implementations of each style, the only condition being that the API Gateway should support both the communication mechanisms. We are going to discuss about Inter-service communications in a future article of our microservice series.

Number 4: Service Discovery

For handling requests and communication, the API Gateway should know the location of each and every microservice. Here, the location refers to the IP address and port address. Talking about the traditional applications, in some of them, one could hardwire the locations. In modern cloud-based microservices, this is a non-trivial problem. Infrastructure services like message broker have a static location. It can be specified via OS environment variables. But determining the location of an application service is not easy because they have dynamically assigned locations.

The set of instances for a service could change dynamically due to auto scaling and upgrading. The API Gateway like any other service client uses the system’s service discovery mechanism, either Server-Side Discovery or Client-Side Discovery. If the system is using the Client-Side Discovery then the API Gateway should be able to query the Service Registry. It is a sort of meta-data or a database of all microservice instances. It also has their locations. We are going to discuss more about service discovery in the next article. It is an important article of our microservice architecture series.

Number 5: Handling Partial Failures

The problem of partial failure is quite a concern while implementing an API Gateway. This problem occurs in distributed systems during a situation when one service calls another service and the other service is unavailable or slow in responding. Due to this, the API Gateway gets blocked for an indefinite time which further leads to downstream service. So the main thing is to handle this failure in such a scenario by focusing on the service that is creating this problem. For example, the review service isn’t responding to the product details scenario. The API Gateway should instantly return the rest of the product details to the client because the client shouldn’t be waiting. For the client, even other things like product size, product prices, product recommendations, etc are important. The review part can be kept empty or replaced by something else. If the overall product information service isn’t responding, then the API Gateway should return an error and the error page/message shall be displayed to the client.

The other option is that the API Gateway can also return cached data, but it depends on its availability. For example, product prices change infrequently and the API Gateway can return cached product price data if the price service isn’t responding. The data can be stored in some external cache like Redis or can be cached by the API Gateway. If the cached data is returned to the user, the service failure doesn’t impact the user experience. But as mentioned, it depends on the availability. Another factor is that cached data won’t work if the value keeps changing. For example, in the stock market, the rates change anytime! Here, returning the cached data isn’t a good idea!

Let us know discuss a known use of an API Gateway : The Netflix API Gateway

We all know about Netflix right? Well, most of us must be knowing. It handles thousands of requests per day from all around the world. How does it handle all this? There is an amazingly useful library for writing code, it is known as the Netflix Hystrix. It helps in invoking the remote services. Hystrix is an expert at timing out calls that exceed a specified threshold. This is made possible with the help of a Circuit Breaker pattern. We are going to discuss about circuit breaker in a future article of our ongoing microservice architecture series. It stops the client from waiting needlessly because it is useless to wait for an unresponsive service. If the error rate for a service exceeds a pre-defined threshold, the circuit trips. All the requests fail immediately for a certain period of time. Hystrix also has a fallback action whenever such fails occur. It reads the value from a cache or returns some default value. Hystrix is a good replacement for JVM and for a non-JVM environment, one can use an equivalent library.

Guys, if you wish to know more about Hystrix, please visit the link below in the transcript.


There is a good demand for API Gateway developers in the industry. This is because API Gateway is very important. That is the reason why we are discussing API Gateway. We begin the article by discussing “what is an API Gateway” and then move on to “why do we need an API Gateway?”. Here we also understand the tasks of an API Gateway. It is important for inter-service communications and handling the common tasks. Without an API, things could get worse, especially with the large applications.

But having knowledge of an API Gateway isn’t enough! In fact, one would truly understand the importance of an API Gateway after implementing it. So for implementation, we discuss a few considerations like authentication, performance and stability, service invocation, service discovery, and handling partial failures. We end our article by discussing a real-time known example of Netflix and the use of API Gateway. So these are the basics of API Gateway. I find it quite interesting and worth learning/practicing. So don’t stop yourself from, dive deep into the ocean of Microservice Architecture and its essentials like API Gateway.

Here is the link to the previous article of this series.

Tao is a passionate software engineer who works in a leading big data analysis company in Silicon Valley. Previously Tao has worked in big IT companies such as IBM and Cisco. Tao has a MS degree in Computer Science from University of McGill and many years of experience as a teaching assistant for various computer science classes.

Leave a Reply

Your email address will not be published.


get free access to this free guide, downloaded over 200,00 times !

You have Successfully Subscribed!

Level Up Big Data Pdf Book


get free access to this free guide, downloaded over 200,00 times !

You have Successfully Subscribed!

Jenkins Level Up

Get started with Jenkins!!!

get free access to this free guide, downloaded over 200,00 times !

You have Successfully Subscribed!