Where is my cab? What time will my train arrive? Was there a price drop in the stock that I invested in? Today, users increasingly demand ever more interactive experiences, and they expect to be automatically informed when something has changed without having to hit the refresh button.
Receiving notifications about someone liking your picture or reacting to your story on Instagram, or your Facebook news stream showing the latest updates about what your friends have been up to, are just a few examples of how frontend applications react to events in the backend. It is all made possible through event-driven APIs. These are supported through various modes of asynchronous event-driven communication between the client application and the backend.
So, the question is, how do you build apps that require (near) real-time updates based on remote events? How should developers and architects who create reactive web applications or expose APIs for the consumption of such applications handle asynchronous or event-driven communication in the world of APIs where synchronous communication is predominant and most firewalls block non-HTTP traffic?
The answer? It is all about having an event-driven backend, choosing an asynchronous web API technology based on exact async requirements, and a client that can work with remote events.
Synchronous APIs versus asynchronous web API technology
Many APIs that make up the web today are synchronous APIs. This is where the client initiates all communication between the client and the backend. It does this by sending a request to the backend and waiting for a response. However, these request/reply API calls happen one at a time, in a pre-arranged sequence. Each interaction blocks the progress of the process until its completion.
If the client application wishes to know about an update, it must continuously invoke the API for updates at a regular interval. This is known as polling. It has been a common approach for client apps that need to become aware of new data. But this method is largely inefficient and less reactive than if the backend immediately notified the client when an event occurs. The term “polling madness” was coined because most of these polling calls are wasted because the data has not changed.
When should you use event-driven APIs?
Event-enabling APIs is a relatively recent phenomenon compared to event-driven architecture (EDA). It uses EDA to support scalable, real-time or near-real-time, push-based communication in APIs published to third parties. Being event-driven is about taking advantage of events as they happen. Consumers can subscribe and register their interest and react to events.
Making APIs event-driven or asynchronous (along with an underlying server-side event-driven architecture) can eliminate the need for inefficient polling requests. It sends updates to the client or event subscriber as soon as they occur. It provides a much better experience for users.
So, in a nutshell, event-driven APIs should be used if the application:
- needs to push changes from the server to the client rather than waiting for a client request
- must provide users with a highly active two-way communication channel
- requires many continuous interactions between the client and the backend that create scaling issues for the backend if synchronous APIs are used
- needs to be able to monitor system-wide events
Choosing asynchronous API technology
Event-enabling APIs do come with multiple complexities. It starts with what frameworks and networking protocols to choose. Then, building the reliability of delivery and finally ensuring the scalability of the solution. The asynchronous protocols commonly used today solve the problem of real-time communication. They solve different aspects of this problem in different ways. It means that some protocols serve different purposes better than others:
- For multiplexed, bidirectional streaming and for applications that need a considerable number of messages from both ends of the connection, WebSockets is ideal.
- Server-Sent Events are especially useful when a browser client is just interested in subscribing to a stream of data without needing to communicate back with the server in the same connection.
- Webhooks can be used for a simple implementation of pushing notifications to one or a small number of servers.
- The newer async variants of GraphQL (Subscriptions and Live Queries) come with the benefits associated with GraphQL. However, the implementation is relatively complex and entails a considerable learning curve.
- gRPC and Kafka are generally used for communication between internal microservices.
Understanding what the backend architecture should look like
The client architecture subscribes to state changed events that originate from the backend. The typical backend architecture can be extended to create remote event connections with the clients via:
- Event-driven APIs (ideally exposed through an API management platform that supports eventing semantics).
- A message broker (if one is not already present).
- Microservices that publish and process state changed events.
Although event-driven APIs do not explicitly require a broker, using an intermediary between event producers and consumers helps implement the required patterns to deliver more manageable and scalable solutions. The broker receives events from IoT devices, change data capture (CDC) tools, other backend systems and services, and from client applications if two-way communication is enabled. It can then alert the services that subscribe to those events.
A scalable microservices architecture (MSA) is the optimal architecture for complex event-driven backend services. These event-driven microservices can act as event subscribers or publishers to process events, handle errors, and persist event-driven states.
The remote event connection between the backend and web clients can be established through event-driven APIs powered by various asynchronous API technologies — such as Webhooks, WebSockets, Server-Sent Events, GraphQL subscriptions, etc. The backend uses these APIs to send events pertinent to the clients and receive events that originate from clients.
Managing event-driven APIs
An event-driven backbone will manage the overall real-time data flow securely and at scale. Asynchronous APIs can be managed for external and internal consumption with a traditional API management solution. It comes with inherent or extended capabilities to support event-driven semantics. Most organisations have basic event processing infrastructure, but many do not have the capabilities to design, develop, test, and manage event-centric APIs.
Essential API management capabilities — particularly governance, access control, monitoring, analytics, and monetisation — provided by API management solutions can be used to manage asynchronous and REST APIs. Some API management solutions already support Webhooks or/and WebSockets.
Expanding business reach and adoption
Synchronous APIs are ideal for many scenarios, particularly if you need an instant response. In other cases, when clients need to be informed of events or the processing required for the response happens at a different time, ordinary synchronous messaging becomes tricky. This is where event-driven APIs can aptly address such needs.
EDA is flexible enough that you can start with simple notifications. You can also combine two-way event communication with a synchronous approach to deliver an optimal architecture. Using an event-driven design alongside a traditional request/response design for your APIs gives you the best of both worlds to build highly scalable and reactive web applications. Furthermore, combining traditional API management capabilities with an event-driven.
Founded in 2005, WSO2 is one of the world’s best open source integration vendors, helping digitally driven organisations become integration agile. WSO2 is a global organisation with offices in Europe, the Americas, Sri Lanka and Australia.
WSO2 solutions give enterprises the flexibility to deploy applications and services on-premises, on private or public clouds, or in hybrid environments and easily migrate between them as needed. All of the products are pre-integrated allowing enterprises to focus on value-added services and get to market faster.