Sticky sessions with Apache APISIX - Theory

Sticky sessions became popular when we stored the state on the upstream node, not the database. I’ll use the example of a simplified e-commerce shop to explain further.

The basic foundations of a small e-commerce site can consist of a web application and a database.

If the business is successful, it will grow, and you’ll need to scale this architecture at some point. Once you cannot scale vertically (bigger machines), you must scale horizontally (more nodes). With additional apps node, you’ll also need a load balancer mechanism in front of the web app nodes to distribute the load among them.

Going to the database every time is an expensive operation. It’s ok for data that is accessed infrequently. However, we want to display the cart’s content for every request. A couple of alternatives are available to speed things up. If we assume that the web app uses Server-Side Rendering, the classical solution is to keep cart-related data in memory on the web app node.

However, if we store user X’s cart on node 1, we need to ensure that we forward every request of user X to the same node. Otherwise, they will feel as if they lost their cart’s content. Sticky sessions, or session affinity, is the mechanism that consistently routes the same user to the same node.

Before going further, I must explain a significant limitation of sticky sessions. If the webapp node that stores the data goes down for any reason, the data are irremediably lost. For the e-commerce scenario above, it means users will lose their cart occasionally, which is unacceptable from a business point-of-view.

For this reason, sticky sessions must go hand-in-hand with session replication: data stored on a node must be copied and kept in synch with all other nodes.

While session replication exists in all tech stacks, there’s no related specification. I’m familiar with the JVM, so here are a couple of options:

When data is replicated on all nodes (or a remote cluster), you may think you no longer need sticky sessions. It’s true if one accounts only for availability and not for performance. It’s about data locality: fetching data on the current node than from somewhere else via the network is faster.

Sticky sessions are a must-have for any Load Balancer, Reverse Proxy, and API Gateway worth their salt. However, I must admit that Apache APISIX’s documentation needs an easy entry point into the subject.

Apache APISIX binds a route to an upstream. An upstream consists of one or more nodes. When a request matches the route, Apache APISIX must choose among all available nodes to forward the request to. By default, the algorithm is weighted round-robin. Round-robin uses one node after the other, and after the last one, get back to the first one. With weighted round-robin, the weight affects how many requests Apache APISIX forwards to a node before it switches to the next one.

However, other algorithms are available:

Consistent hashing allows forwarding to the same node depending on some value: an NGINX variable, an HTTP header, a cookie, etc.

Remember that HTTP is a stateless protocol, so application servers set a cookie on the first response to track the user across HTTP requests. It’s what we call a "session". We need to know the underlying session cookie name. Different application servers hand out different cookies:

I shall use a regular Tomcat, so the session cookie is JSESSIONID. Henceforth, the Apache APISIX documentation for two nodes is the following:

In this post, we detailed sticky sessions, that you should always use session replication with sticky sessions, and how to implement sticky sessions on Apache APISIX.