Microservices are not a trend anymore. Like it or not, they are here to stay. Yet, there’s a huge gap before embracing the microservice architecture and implementing them right. As a reminder, one might first want to check the many fallacies of distributed computed. Among all requirements necessary to overcome them is the ability to follow one HTTP request along microservices involved in a specific business scenario - for monitoring and debugging purpose.
One possible implementation of it is a dedicated HTTP header with an immutable value passed along every microservice involved in the call chain. This week, I developed this monitoring of sort on a Spring microservice and would like to share how to achieve that.
Headers forwarding out-of-the-box
In the Spring ecosystem, the Spring Cloud Sleuth is the library dedicated to that:
Spring Cloud Sleuth implements a distributed tracing solution for Spring Cloud, borrowing heavily from Dapper, Zipkin and HTrace. For most users Sleuth should be invisible, and all your interactions with external systems should be instrumented automatically. You can capture data simply in logs, or by sending it to a remote collector service.
Within Spring Boot projects, adding the Spring Cloud Sleuth library to the classpath will automatically add 2 HTTP headers to all calls:
X-B3-Traceid-
Shared by all HTTP calls of a single transaction i.e. the wished-for transaction identifier
X-B3-Spanid-
Identifies the work of a single microservice during a transaction
Spring Cloud Sleuth offers some customisation capabilities, such as alternative header names, at the cost of some extra code.
Diverging from out-of-the-box features
Those features are quite handy when starting from a clean slate architecture. Unfortunately, the project I’m working has a different context:
- The transaction ID is not created by the first microservice in the call chain - a mandatory façade proxy does
- The transaction ID is not numeric - and Sleuth handles only numeric values
- Another header is required. Its objective is to group all requests related to one business scenario across different call chains
- A third header is necessary. It’s to be incremented by each new microservice in the call chain
A solution architect’s first move would be to check among API management products, such as Apigee (recently bought by Google) and search which one offers the feature matching those requirements. Unfortunately, the current context doesn’t allow for that.
Coding the requirements
In the end, I ended up coding the following using the Spring framework:
- Read and store headers from the initial request
- Write them in new microservice requests
- Read and store headers from the microservice response
- Write them in the final response to the initiator, not forgetting to increment the call counter
Headers holder
The first step is to create the entity responsible to hold all necessary headers.
It’s unimaginatively called HeadersHolder.
Blame me all you want, I couldn’t find a more descriptive name.
private const val HOP_KEY = "hop"
private const val REQUEST_ID_KEY = "request-id"
private const val SESSION_ID_KEY = "session-id"
data class HeadersHolder (var hop: Int?,
var requestId: String?,
var sessionId: String?)The interesting part is to decide which scope is more relevant to put instances of this class in.
Obviously, there must be several instances, so this makes singleton not suitable.
Also, since data must be stored across several requests, it cannot be prototype.
In the end, the only possible way to manage the instance is through a ThreadLocal.
Though it’s possible to manage ThreadLocal, let’s leverage Spring’s features since it allows to easily add new scopes.
There’s already an out-of-the-box scope for ThreadLocal, one just needs to register it in the context.
This directly translates into the following code:
internal const val THREAD_SCOPE = "thread"
@Scope(THREAD_SCOPE)
annotation class ThreadScope
@Configuration
open class WebConfigurer {
@Bean @ThreadScope
open fun headersHolder() = HeadersHolder()
@Bean open fun customScopeConfigurer() = CustomScopeConfigurer().apply {
addScope(THREAD_SCOPE, SimpleThreadScope())
}
}Headers in the server part
Let’s implement requirements 1 & 4 above: read headers from the request and write them to the response. Also, headers need to be reset after the request-response cycle to prepare for the next.
This also mandates for the holder class to be updated to be more OOP-friendly:
data class HeadersHolder private constructor (private var hop: Int?,
private var requestId: String?,
private var sessionId: String?) {
constructor() : this(null, null, null)
fun readFrom(request: HttpServletRequest) {
this.hop = request.getIntHeader(HOP_KEY)
this.requestId = request.getHeader(REQUEST_ID_KEY)
this.sessionId = request.getHeader(SESSION_ID_KEY)
}
fun writeTo(response: HttpServletResponse) {
hop?.let { response.addIntHeader(HOP_KEY, hop as Int) }
response.addHeader(REQUEST_ID_KEY, requestId)
response.addHeader(SESSION_ID_KEY, sessionId)
}
fun clear() {
hop = null
requestId = null
sessionId = null
}
}To keep controllers free from any header management concern, related code should be located in a filter or a similar component. In the Spring MVC ecosystem, this translates into an interceptor.
abstract class HeadersServerInterceptor : HandlerInterceptorAdapter() {
abstract val headersHolder: HeadersHolder
override fun preHandle(request: HttpServletRequest,
response: HttpServletResponse, handler: Any): Boolean {
headersHolder.readFrom(request)
return true
}
override fun afterCompletion(request: HttpServletRequest, response: HttpServletResponse,
handler: Any, ex: Exception?) {
with (headersHolder) {
writeTo(response)
clear()
}
}
}
@Configuration open class WebConfigurer : WebMvcConfigurerAdapter() {
override fun addInterceptors(registry: InterceptorRegistry) {
registry.addInterceptor(object : HeadersServerInterceptor() {
override val headersHolder: HeadersHolder
get() = headersHolder()
})
}
}Note the invocation of the clear() method to reset the headers holder for the next request.
The most important bit is the abstract headersHolder property.
As its scope - thread, is smaller than the adapter’s, it cannot be injected directly less it will be only injected during Spring’s context startup.
Hence, Spring provides lookup method injection.
The above code is its direct translation in Kotlin.
Headers in client calls
The previous code assumes the current microservice is at the end of the caller chain: it reads request headers and writes them back in the response (not forgetting to increment the 'hop' counter). However, monitoring is relevant only for a caller chain having more than one single link. How is it possible to pass headers to the next microservice (and get them back) - requirements 2 & 3 above?
Spring provides a handy abstraction to handle that client part - ClientHttpRequestInterceptor, that can be registered to a REST template.
Regarding scope mismatch, the same injection trick as for the interceptor handler above is used.
abstract class HeadersClientInterceptor : ClientHttpRequestInterceptor {
abstract val headersHolder: HeadersHolder
override fun intercept(request: HttpRequest,
body: ByteArray, execution: ClientHttpRequestExecution): ClientHttpResponse {
with(headersHolder) {
writeTo(request.headers)
return execution.execute(request, body).apply {
readFrom(this.headers)
}
}
}
}
@Configuration
open class WebConfigurer : WebMvcConfigurerAdapter() {
@Bean open fun headersClientInterceptor() = object : HeadersClientInterceptor() {
override val headersHolder: HeadersHolder
get() = headersHolder()
}
@Bean open fun oAuth2RestTemplate() = OAuth2RestTemplate(clientCredentialsResourceDetails()).apply {
interceptors = listOf(headersClientInterceptor())
}
}With this code, every REST call using the oAuth2RestTemplate() will have headers managed automatically by the interceptor.
The HeadersHolder just needs a quick update:
data class HeadersHolder private constructor (private var hop: Int?,
private var requestId: String?,
private var sessionId: String?) {
fun readFrom(headers: org.springframework.http.HttpHeaders) {
headers[HOP_KEY]?.let {
it.getOrNull(0)?.let { this.hop = it.toInt() }
}
headers[REQUEST_ID_KEY]?.let { this.requestId = it.getOrNull(0) }
headers[SESSION_ID_KEY]?.let { this.sessionId = it.getOrNull(0) }
}
fun writeTo(headers: org.springframework.http.HttpHeaders) {
hop?.let { headers.add(HOP_KEY, hop.toString()) }
headers.add(REQUEST_ID_KEY, requestId)
headers.add(SESSION_ID_KEY, sessionId)
}
}Conclusion
Spring Cloud offers many components that can be used out-of-the-box when developing microservices. When requirements start to diverge from what it provides, the flexibility of the underlying Spring Framework can be leveraged to code those requirements.