Every service needs a health check! Health checks are used to test if a service is alive and able to handle requests. If a service is unresponsive or reports an unhealthy status, the scheduler will not route any requests to the service. Depending on the actions, the scheduler might restart the unhealthy service for recovery.

The gRPC defines a simple health checking protocol, which your service should expose. This has the benefit, that there are several tools already existing to perform the health checks using this protocol.

The most prominent tool is the grpc-health-probe, a command line tool to invoke the health check of a grpc service.

Before Kubernetes version 1.23, this was the preferred method, to perform health checks. You ship the grpc_heath_probe executable with your docker image and use it as an exec probe.

Since Kubernetes version 1.23, you use the native grpc health check instead. There is no need to ship grpc_health_probe  with your docker image anymore.

Let's see an example for an grpc service written in Scala:

import io.grpc.ServerBuilder
import io.grpc.protobuf.services.HealthStatusManager

object Main extends App {
  // Part of the grpc-services package
  val healthStatusManager = new HealthStatusManager()
  
  val server = ServerBuilder
      .forPort(8000)
      .addService(healthStatusManager.getHealthService)
      .build
      
  // ...
}

The HealthStatusManager is a simple implementation of the health checking protocol. The initial status is SERVING by default. You can change the status by calling the method setStatus.

To activate the health check for your deployment in Kubernetes, add the following livenessProbe section to your deployment spec:

spec:
  containers:
    - name: my-service
      # ...
      livenessProbe:
        grpc:
          port: 8000
        initialDelaySeconds: 10

That's it. Health checking is now enabled and Kubernetes will restart your service in case it becomes unresponsive or changes its status to NOT_SERVING.

Do not implement your own health checking protocol. Use the default one to benefit from the ecosystem. Only use grpc_heath_probe with a exec probe in case you can't use Kubernetes 1.23 or later. Prefer the native gRPC health checking method provided by Kubernetes.