Dev Learning Lab

The labs on this site need JavaScript to run. The same content is written out below.

Learn › Architecture and operations › Deployment and operations

Deployment strategies

How to switch to a new version without interrupting the service

Four servers are running v1. Depending on how you switch them to v2, it changes whether the service is interrupted during the deployment, how many more servers you need, and how many users are affected if the new version has a problem.

Checked against documentation and standards · The official Kubernetes and Spring Boot documentation

Concepts in this lab

Rolling, blue-green, and canary deployments
Ways to deploy a new version without interrupting the service. A rolling deployment replaces servers with the new version a few at a time, in turn. A blue-green deployment prepares a complete, separate environment for the new version and then switches the traffic all at once. A canary deployment applies the new version to only some users first, checks that there are no problems, and then expands it gradually.
Graceful shutdown
Stopping an application by accepting no new requests, finishing the requests that are already in progress, and only then exiting. It prevents errors caused by in-progress requests being cut off when a server is taken down, for example for a deployment.
Health check (liveness, readiness)
A periodic check of whether a server is healthy. Liveness checks whether the process is alive, so that it is restarted if there is a problem. Readiness checks whether the server is ready to take requests, so that traffic is not sent to a server that is not ready. This keeps requests from reaching a server while it is starting up or shutting down.
Load balancer
A device that distributes incoming requests across several servers. It checks the state of the servers periodically (health check) and does not send requests to a server that has a problem. It keeps load from piling up on one server and keeps the service running even when some servers fail.
CI/CD
CI (continuous integration) means merging code changes often and automatically running a build and tests on every merge to find problems early. CD (continuous delivery or continuous deployment) means using an automated process to get verified code into a deployable state or to deploy it all the way to production. Automating the build, test, and deployment work that people used to do by hand reduces mistakes and shortens the release cycle.

Try these

Running with the default settings

  1. Current state: all 4 servers are v1, and the load balancer spreads the requests evenly.
  2. Server 1: The load balancer stops sending new requests to this server, and a termination signal is sent to the server. The server finishes the requests in progress and then exits.
  3. Server 1: Bringing it up as v2. It receives no requests until it passes the readiness check. In the meantime the other 3 servers carry the load.
  4. Server 1: Ready. It receives requests again. Now v1 and v2 are serving together, and 25% of requests go to v2.
  5. Server 2: The load balancer stops sending new requests to this server, and a termination signal is sent to the server. The server finishes the requests in progress and then exits.
  6. Server 2: Bringing it up as v2. It receives no requests until it passes the readiness check. In the meantime the other 3 servers carry the load.
  7. Server 2: Ready. It receives requests again. Now v1 and v2 are serving together, and 50% of requests go to v2.
  8. Server 3: The load balancer stops sending new requests to this server, and a termination signal is sent to the server. The server finishes the requests in progress and then exits.
  9. Server 3: Bringing it up as v2. It receives no requests until it passes the readiness check. In the meantime the other 3 servers carry the load.
  10. Server 3: Ready. It receives requests again. Now v1 and v2 are serving together, and 75% of requests go to v2.
  11. Server 4: The load balancer stops sending new requests to this server, and a termination signal is sent to the server. The server finishes the requests in progress and then exits.
  12. Server 4: Bringing it up as v2. It receives no requests until it passes the readiness check. In the meantime the other 3 servers carry the load.
  13. Server 4: Ready. It receives requests again. Now v1 and v2 are serving together, and 100% of requests go to v2.
  14. All 4 servers are now v2. The service was not interrupted during the deployment.

Result of this run

Code

# Kubernetes Deployment (replicas: 4)
strategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 1           # takes down at most 1 server at a time
    maxSurge: 0

readinessProbe:                 # receives requests only when ready
  httpGet: { path: /actuator/health/readiness, port: 8080 }

# application.yml
server.shutdown: graceful         # finishes the requests in progress, then exits
spring.lifecycle.timeout-per-shutdown-phase: 30s

Assumptions of this model

This is a simple setup with 4 servers, replaced one at a time. The bug is assumed to be of a kind that the health check does not filter out and that shows up only after the server receives real requests, and in the rolling deployment it is assumed to be found at the point when 2 servers have been replaced. The requests in progress when a server is taken down are set to 2 per server. The definition of each strategy follows the explanations in the official Kubernetes and Spring Boot documentation.

Previous lab: CI/CD pipelines · Next lab: Containers and virtual machines

Concepts in this lab

What this lab shows

Settings

Code


      
    


    

      
Diagram

      

      

    


    

      

      

      

    


  


  

    

      
Run comparison

      

      

    

    

      
Try these