The labs on this site need JavaScript to run. The same content is written out below.

Fundamentals › Architecture and operations

Caches

What is a cache, and what do you gain and lose by using one?

A cache is a device that writes down a result that took a long time to get, in a place close by, and hands out what it wrote down when the same thing is asked for again. In the code of this lab, the Map named cache is the place where results are written down, and the cache lines inside the find method do this: "if there is a written-down value, give it, and only if there is none, do the slow lookup (slowLoad) and write the result down." slowLoad is the slow job of reading a value from the source, and slowCount counts how many times it was called. The lab does not measure real time. It shows "how much slow work was done" by the number of calls to slowLoad. In this lab you remove the cache lines, change the value in the source between the two lookups, or erase the cached value with the cache.remove(1) line. The same code is compiled and run, and you see how many slow lookups happen and whether the second value you receive is the same as the value in the source. (static is read as a keyword that lets main call these without creating an object, and it is not covered here.)

Checked against real execution · Compiled five combinations on JDK 21 with javax.tools, ran them, and checked against the model

Concepts in this lab

Cache

Storing frequently used data or computed results ahead of time in a place that can be accessed quickly, and reusing them. Responses get faster and the load on the source drops, because the source (usually a database) is not queried every time. In return, you must manage the problem of stale values that remain when the source changes.

Cache hits and misses

When you look for a value in a cache, one of two things happens. If what you look for is already written in the cache, it is called a hit (cache hit), and the written-down value is returned as it is without going to the source. If it is not written, it is called a miss (cache miss), and the value is read from the source, written in the cache, and returned. The cache abstraction of Spring works in the same order: if the method was already called with the same arguments, it returns the cached result without executing the method, and otherwise it executes the method and puts the result in the cache and returns it. The first lookup is always a miss, and the more hits there are, the less slow work goes all the way to the source.

Cache-aside pattern

The most common approach, in which the application manages the cache directly. To read data, the application checks the cache first. If the data is not there, it reads from the database, stores the value in the cache, and returns it. To change data, it updates the database and deletes the matching cache entry.

Stale data in a cache

Stale data in a cache is a value that stays in the cache after the value in the source has changed, so that it differs from the source. The side that uses the cache does not look at the source when there is a hit, so it receives the old value without knowing that it changed. If you also erase the value written in the cache when you change the source, the next lookup is a miss and reads the changed value again (only that one lookup is slow). Another way is to give a written-down value a validity time. The HTTP caching specification (RFC 9111, section 4.2) calls a stored response stale when it is past its determined freshness lifetime, and this word is the term of the specification for a response whose time has passed.

Try these

• Default settings — The cache lines are there. You looked up twice, but the slow lookup happens only once.
• Looking up twice without a cache — Each lookup goes all the way to the source, so the slow lookup happens twice.
• The source changes while the cache is there — The slow lookup happens once, but the second value you receive is different from the source.
• Changing the source and also erasing the cached value — The second find cannot find the value in the cache and reads the changed value again.
• The source changes without a cache — Every lookup reads the source, so you receive the changed value, but the slow lookup happens twice.

Running with the default settings

1. main starts and the first line, source.put(1, "keyboard");, runs. The source gets the value "keyboard" for key 1. The source is where the values originally are, and it is where slowLoad reads from. cache is an empty Map created by calling new HashMap<>(), so nothing is written in it yet.
2. The line find(1); runs. First, find calls cache.containsKey(1) to see whether a value for key 1 is written in the cache. The cache is empty, so there is none. This case, where the value is not written in the cache, is called a miss.
3. So find calls the slowLoad method. slowLoad is the slow job of reading the value "keyboard" for key 1 from the source, and it increases slowCount to count how many times it was called. So far slowLoad has been called 1 time.
4. find writes the value it read, "keyboard", into the cache with the call cache.put(1, v), and returns that value. Now the cache holds the value "keyboard" for key 1.
5. The line String second = find(1); runs. find calls cache.containsKey(1) to look at the cache, and a value for key 1 is written there. This case, where the value is written in the cache, is called a hit. find does not call slowLoad and returns the written-down value "keyboard" as it is. slowLoad has still been called 1 time.
6. The last line, System.out.println, prints slowCount, second, and source.get(1) in that order. The output is "1 / keyboard / keyboard". The value you received is the same as the value in the source.

Result of this run

• A cache writes down a value it read in a Map, and when the same thing is looked up again it gives the written-down value. The cache lines in find (checking with containsKey, writing with put, taking out with get) did this, and although key 1 was looked up twice, the slow lookup slowLoad was called only once.
• The second find gave the written-down value right away and did not go to the source. The source did not change between the two lookups, so the value you received was the same as the source.

Code

import java.util.*;

public class Main {
    // The source: where the values originally are
    static Map<Integer, String> source = new HashMap<>();
    static int slowCount = 0;

    // Slow lookup: reads from the source. Counts how many times it is called
    static String slowLoad(int id) {
        slowCount++;
        return source.get(id);
    }

    // Cache: the place where values that were read are written down
    static Map<Integer, String> cache = new HashMap<>();

    static String find(int id) {
        // If a value is written down, give it; if not, read it and write it down
        if (cache.containsKey(id)) return cache.get(id);
        String v = slowLoad(id);
        cache.put(id, v);
        return v;
    }

    public static void main(String[] args) {
        source.put(1, "keyboard");
        // Look up key 1 for the first time
        find(1);
        // Look up key 1 again
        String second = find(1);
        // Slow lookups / second value received / value in the source
        System.out.println(
            slowCount + " / " + second + " / " + source.get(1));
    }
}

Assumptions of this model

The compiler and runtime are JDK 21.0.12.1 (javac 21). All five combinations (the two choices of having a cache or not, times the three choices of what happens between the two lookups, minus the one that is left out because you cannot use the cache.remove(1) line without a cache) were checked against real runs: the same source as the code on the screen (identical line by line, including the import line) was compiled with javax.tools, the class file was read by a new class loader, Main.main was run, and the standard output that was captured was compared with the model. What was compared is whether the compile succeeded, the three printed values (the number of calls to slowLoad, the value the second find returned, and the value in the source), and the output text that the last step shows. The fact that the step sentences rely on, that "after one call to find(1) at first, slowLoad has been called 1 time and the value received is keyboard", was checked by running separate source (one with a cache and one without) changed to print the result of the first find, in the same way. The code on the screen is a program that is really compiled and run, and nothing was shortened. slowLoad does not really take long and no time was measured. "Slow" is shown only by the number of calls to slowLoad, and the time of a real database or network was not measured. This cache is one Map, and it has no rule for when to throw away a written-down value (expiry time, size limit), so a value stays until it is erased with the cache.remove(1) line. I opened the documentation myself. The introduction of the Cache Abstraction chapter of the Spring Framework Reference (https://docs.spring.io/spring-framework/reference/integration/cache.html) opened only as far as the explanation that caching is support that is added transparently to an existing application, and in the lower page of the same documentation (https://docs.spring.io/spring-framework/reference/integration/cache/strategies.html) I confirmed that if the method has already been invoked with the same arguments, the cached result is returned without executing the method, and otherwise the method is executed and its result is put in the cache, and that there are features to update the contents of the cache or evict entries. For RFC 9111 (HTTP Caching), I confirmed section 1 (a cache stores responses to reduce response time and network use), section 4 (conditions for reusing a stored response), and sections 4.2 and 4.2.4 (a stored response that is past its freshness lifetime is called stale), and I did not read the last 7 thousand or so characters of the document. I did not confirm whether those documents use the words hit and miss, and the glossary uses these words only as general terms. The Map cache in this lab is just one example of the approach the documents describe, and it was not checked against real execution of Spring or of an HTTP cache. Not covered: a cache that several threads write at the same time, what to throw away when the cache is full, the validity time (TTL), many requests rushing to the source when a popular value disappears all at once, @Cacheable of Spring, the HTTP cache between a browser and a server (ETag), and Redis. These are covered in the in-depth labs "Caching and TTL", "Cache abstraction", "HTTP caching and ETag", and "Redis single-threaded execution".

Previous lab: Load balancers · Next lab: Message queues