How Does a Cache Work?

A cache is a pretty simple concept that shows up in a lot of different areas in computing. In almost all cases, a cache serves basically one main purpose. The key idea is to speed up access to data that is needed frequently. What that data actually is could vary depending on the context.

One example most people are familiar with is the apps on your phone. So let’s start there. Let’s say you’re listening to a song on your favorite music playing app. You’re absolutely hooked on to the song and you can’t get enough of it. So you keep mashing that repeat button and listening to the same song over and over again. Now, the actual contents of the song are not stored on your phone by default. When you install a music streaming service, you’re not installing the millions of songs in their library onto your phone automatically. That would be ridiculous.

Instead, once you listen to a song, the app is smart enough to make a copy of it on your phone. So the next time you try to listen to the same song, the app uses the copy for playback rather than downloading it again. This is great for two reasons. Not only are you doing the streaming service a favor by relieving some pressure from their servers, but you’re also saving some of your own precious bandwidth and data charges you’d spend grabbing the song each time.

You can already start to see how this approach would be useful for other things as well, such as images, videos, your message history, or anything else that doesn’t change too frequently.

Another example you’ve probably already experienced is the browser cache on your computer. The idea, again, is very similar. Websites are rarely just blobs of text anymore. They’re comprised of images, videos, and a whole lot of data that can be expensive for your network. So instead of fetching every single resource every time you visit the website, some of theses resources are cached by your browser the first time you visit. Every subsequent visit then relies on reading at least some of the information from this cache.

This is all great, of course. Caches are great. They speed up access to things and save you some $$$ in network costs. They make lives easier for developers. Everybody wins. So what’s the catch?

Well, the problem is that at the end of the day, the data stored on your local device (whether your browser or your phone) is just a copy of whats on the server. It’s not the real thing.

So if the data on the server has changed since the last time your cache was updated, the thing you’re seeing is likely out of date. This isn’t usually the end of the world. Sometimes, it just means you don’t see your friends’ latest profile pictures as soon as they‘re updated. But this is a conscious tradeoff that needs to be made by the developers, and it might not always be worth it. You might have noticed that clearing your browser and app caches can sometimes fix weird bugs.

The other downside is that caches take up space. A lot of space. In fact, you can see just how much space some of the apps on your phone are using by taking a casual look through your phone’s settings. This isn’t a problem if you have a lot of storage at your disposal, but if you’re running even slightly out of date hardware, this can be noticeable.

And lastly, even though we’ve talked about caches only in the contexts of your apps and browsers so far, they’re a critical building block for computers at the hardware level. The CPU in your computer executes, to use the scientific term, a crap ton of instructions every second.

Just like before, in order to speed up operations, the CPU maintains a local copy of data that it needs frequently in a hardware cache. Also, the CPU might have multiple levels of caches. The levels closest to the CPU are the fastest but can store the least amount of data. Each successive layer is slower but can store more data. For example, you might have heard of the terms L1, L2, and L3 caches when shopping for a new computer.

Caches are a powerful concept that is seemingly everywhere once you start looking for it!