Content Distribution Networks

Caching

Previously, when we talked about the "P"s to improve HTTP performance, which mentioned the strategy of caching.

Another way to improve HTTP performance is to implement caching, which exploits large overlaps in content. Caching works up to a limit, that is the number of unique requests. The empirical result says that the effectiveness of caching (cache hit ratio) grows logarithmically with user size.

Limitations of caching include

• a lot of content is not cacheable, including dynamic data (stock prices, web cams), cookies (results depend on passed data), SSL (encrypted data is not cacheable)
• stale data, overhead of refreshing the cached data, or overhead of refreshing the cached data.

How to cache? Add a modifier to the GET request, If-modified-since, which is specified by the client in the request:

GET /somedir/page.html HTTP/1.1  
Host: www.someschool.edu  
User-agent: Mozilla/4.0  
If-modified-since: Wed, 18 Jan 2017 10:25:50 GMT  
(blank line)

The server compares if-modified-since against the last-modified time of the resource, and it'll return "not modified" if resource has not changed, or a "OK" with the latest version otherwise.

The response header can also include expires, which is how long it's safe to cache the resource, and/or no-cache which ignores all caches; always get resource directly from server.

Where is caching used? Caching can be used for the client (browser), forward proxies, and reverse proxies.

Proxies

Many clients transfer the same information, so it generates a lot of unnecessary server and network load, causing unnecessary latency. You have to maintain many persistent TCP connections.

Caching with reverse proxies means you cache documents close to the server. This decreases server load and is done by the content provider.

Caching with forward proxies means you cache close to the clients. This reduces network traffic and decreases latency. This is done by the ISPs or enterprises.

Replication

Replicate popular websites across many machines.

• spreads the load across servers
• places content closer to the clients
• helps with performance when the content isn't cacheable.

Replication is done by CDNs.

Content Distribution Network

A content distribution network (CDN) does caching and replication as a service.

• It is usually administered by one entity and is a large-scale distributed storage infrastructure.
• It can do some processing to handle dynamic webpage content.

Typically the origin server communicates with a CDN distribution node, which communicates with multiple CDN servers in different geographies.

For example, Akamai has servers in 20,000+ locations. They create new domain names for each client, like a128.g.akamai.net for cnn.com. Then, the client content provider modifies content so that embedded URLs reference new domains, let's call it "akamaizing" the content: cnn.com/image-of-the-day.gif becomes a128.g.akamai.net/image-of-the-day.gif. The requests are now sent to CDN's infrastructure.

Server selection

An application can implement HTTP Redirection: you do a GET and it returns a Redirect, which makes you send a GET to a different server, which will give you your desired content.

Using naming, which is a type of DNS-based server selection. Make a DNS query to a local DNS server using a name, and you will get the address, which you can then communicate with.

	HTTP Redirection	DNS-based server selection
TCP set-up delay	Extra round-trips for TCP connection to servers	Avoid TCP set-up delay
Overhead on server	High	DNS caching reduces overhead
Level of control	Fine-grain control	Relatively fine control

An example of CDN server selection with Akamai:

• Akamai will create a new domain name for each client. For example cnn.com is a128.g.akamai.
• The client content provider will modify the content so that the embedded URLs reference new domains. This process is called "akamaizing" content.
• The requests are now sent to CDN's infrastructure

Server selection policy requires continuous monitoring of liveness, load, and performance:

• live server for availability
• lowest load to balance load across the server
• closest or the nearest geographically, or in round-trip time
• best performance, considering throughput, latency, ...
• cheapest bandwidth, electricity, etc.