Scenario: NetCache as a distributed Web site accelerator

About this scenario

In this scenario, a large media site/hosting provider is interested in caching for the following reasons:

Bandwidth use is growing. The organization wants to constrain the cost of bandwidth.
The organization needs a way to scale sites that are growing rapidly.

A large amount of the data that clients request from this provider is static. Therefore, caching that data would result in significant savings in bandwidth use.

Note
This example shows a deployment with Web accelerators. Streaming accelerators can also be distributed as shown in this example.

Organization's requirements

The organization's requirements are as follows:

The caching system must be reliable.
The content served must be fresh; that is, if the content of a Web page changes, users requesting that page must be able to get the newest version very soon after it is changed.
The media site/hosting provider must be able to log access to the data, for example, to bill an advertiser or perform a security audit.

Deployment illustrated

The following illustration shows a deployment for this provider that includes two NetCache appliances running as Web accelerators.

Goals for locating the Web accelerators:

The U.S.A. side of the illustration is like that in Scenario: an accelerator outside the firewall, with the accelerator outside the firewall. In this scenario also, the goal of deploying the Web accelerator outside the firewall and the Web server inside the firewall is to secure the Web site content.

A Web accelerator is deployed in Europe so that data from the Web site can be cached locally, thereby saving expensive bandwidth. Initially, the European Web accelerator must obtain the Web objects, and incur bandwidth cost. However, after the objects are in the cache in Europe, the objects can be served locally until they become stale. If the Web accelerator determines that an object is stale, it fetches a newer version of the object.

About the European Web accelerator:

The provider can configure the European NetCache appliance in two different ways:

Identify the Web server www-internal.abc.com as the Web server for the European Web accelerator, just as it did for the U.S.A. Web accelerator. In this case, if the European Web accelerator does not have a requested object in its cache, it goes to the Web server to obtain the object.
Identify the U.S.A. Web accelerator as the "Web server" for the European Appliance Web accelerator. In this case, if the European Web accelerator does not have a requested object in its cache, it goes to the U.S.A. Web accelerator to obtain the object. The European Web accelerator does not go to the real Web server, www-internal.abc.com. This method is preferable because the provider does not have the cost of the slow firewall or the bottleneck that prompted the provider to deploy the U.S.A. Web accelerator.

Sending requests to the appropriate Web accelerator

The provider needs to determine the method to use so that user requests are sent to the appropriate Web accelerator, that is, the accelerator that the provider wants the each client to use. This request distribution is outside the control of NetCache.

Different methods for ensuring that requests are sent to the correct Web accelerator are as follows:

Notify users about the URL they must enter so that the request is sent to the Web accelerator that the provider wants.
Users in the U.S.A. access the Web server by entering www.abc.com. Users in Europe must be instructed to enter the URL for the European Web accelerator, www-europe.abc.com, not the URL for the U.S.A. Web accelerator. Otherwise, their requests are sent to the U.S.A. Web accelerator, with a higher bandwidth cost.
Use a product, such as a switch or Server Load Balancer (SLB), that can distribute requests to the correct Web accelerator.
Use DNS routing.
3DNS (from F5 Lab), Distributed Director (from Cisco), and WSD-DS (from RND Network) are examples of DNS servers that assign IP addresses to ensure that the request is sent to the closest Web accelerator.

Logging:

Logging features in NetCache enable system administrators to identify clients that are requesting objects and to use that information for billing. ContentReporter software can be used to gather logs from NetCache appliances. That data can then be imported into third-party reporting applications, for example, to create billing reports.