This section gives an overview of the tools we have developed to evaluate the performance of the various directory structures implemented in Crispy Squid.
Our approach to evaluating performance of Web cache structures is rooted in the wide range of proxy traces available today. Because these traces are indicative of current real-life proxy usage patterns, they are a powerful basis for predicting performance of collective cache structures in actual use. [13,16] Used in conjunction with continuous online trace gathering, trace-based performance studies can track the effects of Web usage patterns as they evolve.
However, we have found that the tools available to analyze these traces are inadequate for our needs. Therefore, we developed a collection of tools to process trace files and generate trace-driven Web access traffic for testing proxy caches. We later added tools to simulate the behavior of proxy caches under workloads driven by various proxy traces. We have consolidated these programs into a carefully-designed suite of tools and reusable libraries called Proxycizer. [7] Proxycizer is written in C++, will compile with any recent version of g++ (as well as many vendor-specific compilers) and has been tested on platforms running FreeBSD, Digital Unix, and Solaris.
The simclient and webulator make up the ``proxy exercising'' component of Proxycizer. simclient generates a request stream to a proxy from a log file. It can wait a specified delay between requests, and/or simulate several simultaneous clients each reading from separate log files. simclient can also generate a ``real-time'' request stream based on timestamps given in the log file. This feature can be used to accurately replay a specified trace in order to replicate a traffic pattern seen in the past.
webulator provides the server side of the testing infrastructure. webulator will respond to simple HTTP requests and return HTTP replies (content-filled with `*') of a size specified by a previously constructed on-disk database. webulator can also use duration information stored in the database to delay the return of data for a given amount of time (keeping the connection open while other connections are serviced).
Proxycizer provides an abstract representation of Web trace entries, accessed through a strict procedural interface, to enable maximum code reuse for analysis of traces from different locations. The abstract log entry class currently exports common fields including request time, request duration (both at microsecond granularity), reply size, HTTP method, and URL, which is itself an abstract type that has both character string and numeric forms for extremely fast comparisons in those traces that provide numeric URLs. Proxycizer automatically senses the log format and selects the correct filter. The currently supported log formats are Squid access logs [20], Crispy Squid access logs [8], Harvest access and hierarchy logs [3], UC Berkeley Home IP traces [11], DEC traces [12], and simclient logs. Proxycizer also implements several proxy simulator classes, including those for Harvest and the CRISP cache architectures described in Section 2.
Several other useful packages have recently been made available, each addressing different aspects of Internet server simulation and benchmarking. While Proxycizer is purely trace-driven, the Wisconsin Proxy Benchmark (WPB) [1] uses synthetic workloads modeled to emulate typical temporal locality patterns to test proxies under load. hbench:Web [14] is a benchmark for Web servers that is not explicitly trace-driven, but does preprocess Web server logs and automatically generates an appropriate traffic model for driving a Web server. S-Clients [2] introduces a scalable mechanism for driving Web servers to overload conditions. Squid Proxy Analysis (SPA) [5] provides trace-driven simulations that are similar in nature to our simulation programs, but are tailored to emulate the replacement behavior of Squid caches.