| Setting Other Cache Parameters with the afsd program | ||
|---|---|---|
 | Chapter 10. Administering Client Machines and the Cache Manager |  |
| Setting Other Cache Parameters with the afsd program | ||
|---|---|---|
| | Chapter 10. Administering Client Machines and the Cache Manager | |
There are only three cache configuration parameters you must set: the mount directory for AFS, the location of the disk cache directory, and the cache size. They correspond to the three fields in the /usr/vice/etc/cacheinfo file, as discussed in Determining the Cache Type, Size, and Location. However, if you want to experiment with fine-tuning cache performance, you can use the arguments on the afsd command to control several other parameters. This section discusses a few of these parameters that have the most direct effect on cache performance. To learn more about the afsd command's arguments, see its reference page in the OpenAFS Administration Reference.
In addition, the AFS initialization script included in the AFS distribution for each system type includes several variables that set several afsd arguments in a way that is suitable for client machines of different sizes and usage patterns. For instructions on using the script most effectively, see the section on configuring the Cache Manager in the OpenAFS Quick Beginnings.
The cache configuration parameters with the most direct effect on cache performance include the following:
total cache size. This is the amount of disk space or machine memory available for caching, as discussed in detail in Determining the Cache Type, Size, and Location.
number of cache chunks. For a disk cache, each chunk is a Vn file in the local cache directory (see Cache-Related Files). For a memory cache, each chunk is a set of contiguous blocks allocated in machine memory.
This parameter does not have as much of an effect on cache performance as total size. However, adjusting it can influence how often the Cache Manager must discard cached data to make room for new data. Suppose, for example, that you set the disk cache size to 50 MB and the number of chunks (Vn files) to 1,000. If each of the ten users on the machine caches 100 AFS files that average 20 KB in size, then all 1,000 chunks are full (a chunk can contain data from only one AFS file) but the cache holds only about 20 MB of data. When a user requests more data from the File Server, the Cache Manager must discard cached data to reclaim some chunks, even though the cache is filled to less than 50% of its capacity. In such a situation, increasing the number of chunks enables the Cache Manager to discard data less often.
chunk size. This parameter determines the maximum amount of data that can fit in a chunk. If a cached element is smaller than the chunk size, the remaining space in the chunk is not used (a chunk can hold no more than one element). If an element cannot fit in a single chunk, it is split across as many chunks as needed. This parameter also determines how much data the Cache Manager requests at a time from the File Server (how much data per fetch RPC, because AFS uses partial file transfer).
The main reason to change chunk size is because of its relation to the amount of data fetched per RPC. If your network links are very fast, it can improve performance to increase chunk size; if the network is especially slow, it can make sense to decrease chunk size.
number of dcache entries in memory. The Cache Manager maintains one dcache entry for each cache chunk, recording a small amount of information, such as the file ID (fID) and version number of the AFS file corresponding to the chunk.
For a disk cache, dcache entries reside in the /usr/vice/cache/CacheItems file; a small number are duplicated in machine memory to speed access.
For a memory cache, the number of dcache entries equals the number of cache chunks. For a discussion of the implications of this correspondence, see Controlling Memory Cache Configuration.
For a description of how the Cache Manager determines defaults for number of chunks, chunk size, and number of dcache entries in a disk cache, see Configuring a Disk Cache; for a memory cache, see Controlling Memory Cache Configuration. The instructions also explain how to use the afsd command's arguments to override the defaults.
The default number of cache chunks (Vn files) in a disk cache is calculated by the afsd command to be the greatest of the following:
100
1.5 times the result of dividing cache size by chunk size (cachesize/chunksize * 1.5)
The result of dividing cachesize by 10 MB (cachesize/10240)
You can override this value by specifying a positive integer with the -files argument. Consider increasing this value if more than 75% of the Vn files are already used soon after the Cache Manager finishes initializing. Consider decreasing it if only a small percentage of the chunks are used at that point. In any case, never specify a value less than 100, because a smaller value can cause performance problems.
The following example sets the number of Vn files to 2,000:
/usr/vice/etc/afsd -files 2000
It is conventional to place the afsd command in a machine's AFS initialization file, rather than entering it in a command shell. Furthermore, the values specified in this section are examples only, and are not necessarily suitable for a specific machine.
The default chunk size for a disk cache is 64 KB. In general, the only reason to change it is to adjust to exceptionally slow or fast networks; see Setting Cache Configuration Parameters. You can use the -chunksize argument to override the default. Chunk size must be a power of 2, so provide an integer between 0 (zero) and 30 to be used as an exponent of 2. For example, a value of 10 sets chunk size to 1 KB (210 = 1024); a value of 16 equals the default for disk caches (216 = 64 KB). Specifying a value of 0 (zero) or greater than 30 returns chunk size to the default. Values less than 10 (1 KB) are not recommended. The following example sets chunk size to 16 KB (214):
/usr/vice/etc/afsd -chunksize 14
For a disk cache, the default number of dcache entries duplicated in memory is one-half the number of chunks specified with the -files argument, to a maximum of 2,000 entries. You can use the -dcache argument to change the default, even exceeding 2,000 if you wish. Duplicating more than half the dcache entries in memory is not usually necessary, but sometimes improves performance slightly, because access to memory is faster than access to disk. The following example sets the number to 750:
/usr/vice/etc/afsd -dcache 750
When configuring a disk cache, you can combine the afsd command's arguments in any way. The main reason for this flexibility is that the setting you specify for disk cache size (in the cacheinfo file or with the -blocks argument) is an absolute maximum limit. You cannot override it by specifying higher values for the -files or -chunksize arguments, alone or in combination. A related reason is that the Cache Manager does not have to reserve a set amount of memory on disk. Vn files (the chunks in a disk cache) are initially zero-length, but can expand up to the specified chunk size and shrink again, as needed. If you set the number of Vn files to such a large value that expanding all of them to the full allowable size exceeds the total cache size, they simply never grow to full size.
Configuring a memory cache differs from configuring a disk cache in that not all combinations of the afsd command's arguments are allowed. This limitation results from the greater interaction between the configuration parameters in a memory cache than a disk cache. If all combinations are allowed, it is possible to set the parameters in an inconsistent way. A list of the acceptable and unacceptable combinations follows a discussion of default values.
The default chunk size for a memory cache is 8 KB. In general, the only reason to change it is to adjust to exceptionally slow or fast networks; see Setting Cache Configuration Parameters.
There is no predefined default for number of chunks in a memory cache. The Cache Manager instead calculates the correct number by dividing the total cache size by the chunk size. Recall that for a memory cache, all dcache entries must be in memory. This implies that the number of chunks equals the number of dcache entries in memory, and that there is no default for number of dcache entries (like the number of chunks, it is calculated by dividing the total size by the chunk size).
The following are acceptable combinations of the afsd command's arguments when configuring a memory cache:
-blocks alone, which overrides the cache size specified in the /usr/vice/etc/cacheinfo file. The Cache Manager divides the value of this argument by the default chunk size of eight KB to calculate the number of chunks and dcache entries. The following example sets cache size to five MB (5,120 KB) and the number of chunks to 640 (5,120 divided by 8):
/usr/vice/etc/afsd -memcache -blocks 5120-chunksize alone, to override the default of eight KB. The chunk size must be a power of two, so provide an integer between 0 (zero) and 30 to be used as an exponent of two. For example, a value of ten sets chunk size to 1 KB (210 = 1024); a value of 13 equals the default for memory caches (213 = 8 KB). Specifying a value of 0 (zero) or greater than 30 returns the chunk size to the default. Values less than ten (equivalent to 1 KB) are not recommended. The following example sets the chunk size to four KB (212). Assuming a total cache size of four MB (4,096 KB), the resulting number of chunks is 1024.
/usr/vice/etc/afsd -memcache -chunksize 12-blocks and -chunksize together override the defaults for cache size and chunk size. The Cache Manager divides the first by the second to calculate the number of chunks and dcache entries. For example, the following example sets the cache size to six MB (6,144 KB) and chunksize to four KB (212), resulting in 1,536 chunks:
/usr/vice/etc/afsd -memcache -blocks 6144 -chunksize 12The following arguments or combinations explicitly set the number of chunks and dcache entries. It is best not to use them, because they set the cache size indirectly, forcing you to perform a hand calculation to determine the size of the cache. Instead, set the -blocks and -chunksize arguments alone or in combination; in those cases, the Cache Manager determines the number of chunks and dcache entries itself. Because the following combinations are not recommended, no examples are included.
The -dcache argument alone explicitly sets the number of chunks and dcache entries. The Cache Manager multiples this value times the default chunk size of 8 KB to derive the total cache size (overriding the value in the cacheinfo file).
The combination of -dcache and -chunksize sets the chunk number and size. The Cache Manager sets the specified values and multiplies them together to obtain total cache size (overriding the value in the cacheinfo file).
Do not use the following arguments for a memory cache:
-files alone. This argument controls the number of Vn files for a disk cache, but is ignored for a memory cache.
-blocks and -dcache. An error message results, because it is possible to provide values such that dividing the first (total size) by the second (number of chunks) results in a chunk size that is not a power of two.
Tuning the parameters of the OpenAFS cache for optimal performance is highly dependent on the behavior of applications and users on a client machine. The default options may perform poorly under certain conditions.
The xstat_cm_test command is useful for measuring how effectively the cache is operating. The following procedure may be used to aide in tuning the parameters for the data cache (dcache) and the stats cache (vcache):
Run the following command and replace "hostname" with the hostname of the machine to be measured:
xstat_cm_test hostname 2 -onceonly
Take note of the following fields: dcacheHits, dcacheMisses, vcacheHits, and vcacheMisses. Saving the above command output to a file or filtering it using grep is advised.
Using the noted fields, compute the miss ratios for the dcache and vcache using the following formulas:
dcache miss ratio = dcacheMisses / ( dcacheMisses + dcacheHits )
vcache miss ratio = vcacheMisses / ( vcacheMisses + vcacheHits )
As a guideline, a miss ratio of 0.05 (5 percent) or less is acceptable and a miss ratio of 0.01 (1 percent) or less is recommended.
If your dcache miss ratio is too large, then cache performance is likely to improve if the data cache is made larger. If the vcache miss ratio is too large, then increase the size of the stat cache using the -stat parameter to afsd for a Unix-based client or using the Control Panel or registry interfaces on Microsoft Windows-based clients. The default size of the stat cache is 10,000 entries on windows platforms and 300 entries on Unix platforms. There may be a significant performance penalty when the vcache size is much smaller than the working set of commonly accessed files. On the fileserver, the number of callbacks should be more than the size of the vcache of any client that connects to the server. If the cache is too small or there aren't enough callbacks (-cb) on the fileserver, then the cached entries will be discarded prematurely, causing thrashing.
As an example of how the wrong vcache size can degrade performance, one OpenAFS site had performance issues with the Apache and mod_php software on a Unix web server serving web pages directly out of AFS. During peak times, the load on the server would spike with an excess of Apache processes. After profiling, it was found that Apache and PHP made lots of stat() library calls and that the default vcache size of 300 was too small. After some experimentation, a vcache size of 50,000 was found to improve performance. This size makes sense in light of that fact that the total number of files in the website exceeded 350,000, including 50,000 PHP files. The number of callbacks configured on the fileserver was 1,500,000, so the vcache size was not too large.
After changing your configuration appropriately and restarting the AFS client service, wait until enough data has been collected before changing the configuration further. The sum of the hits and misses should be at least five times the value of the configured parameter before making further adjustments. Repeat this process until the desired miss ratio is achieved. Take note that the numbers from the xstat_cm_test command only reset when the client is restarted. If multiple samples are taken, then subtract the previous measurement from the current measurement to accurately measure the activity that happened between the samples.
| |  | |
| Determining the Cache Type, Size, and Location |  | Maintaining Knowledge of Database Server Machines |