HP P4535A Administrator's Manual
HP Cache Server Appliance
Administrator Guide
HP Part Number 5971-3045
Printed in June 2001
ii
Notice
The information contained in thisdocument is subject to change without notice.
Hewlett-Packard makes no warrantyof any kind with regardtothis material, including, but notlimited
to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall
not be liable for errors contained herein or for incidental or consequential damages in connection with the
furnishing, performance, or use of this material.
Hewlett-Packard assumes no responsibility for the use or reliability of its software on equipment that is not
furnished by Hewlett-Packard.
This document contains proprietaryinformation that is protected by copyright. All rights are reserved. No part
of this document may be photocopied, reproduced, or translated to another language without the prior written
consentof Hewlett-Packard Company.
UltraServer and SPARCstorage are trademarks of Sun Microsystems, Inc.; Myrinet is a trade name of Myricom, Inc.; UNIX is a registered trademark of AT&T. All other trademarksare the property of their respective
owners.
Hewlett-Packard Company
Network Server Division
Technical Communications/MS 45SLE
10955 Tantau Avenue
Cupertino, CA 95014 USA
© Copyright 2001, He wlett-Packard Company.
Audience Assumptions
This guide is for the person who installs, administers, and troubleshoots network infrastructure products.
Hewlett-Packard Company assumes you are qualified in the servicing of computer equipment and trained in
recognizing hazards in products with hazardous energy levels.
iii
Preface ......................................................................... xii
Who should read this manual . . . . . ................................ xii
Conventionsusedinthismanual................................... xii
1 Overview....................................................................... 1
WhatisTrafficServer?.........................................1
Traffic Server deployment options . . .................................1
TrafficServerasawebproxycache.................................2
TrafficServerasareverseproxy...................................2
TrafficServerinacachehierarchy..................................2
TrafficServerinacluster.......................................2
Traffic Server components . . . . . . .................................3
TheTrafficServercache.......................................3
The Adaptive Redirection Module (ARM) . .............................3
The Host Database. . ........................................3
TheDNSResolver..........................................4
TrafficServerprocesses.......................................4
Administrationtools..........................................5
Trafficanalysisoptions.........................................5
TrafficServersecurityoptions .....................................5
2 GettingStarted.................................................................. 7
Overview of Access Methods . . . . . .................................7
Verifying that Traffic Serv er is up and running .............................8
Accessing Traffic Manager . . . . . . .................................8
UsingtheMonitorandConfiguretabs................................9
Usingonlinehelp...........................................9
StartingTrafficLine..........................................10
RestartingTrafficServer....................................... 10
3 WebProxyCaching ............................................................. 11
Understanding web proxy caching . . ................................11
Adayinthelifeofacacherequest.................................11
Ensuring cached object freshness . . ................................12
RevalidatingHTTPobjects.....................................12
HTTP object freshness tests . . . . ................................13
DecidingwhethertoserveHTTPobjects..............................13
ConfiguringHTTPfreshnessoptions................................14
CachingHTTPalternates...................................... 14
Scheduling updates to local cache content. ............................16
Newsarticlecaching......................................... 16
TrafficServerasanewsserver..................................17
TrafficServerasacachingproxynewsserver...........................17
Supporting several parent news servers . . ............................17
Contents
iv
Contents
Blocking particular groups . . . . . ................................18
Clustering..............................................18
Transparency............................................19
Posting...............................................19
Maintaining the cache: updates and feeds . ............................19
Configuringaccesscontrol.....................................20
Using enhanced NNTP authentication . . . ............................20
Obeying NNTP control messages . ................................ 21
Clientbandwidththrottling.....................................21
4 TransparentProxyCaching....................................................... 22
Serving requests transparently . . . . ................................22
ARMredirection............................................22
Interceptionstrategies.........................................23
Using a layer 4 switch with cache switching functionality to filter transparency requests . . . . . 23
Using a WCCP-enabled router for transparency . . ........................ 24
Usingpolicy-basedroutingtofiltertransparencyrequests.....................27
Interceptionbypass..........................................28
Dynamicbypassrules.......................................29
Staticbypassrules......................................... 32
Viewingthecurrentsetofbypassrules ..............................32
ConfiguringARMsecurity .....................................32
5 ReverseProxyandHTTPRedirects................................................ 33
Understanding reverse proxy caching ................................33
Reverseproxysolutions......................................33
Howdoesreverseproxycachingwork?..............................34
HTTPReverseProxy......................................... 35
Handling origin server redirect responses . ............................35
Using mapping rules . .......................................36
SettingHTTPreverseproxyoptions................................38
FTPReverseProxy.......................................... 39
ConfiguringFTPReverseProxy..................................40
SettingFTPMappingRules....................................40
Enabling FTP Reverse Proxy. . . . ................................40
ModifyingFTPOptions.......................................41
Redirecting HTTP Requests . . . . . ................................43
6 TrafficServerClusters........................................................... 45
Understanding Traffic Server clusters. ................................45
Management-only clustering . . . . ................................45
Fullclustering............................................45
Changingclusteringmode.....................................46
Adding and deleting nodes in a clus ter ................................ 46
Adding nodes to a c luster . . . . . ................................46
Deleting nodes from a cluster . . . ................................47
VirtualIPfailover...........................................47
Setting virtual IP address options . ................................48
v
Contents
7 HierarchicalCaching ............................................................ 51
Understanding cache hierarchies . . . ................................51
HTTPcachehierarchies .......................................51
Parentfailover...........................................52
ConfiguringTrafficServertouseanHTTPparentcache.....................52
ICPcachehierarchies.........................................54
ConfiguringTrafficServertouseanICPcachehierarchy.....................54
8 Configuring the cache . . . ........................................................ 59
TheTrafficServerCache....................................... 59
RAMcache.............................................59
Partitioningthecache......................................... 59
Creatingcachepartitionsforspecificprotocols ..........................60
Partitioningthecacheaccordingtooriginserverordomain....................61
Clearingthecache..........................................62
9 MonitoringTraffic............................................................... 63
TrafficServermonitoringtools ....................................63
Viewing statistics from Traffic Manager ................................63
Starting Traffic Manager Monitor mode . . ............................63
UsingMonitormode........................................64
WorkingwithTrafficManagerAlarms.................................67
Resolvingalarms..........................................67
ConfiguringTrafficServertoE-mailalarms............................ 68
ViewingStatisticsfromTrafficLine..................................68
Viewing groups of statistics. . . . . ................................68
Retrievingindividualstatistics...................................69
UsingMRTG .............................................70
AccessingMRTG..........................................70
NavigatingMRTG .........................................70
UsingSNMP .............................................71
Enabling SNMP...........................................71
ConfiguringSNMPtrapdestinations................................72
10ConfiguringTrafficServer........................................................ 73
Configuring Traffic Server us ing Traffic Manager. . . ........................73
Starting Traffic Manager Configure mode . ............................73
UsingConfiguremode.......................................74
ConfiguringTrafficServerusingTrafficLine.............................76
StartingConfiguremode......................................76
NavigatingConfiguremode ....................................77
SettingconfigurationoptionsinConfiguremode..........................77
Settingconfigurationoptionsinbatchmode............................78
ConfiguringTrafficServerusingconfigurationfiles..........................78
11SecurityOptions................................................................ 80
TrafficServersecurityoptions ....................................80
ControllingclientaccesstotheTrafficServerproxycache...................... 81
ControllinghostaccesstotheTrafficServermachine(ARMsecurity)................81
vi
Contents
Controlling access to Traffic Manager ................................ 83
SettingtheadministratorIDandpassword.............................83
Creating a list of administrator accounts . . ............................85
Controlling host access to Traffic Manager. ............................86
UsingSSLforsecureadministration................................87
ConfiguringSOCKSfirewallintegration................................89
SettingSOCKSconfigurationoptions...............................89
ConfiguringDNSserverselection(splitDNS).............................91
Configuring LDAP-based proxy authentication ............................92
Configuring LDAP Authentication Bypass . ............................92
UsingSSLTermination........................................93
Client and Traffic Server connec t ions . . . ............................94
TrafficServerandoriginserverconnections............................96
12WorkingwithLogFiles........................................................... 98
Understanding Traffic Server log f iles. ................................98
Understanding event log files . . . . . ................................98
Managing event log files .......................................99
Choosingtheloggingdirectory...................................99
Controlling logging space. . . . . . ................................99
Setting log file management options ............................... 100
Choosingeventlogfileformats................................... 101
Using standard formats...................................... 101
Usingcustomformats...................................... 103
ChoosingbinaryorASCII .................................... 108
UsinglogcattoconvertbinarylogstoASCII........................... 108
Rollingeventlogfiles........................................ 109
Rolledlogfilenameformat.................................... 109
Rollingintervals.......................................... 110
Settinglogfilerollingoptions................................... 110
Splittingeventlogfiles ....................................... 112
NNTPlogsplitting ........................................ 112
ICPlogsplitting.......................................... 112
HTTPhostlogsplitting...................................... 112
Settinglogsplittingoptions.................................... 113
Editing the log_hosts.config file . . ............................... 114
Collatingeventlogfiles....................................... 115
Settinglogcollationoptions ................................... 116
Usingastand-alonecollator................................... 118
Recoveringlogfiles........................................ 119
Viewing logging statistics ...................................... 119
A TrafficManagerStatistics ....................................................... 120
The Dashboard page . . ...................................... 120
The Node page . .......................................... 121
The Graphs page .......................................... 121
The Protocols page . . . ...................................... 122
TheCachepage .......................................... 125
vii
Contents
TheOtherpage........................................... 126
The MRTG page .......................................... 127
B TrafficManagerConfigurationOptions ............................................ 128
The Server Basics page. ...................................... 128
The Protocols page . . . ...................................... 131
TheCachepage .......................................... 135
The Security page.......................................... 137
The Routing page .......................................... 138
The Host Database page ...................................... 139
The Logging page.......................................... 140
The Snapshots page . . ...................................... 143
The Plugins page .......................................... 143
The Content Management page . . . ............................... 143
C TrafficLineCommands......................................................... 145
TrafficLinebatchmodecommands................................. 145
TrafficLineinteractivemodecommands.............................. 146
TrafficLinevariables........................................ 147
Statistics............................................. 147
ConfigurationOptions ...................................... 151
D ConfigurationFiles............................................................. 159
arm_security.config......................................... 160
Format.............................................. 160
Example............................................. 160
bypass.config............................................ 161
Format.............................................. 162
Example............................................. 162
cache.config............................................. 162
Format.............................................. 163
Example............................................. 164
filter.config.............................................. 164
Format.............................................. 164
Example............................................. 165
ftp_remap.config .......................................... 166
Format.............................................. 166
Example............................................. 166
hosting.config............................................ 166
Format.............................................. 167
Example............................................. 167
icp.config.............................................. 167
Format.............................................. 167
Example............................................. 168
ip_allow.config ........................................... 168
Format.............................................. 168
Example............................................. 169
ldapsrvr.config............................................ 169
Format.............................................. 169
viii
Contents
Example............................................. 169
logs.config.............................................. 169
Format.............................................. 170
Example............................................. 170
WELF (WebTrends Enhanced Log Format) ........................... 171
log_hosts.config. .......................................... 171
Format.............................................. 171
Example............................................. 171
logs_xml.config........................................... 171
Format.............................................. 172
Example............................................. 175
WELF (WebTrends Enhanced Log Format) ........................... 175
mgmt_allow.config ......................................... 176
Format.............................................. 176
Example............................................. 176
nntp_access.config......................................... 176
Format.............................................. 176
nntp_servers.config......................................... 177
Format.............................................. 177
Example............................................. 179
parent.config ............................................ 179
Format.............................................. 180
Example............................................. 181
partition.config............................................ 181
Format.............................................. 182
Example............................................. 182
records.config............................................ 182
Format.............................................. 182
Example............................................. 183
Configurationvariables...................................... 183
remap.config ............................................ 212
Format.............................................. 212
Example............................................. 213
snmpd.cnf ............................................. 214
Format.............................................. 214
Configuringtrapdestinations................................... 215
Accesscontrol.......................................... 215
socks.config. . . .......................................... 216
Format.............................................. 216
Example............................................. 216
splitdns.config............................................ 217
Format.............................................. 217
Example............................................. 217
storage.config . . .......................................... 218
Format.............................................. 218
Example............................................. 218
update.config............................................ 218
Supported tag/attribute pairs . . . . ............................... 219
ix
Contents
Format.............................................. 219
Example............................................. 220
Specifying URL Regular Expressions (url_regex). . . ....................... 220
Example............................................. 221
E EventLoggingFormats......................................................... 222
HP custom logging fields ...................................... 222
Loggingformatcrossreference................................... 224
Squidloggingformats...................................... 224
NetscapeCommonloggingformats............................... 225
Netscape Extended logging formats ............................... 225
Netscape Extended-2 logging formats . . . ........................... 225
F TrafficServerErrorMessages.................................................... 227
Traffic Server error m es s ages . . . . ............................... 227
TrafficServerNotes....................................... 227
TrafficServerProcessfatal.................................... 228
TrafficServerWarnings..................................... 228
Traffic Server alarm m ess ages . . . . ............................... 229
HTML messages sent to clients. . . . ............................... 230
Standard HTTP response messages . ............................... 232
Glossary ........................................................................ 233
Index............................................................................236
x
To verify that T raffic Server is up and running: 8
To access Traffic Manager: 8
To start a Traffic Line session: 10
To enable WCCP 1.0 af t er Traffic Server installation: 25
To enable WCCP 2.0 after installation: 26
To set dynamic bypass rules: 29
To view dynamic bypass statistics: 31
To view all current dynamic and static bypas s rules: 32
To create a mapping rule from Traffic Manager: 36
To create a mapping rule manually: 37
To set reverse proxy options from Traffic Manager: 38
To set reverse proxy options manually: 38
To set FTP mapping rules: 4 0
To enable FTP reverse proxy: 40
To modify FTP options: 41
To set redirect rules: 44
To change clustering mode: 46
To add a cache appliance to a cluster: 46
To delete a node from a c lust er: 47
To enable/disable virtual IP addressing from Traffic Manager: 48
To enable/disable virtual IP addressing manually: 49
To add or edit virtua l IP addresses from Traffic Manager: 49
To add or edit v irtual IP addresses manually: 50
To enable the HTTP parent caching option from Traffic Manager: 52
To enable HTTP parent caching manually: 53
To identify an HTTP parent cache from Traffic Manager: 53
To set ICP options from Traffic Manager: 55
To set ICP options manually: 55
To identify an ICP peer from Traffic Manager: 56
To identify an ICP peer manually: 57
To partition the cache according to protocol: 60
To partition the cache according to hostname and domain: 61
To clear the cache: 62
To start Traffic M anager Monitor mode: 63
To start Monitor mode: 68
Toretrieveasinglestatistic:69
To access MRTG: 70
To enable the SNMP agent from Traf fic Manager: 71
To enable the SNMP agent manually: 71
To start Traffic M anager Configure mode: 73
To start Configure mode: 76
List of Procedures
xi
List of Procedures
To set configuration options in Configure m ode: 77
To set configuration options in ba tch mode: 78
To specify the clients allowed to use Traffic Server as a proxy cache: 81
To edit the arm_security.config file and enable the ARM security option: 82
To change the administrator ID and password: 83
To clear and re-enter the administrator password: 84
To create a list of administrator accounts: 85
To control which hosts can access Traffic Manager: 86
To enable SSL from Traffic Manager: 87
To enable SSL manually: 88
To set SOCKS options from Traffic Manager: 89
To set SOCKS options manually: 91
To configure DNS server selection: 91
To configure LDAP-based proxy authentication: 92
To enable clients to access specific sites without LDA P auth enti c ation: 92
To set SSL termination conf iguration variables for c lient/Traffic Server connections: 95
To set SSL termination configuration variables for Traffic Server/origin server connections: 97
To set log management options from Traffic Manager: 100
To set log management options manually: 100
To select a st andard event log file format from Traffic Manager: 101
To select a standard event log file format manual ly: 102
To create traditional custom log format s: 104
To generate XML-based custom log files: 106
To create a summary log file: 107
To convert a binary log file to ASCII: 108
To set log file rolling options from Traffic Manager: 110
To set log file rolling options manually: 111
To set log splitting o ptions from Traffic Manager: 113
To set log splitting o pti ons manually: 113
To edit the log_hosts.config file: 114
To configure a Traffic Serv er node to be a collation server fr om Traffic Manager: 116
To configure a Traffic Serv er node to be a collation client from Traffic Manager: 116
To set log collation opt ions ma nually: 117
To run a s tand-alone collator: 118
To move information from t he orphan files into your central log files : 119
xii
This manual describes how to use and configure an HP Traffic Server™ system.
For information about installing Traffic Server and unsupported features and last minute information not
available in thismanual, refer to the HP Web Cache Server Appliance (sa2100 or sa2200) Getting Started
Guide.
The manual discusses the following topics:
• Chapter 1 provides an overview of the Traffic Server features and components
• Chapter 2 through Chapter 12 provide procedural information about starting, monitoring, configuring,
and maintaining the Traffic Server
• Appendix A through Appendix F provide Traffic Server reference information
Who should read this manual
This manual is intended for T raffic Serversystem administrators who configure, run, and administer Traffic
Server systems.
The manual assumes that you have experience in UNIX or Windows and Web server administration, and that
you are comfortable performing complex system configuration tasks, such as partitioning and formatting
disks, setting up TCP/IP ports, and establishing DNS round robin services.
Conventions used in this manual
This manual uses the following typographic conventions.
Preface
Convention Purpose
italics
Represent emphasis and introduce terms; for example, “the reverse proxy
option.”
bold
Represents graphical user interface options and menu names; for example,
click the Protocols button.
monospaced face
Represents commands, file names, file content, and computer input and
output; for example, “ use the
reconfigure command.”
monospaced italic
Represents variables for which you should substitute a value; for example,
“enter
filename.”
brackets[ ]
Enclose optionalcommandarguments in command syntax; for example, add
pathname [size].
verticalbar|
Separates value options in command syntax; for example,
open tcp|udp ports o_ports.
1
Welcome to a faster network.
Traffic Server speeds Internet access, enhances web site performance, and delivers unprecedented web
hosting capabilities.
This chapter discusses the following topics:
• What is Traffic Server?‚ on page 1
• Traffic Server deployment options‚ on page 1
• Traffic Server components‚ on page 3
• Traffic analysis options‚ on page 5
• Traffic Server security options‚ on page 5
What is Traffic Server?
The dream of global data networking has come true. Internet users request billions of documents each day all
over the world. Unfortunately, this dream ofglobal data networking has become a nightmarefor information
systems professionals as they struggle with overloaded servers and congested networks, trying to keep pace
with society’s growing data demands.
Traffic Server is a high-performance web proxy cachethat improves network efficiency and performance by
caching frequently accessed information at the edge of the network.This brings content physically closer to
end users for faster delivery and dramatically reduces bandwidth usage.
Traffic Server is designed to improve content delivery for enterprises, Internet Service Providers (ISPs),
backbone providers, and large intranets by m aximizing existing bandwidth.
Traffic Server deployment options
Traffic Server can be deployed i n different ways to best suit your needs and your environment:
• As a web proxy cache
• As a reverse proxy
• In a cache hierarchy
• In a T raffic Server cluster
The following sections provide a summary of the Traffic Server deployment options.
1Overview
IMPORTANT The paths to files used in the manual are the following:
For Command files:
/home/TS4/<version>/bin
For Configuration files: /home/TS4/<version>/config
For Log files: /home/TS4/<version>/logs
In each path reference, replace <version> with the version number that was installed
in the
/home/TS4 directory.
2
Chapter 1 Overview
TrafficServerasawebproxycache
As a web proxycache, user requestsforweb content go to Traffic Server on theway to the destinedwebserver
(origin server). If Traffic Server containsthe requested content, it serves it directly. If Traffic Server does not
have the requested content, Traffic Server acts as a proxy, fetching the content from the origin server on the
user’s behalf, while keeping a copy to satisfy future requests.
Traffic Server provides two proxy caching options:
• Transparent proxy caching, where user requests are automatically injected into a Traffic Server cache on
their way to the eventual destination. Users request Internet content as u sual without any browser
configuration and Traffic Server automatically serves their requests. The user’s client software (typically
abrowser)isunaware thatit iscommunicatingwith Traffic Server. Transparentproxycaching isdescribed
in more detailin Chapter 3‚ Web Proxy Caching.
• Explicit proxycaching,where the user’s client software must be configuredto sendrequestsdirectly to the
Traffic Server.
Traffic Server as a reverse proxy
As a reverse proxy, T raffic Server is configured to be the origin server the user is trying to connect to (the
origin server’s advertised host name resolves to Traffic Server, which is acting as the real origin server). The
reverse proxy feature is also called server acceleration. Reverse proxy is described in more detail in Chapter
5‚ Reverse Proxy and HTTP Redirects.
Traffic Server in a cache hierarchy
Traffic Server can participate in flexible cache hierarchies, where Internet requests not fulfilled in one cache
can be routed to other regional caches, taking advantage of the contents and proximity of ne arby caches.In a
hierarchy of proxy servers, Traffic Server can act either as a parent or child cache, eitherto other Traffic
Serversor to other caching products.
Traffic Server supports the standard Internet Cache Protocol (ICP) to interoperate with existing ICP cache
hierarchies.
Hierarchicalcaching is described in more detail in Chapter 7‚ Hierarchical Caching.
Traffic Server in a cluster
Traffic Server scales from a single node into multiple nodes that form a cluster allowing you to improve
system performance and reliability. Traffic Server detects the addition or removal of nodes automatically. If
Traffic Server’s virtual IP failover option is enabled, Traffic Server maintainsa pool of virtual IP addresses
that it assignsto the nodes of the cluster. Traffic Server can detect ha rd node failures (such as power supply
or CPUfailures) andreassignIPaddressesof the failednodeto the remainingoperationalnodesautomatically.
Traffic Server has two clustering modes:
• Management-only mode, where you can administerall the nodesin a cluster at the same time. Nodes
automaticallyshare configuration information.
• Full-clustering mode, where the node caches act as a single aggregate cache. A T raffic Server cluster
distributesits cacheacrossits nodesintoasingle, virtualobjectstore,rather thanreplicatingthecache node
by node.
A fullyclustered TrafficServer providesasinglesystemimageto both users and administrators,appearing
as a single virtual server. Full-clustering mode includes management-only mode.
Traffic Server clusters are described in more detail in Chapter 6‚ Traffic Server Clusters.
3
Chapter 1 Overview
Traffic Server com pon ents
Traffic Server consists of several components that work together to form a web proxy cache you can easily
monitor and configure. The main components are described below.
The Traffic Server cache
The Traffic Server cache consists of a high speed object database called the object store. The object store
indexesobjectsaccording toURLsandassociatedheaders.Usingsophisticatedobjectmanagement,theobject
store can cache alternate versions of the same object, varying on spoken language or browser type, and can
efficientlystoreverysmall andverylarge documents, minimizingwastedspace.Oncethecache beginstofill,
the Traffic Server mobilizes garbage collectors to remove staledata,ensuringthat the most requested objects
are kept on-hand and fresh.
Traffic Server is designed to tolerate total diskfailureson any of the cache disks. If the disk fails completely,
TrafficServerm arks the entirediskas corrupt and continuesusing theremaining disks.Ifallof the cache disks
fail, Traffic Server goesinto proxy-only mode.
You can partitionthe cachetoreservea certainamountofdiskspace for storing dataforspecific protocolsa nd
origin servers.
The Traffic Server cache is described in more detail in Chapter 8‚ Configuring the cache.
RAM cache
Traffic Server m aintains a small RAM memory cache of extremely popular objects. This RAM cache serves
the most popular objects as fast as possible and reduces load on disks, especially during temporary traffic
peaks. You can configure the RAM cache size to suit yourneeds.
The Adaptive Redirection Module (ARM)
The Adaptive Redirection Module (ARM) is used in transparent proxy caching to redirect intercepted user
requests destined for an origin server to the Traffic Server. Before the traffic is redirected by the ARM, it is
interceptedbyanL4switchorrouter.
To redirect user requests to Traffic Server,the ARM changes an incoming packet’saddress. The packet’s
destination IP address is changed to the IP addressof Traffic S erver and the packet’s destination port is
changed according to the protocol used. For example, for HTTP, the packet’s destination port is changed to
Traffic Server’s HT TP port (usually 8080).
The ARM supports automatic bypass of sites that do not function properly with proxy caches.
Traffic Server can respond to client request overloads by forwarding requests directly to origin servers. This
feature is called load shedding. Overload conditions, such as network outages, misconfigured routers, or
security attacks, can slow down Traffic Server’s response time. In transparent configurations, Traffic Server
can use its ARM bypass functionality to forward overload requests directly to origin servers, bypassing the
cache. When the overload condition dissipates, Traffic Server automatically returns to full caching mode.
The Host Database
The Traffic Server host database stores the Domain Name S erver (DNS) entries of origin servers to which
Traffic Server connects t o fulfill user requests. This information is used to adapt future protocol interactions
to optimize performance.
Among other information, the host database tracks:
• DNS information (for fast conversion of host names to IP a ddresses)
• The HTTP version of each host (so advanced protocol features can be used with hosts running modern
servers)
4
Chapter 1 Overview
• Host reliability and availability information (to avoid making the user wait for non-functional servers)
The DNS Resolver
Traffic Server includes a fast, a synchronous DNS resolver to streamline conversion of host names to IP
addresses. Traffic Server implements the DNS resolver natively, directly issuing DNS command packets,
rather than relyingon slower,conventional resolverlibraries. Many DNS queries can be issued in parallel and
a fast DNS cache maintains popular bindingsin memory, significantly reducing DNS traffic.
Traffic Server processes
Traffic Server c ontains three processes that work together to process Traffic Server requests and manage,
control, and monitorthe health of the Traffic Server. The three processes are described below:
• The
traffic_server process is the transactionprocessingengine of Traffic Server. It is responsiblefor
accepting connections, processing protocol requests, and serving documents from the cache or origin
server.
• The
traffic_manager process is the commandandcontrolfacilityofthe Traffic Server, responsiblefor
launching, monitoring, and reconfiguring the
traffic_server process. The traffic_manager
process is also responsible for T raffic Manager,the proxy auto configuration port, the statistics interface,
cluster administration, and virtual IP failover.
If the
traffic_manager process detectsa traffic_server process failure, it instantly restarts the
process but also maintains a connection queue of all incoming requests. All incoming connections that
arrive in the several seconds before full server restart a re saved in the connection queue and processed in
first-come, first-served order. This connection queueing shields users from any server restart downtime.
• The
traffic_cop process monitorsthe health of both the traffic_server and traffic_manager
processes. The traffic_cop process periodically (several times each minute) queries the
traffic_server and traffic_manager process by issuing heartbeat requests to fetch synthetic web
pages. In the event of failure (if no response is received withina time-out interval or if an incorrect
response is received),
traffic_cop restarts the traffic_manager and traffic_server processes.
Figure 1-1. illustrates the three Traffic Server processes.
Figure 1-1. Traffic Server processes
crontab process ensures
that traffic_cop process
is running
traffic_server
traffic_manager
traffic_cop
operating
system
Traffic
Server
Processes
Traffic Server Node
5
Chapter 1 Overview
Administration tools
Traffic Server offers several administration alternatives to suit the needs of many environments:
• T raffic Manager User Interface (UI) isa web based interfaceconsisting of a seriesofweb pagesaccessible
through a browser. Traffic Manager provides a rich set of graphs and statistical displays for monitoring
Traffic Server pe rformance and network traffic, a nd a set of options for configuringand fine-tuning the
Traffic Server s ystem. Traffic Manager offers password-protected, SSL-encrypted, single-point
administrationfor an entire Traffic Server c luster.
• The Traffic Line command-line interface is a textbased interface that provides equivalent functionality to
that of Traffic Manager. From the command line, you can e xecute individual commands or script a series
of commands in a shell.
• Various Configuration files allow complete administration through a simple file editing and signal
handling interface. You can change configuration optionsby editing configuration files manually instead
of usingTrafficManageror TrafficLine.(Anychangesyoum ake through Traffic Manager orTrafficLine
are automatically made to the configuration files.)
Traffic analysis options
Traffic Server provides several options for network traffic analysis and monitoring:
• Traffic Manager statistics and graphs show network traffic information. You can view graphs and
statistics from Traffic Manager or collect and processstatistics using the command-lineinterface, Traffic
Line.
• MRTG (Multi Router Traffic Grapher) is a graphing tool that provides a variety of graphs showing
historicalinformation about virtual memory usage, client connections,documenthit rates,and so on. You
can access MRTG from Traffic Ma n ager.
• SNMP Network Management support letsyoumonitorandm anage Traffic Server throughSNMPnetwork
management facilities. TrafficServersupports two management informationbases(MIBs): MIB-2, a well
known standard MIB, and the HP proprietary Traffic Server MIB that provides more specific node and
cluster information.
• TrafficManageralarms are presentedinTrafficManager.TrafficServersignalsanalarmfor any detected
failure condition. You can configure Traffic Server to send E-mail or page support personnel when an
alarm occurs.
• Transaction logging lets you record information in a log file about every request that Traffic Server
receives and every error it detects. By analyzingthelog files,you can determine how many people use the
TrafficServercache, how muchinformatione ach person requested,andwhatpagesare most popular. You
canalsosee why aparticulartransactionwasin error andwhatstate theTrafficServerwasin at a particular
time. For example, you can see that Traffic Server was restarted or that cluster communication timed out.
Traffic Server supports several standard log file formats, suchasSquid and Netscape, and its own custom
formats. You can analyze the standardformat log files with off-the-shelf analysispackages.To help with
log file analysis, you can separate log filesso that they contain information specific to protocol or hosts.
Trafficanalysis options are described in more detail in Chapter 9‚ Monitoring Traffic. Traffic Server logging
options are described in Chapter 12‚ W orking with Log Files.
Traffic Server security options
Traffic Server provides numerous options that enable you to establish securecommunication betweenthe
Traffic Server system and other computers on the network. Using the security options, you can:
• Control client access to the Traffic Server proxy cache.
• Control which hosts are allowed to access the Traffic Server machine (ARM security).
6
Chapter 1 Overview
• Configure T raffic Server integration into your firewall and control trafficthrough a SOCKS server.
• Configure T raffic Server to use multiple DNS servers to match your site’s security configuration. For
example, you might choose to have Traffic Server use different DNS serversdepending on whe ther it
needs to resolve host names located inside or outside a firewall. This enables you to keep your internal
network configuration secure while continuing to provide transparent access to external sites on the
Internet.
• Use LDAP-based proxy authentication that enables you to leverage e xisting directory services by
supportingasynchronous match and bind requests to LDAP s ervers. This enables you to maintain policies
thatrequireusers to log in andbeauthenticatedbythe proxy before goingoutonto the Internet.Inaddition,
you can enable TrafficServerclients to access specificsiteson theInternetwithoutbeing authenticatedby
theLDAPserver.TrafficServer usesalocaldatabasetoimprovetheperformanceof LDAP authentication
and, upon completion, logs successfully authenticated users.
• Secure connections in reverse proxy mode betweena client and Traffic Server, and TrafficServer and the
origin server, using the SSL termination option.
• Control access to Traffic Manager using:
o SSL (Secure Sockets Layer) protection for encrypted, authenticated access
o An access control list (ACL) that defines which hosts are allowed to access Traffic Manager
o Administrator accounts that define which users can access Traffic Manager and which activities they
can perform (for example, view statistics only or view statistics and configure the Traffic Server)
• Set NNTP specific security options that:
o Control user access to ne ws articles cached by Traffic Server by defining access privileges for a
particular group of clients
o Enable external program-based NNTP authentication providing enterprise-wide control over new s
access, posting behavior, and other related privileges
Traffic Server security options are described in more detail in Chapter11‚ Security Options.
7
You are now ready to begin using Traffic Server.
This chapter contains the following sections:
• Overview of Access Methods‚ on page 7
• Verifying that Traffic Server is up and running‚ on page 8
• Accessing Traffic Manager‚ on page 8
• Starting Traffic Line‚ on page 10
Overview of Access Methods
The instructionsin this guide refer to different methods of a ccessing the functionalityof the cache appliance.
As a general rule all tasks should be accomplished with Traffic Manager since it has the mostintuitive user
interface and automates many of the administrative tasks that would otherwise have to be completed
manually. The access methods are as follows:
1. Traffic Manager. Traffic Manager is a web-based tool that will allow customization of most aspects of
the cache appliance functionality. This is the preferred method for working with Traffic Server. Traffic
Manager can be accessed by a client web browser directed to the URL
http://<ip.address.of.appliance>:<adminport>
. T he IP address of the appliance was set during the initial
configuration as described in the HP Web Cache Server A ppliance Getting Started Guide. The
adminport is configurable, but will be 8081 after initial configuration until a different port is manually
specified. The user will be prompted for a user name and password, which are both admin after initial
configuration unless manually specified otherwise.
2. General Telnet access. By using telnet to access the appliance with the Telnet access user name and
password entered during the appliance initial network configuration session, you will be presented with a
menu listing a number of configuration and maintenance tasks. There are some Traffic Server
configurationtasks that require shellaccess to the cacheappliance filesystem, and youwillhavetoselect
shell access from the menu to complete these tasks. Instructions in this guide include T raffic Manager
instructions as well as manual editing of c onfiguration files throughshell access for completeness;
however, it is highly recommended that Traffic Manager be used whenever possible.
3. Telnet access to the appliance initial network configuration utility. T his is covered i n the HP Web
Cache Server Appliance Getting Started Guide. If you telnet into a previously unconfigured appliance or
an appliance that has just been reset to factory defaults with a user nameof hpsaconfig a nd a passwordof
hpsa, you will access the appliance initial network configuration utility where you can enter the IP
addresses for the LAN adapters, c onsole user name and password. After successfully completing the
appliance initial network configuration task, the
hpsaconfig user will be deleted and you must access
the appliance with the console user name and password you entered during the initial network
configuration.
4. Serial port access for initial configuration. T his is covered in the HP Web Cache ServerAppliance
Getting Started Guide. This allows a user to initially access the cache appliance before it has any IP
addressto perform initialnetwork setup.
2 Getting Started
8
Chapter 2 Getting Started
Verifying that Traffic Server is up and ru nning
When youcompletedtheinitial configurationof the cache applianceasdescribedintheHPW eb Cache Server
Appliance sa2100/sa2200 Getting Started Guide, Traffic Server was automatically started.
To verify that Traffic Server i s up and running:
1. Access Traffic Manager (refer to Overview of Access Methods‚ on page 7).
2. From the Monitor tab, click the Protocols button.
3. Make a note of the current HTTP ClientTotal Document Bytes statistic.
4. Set your browser proxy setting to the Traffic Server name and proxy port.
5. Browse the internet.
6. Recheck the HTTP Client Total Document Bytes statistic.
This value increases as Traffic Server processes HTTP requests.
Accessing Traffic Manager
Traffic Manager is Traffic Server’s browser-based user interface, consistingof a series of web pages. Traffic
Manager provides a rich set of graphs and statistical displays for monitoring Traffic Server performance and
network traffic, plus a set of optionsfor configuring and fine-tuning yoursystem. All common Traffic Server
configuration tasks should be performed using this UI.
You access Traffic Manager through your web browser.
To access Traffic Manager:
1. Open your web browser.
Traffic Manager requiresJava and JavaScript;be sure to enable Java and JavaScript in your browser.
2. Type one of the following locations in your browser:
Standard
http://nodename:adminport/
SSL https://nodename:adminport/
where nodename is the name of the applianceand adminport is the number assignedto Traffic Manager
port (the defaultvalue for adminport is 8081).
NOTE Use the following
https addressinstep 2 to reach Traffic Manager only if you have
restricted access to Traffic Manager via SSL connections; otherwise, use the
standard
http address.
9
Chapter 2 Getting Started
Traffic Manager opens in your web browser and displays the Dashboard, s hown in Figure 2-1.
Using the Monitor and Configure tabs
Traffic Manager has two tabs:
• The Monitor tab lets you view Traffic Server performance and network traffic statistics. For more
information, refer to Viewing statistics from TrafficManager‚ on page 63.
• The Configuretabletsyou view and modifyTrafficServer’sconfigurationoptions.For more information,
refer to Appendix 10‚ Configuring Traffic Server.
By default, Traffic Manager starts by displaying the Monitor tab. To display the Configure tab, click the
Configure tab to the right of the Monitor tab.
Using online help
Both the Monitor and Configure tabs provide a Help button. When you click the Help button, the Traffic
Server online help opens in another browser window. The online help describes each page that opens when
you click a button on the Monitor or Configure tab.
Figure 2-1. Traffic Manager: The Monitor Dashboard
The Monitor
tab contains
seven
buttons.
Click a
button to
display a
page of
Click the
Configure tab
todisplaythe
Configure
buttons and
set
configuration
Click the Help
button to
display a
description of
the current
page
This shows t he current
user loggedin to Traffic
Manager
10
Chapter 2 Getting Started
Starting Traffic Line
Traffic Line is a text-based interface that can be accessed t hrough a Telnet session. You can use Traffic Line
to perform many of the tasks you can perform in Traffic Manager. For ease of use, it is recommended that
Traffic Manager be used unless a function only available in Traffic Line is required.
Traffic Line has two command-line modes:
• TrafficLinebatchmode
You can use the batchmode to execute individual commands or to script multiple commands in a shell.
Refer to Appendix C‚ Traffic Line Commands for a listof commands.
• TrafficLineinteractivemode
You can use interactive mode to retrieve statistics and to configure Traffic Server. TrafficLineinteractive
mode consists of several levels of commands. The Traffic Line interactive levels consist of the same
commands available on Traffic Manager Monitor and Configure tabs.
To start a Traffic Line session:
1. Telnet into the HP web cache appliance and select Shell Access as described in Overview of Access
Methods‚ on page 7.
You are now ready to enter Traffic Line commands.
Traffic Line commands take the following form:
traffic_line -flag argument
2. For a list of traffic_line commands, enter:
traffic_line -h
3. To enter Traffic Line interactive mode, enter the followingcommand:
traffic_line -i
For information about monitoring Traffic Server using Traffic Line interactive mode, refer to Viewing
Statistics from Traffic Line‚ on page 68.
For informationabout configuringTrafficServer usingTrafficLine interactive mode,referto Configuring
Traffic Server using Traffic Line‚ on page 76.
Restarting Traffic Server
Various procedures throughout thismanual instruct you to restart Traffic Server.
YoucanrestartTrafficServer by scrollingto the Web ManagementsectionoftheServer BasicspageinTraffic
Manager, and click the restart button. (For more information, see Configuring Traffic Server using Traffic
Manager‚ on page 73.)
11
The idea behind web proxycaching is to store copies of frequently accessed documents close to users and
serve this information to them on demand.Internet users get their information faster and Internet bandwidth
is freed up for other tasks.
This chapter discusses the following topics.
• Understanding web proxy caching‚ on page 11
• News article caching‚ on page 16
Understanding web proxy cach ing
Internet users direct their requests to web servers all over the Internet. For a caching server to serve these
requests,itmustactasaweb proxy server. A web proxy server fields user requests to arbitrary web servers
and either servesthe requests, or forwards them on to the origin server (the web server that contains the
original copy of the requested information).
The Traffic Serverproxy supportsboth transparentproxyc aching, where the user’s client software (typically
a browser) is unaware that it is communicating with a proxy, and explicit proxy caching, where the user’s
client software must be configured to send requests directly to the traffic Server proxy.
A day in the life of a cache request
Here is an overview of the steps that take place as a Traffic Server proxy cache serves a user request.
1. Traffic Server receivesa user request for a document, image, news article, or other web object.
2. With the object address in hand,TrafficServerlooksup therequested object in its objectdatabase(cache).
3. If the object is in the cache, Traffic Server checks to see if the object is fresh enough to serve. (See
Ensuring cached object freshness‚onpage 12 for details.) If the objectis fresh, Traffic Server serves it to
the user as a cache hit (Figure 3-1.).
Figure 3-1. A cache hit
4. If thedata in thecacheisstale,T raffic Server connects to the origin server and asks if the document is still
fresh. If the document is stillfresh, Traffic Server sends the cached copy to the user immediately.
3 Web Proxy Caching
cache
local
request
a cache hit
returned document
Traffic Server
hit
1
2
3
client
12
Chapter 3 Web Proxy Caching
5. If the object isnotinthecache (a cache miss)or the serverindicates thatthecached copy isnolongervalid,
Traffic Server gets the document from the origin server, simultaneously streaming i t to the user and the
cache (Figure 3-2.). Subsequentrequests for the object will be served faster.
Figure 3-2. A cache miss
Caching is more complex than the preceding overview suggests. In particular, the overview does not answer
these questions:
• How does Traffic Server ensure freshness?
• How does Traffic Server serve correct HTTP alternates?
• How does Traffic Server treat requests for objects that cannot or should not be cached?
The following sections discuss these questions.
Ensuring cached object freshness
Traffic Server ha ndles object freshness differently depending on protocol.
Revalidating HTTP objects
If an HTTP object is stale, Traffic Server revalidates the object. A revalidation is a query to the origin server
that asks if the object is unchanged. The result of a revalidation could be:
• The object is still fresh; Traffic Server resets its freshness limit and serves the object.
• A new copy of the object is available; Traffic Server caches the new object,replacing the stale copy, and
serves the objectto the user simultaneously.
• The object no longerexists on the origin server; Traffic Server does not serve the cached copy.
• The origin server does not respond to the revalidation query. The Traffic Server s erves the stale object
along with a
111 Revalidation Failed warning.
HTTP Web documents support optional author-specified expiration dates. Traffic Server adheres
to these expiration dates; otherwise it picks an expiration date based on how frequently the
document is changingand on administrator-chosen freshness guidelines. In addition,
documents can be revalidated, checking with the server if a document is still fresh.
FTP FTP documents s tay in the cache for a time period s pecified in the Freshness section of
the Cache page in Traffic Manager’s Configure Mode.
NNTP News articles are refreshed each time Traffic Server polls parent news servers for c hanges
in group lists, article overview lists, and article updates. See Maintaining the cache:
updates and feeds‚ on page 19.
request
a cache miss
Traffic Server simultaneously caches
and serves the document to the client
Traffic Server
1
2
3
client
cache
local
server
origin
miss
13
Chapter 3 Web Proxy Caching
HTTP object freshness tests
Here is how Traffic Server determines an HTTP document’s freshness:
Expires header test. Some documents come with Expires headers or max-age headers that e xplicitly
define how long the document may be c ached. A simple comparison of the current time with the e xpiration
time tells T raffic Server whether or not the document is fresh.
Last-Modified / Date header test. If there is no expiration information, a freshness limit can be estimated
from the
Last-Modified and Date headers. The Last-Modified header indicates how long ago a
document was modified. If a document was last modified two years ago, it is unlikely to suddenlychange, so
Traffic Server can cache it safely for a while.B ut ifthedocument just changed 5 minutes ago, it’s likely to be
volatile,and Traffic Server should not cache it forlong.TrafficServer uses a formula to determine the length
of time it considers a stored object to be fresh. Traffic Server stores an object for s ome percentage (F) of the
time that elapsed since it last changed, 10% by default:
freshnesslimit=F*(Date-Last-Modified)
Where the Date header provides the date the object was sent to Traffic Server, and the Last-Modified
header provides the date the object was last modified on the origin server.
For example, if a document was last m odified 32 days ago and was sent to Traffic Server 2 days ago, it is
considered fresh in cachefor 3 days after it was sent, assuming a factor of 10%. It is considered fresh for one
more day.
Because this method might select large freshness times for documents that have not changed for a long time,
cache administratorsmaywant to place an upper boundary on the freshness limit. The freshness limit, then, is
the minimum of this upper boundary and the computed freshness limit.You configure this upper boundary in
the Freshness sectionof the Configure:Cachepageof TrafficManager. (Minimum freshness information
for a document to be cacheable is one of the configuration options under Freshness.)
Default test. For documents that do not have Expires headers or do not have both Last-Modified and
Date headers, you can specifyan absolute freshness limit in t he Freshness section of the Configure: Cache
page.
Revalidate rules in the cache.config file. Revalidate rules apply specific freshness limits to specific HTTP
or FTP objects. You can set freshness limits for objects originating from particular domains or IP addresses,
objects with URLs that contain specified regular expressions, objects requested by particular clients, and so
on. See cache.config‚ on page 162.
Deciding whether to serve HTTP objects
Even though a document may be fresh i n the cache, clients or servers may have their own constraints that
prevent them from retrieving the document from the cache. For example, a client might request that a
document not c ome from a cache, or if it does, it cannot have been cached for more than 10 minutes.
Traffic Server bases the servability of a cached document on
Cache-Control header fields. Cache-
Control
headers can appear in both client requests and server responses.
The following cache-control header fields affect whether objects are served:
• The
no-cache field, sentbyclients,tellsTrafficServerto servenoobjects directly from the cache; always
revalidate. You can configure Traffic Server to ignore client
no-cache fields.
• The
max-age field, sent by servers, is compared to the document age; ifthe age is less than the max-age,
the document is fresh and can be served.
• The
min-fresh field, sent by c lients, is an acceptable freshness tolerance.The client wants the object to
be at least this fresh. If a cached document does not remain fresh at least this long in the future, it is
revalidated.
• The
max-stale field, sent by clients, permits Traffic Server to serve stale documents provided they are
not too old. Some browsers may be willing to take slightly old documents in exchange for improved
performance, especially during periods of poor Internet availability.
14
Chapter 3 Web Proxy Caching
Traffic Server applies Cache-Control servability criteria after HTTP freshness criteria. For example, a
document might be considered fresh, but if its age is greater than its
max-age, it is not served.
Configuring HTTP freshness options
You can configure the following freshness guidelines for Traffic Server:
• How oftentorevalidate(whentoconsiderobjectsstale). See Configuring HTTP revalidation‚on page 14.
• Whether to cache documents without freshness information. See ConfiguringHTTP cachability‚ on
page 14.
• The upper boundary used to determine if the Last-Modified /Date freshness limit is too long.
• The absolute freshness lifetime used to estimate the freshness of documents without
Expires or Last-
Modified
headers.
In the
cache.config file,you can configure Traffic Server to revalidate objects from specific origin servers
at specific times. Refer to cache.config‚on page 162.
Configuring HTTP revalidation
The following HTTP revalidation options are available:
• Always revalidate(everythingis considered stale).
• Never revalidate (everything is considered fresh).
• Revalidate all objects without
Expires headers. Evaluate the freshness of objects with Expires headers
by first checking the
Expires header, and then checking Cache-Control headers.
• E valuate freshness as follows:
1. Use the
Expires header test, if applicable,otherwise go tostep 2. If the object is stale, revalidate. If
it is fresh, check the
Cache-Control headers.
2. Use the
Last-Modified / Date header test, if applicable, otherwisegoto step 3. If the object is fresh
according to the
Last-Modified / Date test, check the Cache-Control headers for any freshness
restrictions.
3. Use the absolute freshness limit specified in the Freshness section of the Configure: Cache page.
Revalidate if the age is past the freshness limit.
Configuring HTTP cachability
The following HTTP cachability options are available:
• Cache only documents that have
Expires headers
• Cache only documents that have
Expires or Last-Modified headers
• Do not restrict caching
Caching HTTP alternates
Some origin servers answer requests to the same URL with a variety of objects. The content of these objects
can vary widely, according to whe ther a server delivers content for different languages, targets different
browsers with differentpresentation styles,or deliversvariablec ontent at different times of the day.Different
versions of the same object are called alternates.
15
Chapter 3 Web Proxy Caching
Alternates are identified by header information. You can configure Traffic Server to cache a ll alternates
according to a particular header. For example, if you tell Traffic Server to vary on the
User-Agent header,
Traffic Server c aches all the different user-agent versions of documents it encounters. You configure the
cachingof alternates in the Variable Content section of t he Cache page in Traffic Manager’s Configure
mode.
To cache or not to cache?.
The following table lists the HTTP caching directives that Traffic Server follows.
NNTP You can limit article caching to specific news groups. See Blocking particular groups‚ on
page 18.
FTP You can specify never-cache rules for specifictypes of FTP documents in the
cache.config fi le. See cache.config‚ on page 162.
HTTP Traffic Server responds to caching directives from clients and o rigin servers, as well as
configurable options inTraffic Manager and the
cache.config file.
Directive source Caching directives
administration
options
Traffic Server has the following administration options for caching:
• Configure Traffic Server not to cache objects with URLs containing the
following:
?
;
/cgi
end in
.asp
• Configure Traffic Server not to cache objects served in response to the Cookie:
header.
• Use never-cache rulesin the cache.config file. Refer to cache.config‚ on
page 162.
client Traffic Server does not cache objects with the following request headers. Note that
some of these directives can be overridden by Traffic Server administration options.
• Cache-Control: no-store header
• Cookie: header
• Authorization: header
origin server TrafficServerdoes not cache objects with the following response headers. Note that
some of these directives can be overridden by Traffic Server administration options.
• Cache-Control: no-store
• www-Authenticate: header
• Set-Cookie: header
• Cache-Control: no-cache header
• Pragma: no-cache header
• Expires: header with value of 0 (zero) or a past date
16
Chapter 3 Web Proxy Caching
Scheduling updates to local cache content
To further increasethe performance of Traffic Server, you can configure it to perform scheduled updates to
the local cache content. This enables you to instruct Traffic Server to explicitly load specific objects into
cache. You might find this especially beneficial when using Traffic S erver as a reverse proxy for server
acceleration,enablingyou to preloadcontent thatyouanticipatewillbe in demand.SeeUnderstandingreverse
proxy caching‚ on page 33.
You do this by inputtinga list of URLs inthe Content Management page in Traffic Manager. This modifies
the
update.config file, specifying objects that you want to schedule for update along with the time and
interval of when this update should take place. The UI also enables you to specify a recursion depth for the
URL.
Traffic Server uses this information to determine the URLs for which it is responsible and, for each URL,
derives all recursive URLs if applicable. It then generates a unique,sorted URL list. Using this list, Traffic
Server initiates an HTTP
GET for each un-accessed URL, ensuring that it remains within the user-defined
limits for HTTP concurrency at any giventime. The system logs the completion of allHTTP
GET operations,
enabling you to monitor the performance of this feature.
News article caching
Traffic Server c an function as a news server or a caching news server. T his section provides background
information about Traffic Server news server and the Ne twork News Transfer Protocol (NNTP) caching
features.
News, alsoknownasUSENETanddiscussions, isa systemofonlinediscussion groups. NNTP is theprotocol
used to retrieve and distribute these discussion groups. News groups exist to discuss just about any subject;
for example,
rec.humor, talk.religion, news.answers, rec.food.recipes,andcomp.std.unix.
The articles posted to these groups are propagated around the world. Traffic Server supports NNTP as
specified in RFC 977 and many common extensions and proposed extensions.
To read news articles, users need a news reader, such as Netscape Communicator or Microsoft Internet
Explorer, and a ccess to a news server. Traffic Server is a caching news server. It can be configured tosit
transparently between users and a parent or backing news server, increasing responsiveness for the user and
decreasing network bandwidth use and the load on the parent news server.
Figure 3-3. Traffic Servers caching news articles for a distant NNTP server
Parent
NNTP
Server
Traffic
Server
users
users
Traffic
Server
17
Chapter 3 Web Proxy Caching
Traffic Server provides many configurable options for supporting parent NNTP servers. The following
sections describe Traffic Se rver’s NNTP features.
Traffic Server as a news server
When clients want to read news, they access a news server. The news server offers a list of groups to which
clients can subscribe. For each subscribed group, the clients read an overview list of the articles in the group,
and then select an articleto read. When Traffic Server acts as a news server,it:
• Maintains lists of supported news groups
• Accepts news feeds for each supported news group
• Serves requested articles to users
• Accepts and numbers user postings to its supported news groups
Traffic Server as a caching proxy news server
When Traffic Serveracts as a caching proxy news server for a particular news server,it:
• Maintains l ists of the news groups on its parent NNT P servers. You c an configure the frequency that
Traffic Server updates its copies of group lists.
• Caches and serves article overview listson demand. You can also tell Traffic Server to pull article
overview lists from the parent ne ws server periodically.
• Caches and serves articles on demand. Traffic Server can also accept news feeds, like any news server.
• Caches and serves miscellaneous LIST files, such as subscription files.
• Sends user postings to the parent news server.
When clients issue news requests, Traffic Server intercepts these requests and serves them from its cache,
reducing traffic to parent news servers. If a particular overview or article is not in the cache, Traffic Server
forwards requests to the parent server.
Supporting several parent news servers
TrafficServercancache articles forseveralnewsservers. You specifyallof the parent news serversforTraffic
Server in the
nntp_servers.config file. See nntp_servers.config‚ on page 177. For each parent news
server, you can have Traffic Server cache some or all of that server’s news groups.
Some of the possible parent configurations that T raffic Server supports are as follows:
Several news servers supplying the same groups
Several news servers can be configured to redundantly serve the same groups, providing enhanced reliability.
Traffic Server provides the following features for managing these configurations:
• Priorities
If TrafficServerhastocontact a parentnewsserver for informationabout a group suppliedby severalnews
servers, TrafficServer contacts the news server with the highest priority.
• Round-robin
If several parent news servers supplying the same group have the same priority, Traffic Server selects a
parent news server in round-robin fashion.
• Failover
If a request to a parent server fails, Traffic Server tries the next server in the round-robin (of the same
priority), and then servers of lower priority.
18
Chapter 3 Web Proxy Caching
• Background re tries
Failed servers a re retried in the background and are used (restored to their specified priority) when they
become available.
Several servers supplying different groups
Several news servers can be configured with news servers supplying different (disjoint) groups.
Administrators can use this feature to spread the load based on group.
NNTP cache hierarchies
Using a TrafficServer as parent to a groupof Traffic Serverscanreduceloado n aparentnewsserverandtake
advantage of the large number of concurrent connections Traffic Server supports.
Figure 3-4. Hierarchy of news caching servers
In Figure 3-4. above, the parent news server for each of the child Traffic Servers isthe parent Traffic Server.
The parent Traffic Server is a child cache to the distant parent news server.
Nonstandard ports and network interfaces
You can configure the interface from which to connect to a parent news server port. You can also configure
the port on the parent server to which Traffic Server connects.
Blocking particular groups
You can block particular groups on specified news servers.Clientsdo not see blocked groups in news server
group lists. You list all blocked groups in the
nntp_servers.config file; see page 177.
Clustering
Articles, overview lists, group lists, and LIST files are all m aintained in HP ’s high performanceobject store.
This information is updated at configurable intervals so that users and child caches see a consistent view of
news.
Large clusters of Traffic Servers can be configured to act as a single l arge virtual cache with all the storage
and serving power of the aggregate. See Chapter 6‚ Traffic Server Clusters. Article numbers and group
information are maintained consistently across the cluster.
Zurich
Traffic Server Child Caches
London
Traffic Server
Parent Cache
Parent
NNTP
Server
Bomba
y
Paris Madrid Oslo
Loading...