PICS Labels
Platform for Internet Content Selection Version 1.1
PICS Label Distribution
Label Syntax and Communication Protocols
by Tim Krauskopf (
timk@spyglass.com
),
Jim Miller (
jmiller@w3.org
), Paul Resnick
presnick@research.att.com
),
and Win Treese
treese@OpenMarket.com
Revision 5 Last modified on Sun. May 5, 1996
Overview
This document has been prepared for the technical subcommittee of PICS (Platform
for Internet Content Selection). It defines a general format for labels and
three methods by which these labels may be transmitted:
In an HTML document.
We specify a mechanism, using the existing META tag, for embedding one or
more labels in (the header of) an HTML document.
With a document transported via a protocol that uses RFC-822 headers.
Labels can be transmitted using
any
protocol that uses RFC-822-style
headers. In addition, we define an extension specific to the HTTP protocol
that allows an HTTP client (Web browser) to request which labels (if any)
it would like to have sent along with a document. The PICS committee hopes
that other network protocols will be extended in a similar way.
Separately from the document.
A client can request labels from a "label bureau" that runs the HTTP protocol.
The labels may refer to any document that has a URL (see
RFC-1738
), including
those available through protocols other than HTTP, such as ftp, gopher, or
netnews. Notice that PICS defines a new URL scheme for referencing IRC chat
rooms (see
Rating Services and
Rating Systems
). The simplest implementation of a label bureau is an
off-the-shelf HTTP server running a special CGI script.
General Format
A label consists of a
service identifier
label options
, and
rating
. The service identifier is the URL chosen by the rating service
(see
Rating Services and Rating
Systems
) as its unique identifier. Label options give additional properties
of the document being rated as well as properties of the rating itself, such
as the time the document was rated. The rating itself is a set of attribute-value
pairs that describe a document along one or more dimensions. One or more
labels may be distributed together as a list. The general form for a label
list (formatted for presentation, and not showing error status codes) is:
(PICS-1.1
option...
labels [
option...
] ratings (
[option...] ratings (
...
service url
> [
option...
labels [
option...
] ratings (
[option...] ratings (
...
...)
specific
label applies to a single document. If the document is
in HTML format, it may refer to other documents, either by external reference
(for example, using the tag) or by requesting that they
be displayed in-line (for example, using the or
...contents of foo.html...
Explanation of example
The client requests the document foo.html. In addition, the client requests
the full label of the document from the rating service "http://www.gcf.org/v2.5".
The server responds by sending back the label, in the PICS-Label header,
as well as the document. The format of the PICS-Label header field (a
labellist
) allows the server to respond either with a label or an
explanation of why the label is not available, since it would be inappropriate
for the server to generate an HTTP error status if the document is available
but (some of) the labels are not.
Following the usual HTTP distinction between HEAD and GET, a client that
wishes to examine a rating before retrieving the full document can substitute
the word HEAD for GET in the request. The server responds with exactly the
headers shown above, but does not send back the document foo.html.
Detailed Syntax of HTTP Requests for Labels With Document
The following grammar, in modified BNF, describes the syntax of the additional
header line to be included in an HTTP request for a document and associated
labels.
request-header
::
'Protocol-Request: {PICS-1.1 {params ' [
completeness
extension
services
'}}'
completeness ::
'minimal' | 'short' | 'full' | 'signed'
extension ::
'{'
token-or-quoted-string
+ '}'
where the first
token-or-quoted-string
is not '
services
'.
token-or-quoted-string ::
token
quotedname
token ::
alphanumpm
services
:: '{' 'services'
quotedURL
+ '}'
A request for a
minimal
label asks that all options be omitted, unless
a generic label is returned, in which case the
generic
and
for
options must also be included in the label. A
short
label includes
everything that is included in a
minimal
label, plus additional options
that the server deems appropriate. A request for a
full
label asks
that as much information as possible should be sent back in the label, either
directly or through the use of a
complete-label
(or
full
) option,
but no
signature-RSA-MD5
option is needed.
A request for
signed
labels asks that all the information in a
full
label should be sent, along with a digital signature on the label
itself. In a signed label the information must be transmitted directly as
part of the label (and included in the computation of the signature); the
complete-label
(or
full
) option may be sent, but it would be
redundant. Details of signing labels are included in the section
MICs and Digital Signature
It is acceptable for a server to ignore the
completeness
, either by
delivering more or fewer options than requested. If the
completeness
is omitted, it should be treated as though
minimal
had been supplied.
For future extensibility, any alphanumeric string may be used for a value
of the
completeness
option. Servers which receive a value of
completeness
that they do not recognize must treat it as though
minimal
had been specified.
The
extension
s are for future extensions to the protocol; any extensions
which are not understood by the server must be ignored by it. It is recommended
that experimental extensions use a URL, which dereferences to a description
of the extension, as the initial
token-or-quoted-string
Each
quotedURL
in a
service
specifies a rating service from
which the client is requesting a label for the document. There may be as
many repetitions of the
quotedURL
part of the
service
as desired,
so it is possible to request labels from any number of rating services in
a single HTTP request.
Detailed Syntax For HTTP Response Headers For Labels With Document
Two additional headers are specified:
protocol-header ::
'Protocol: {PICS-1.1 {headers PICS-Label}}'
label-header ::
'PICS-Label: '
labellist
Requesting Labels Separately
PICS labels can also be retrieved separately from the documents to which
they refer. To request labels in this way, a client contacts a
label
bureau
. A label bureau is an HTTP server that understands a particular
query syntax, defined below. It can provide labels for documents that reside
on other servers, and, indeed, for documents available through protocols
other than HTTP. It is anticipated that there will be "well-known" label
bureaus which dispense (possibly for a fee) labels created by many rating
services.
Rating services are also encouraged to act as label bureaus, providing on-line
access to their own labels. By default, the URL that identifies a rating
service also identifies its label bureau. If a client requests the URL that
identifies a rating service, a human-readable description of the service
is returned, as specified in
Rating
Services and Rating Systems
. If, on the other hand, a client requests
the same URL and includes query parameters as defined below, it should be
interpreted as a request for labels. A rating service, however, is not required
to act as a label bureau, and it may choose a different URL (perhaps even
on a different HTTP server) to act as its label bureau.
Sample Query
(For more complex queries and responses, see
Appendix
B.
Imagine a rating service, identified by the URL http://www.labels.org/Ratings,
which decides to run a label bureau to dispense (at least) its own labels
for documents. The following sample request, made to the HTTP server
www.labels.org, is illustrative (line breaks are inserted for presentation
purposes only):
GET /Ratings?opt=generic&
u="http%3A%2F%2Fwww.questionable.org%2Fimages"&
s="http%3A%2F%2Fwww.gcf.org%2Fv2.5"
HTTP/1.0
The query asks the label bureau http://www.labels.org/Ratings to send a single
label that applies to everything in the images hierarchy at site
www.questionable.org. The desired label should have been created by the service
for "/." This is required for encoding characters within a URL. See
RFC-1738
The label bureau responds by sending back a document of type
"application/pics-labels." The labels should be as complete as possible,
either by including as many options as possible or by supplying the
complete-label
(or
full
) option.
Detailed Syntax of HTTP Query for Labels Separate From Documents
The following grammar, in modified BNF, describes the syntax of GET and POST
requests to a label bureau. The use of the POST request is specified only
for backward compatibility with HTTP servers that cannot handle a long GET
query. Its use, while described in the
HTML 2.0
specification
(for use in submitting forms, see section 8.2.1 and 8.2.3), is deprecated.
request ::
get
post
get ::
'get'
url-fragment
'?' [
opt
] [
format
extension
url
service
post ::
'post'
url-fragment crlf crlf formencodeddata
url-fragment ::
the part of the original URL after the host
name, as specified in HTTP 1.0.
crlf ::
carriage return (hex D) followed by line feed (hex A)
opt ::
'opt='
option
option ::
'generic' | 'normal' | 'tree' | 'generic+tree'
format ::
[and] 'format='
form
form ::
'minimal' | 'short' | 'full' | 'signed'
extension ::
token
'='
token-or-quoted-string
where the
token
is not one of
opt
format
, or
; and
token-or-quoted-string
follows
the quoting conventions specified in
RFC-1738
token-or-quoted-string ::
token
quotedname
token ::
alphanumpm
url ::
[and] 'u=' encodedURL
service ::
[and] 's=' encodedURL
boolean ::
't' | 'f' | 'true' | 'false'
and ::
'&' this must be included unless it immediately
follows the ? in the query.
encodedURL ::
a URL, with quotation as required for inclusion
within another URL. According to
RFC-1738
, quotation is done
using "%xx" notation. Alphabetic characters, digits,
and the special characters $_-.+!*'(), need not be quoted,
but other characters must be. This
does
imply that the
colon (:) must be encoded as %3A and slash (/) as %2F.
formencodeddata ::
The query as specified for
get
but encoded into
MIME type application/x-www-form-encoded as described in
sections 8.2.1 and 8.2.3 of
HTML 2.0
Response to Query for Labels Separate From Documents
The label bureau responds by sending back a document of type
"application/pics-labels."
Unless the document indicates an overall error, there should be one
service-info
for each rating service requested in the query. Each
service-info
should have an error message or a label (or list of labels,
in the case of a "tree" query) for each requested URL.
The query's ordering must be preserved in the response. That is, the information
from the rating services must be presented in the same order the rating services
appear in the query, and the labels from each service must be presented in
the same order the URLs appear in the query. If a rating service or label
is not provided, the error message should appear in the same position that
the
service-info
or label would appear. Because order is preserved,
it is acceptable (except where indicated below) to omit from the labels the
for
" option which indicates the URL being rated. The client should
match the label positionally with the URL for which it requested a rating.
Definitions.
Given a URL (e.g., "http://www.greatdocs.com/foo/"),
descendant
URL is any URL that contains the original as a prefix
(e.g., "http://www.greatdocs.com/foo/bar/bat.htm"). A
child
URL is
any descendant URL that does not contain any additional '/' characters (e.g.,
"http://www.greatdocs.com/foo/ba"). An
ancestor
URL is any URL that
is a prefix of the original (e.g., "http://www.greatdocs.com/f"). Note that
ancestry and descendence is determined strictly by case-sensitive string
matching on URLs, not by any links that may appear in html documents retrieved
using those URLs. Note that any quotation such as %3A for colon (:) or %2F
for slash (/) is unencoded prior to comparing URL strings.
opt=normal
, or omitting the
opt
completely, requests specific
labels for the URLs specified. If no specific label is available for a requested
URL, the server may choose to send a generic label for the requested URL
or for an ancestor URL. For example, in response to a label request for URL
"http://w3.org/PICS/Overview.html" a generic label for the URL
"http://w3.org/PICS" (or even "http://w3.org") may be returned. In this case,
it is required that the "
for
" and "
generic
" options be included
in the label, to specify exactly what rating is being returned. Note that
the "for" option may specify a URL string which does not appear to match
the request URL, perhaps due to the server knowing about the existence of
an alternative URL for the same document. In that case, the server is suggesting
that the label applies to the request URL, though a suspicious client may
choose not to believe the suggestion.
opt=generic
requests generic labels. It is useful for requesting a
rating of a site or subpart of a site. For each requested URL, the desired
response is a generic label that applies to the requested URL and all descendant
URLs. A generic label for the requested URL, or a generic label for any ancestor
URL, would satisfy this request, as such a generic label would apply to all
URLs containing the requested URL as a prefix. If no such generic label is
available, the server should include the "no-label" message rather than sending
back a specific label.
opt=tree
requests a tree of labels. This is a way to request all the
labels that apply to items in a site or subpart of a site. For each requested
URL, the desired response is a set of labels (both specific and generic)
that apply to descendants of the requested URL. In the response, everywhere
label
would normally be expected in the response, a set of
simple-label
s will be returned, surrounded by parentheses. This enables
the client to match the entire set positionally with the single request URL.
All labels produced in response to this query must include a
for
option.
The minimum response expected is the set of labels that would have been generated
if a query had been issued, with
opt=normal
specified, for each known
child of the requested URL. Additional labels may also be returned, typically
either generic labels for ancestor URLs or labels for descendant URLs farther
down the hierarchy than children.
opt=generic+tree
is similar to the
opt=tree
request, but returns
only generic labels. As with
opt=tree
, the server can choose the amount
of detail. The minimum response expected is the set of labels that would
have been generated if a query had been issued, with
opt=generic
specified, for each known child of the requested URL. Additional labels may
also be returned, typically generic labels for ancestor URLs. All labels
produced in response to this query must include a
for
option.
It is permitted to include more than one URL and/or service in the request.
Requesting
URLs and
services results in a total of
labels being generated (or label sets in the case of tree and
generic+tree queries.)
The
format=
specifies the optional information that should be transmitted
with the labels. It is treated precisely as the similar keywords would be
when sent to a document server as the
completeness
(see
Detailed Syntax of HTTP Requests for Labels With Document
),
except that the default is
full
(rather than
minimal
). Servers
which receive a value of
completeness
that they do not recognize must
treat it as though the default,
full
, had been specified. All labels
produced in response to this query must include a
for
option.
MICs and Digital Signatures
This specification includes two independent security features, each intended
to prevent a different problem that can arise in a PICS system. They may
be used independently or together. Both features rely on patented cryptographic
technology whose use is subject to a variety of legal restrictions (including
possible U.S. export controls). The PICS technical committee cannot provide
any information about the exact legal status of the code or algorithms.
Within the United States, RSA Laboratories (100 Marine Parkway, Redwood City,
CA, 94065-1031) distributes a source code kit called
RSAREF
which provides
all of the code required to implement the cryptographic components of the
PICS spec. The president of RSA Data Security, Inc., Mr. Jim Bidzos, has
advised us that RSAREF will be made available at no cost for use in implementing
the PICS specifications. Questions about the legal status, etc., should be
directed to Mr. Bidzos.
The first problem arises when a document has been examined and a label generated,
and then the document is modified without updating the label. While this
can happen legitimately (as when Time-Warner updates the page containing
the current issue of Time Magazine and believes that the label is still valid)
it can also happen as a result of tampering with the document by an unauthorized
party. PICS labels contain three option fields intended to help deter this
kind of problem:
At
If the objective is to simply detect accidental changes, then the date of
last modification of the document can be calculated when the label is created
and stored in the
at
field. Assuming that the last modification time
is accurately maintained, this will detect updates to the document made after
the label was created.
Until
or
exp
If the document is expected to be updated infrequently or periodically, the
label can contain an expiration date that should cause the label to be invalid
before the document is next updated. This, too, does not guard against a
concerted malicious attack.
MIC-md5
or
md5
If the label is intended to apply only to the data that was actually rated,
then a form of checksum (called a "message digest") can be applied to the
data when the label is created. The message digest is converted into US-ASCII
characters using
MIME
base-64 encoding and stored in the
MIC-md5
(also called
md5
field. When the document is later retrieved, the same algorithm can be used
to recompute the message digest and the two digests can be compared. The
MD5 algorithm is designed so that it is extremely unlikely that the two digests
will be the same if the document has been tampered with in any way.
This technique is well-known in the cryptographic community and has been
adopted by the electronic mail community, where it is part of the
MOSS
specification. For
use with electronic mail, an elaborate technique is required to assure that
the two message digests will match, since electronic mail gateways can modify
the data before it is delivered (by wrapping lines, for example). We have
chosen
not
to adopt MOSS directly for PICS, largely because of this
complexity.
Instead, we recommend the direct use of the MD5 algorithm on the source document
and conversion of the result to base64 encoding. This resulting string is
included directly in the
mic-md5
md5
) label option. The MD5
algorithm and the conversion of the result into US-ASCII characters is provided
by the RSAREF (version 2.0) software.
Because PICS labels can be embedded inside of the documents they label, care
must be taken to ensure that the message digest is computed excluding
all
PICS labels in the document. For HTML documents, this means
that the digest must be computed after removing all META elements that include
PICS labels (and any whitespace immediately following the end of each of
these meta elements).
The second problem is that of tampering with or forging labels. Here the
problem is that the end user needs some way of being reassured that the label
they receive was created by the rating service they expected and that it
has not been altered since it was created. PICS addresses this problem by
allowing labels to be "digitally signed". A digital signature, while not
currently legally recognized, is a cryptographic technique to provide exactly
this assurance. The RSA signature technique works as follows:
In order to sign a label, the rating service (or people authorized to generate
labels on behalf of the service) needs a "public key pair." (The RSAREF software
includes routines to create these pairs.) One of these (the private key)
must be kept secret by the service; the other (the public key) must be
distributed to anyone who is interested in verifying the signatures on the
service's labels.
After creating a label, the service converts it to a special form specified
below and computes the MD5 message digest of the label. It then uses the
service's private key to encrypt the digest. This encrypted digest is the
digital signature, and it is converted to US-ASCII using the same base64
encoding technique mentioned above. The US-ASCII version (split into 60 character
lines) is stored in the
signature-rsa-md5
option of the label when
it is transmitted to the client. (The RSAREF software includes routines to
generate the signature and convert it to US-ASCII.)
When the client receives a label and wants to verify the signature it takes
the label it received and converts it back into the same special form in
which it was originally signed. The client recomputes the message digest
on this special form. It also takes the contents of the
signature-rsa-md5
option, combines all of the lines back into a single
string of US-ASCII characters, converts these from base64 into their original
(binary) form, and decrypts them using the service's public key. If the result
isn't the same as the message digest it computed the signature is invalid.
(RSAREF contains routines to do all of this work except for the combining
of the lines into a long string.)
The problem of distributing these keys (and invalidating them in case the
service's key is compromised) is an active area of commercial competition.
Since there is no clearly established solution available today, PICS assumes
that each service will distribute the public keys in some way it chooses.
It also assumes that no keys will ever have to be invalidated. While this
is clearly not a perfect solution, it seems to be the limit of what can be
done today without committing to specific proprietary technology.
There is one additional problem with the digital signature solution outlined
above. If a rating service allows other people to generate labels under its
name (for example, a service that supports self-ratings by content producers)
then the labels may need to be signed by
both
the service and the
content producer. This can be done (each signs the label without the other's
signature), but it becomes quite difficult to distribute the public keys
needed to verify the signature. The PICS specification does not propose a
solution to this problem (it, too, is part of active commercial competition).
Signature Details
PICS specifically requires the use of the RSA signature algorithm with the
MD5 message digest. Should this system become outdated, the PICS specification
can be easily updated to add a new label option that supports a different
pair of algorithms.
PICS does not specify the key length to be used for the digital signatures.
Individual services will need to investigate the legal and technical
ramifications involved and make a choice. Should a single answer become common,
this specification may be re-issued with this detail filled in.
The special form of the label that is used for signatures is computed as
follows:
The service must decide which options it will include in the signed label
when it is transmitted. Any options not transmitted with the signature cannot
be used in the computation of the signature. We recommend that
all
options with known values be included with the exception of
signature-rsa-md5
. Any option may be omitted, but it will be common
for the options
mic-md5
(or
md5
) and
full
(or
complete-label
) to be omitted. The
signature-rsa-md5
option
is
never
included in the list of options.
The selected options are sorted alphabetically by their shortest name (i.e.
use
full
instead of
complete-label
). If a selected option has
a default value and it is the same as the value to be used in the label,
the option is omitted from this list.
For each option in the list (in order), the short name is put into the label
followed by a single space followed by the value of the option, followed
by a space. The shortest form of a value is used, and strings are output
in lower case if they are case insensitive.
After all of the options has been output, output the characters "r (".
Output the transmission names and their values, in alphabetical order by
transmission name (using the US-ASCII character collating sequence for
"alphabetical order"), separating the transmission name from the value by
a single space. In outputting the value, no whitespace is permitted except
for a single space used to separate items in a
multi-value
Output a ")"
When the client computes the special label format described above, it will
use all options available to it: both those in the
single-label
and in the
service-info
. This implies a constraint on the server
when it decides what options to include in the transmitted set. The transmitted
set must include any options that the server ships as part of the
service-info
, unless either the value specified in the
service-info
or the value of the option for this label is the default
value of the option.
Glossary
application/pics-labels
A new MIME data type used to transmit one or more
labels
, defined
in this document.
application/pics-service
A new MIME data type used to describe a
rating service
, defined in
Rating Services and Rating
Systems
BNF
Backus-Naur Form (or Backus Normal Form). A notation for describing a formal
syntax, used extensively in describing programming languages and
computer-readable data formats.
category
The part of a rating system which describes a particular criterion used for
rating. For example, a rating system might have three categories named "sexual
material," "violence," and "vocabulary." Also called a
dimension
content label
A data structure containing information about a given document's contents.
Also called a
rating
or
content rating
. The content label may
accompany the document it is about or be available separately.
content rating
See
content label
dimension
See
category
document
Any item that can be referred to by a URL. Also known, in other contexts,
as a "hypertext page" or a "resource."
HTML
HyperText Markup Language. A means of representing
hypertext
documents.
Based on
SGML
. See the
HTML 2.0 Proposed
Standard
HTTP
HyperText Transfer Protocol. Used for retrieving document contents and/or
descriptive header information. See the
draft
HTTP specification
hypertext
Text, graphics, and other media connected through links.
label
See
content label
label bureau
A computer system which supplies, via a computer network, ratings of documents.
It may or may not provide the documents themselves.
MD5
An algorithm, see
RFC1321
, that can be
used to compute a
MIC.
PICS specifies this particular algorithm for
use in PICS labels.
MIC
Message Integrity Check. Also known as a "cryptographic checksum." For PICS,
the importance of a MIC is that a rating service can compute the MIC of a
piece of information when the label is created and that MIC can be put into
the label itself. A client can retrieve the label and the information to
which it is supposed to be attached, recompute the MIC and compare it to
the one in the label. If they match, for all practical purposes, it is a
proof that the label really belongs to the information that has been retrieved.
The particular algorithm specified by PICS to compute the MIC is
MD5.
MIME
Multimedia Internet Message Extension. A technique for sending arbitrary
data through electronic mail on the Internet. See
RFC-1521
PICS
Platform for Internet Content Selection, the name for both the suite of
specification documents of which this is a part, and for the organization
writing the documents. For more information, see http://w3.org/PICS
rating
See
content label
rating server
See
label bureau
rating service
An individual or organization that assigns labels according to some rating
system, and then distributes them, perhaps via a label bureau or via CD-ROM.
rating system
A method for rating information. A rating system consists of one or more
categories
scale
The range of permissible values for a category.
SGML
Standard Generalized Markup Language. See
ISO 8879
transmission name
(of a
category
) The short name intended for use over a network to
refer to the category. This is distinct from the category name in as much
as the transmission name must be language-independent, encoded in US-ASCII,
and as short as reasonably possible. Within a single
rating system
the transmission names of all categories must be distinct. URLs, while generally
longer than desired, can be used as transmission names. Hence transmission
names are case sensitive.
URL
Uniform Resource Locator. Described in
RFC-1738
. A URL describes
the location and means of retrieval for a single document. It consists of
three components: the "scheme" (protocol used to retrieve a document, like
"http" or "ftp"), a host name, and a hierarchical document name within that
host. For example "http://w3.org/PICS" is the URL of the PICS home page.
The scheme for retrieving it is "http," the host is "w3.org" and the name
within that host is "PICS". Notice that PICS defines an additional scheme
beyond those listed in RFC-1738, described in
Rating Services and Rating
Systems
, which allows Chat (IRC) rooms to be named.
References
PICS,
Rating Services and Rating
Systems
, Internet Draft, "draft-pics-services-00.txt", 11/21/95.
R. Rivest, "The MD5 Message-Digest Algorithm",
RFC 1321
, 04/16/1992.
N. Borenstein, N. Freed, "MIME (Multipurpose Internet Mail Extensions) Part
One: Mechanisms for Specifying and Describing the Format of Internet Message
Bodies",
RFC 1521
09/23/1993.
T. Berners-Lee, D. Connolly, "Hypertext Markup Language - 2.0",
RFC 1866
, 11/03/1995.
T. Berners-Lee, L. Masinter, M. McCahill, "Uniform Resource Locators (URLs)",
RFC 1738
, 12/20/94.
Acknowledgments
Comments and suggestions from the following people are gratefully acknowledged:
Bob Atkinson, Microsoft
Anselm Baird-Smith, W3C
Brenda Baker, Lucent
Scott Berkun, Microsoft
Tim Berners-Lee, W3C
Roxana Bradescu, AT&T
Daniel W. Connolly, W3C
Roy Fielding, W3C
Jay Friedland, SurfWatch
Henrik Frystyk Nielsen, W3C
Philip Gladstone, Raptor Systems
Michael Gordon, Prodigy
Wayne Gramlich, Sun
Woodson Hobbs, NewView
David Karger, MIT
Rohit Khare, W3C
Charlie Kim, Apple
John C. Klensin, MCI
Breen Liblong, IFSI
Ann McCurdy, Microsoft
Rich Petke, CompuServe
Eric Prud'hommeaux, W3C
Dave Raggett, W3C
Gordon Ross, NetNanny
Bob Schloss, IBM
David Singer, IBM
Ray Soular, SafeSurf
Michael Smith, Prodigy
Marcy Swenson, Providence Systems
Jason Thomas, MIT
Appendix A: An Algorithm for Locating a Label Bureau
As the use of PICS grows, we must consider its impact on overall network
performance. In general, the PICS techniques for transmitting labels in or
with documents add only a very small amount of traffic to the net, since
the additional PICS headers will ordinarily contain only a few hundred bytes
of data and the documents themselves are more likely to be several thousand
bytes of data. Furthermore, since the labels come from the same source as
the document itself there is no network hot spot created by PICS (although
popular servers may themselves already be such hot spots).
Label bureaus, however, are a new component proposed by PICS. And if a single
label bureau becomes popular then there is a significant risk of it becoming
a hot spot and hence a performance bottleneck for the PICS system. The Internet
is in need of a good solution to this problem, and there is work (both underway
and proposed) that may solve the problem in the long term.
In the short term, however, there is no truly good solution. The following
suggestion comes from Prof. David Karger at MIT. It is a variant on several
well-known algorithms for distributing load in a system.
First, we assume that popular label bureaus will be able to establish a number
of mirror sites around the network. This is already common practice, and
we have no suggestions for the details of determining the sites or keeping
them updated as new labels are generated. Our algorithm simply assumes that
they exist and are equivalent, and that the network's Domain Name System
(DNS) has records which map the single well-defined name for the label bureau
to multiple Internet addresses, in the usual manner.
When client software starts, it should attempt to resolve the name of the
label bureau it wishes to use (we assume one label bureau, but the algorithm
extends in an obvious manner to multiple bureaus) through DNS. If it receives
more than one host address, it saves the entire list and chooses two at random,
labeling one the "primary" and the other the "secondary" bureau. Alternatively,
these may be configuration parameters of the client software that are then
validated when the software starts. It also divides 60 minutes by the total
number of address it can find for the label bureau, sets a timer to this
value, and remembers this as the "threshold" value.
Every time the client wishes to contact the label bureau it does the following.
If the timer is below the threshold, the primary bureau address is used.
Otherwise, the query is sent to both the primary and the secondary label
bureau address. When the first answer arrives the connection to both label
bureaus is closed down. The bureau which answered first becomes the primary
bureau. In any case, a new secondary bureau address is chosen at random and
the timer is reset to the threshold value.
A simple variant on this algorithm will probably become feasible in the near
future. When the HTTP protocol is updated to allow "keep alive" connections
to a server, the PICS client should keep its connection to the primary label
bureau alive as long as possible. Then, instead of simply accepting the first
response and considering the responder as the primary, a more careful measurement
must be made. The time required to send the query and receive the response
must be measured, rather than the total transaction time: connection setup
costs can be quite high, and would distort the measurement if one compared
the round trip time to the primary bureau through an existing connection
to the time to establish the connection to the secondary bureau plus the
round trip time.
Appendix B: Sample Label Bureau Queries and Responses
The following queries and responses illustrate many of the features of client
interactions with label bureaus that dispense labels separately from documents.
All four queries request labels for the same three documents, provided by
the same three services. They differ only in the query mode (Generic, Normal,
Tree, Generic+Tree).
Labels are requested for the following URLs:
Labels are requested from the following services:
The server has the following relevant labels:
Ages rating service
"http://www.w3.org/pub" (generic)
"http://www.w3.org/pub/WWW/" (generic)
"http://www.w3.org/pub/WWW/Daemon" (generic)
"http://www.w3.org/pub/WWW/PICS" (generic)
"http://www.w3.org/pub/WWW/Overview.html"
RSAC rating service
"http://www.w3.org/pub/WWW" (generic)
"http://www.w3.org/pub/WWW/Daemon" (generic)
"http://www.w3.org/pub/WWW/PICS" (generic)
"http://www.w3.org/pub/WWW/Daemon/Overview.html"
"http://www.w3.org/pub/WWW/TheProject.html"
unknown.com rating service
None.
The query responses have been pretty-printed for readability. Comment lines,
beginning with ';' have been added to explain the responses. Query requests
have been split onto multiple lines for display purposes; they are actually
sent as single (very long) lines.
Generic request
This request is for full generic labels that apply to the three documents.
Client sends request to server:
GET /ratings?opt=generic&format=full&
u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2F+&
u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2FTheProject.html&
u=http%3A%2F%2Fwww.w3.org%2Funknown&
s=http%3A%2F%2Fwww.ages.org%2Four-service%2Fv1.0%2F&
s=http%3A%2F%2Fwww.rsac.org%2Fv1.0&
s=http%3A%2F%2Funknown.com HTTP/1.0
Server responds to client:
HTTP/1.0 200 OK
Content-Length: 550
Content-Type: application/pics-labels
Server: Jigsaw 0/0
Date: 15 Apr 1996 18:20:47 GMT
(PICS-1.1
"http://www.ages.org/our-service/v1.0/"
;first service
labels
for "http://www.w3.org/pub/WWW/"
generic true
by "abaird@w3.org"
ratings (age 11)
;end of first label, since 'ratings' is always
;last part of a label. The same generic label
;applies also to any URL beginning
;http://www.w3.org/pub/WWW/TheProject.html
for "http://www.w3.org/pub/WWW/"
generic true
by "abaird@w3.org"
ratings (age 11)
;end of second label
error (not-labeled "http://www.w3.org/unknown")
;no label available for third document
;three labels requested, so end of first service
"http://www.rsac.org/v1.0"
labels
for "http://www.w3.org/pub/WWW"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
for "http://www.w3.org/pub/WWW"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
error (not-labeled "http://www.w3.org/unknown")
;;no labels for third service
error (no-ratings "unknown service"))
Normal query
This query requests full specific labels for each of the documents.
Client sends request to server:
GET /ratings?opt=normal&format=full&
u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2F+&
u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2FTheProject.html&
u=http%3A%2F%2Fwww.w3.org%2Funknown&
s=http%3A%2F%2Fwww.ages.org%2Four-service%2Fv1.0%2F&
s=http%3A%2F%2Fwww.rsac.org%2Fv1.0&
s=http%3A%2F%2Funknown.com HTTP/1.0
Server responds to client:
HTTP/1.0 200 OK
Content-Length: 569
Content-Type: application/pics-labels
Server: Jigsaw 0/0
Date: 15 Apr 1996 18:20:54 GMT
(PICS-1.1
"http://www.ages.org/our-service/v1.0/"
labels
;;no specific label available, so generic label returned
for "http://www.w3.org/pub/WWW/"
generic true
by "abaird@w3.org"
ratings (age 11)
;;no specific label available, so generic label returned
for "http://www.w3.org/pub/WWW/"
generic true
by "abaird@w3.org"
ratings (age 11)
error (not-labeled "http://www.w3.org/unknown")
"http://www.rsac.org/v1.0"
labels
;;no specific label available, so generic label returned
for "http://www.w3.org/pub/WWW"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
;;here a specific label is returned.
for "http://www.w3.org/pub/WWW/TheProject.html"
generic false
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
error (not-labeled "http://www.w3.org/unknown")
error (no-ratings "unknown service"))
Tree query
This request is for full specific labels for all URLs that have the requested
URLs as a prefix. This label bureau responds to tree queries by sending only
labels for documents in the current directory.
Client sends request to server:
GET /ratings?opt=tree&format=full&
u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2F+&
u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2FTheProject.html&
u=http%3A%2F%2Fwww.w3.org%2Funknown&
s=http%3A%2F%2Fwww.ages.org%2Four-service%2Fv1.0%2F&
s=http%3A%2F%2Fwww.rsac.org%2Fv1.0&
s=http%3A%2F%2Funknown.com HTTP/1.0
Server responds to client:
HTTP/1.0 200 OK
Content-Length: 1075
Content-Type: application/pics-labels
Server: Jigsaw 0/0
Date: 15 Apr 1996 18:21:00 GMT
(PICS-1.1
"http://www.ages.org/our-service/v1.0/"
labels
;;several labels delimited by ()
(for "http://www.w3.org/pub/WWW/"
generic true
by "abaird@w3.org"
ratings (age 11)
for "http://www.w3.org/pub/WWW/Overview.html"
by "abaird@w3.org"
generic false
ratings (age 12)
by "abaird@w3.org"
for "http://www.w3.org/pub/WWW/PICS"
generic true
ratings (age 5)
by "abaird@w3.org"
for "http://www.w3.org/pub/WWW/Daemon"
generic true
ratings (age 5))
;;end of labels for directory http://www.w3.org/pub/WWW/
;;no labels available for URLs containing
;;http://www.w3.org/pub/WWW/TheProject.html as a prefix
error (not-labeled "http://www.w3.org/pub/WWW/TheProject.html")
error (not-labeled "http://www.w3.org/unknown")
"http://www.rsac.org/v1.0"
labels
(for "http://www.w3.org/pub/WWW"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
for "http://www.w3.org/pub/WWW/TheProject.html"
generic false
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
for "http://www.w3.org/pub/WWW/Daemon"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
for "http://www.w3.org/pub/WWW/PICS"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0))
error (not-labeled "http://www.w3.org/pub/WWW/TheProject.html")
error (not-labeled "http://www.w3.org/unknown")
error (no-ratings "unknown service"))
generic+tree
This query requests all generic labels for URLs that contain the requested
URLs as prefixes. A subset of the labels returned for the previous query
are returned here: only those that are generic.
Client sends request to server:
GET /ratings?opt=generic%2Btree&
format=full&u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2F+&
u=http%3A%2F%2Fwww.w3.org%2Fpub%2FWWW%2FTheProject.html&
u=http%3A%2F%2Fwww.w3.org%2Funknown&
s=http%3A%2F%2Fwww.ages.org%2Four-service%2Fv1.0%2F&
s=http%3A%2F%2Fwww.rsac.org%2Fv1.0&
s=http%3A%2F%2Funknown.com HTTP/1.0
Server responds to client:
HTTP/1.0 200 OK
Content-Length: 872
Content-Type: application/pics-labels
Server: Jigsaw 0/0
Date: 15 Apr 1996 18:38:28 GMT
(PICS-1.1
"http://www.ages.org/our-service/v1.0/"
labels
(for "http://www.w3.org/pub/WWW/"
generic true
by "abaird@w3.org"
ratings (age 11)
by "abaird@w3.org"
for "http://www.w3.org/pub/WWW/PICS"
generic true
ratings (age 5)
by "abaird@w3.org"
for "http://www.w3.org/pub/WWW/Daemon"
generic true
ratings (age 5))
error (not-labeled "http://www.w3.org/pub/WWW/TheProject.html")
error (not-labeled "http://www.w3.org/unknown")
"http://www.rsac.org/v1.0"
labels
(for "http://www.w3.org/pub/WWW"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
for "http://www.w3.org/pub/WWW/Daemon"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0)
for "http://www.w3.org/pub/WWW/PICS"
generic true
by "abaird@w3.org"
ratings (v 0 s 0 n 0 l 0))
error (not-labeled "http://www.w3.org/pub/WWW/TheProject.html")
error (not-labeled "http://www.w3.org/unknown")
error (no-ratings "unknown service"))
Comments to
pics-ask@w3.org
Webmaster
Created 21 November 1995 by Jim Miller