draft-ietf-usefor-usepro-05.txt   draft-ietf-usefor-usepro-06.txt 
INTERNET-DRAFT Charles H. Lindsey INTERNET-DRAFT Charles H. Lindsey
Usenet Format Working Group University of Manchester Usenet Format Working Group University of Manchester
January 2006 November 2006
News Article Architecture and Protocols News Article Architecture and Protocols
<draft-ietf-usefor-usepro-05.txt> <draft-ietf-usefor-usepro-06.txt>
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
.QP Internet-Drafts are working documents of the Internet .QP Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working groups. Engineering Task Force (IETF), its areas, and its working groups.
Note that other groups may also distribute working documents as Note that other groups may also distribute working documents as
skipping to change at page 1, line 33 skipping to change at page 1, line 33
documents at any time. It is inappropriate to use Internet-Drafts documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in as reference material or to cite them other than as "work in
progress." progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.html. http://www.ietf.org/ietf/1id-abstracts.html.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire in July 2006. This Internet-Draft will expire in May 2007.
Abstract Abstract
This Draft, together with its companion draft [USEFOR], are This Draft, together with its companion draft [USEFOR], are
intended as standards track documents, together obsoleting RFC intended as standards track documents, together obsoleting RFC
1036, which itself dates from 1987. 1036, which itself dates from 1987.
This Standard defines the architecture of Netnews systems and This Standard defines the architecture of Netnews systems and
specifies the requirements to be met by software which originates, specifies the requirements to be met by software which originates,
distributes, stores and displays Netnews articles. distributes, stores and displays Netnews articles.
Backward compatibility has been a major goal of this endeavour, but Backward compatibility has been a major goal of this endeavour, but
where this standard and earlier documents or practices conflict, this where this standard and earlier documents or practices conflict, this
standard should be followed. In most such cases, current practice is standard should be followed. In most such cases, current practice is
already compatible with these changes. already compatible with these changes.
A companion Best Current Practice document [USEAGE], addressing A companion Best Current Practice document [USEAGE], addressing
requirements which are present for Social rather than Normative requirements which are present for Social rather than Normative
reasons is in preparation. reasons is in preparation.
[This is the latest in the line of USEPRO drafts. However, the USEFOR
Working Group is currently considering the possibility of a complete
rewrite of this document.]
News Article Architecture and Protocols November 2006
[The use of the words "this standard" within this document when [The use of the words "this standard" within this document when
referring to itself does not imply that this draft yet has pretensions referring to itself does not imply that this draft yet has pretensions
to be a standard, but rather indicates what will become the case if and to be a standard, but rather indicates what will become the case if and
News Article Architecture and Protocols January 2006
when it is accepted as an RFC with the status of a proposed or draft when it is accepted as an RFC with the status of a proposed or draft
standard.] standard.]
[Remarks enclosed in square brackets and aligned with the left margin, [Remarks enclosed in square brackets and aligned with the left margin,
such as this one, are not part of this draft, but are editorial notes to such as this one, are not part of this draft, but are editorial notes to
explain matters amongst ourselves, or to point out alternatives, or to explain matters amongst ourselves, or to point out alternatives, or to
assist the RFC Editor.] assist the RFC Editor.]
[In this draft, references to [NNTP] are to be replaced by references to
the RFC arising from the series of drafts draft-ietf-nntpext-base-*.txt,
which has now passed its IETF last call.]
Table of Contents Table of Contents
1. Introduction .................................................. 4 1. Introduction .................................................. 4
1.1. Basic Concepts ............................................ 4 1.1. Basic Concepts ............................................ 4
1.2. Objectives ................................................ 4 1.2. Objectives ................................................ 4
1.3. Historical Outline ........................................ 5 1.3. Historical Outline ........................................ 5
2. Definitions, Notations and Conventions ........................ 5 2. Definitions, Notations and Conventions ........................ 5
2.1. Definitions ............................................... 5 2.1. Definitions ............................................... 5
2.2. Defining the Architecture ................................. 6 2.2. Defining the Architecture ................................. 5
2.3. Identification of news servers ............................ 7 2.3. Identification of news servers ............................ 6
2.4. Variant Header Fields ..................................... 8 2.4. Variant Header Fields ..................................... 8
2.5. Textual Notations ......................................... 8 2.5. Textual Notations ......................................... 8
3. Changes to the existing protocols ............................. 9 3. Transport ..................................................... 9
3.1. Protocol Changes .......................................... 9 4. Definition of new Media Types ................................. 10
3.2. Transitional Arrangements ................................. 10 4.1. Application/news-transmission ............................. 10
4. Transport ..................................................... 11 4.2. Message/news obsoleted .................................... 11
5. Definition of new Media Types ................................. 12 4.3. Application/news-groupinfo ................................ 11
5.1. Application/news-transmission ............................. 12 4.4. Application/news-checkgroups .............................. 12
5.2. Message/news obsoleted .................................... 13 5. Control Messages .............................................. 13
5.3. Application/news-groupinfo ................................ 13 5.1. Digital Signature of Header Fields ........................ 14
5.4. Application/news-checkgroups .............................. 14 5.2. Group Control Messages .................................... 14
6. Control Messages .............................................. 15 5.2.1. The 'newgroup' Control Message ........................ 15
6.1. Digital Signature of Header Fields ........................ 16 5.2.1.1. The Body of the 'newgroup' Control Message ........ 15
6.2. Group Control Messages .................................... 16 5.2.1.2. Initial Articles .................................. 15
6.2.1. The 'newgroup' Control Message ........................ 16 5.2.1.3. Example ........................................... 16
6.2.1.1. The Body of the 'newgroup' Control Message ........ 17 5.2.2. The 'rmgroup' Control Message ......................... 17
6.2.1.2. Initial Articles .................................. 17 5.2.2.1. Example ........................................... 17
6.2.1.3. Example ........................................... 18 5.2.3. The 'mvgroup' Control Message ......................... 17
6.2.2. The 'rmgroup' Control Message ......................... 19 5.2.3.1. Example ........................................... 19
6.2.2.1. Example ........................................... 19 5.2.4. The 'checkgroups' Control Message ..................... 20
6.2.3. The 'mvgroup' Control Message ......................... 19 5.3. Cancel .................................................... 21
6.2.3.1. Example ........................................... 21 5.4. Ihave, sendme ............................................. 22
6.2.4. The 'checkgroups' Control Message ..................... 21 5.5. Obsolete control messages. ............................... 23
6.3. Cancel .................................................... 23 6. Duties of Various Agents ...................................... 23
6.4. Ihave, sendme ............................................. 23 6.1. General principles to be followed ......................... 24
6.5. Obsolete control messages. ............................... 25 6.2. Duties of an Injecting Agent .............................. 24
7. Duties of Various Agents ...................................... 25 6.2.1. Proto-articles ........................................ 25
7.1. General principles to be followed ......................... 26 6.2.2. Procedure to be followed by Injecting Agents .......... 25
7.2. Duties of an Injecting Agent .............................. 26 6.2.3. Procedure for Forwarding to a Moderator ............... 28
7.2.1. Proto-articles ........................................ 27 6.3. Duties of a Relaying Agent ................................ 28
7.2.2. Procedure to be followed by Injecting Agents .......... 27 6.3.1. Path Header Field Example ............................. 32
News Article Architecture and Protocols January 2006 6.4. Duties of a Serving Agent ................................. 33
News Article Architecture and Protocols November 2006
7.2.3. Procedure for Forwarding to a Moderator ............... 29 6.5. Duties of a Posting Agent ................................. 35
7.3. Duties of a Relaying Agent ................................ 30 6.6. Duties of a Followup Agent ................................ 35
7.3.1. Path Header Field Example ............................. 33 6.6.1. Construction of the References header field ........... 36
7.4. Duties of a Serving Agent ................................. 34 6.7. Duties of a Reading Agent ................................. 37
7.5. Duties of a Posting Agent ................................. 35 6.8. Duties of a Moderator ..................................... 37
7.6. Duties of a Followup Agent ................................ 36 6.9. Duties of a Gateway ....................................... 39
7.6.1. Construction of the References header field ........... 36 6.9.1. Duties of an Outgoing Gateway ......................... 40
7.7. Duties of a Reading Agent ................................. 37 6.9.2. Duties of an Incoming Gateway ......................... 40
7.8. Duties of a Moderator ..................................... 37 6.9.3. Example ............................................... 42
7.9. Duties of a Gateway ....................................... 39 7. Security and Related Considerations ........................... 43
7.9.1. Duties of an Outgoing Gateway ......................... 40 7.1. Leakage ................................................... 43
7.9.2. Duties of an Incoming Gateway ......................... 41 7.2. Attacks ................................................... 44
7.9.3. Example ............................................... 43 7.2.1. Denial of Service ..................................... 44
8. Security and Related Considerations ........................... 44 7.2.2. Compromise of System Integrity ........................ 45
8.1. Leakage ................................................... 44 7.3. Liability ................................................. 46
8.2. Attacks ................................................... 44 8. IANA Considerations ........................................... 47
8.2.1. Denial of Service ..................................... 44 9. References .................................................... 47
8.2.2. Compromise of System Integrity ........................ 46 9.1. Normative References ...................................... 47
8.3. Liability ................................................. 47 9.2. Informative References .................................... 48
9. IANA Considerations ........................................... 47 10. Acknowledgements ............................................. 48
10. References ................................................... 47 11. Contact Address .............................................. 49
10.1. Normative References ..................................... 47
10.2. Informative References ................................... 48
11. Acknowledgements ............................................. 49
12. Contact Address .............................................. 49
Appendix A - Obsolete Control Messages ............................ 49 Appendix A - Obsolete Control Messages ............................ 49
Appendix B - Notices .............................................. 50 Appendix B - Differences from the Protocols in RFC 1036 and its
Appendix C - Change Log ........................................... 51 derivatives ....................................................... 50
News Article Architecture and Protocols January 2006 Appendix C - Transitional Arrangements ............................ 50
Appendix D - Notices .............................................. 51
Appendix E - Change Log ........................................... 52
News Article Architecture and Protocols November 2006
1. Introduction 1. Introduction
1.1. Basic Concepts 1.1. Basic Concepts
"Netnews" is a set of protocols for generating, storing and "Netnews" is a set of protocols for generating, storing and
retrieving news "articles" (which resemble email messages) and for retrieving news "articles", as introduced in F-1.1.
exchanging them amongst a readership which is potentially widely
distributed. It is organized around "newsgroups", with the
expectation that each reader will be able to see all articles posted
to each newsgroup in which he participates. These protocols most
commonly use a flooding algorithm which propagates copies throughout
a network of participating servers. Typically, only one copy is
stored per server, and each server makes it available on demand to
readers able to access that server.
"Usenet" is a particular worldwide publicly accessible network based "Usenet" is a particular worldwide publicly accessible network based
upon the Netnews protocols, with the newsgroups being organized into upon the Netnews protocols, with the newsgroups being organized into
recognized "hierarchies". Anybody can join (it is simply necessary recognized "hierarchies". Anybody can join (it is simply necessary
to negotiate an exchange of articles with one or more other to negotiate an exchange of articles with one or more other
participating hosts). participating hosts).
An important characteristic of Usenet is the lack of any requirement An important characteristic of Usenet is the lack of any requirement
for a central administration or for the establishment of any for a central administration or for the establishment of any
controlling host to manage the network. Nevertheless, administrative controlling host to manage the network. Nevertheless, administrative
skipping to change at page 4, line 41 skipping to change at page 4, line 33
"policies" applicable to particular parts of Usenet. "policies" applicable to particular parts of Usenet.
A "policy" is a rule intended to facilitate the smooth operation of a A "policy" is a rule intended to facilitate the smooth operation of a
network by establishing parameters which restrict behaviour that, network by establishing parameters which restrict behaviour that,
whilst technically unexceptionable, would nevertheless contravene whilst technically unexceptionable, would nevertheless contravene
some accepted standard of "Good Netkeeping". Since the ultimate some accepted standard of "Good Netkeeping". Since the ultimate
beneficiaries of a network are its human readers, who will be less beneficiaries of a network are its human readers, who will be less
tolerant of poorly designed interfaces than mere computers, articles tolerant of poorly designed interfaces than mere computers, articles
in breach of established policy can cause considerable annoyance to in breach of established policy can cause considerable annoyance to
their recipients. their recipients.
[Could omit that last sentence.] [Could omit that last sentence, and perhaps even the whole paragraph?]
1.2. Objectives 1.2. Objectives
The purpose of this present standard is to define the overall The purpose of this present standard is to define the overall
architecture and the protocols to be used for Netnews in general, and architecture and the protocols to be used for Netnews in general, and
for Usenet in particular, and to set standards to be followed by for Usenet in particular, and to set standards to be followed by
software that implements those protocols. A companion standard software that implements those protocols. The companion standard
[USEFOR] sets out the canonical format of news articles exchanged [USEFOR] sets out the canonical format of news articles exchanged
between the various agents comprising that architecture. In this between the various agents comprising that architecture. In this
standard, references to individual sections in the companion [USEFOR] standard, references to individual sections in the companion [USEFOR]
are prefixed with "F-". are prefixed with "F-".
A set of hosts within a network which, by mutual arrangement, A set of hosts within a network which, by mutual arrangement,
operates some variant (whether more or less restrictive) of the operates some variant (whether more or less restrictive) of the
Netnews protocols is a "cooperating subnet". Netnews protocols is a "cooperating subnet".
[It is not clear whether we still need that definition.]
News Article Architecture and Protocols January 2006
It is NOT the purpose of this standard to settle matters of policy, It is NOT the purpose of this standard to settle matters of policy,
nor aspects of software behaviour which do not impinge upon the nor aspects of software behaviour which do not impinge upon the
generation, transmission, storage and reception of articles, nor how generation, transmission, storage and reception of articles, nor how
the authority of various agencies to create such policies and to the authority of various agencies to create such policies and to
exercise control or oversight of the various parts of Usenet is exercise control or oversight of the various parts of Usenet is
established. For these purposes, a separate Best Current Practice established. For these purposes, a separate Best Current Practice
document [USEAGE] is being provided. document [USEAGE] is being provided.
News Article Architecture and Protocols November 2006
Nevertheless, it is assumed that such agencies with the necessary Nevertheless, it is assumed that such agencies with the necessary
authority will exist, and tools are provided within the protocols for authority will exist, and tools are provided within the protocols for
their use. their use.
1.3. Historical Outline 1.3. Historical Outline
Network news originated as the medium of communication for Usenet, Network news originated as the medium of communication for Usenet,
circa 1980. Since then, Usenet has grown explosively, and many circa 1980. Since then, Usenet has grown explosively, and many
Internet and non-Internet sites participate in it. In addition, the Internet and non-Internet sites participate in it. In addition, the
news technology is now in widespread use for other purposes, on the news technology is now in widespread use for other purposes, on the
skipping to change at page 5, line 36 skipping to change at page 5, line 29
For an account of the earlier formats used in Netnews prior to [RFC For an account of the earlier formats used in Netnews prior to [RFC
1036], see Henry Spencer's 1994 draft, popularly referred to as "Son 1036], see Henry Spencer's 1994 draft, popularly referred to as "Son
of 1036" [Son-of-1036], which has recently been republished as an of 1036" [Son-of-1036], which has recently been republished as an
Informational RFC. Informational RFC.
[That is a tentative statement, which may need revision.] [That is a tentative statement, which may need revision.]
Although never adopted as a formal standard, [Son-of-1036] had a Although never adopted as a formal standard, [Son-of-1036] had a
considerable effect on the development of Netnews and hence on these considerable effect on the development of Netnews and hence on these
present standards, and it is hoped that we have followed its spirit present standards, and it is hoped that we have followed its spirit
and intentions. and intentions.
.nr H1 1
2. Definitions, Notations and Conventions 2. Definitions, Notations and Conventions
2.1. Definitions 2.1. Definitions
All the technical terms defined in F-1.5 are to be considered as All the technical terms defined in F-1.5 are to be considered as
defined also, with the same meaning, in this standard. In addition, defined also, with the same meaning, in this standard. In addition,
some further terms are defined here, and in the following section. some further terms are defined here, and in the following section.
A "hierarchy" is the set of all newsgroups whose names share a first A "hierarchy" is the set of all newsgroups whose names share a first
<component> (as defined in F-3.1.5). The term "sub-hierarchy" is <component> (as defined in F-3.1.4). The term "sub-hierarchy" is
also used where several initial components are shared. also used where several initial components are shared.
The "semantic content" (often abbreviated to just "content" when the The "semantic content" (often abbreviated to just "content" when the
context is clear) of a header field is its semantic interpretation; context is clear) of a header field is its semantic interpretation;
i.e. what remains after unfolding it and removing its field name with i.e. what remains after unfolding it and removing its field name with
its colon and any leading and trailing whitespace and, in the case of its colon and any leading and trailing whitespace and, in the case of
structured header fields only, ignoring comments and other structured header fields only, ignoring comments and other
semantically invisible items and replacing white space by a single semantically invisible items and replacing white space by a single
SP. SP. See 6.6.1 for the use of this term.
News Article Architecture and Protocols January 2006
2.2. Defining the Architecture 2.2. Defining the Architecture
A Netnews system is a distributed database composed of agents of A Netnews system is a distributed database composed of agents of
various types which, acting together according to the protocols various types which, acting together according to the protocols
defined in section 7 of this standard, causes articles to be defined in section 6 of this standard, causes articles to be
propagated throughout the system and to be made available to its propagated throughout the system and to be made available to its
readers. The protocols ensure that all copies of a given article, readers. The protocols ensure that all copies of a given article,
wherever stored, are identical apart from those header fields defined wherever stored, are identical apart from those header fields defined
as variant (2.4). For explaining the working of the protocols, it is as variant (2.4). For explaining the working of the protocols, it is
convenient to define particular sub-categories of agent as follows: convenient to define particular sub-categories of agent as follows:
News Article Architecture and Protocols November 2006
A "posting agent" is the software that assists posters to prepare A "posting agent" is the software that assists posters to prepare
proto-articles in compliance with [USEFOR]. The proto-article is proto-articles in compliance with [USEFOR]. The proto-article is
then passed on to an "injecting agent" for final checking and then passed on to an "injecting agent" for final checking and
injection into the news stream. If the article is not compliant, or injection into the news stream. If the article is not compliant, or
is rejected by the injecting agent, then the posting agent informs is rejected by the injecting agent, then the posting agent informs
the poster with an explanation of the error. the poster with an explanation of the error.
A "reading agent" is software which presents articles to a reader. A "reading agent" is software which presents articles to a reader.
A "followup agent" is a combination of reading agent and posting A "followup agent" is a combination of reading agent and posting
agent that aids in the preparation and posting of a followup. agent that aids in the preparation and posting of a followup.
An "injecting agent" takes the finished article from the posting An "injecting agent" takes the finished article from the posting
agent (often via the NNTP "POST" command), performs some final checks agent (often via the NNTP "POST" command), performs some final checks
and passes it on to a "relaying agent" for general distribution. and passes it on to a "relaying agent" for general distribution.
A "relaying agent" is software which receives allegedly compliant A "relaying agent" is software which receives allegedly compliant
articles from injecting agents and/or other relaying agents, and articles from injecting agents and/or other relaying agents, and
possibly passes copies on to other relaying agents and "serving possibly passes copies on to other relaying agents and "storage
agents". agents".
A "serving agent" receives an article from a relaying agent and files A "storage agent" receives an article from a relaying agent and files
it in a "news database". It also provides an interface for reading it in a "news database". It also provides an interface for reading
agents to access the news database. agents to access the news database.
[There is a suggestion that "serving agent" should be changed to
"storage agent" throughout.]
A "news database" is the set of articles and related structural A "news database" is the set of articles and related structural
information stored by a serving agent and made available for access information stored by a storage agent and made available for access
by reading agents. by reading agents.
A "gateway" is software which receives news articles and converts A "gateway" is software which receives news articles and converts
them to messages of some other kind (e.g. mail to a mailing list), or them to messages of some other kind (e.g. mail to a mailing list), or
vice versa; in essence it is a translating relaying agent that vice versa; in essence it is a translating relaying agent that
straddles boundaries between different methods of message exchange. straddles boundaries between different methods of message exchange.
The most common type of gateway connects newsgroup(s) to mailing The most common type of gateway connects newsgroup(s) to mailing
list(s), either unidirectionally or bidirectionally, but there are list(s), either unidirectionally or bidirectionally, but there are
also gateways between news networks using the [USEFOR] news format also gateways between news networks using the [USEFOR] news format
and those using other formats. and those using other formats.
Posting, reading and followup agents (which are usually just Posting, reading and followup agents (which are usually just
different services provided by the same piece of software) together different services provided by the same piece of software) together
comprise the "user agents" defined in F-1.5. comprise the "user agents" defined in F-1.5.
News Article Architecture and Protocols January 2006 Likewise, injecting, relaying and storage agents (which are often
Likewise, injecting, relaying and serving agents (which are often
just different services provided by the same piece of software) just different services provided by the same piece of software)
together comprise the "news servers". together comprise the "news servers".
2.3. Identification of news servers 2.3. Identification of news servers
[The format of the Path header is still under discussion (ticket #1047). [There are two alternative texts which have been proposed:]
Hence the following texts are tentative, and will need to be changed (as
will the associated protocols in 7.3). Moreover, there are two
alternative texts which have been proposed:]
In order to record the passage of articles through the network, news In order to record the passage of articles through the network, news
servers need to identify themselves by means of a <path-identity> servers need to identify themselves by means of a <path-identity>
(F-3.1.6), which can appear in Path, Injection-Info and Xref header (F-3.1.5), which can appear in Path, Injection-Info and Xref header
fields. Whatever <path-identity> is used in the Path header field fields. Whatever <path-identity> is used in the Path header field
News Article Architecture and Protocols November 2006
SHOULD be used also in any Injection-Info header field (and it would SHOULD be used also in any Injection-Info header field (and it would
be normal to use it in any Xref header field also). be normal to use it in any Xref header field also).
[Maybe that last sentence moves elsewhere.] [Maybe that last sentence moves elsewhere.]
NOTE: Such <path-identity>s may also be suitable for sending NOTE: Such <path-identity>s may also be suitable for sending
email to news server administrators (see [USEAGE]). email to news server administrators (see [USEAGE]).
[1st alternative] [1st alternative]
<Path-identity>s can take the following forms (in decreasing order of <Path-identity>s can take the following forms (in decreasing order of
preference): preference):
1. 1. A fully qualified domain name (FQDN) that SHOULD be resolvable 1. A fully qualified domain name (FQDN) that SHOULD be resolvable in
in the DNS (whether via an A, AAAA or MX record or an equivalent the DNS (whether via an A, AAAA or MX record or an equivalent
CNAME), thus guaranteeing a unique identity. Ideally, it will also CNAME), thus guaranteeing a unique identity. Ideally, it will also
provide a means to contact the administrators by email (according provide a means to contact the administrators by email (according
to [RFC 2142], the forms "usenet@server" and "news@server" are to [RFC 2142], the forms "usenet@server" and "news@server" are
common addresses for a news server administrator). common addresses for a news server administrator).
2. Some other (arbitrary) name believed to be unique and registered 2. Some other (arbitrary) name in the form of a <path-nodot>, and
at least with all other news servers sending articles directly to believed to be unique and registered at least with all other news
the given one. This option SHOULD NOT be used unless the earlier servers sending articles directly to the given one. This option
option is unavailable (e.g. because the server in question is not SHOULD NOT be used unless the earlier option is unavailable (e.g.
connected to the Internet), or unless it is of longstanding usage because the server in question is not connected to the Internet),
and cessation would be unduly disruptive, or unless the earlier or unless it is of longstanding usage and cessation would be
option is provided as well. unduly disruptive, or unless the earlier option is provided as
well.
[2nd alternative] [2nd alternative]
<Path-identity>s can take the following forms (in decreasing order of <Path-identity>s can take the following forms (in decreasing order of
preference): preference):
1. A fully qualified domain name (FQDN) that can be resolved to an 1. A fully qualified domain name (FQDN) that can be resolved to an
email server via an MX, A or AAAA record according to the email server via an MX, A or AAAA record according to the
procedures of [RFC 2821]; this guarantees that the name is unique, procedures of [RFC 2821]; this guarantees that the name is unique,
and makes it easy to contact the administrators if needed. and makes it easy to contact the administrators if needed.
News Article Architecture and Protocols January 2006
2. A fully qualified domain name (FQDN) that is guaranteed to be 2. A fully qualified domain name (FQDN) that is guaranteed to be
unique by the administrators of the domain; for instance, the unique by the administrators of the domain; for instance, the
uniqueness of "server.example.org" could be guaranteed by the uniqueness of "server.example.org" could be guaranteed by the
administrator of "example.org" even if nothing is stored in the administrator of "example.org" even if nothing is stored in the
DNS for that name. DNS for that name.
3. Some other (arbitrary) name believed to be unique and registered 3. Some other (arbitrary) name in the form of a <path-nodot>, and
at least with all other news-servers sending articles directly to believed to be unique and registered at least with all other
the given one. This option SHOULD NOT be used unless the earlier news-servers sending articles directly to the given one. This
options are unavailable, or unless the name is of longstanding option SHOULD NOT be used unless the earlier options are
usage and cessation would be unduly disruptive, or unless one of unavailable, or unless the name is of longstanding usage and
the earlier options is provided as well. cessation would be unduly disruptive, or unless one of the earlier
options is provided as well.
News Article Architecture and Protocols November 2006
According to [RFC 2142]], the forms "usenet@server" and "news@server" According to [RFC 2142]], the forms "usenet@server" and "news@server"
are common addresses for a news server administrator. are common addresses for a news server administrator.
[end of alternatives] [end of alternatives]
NOTE: A news server administrator who chooses a name which turns NOTE: Although domain names are case insensitive and it is
out not to be unique will have to bear the consequences. intended that <path-nodot>s should also be so, it is customary
to render them all in lowercase, since many implementations
compare them case sensitively for reasons of efficiency.
NOTE: The syntax permits the colon character (which, prior to NOTE: A news server administrator who chooses a <path-nodot>
this standard, was a <path-delimiter>) within any <path- which turns out not to be unique (disregarding case) will have
identity> which is in the form of an <IPv6address>. It would to bear the consequences.
therefore be unwise to choose, as such a name, anything composed
solely from four (or less) hexadecimal digits. NOTE: An IP address is not permitted as a <path-identity>,
although it may still appear in a <diag-identity>. Since the
syntax permits a colon (":" which, prior to this standard, was
an alternative to the "!" delimiter) within any <diag-identity>
which takes the form of an <IPv6address>, it would be unwise to
choose as a <path-nodot> anything composed solely from four or
less hexadecimal digits.
2.4. Variant Header Fields 2.4. Variant Header Fields
Header fields with the variant property may differ between (or even Header fields with the variant property may differ between (or even
be completely absent from) copies of the same article as stored or be completely absent from) copies of the same article as stored or
relayed throughout a Netnews system. The manner of the difference (or relayed throughout a Netnews system. The manner of the difference (or
absence) MUST be as specified in this (or some future) standard. absence) MUST be as specified in this (or some future) standard.
Typically, these header fields are modified as articles are Typically, these header fields are modified as articles are
propagated, or they reflect the status of the article on a particular propagated, or they reflect the status of the article on a particular
serving agent, or cooperating group of such agents. A variant header storage agent, or cooperating group of such agents. A variant header
field MAY be placed anywhere within the header fields (though placing field MAY be placed anywhere within the header fields (though placing
it first is recommended). it first is recommended).
The following header fields are classified as "variant": The following header fields are classified as "variant":
o Path (F-3.1.6) - augmented at each relaying agent that an article o Path (F-3.1.5) - augmented at each relaying agent that an article
passes through. passes through.
o Xref (F-3.2.11) - used to keep track of the <article-locator>s of o Xref (F-3.2.14) - used to keep track of the <article-locator>s of
crossposted articles so that reading agents serviced by a crossposted articles so that reading agents serviced by a
particular serving agent can mark such articles as read. particular storage agent can mark such articles as read.
o Injection-Info (F-3.2.14) is also considered variant in some o Injection-Info (F-3.2.8) is also considered variant in some
special situations involving reinjection (7.2 and 7.2.2). special situations involving reinjection (6.2 and 6.2.2).
2.5. Textual Notations 2.5. Textual Notations
This standard contains explanatory NOTEs using the following format. This standard contains explanatory NOTEs using the following format.
These may be skipped by persons interested solely in the content of These may be skipped by persons interested solely in the content of
the specification. The purpose of the notes is to explain why choices the specification. The purpose of the notes is to explain why choices
were made, to place them in context, or to suggest possible were made, to place them in context, or to suggest possible
implementation techniques. implementation techniques.
News Article Architecture and Protocols January 2006
NOTE: While such explanatory notes may seem superfluous in NOTE: While such explanatory notes may seem superfluous in
principle, they often help the less-than-omniscient reader grasp principle, they often help the less-than-omniscient reader grasp
the purpose of the specification and the constraints involved. the purpose of the specification and the constraints involved.
Given the limitations of natural language for descriptive Given the limitations of natural language for descriptive
News Article Architecture and Protocols November 2006
purposes, this improves the probability that implementors and purposes, this improves the probability that implementors and
users will understand the true intent of the specification in users will understand the true intent of the specification in
cases where the wording is not entirely clear. cases where the wording is not entirely clear.
"US-ASCII" is short for "the ANSI X3.4 character set" [ANSI X3.4]. "US-ASCII" is short for "the ANSI X3.4 character set" [ANSI X3.4].
US-ASCII is a 7 bit character set. Please note that this standard US-ASCII is a 7 bit character set. Please note that this standard
requires that all agents be 8 bit clean; that is, they must accept requires that all agents be 8 bit clean; that is, they must accept
and transmit data without changing or omitting the 8th bit. and transmit data without changing or omitting the 8th bit.
Certain words, when capitalized, are used to define the significance Certain words, when capitalized, are used to define the significance
of individual requirements. The key words "MUST", "REQUIRED", of individual requirements. The key words "MUST", "REQUIRED",
"SHOULD", "RECOMMENDED", "MAY" and "OPTIONAL", and any of those words "SHOULD", "RECOMMENDED", "MAY" and "OPTIONAL", and any of those words
associated with the word "NOT", are to be interpreted as described in associated with the word "NOT", are to be interpreted as described in
[RFC 2119]. [RFC 2119].
NOTE: A requirement imposed on a relaying or serving agent NOTE: A requirement imposed on a relaying or storage agent
regarding some particular article should be understood as regarding some particular article should be understood as
applying only if that article is actually accepted for applying only if that article is actually accepted for
processing (since any agent may always reject any article processing (since any agent may always reject any article
entirely, for reasons of site policy). entirely, for reasons of site policy).
Wherever the context permits, use of the masculine includes the Wherever the context permits, use of the masculine includes the
feminine and use of the singular includes the plural, and vice versa. feminine and use of the singular includes the plural, and vice versa.
Throughout this standard we will give examples of various Throughout this standard we will give examples of various
definitions, header fields and other specifications. It needs to be definitions, header fields and other specifications. It needs to be
remembered that these samples are for the aid of the reader only and remembered that these samples are for the aid of the reader only and
do NOT define any specification themselves. In order to prevent do NOT define any specification themselves. In order to prevent
possible conflict with "Real World" entities and people the top level possible conflict with "Real World" entities and people the top level
domain ".example" is used in all sample domains and addresses. The domain ".example" is used in all sample domains and addresses. The
hierarchy "example.*" is also used as a sample hierarchy. hierarchy "example.*" is also used as a sample hierarchy.
Information on the ".example" top level domain is in [RFC 2606]. Information on the ".example" top level domain is in [RFC 2606].
3. Changes to the existing protocols 3. Transport
This standard prescribes many changes, clarifications and new
features since the protocols described in [RFC 1036] and [Son-of-
1036]. It is the intention that they can be assimilated into Usenet
as it presently operates without major interruption to the service
(3.2), though some of the new features may not begin to show benefit
until they become widely implemented. Changes in the syntax and
format are documented in F-Appendix B and changes to control messages
and the protocols are documented below.
3.1. Protocol Changes
o There is a new Control message 'mvgroup' to facilitate moving a
group to a different place (name) in a hierarchy.
o Certain Control messages (Appendix A) have been made obsolete,
and the special significance of "cmsg" when at the start of a
News Article Architecture and Protocols January 2006
Subject header field has been removed (section 6).
o Additional media types are defined for better structuring of
control messages (5.3 and 5.4).
o Distributions are expected to be checked at the receiving end, as
well as the sending end, of a relaying link.
o There are numerous other small changes, clarifications and
enhancements.
3.2. Transitional Arrangements
An important distinction must be made between news servers, which are
responsible for the distribution and storage of news articles, and
user agents, which are responsible for interactions with users. It is
important that the former should be upgraded to conform to this
standard as soon as possible to provide the benefit of the enhanced
facilities. Fortunately, the number of distinct implementations of
such servers is rather small, at least so far as the main "backbone"
of Usenet is concerned, and many of the new features are already
supported. Contrariwise, there are a great number of implementations
of user agents, installed on a vastly greater number of small sites.
Therefore, the new functionality has been designed so that existing
user agents may continue to be used, although the full benefits may
not be realised until a substantial proportion of them have been
upgraded.
In the list which follows, care has been taken to distinguish the
implications for both kinds of agent.
o [RFC 2822] style <comment>s have been prohibited in the case of
those header fields of particular concern to news servers. They
are unlikely to hinder their proper display in existing reading
agents except in the case of the References header field in
agents which thread articles. [USEFOR] therefore provides that
they SHOULD NOT be generated in that case.
o Because of its importance to all serving agents, the whitespace
and folding in Newsgroups header fields newly permitted by
[USEFOR] SHOULD NOT be generated (though it MUST be accepted);
this restriction may well be removed in a future version of this
standard.
[That last bit needs discussion. It should probably be moved to USEFOR
if it is to be retained.]
o The new style of Path header field, using "!!" as a <path-
delimiter>, is already consistent with the previous standards.
However, the intention is that relaying agents should eventually
reject articles in the old style, and so this possibility should
be offered as a configurable option in relaying agents. User
agents are unaffected.
o The introduction by [USEFOR] of MIME reflects a practice that is
already widespread. Articles in strict compliance with the
previous standards (using strict US-ASCII) will be unaffected.
Many user agents already support it, at least to the extent of
widely used charsets such as ISO-8859-1. Users expecting to read
articles using other charsets will need to acquire suitable
reading agents. It is not intended, in general, that any single
News Article Architecture and Protocols January 2006
user agent will be able to display every charset known to IANA,
but all such agents MUST support US-ASCII. Serving and relaying
agents are not affected.
o The new Control: mvgroup command will need to be implemented in
serving agents. For the benefit of older serving agents it is
therefore RECOMMENDED that it be followed shortly by a
corresponding newgroup command and it MUST always be followed by
a rmgroup command for the old group after a reasonable overlap
period. An implementation of the mvgroup command as an alias for
the newgroup command would thus be minimally conforming. User
agents are unaffected.
o Provision is made for relaying and serving agents to use the Date
header field in the case of articles injected through existing
agents which do not yet provide an Injection-Date header field.
o All the header fields newly introduced by [USEFOR] can safely be
ignored by existing software, albeit with loss of the new
functionality.
4. Transport
As in this standard's predecessors, the exact means used to transmit As in this standard's predecessors, the exact means used to transmit
articles from one host to another is not specified. NNTP [NNTP] is articles from one host to another is not specified. NNTP [RFC 3977]
the most common transmission method on the Internet, but much is the most common transmission method on the Internet, but much
transmission takes place entirely independent of the Internet. Other transmission takes place entirely independent of the Internet. Other
methods in use include the UUCP protocol [RFC 976] extensively used methods in use include the UUCP protocol [RFC 976] extensively used
in the early days of Usenet, FTP, downloading via satellite, tape in the early days of Usenet, FTP, tunneling through email using
application news/transmission, downloading via satellite, tape
archives, and physically delivered magnetic and optical media. archives, and physically delivered magnetic and optical media.
Transmission paths for news articles MUST treat news articles as Transmission paths for news articles MUST treat news articles as
uninterpreted sequences of octets, excluding the values 0 (US-ASCII uninterpreted sequences of octets, excluding the values 0 (US-ASCII
NUL) and 13 and 10 (US-ASCII CR and LF, which MUST ONLY appear in the NUL) and 13 and 10 (US-ASCII CR and LF, which MUST ONLY appear in the
combination CRLF which denotes a line separator). combination CRLF which denotes a line separator).
NOTE: this corresponds to the range of octets permitted for MIME NOTE: this corresponds to the range of octets permitted for MIME
"8bit data" [RFC 2045]. Thus raw binary data cannot be "8bit data" [RFC 2045]. Thus raw binary data cannot be
transmitted in an article body except by the use of a Content- transmitted in an article body except by the use of a Content-
Transfer-Encoding such as base64. Transfer-Encoding such as base64.
News Article Architecture and Protocols November 2006
In particular, transmission paths MUST convey all header fields In particular, transmission paths MUST convey all header fields
(including body part header fields and header fields within (including body part header fields and header fields within
message/rfc822 objects) intact, even if they contain octets in the message/rfc822 objects) intact, even if they contain octets in the
range 128 to 255. Furthermore, relaying agents MUST, and other range 128 to 255. Furthermore, relaying agents MUST, and other
agents SHOULD, convey lines even if they exceed 998 characters in agents SHOULD, convey lines even if they exceed 998 characters in
length, especially in article bodies. These requirements include the length, especially in article bodies. These requirements include the
transmissiom paths between posting agents, injecting agents, relaying transmissiom paths between posting agents, injecting agents, relaying
agents, serving agents and reading agents, but NOT the paths agents, storage agents and reading agents, but NOT the paths
traversed by Netnews articles that have been gatewayed into Email traversed by Netnews articles that have been gatewayed into Email
(7.9.1). (6.9.1).
[At some point it will be necessary for the IMAP standards to catch up [At some point it will be necessary for the IMAP standards to catch up
with these requirements.] with these requirements.]
News Article Architecture and Protocols January 2006
5. Definition of new Media Types 4. Definition of new Media Types
This standard defines (or redefines) several new Media Types, which This standard defines (or redefines) several new Media Types, which
require to be registered with IANA as provided for in [RFC 2048]. require to be registered with IANA as provided for in [RFC 4288].
5.1. Application/news-transmission 4.1. Application/news-transmission
The Media Type "application/news-transmission" is intended for the The Media Type "application/news-transmission" is intended for the
encapsulation of complete news articles where the intention is that encapsulation of complete news articles where the intention is that
the recipient should then inject them into Netnews. This Application the recipient should then inject them into Netnews. This Application
type provides one of the methods for mailing articles to moderators type provides one of the methods for mailing articles to moderators
(see 7.2.2) and it is also the preferred method when sending to an (see 6.2.2) and it is also the preferred method when sending to an
email-to-news gateway (see 7.9.2). email-to-news gateway (see 6.9.2).
NOTE: The benefit of such encapsulation is that it removes NOTE: The benefit of such encapsulation is that it removes
possible conflict between news and email header fields and it possible conflict between news and email header fields and it
provides a convenient way of "tunnelling" a news article through provides a convenient way of "tunnelling" a news article through
a transport medium that does not support 8bit characters. a transport medium that does not support 8bit characters.
The MIME Media Type definition of "application/news-transmission" is: The MIME Media Type definition of "application/news-transmission" is:
MIME type name: application MIME type name: application
MIME subtype name: news-transmission MIME subtype name: news-transmission
skipping to change at page 12, line 47 skipping to change at page 11, line 4
transmission over some 7bit transport transmission over some 7bit transport
medium. medium.
Security considerations: A news article may be a "control message", Security considerations: A news article may be a "control message",
which could have effects on the recipient which could have effects on the recipient
host's system beyond just storage of the host's system beyond just storage of the
article. However, such control messages article. However, such control messages
also occur in normal news flow, so most also occur in normal news flow, so most
hosts will already be suitably defended hosts will already be suitably defended
against undesired effects. against undesired effects.
Published specification: [USEPRO] Published specification: [USEPRO]
News Article Architecture and Protocols November 2006
Body part: A complete article or proto-article, ready Body part: A complete article or proto-article, ready
for injection into Netnews, or a batch of for injection into Netnews, or a batch of
such articles in the batch format described such articles in the batch format described
in section 6.4. in section 5.4.
NOTE: It is likely that the recipient of an "application/news- NOTE: It is likely that the recipient of an "application/news-
transmission" will be a specialized gateway (e.g. a moderator's transmission" will be a specialized gateway (e.g. a moderator's
submission address) able to accept articles with only one of the submission address) able to accept articles with only one of the
three usage parameters "moderate", "inject" and "relay", hence three usage parameters "moderate", "inject" and "relay", hence
the reason why they are optional, being redundant in most the reason why they are optional, being redundant in most
situations. Nevertheless, they MAY be used to signify the situations. Nevertheless, they MAY be used to signify the
originator's intention with regard to the transmission, so originator's intention with regard to the transmission, so
removing any possible doubt. removing any possible doubt.
News Article Architecture and Protocols January 2006
When the parameter "relay" is used, or implied, the body part MAY be When the parameter "relay" is used, or implied, the body part MAY be
a batch of articles to be transmitted together, in which case the a batch of articles to be transmitted together, in which case the
batch format defined in section 6.4 MUST be used. batch format defined in section 5.4 MUST be used.
5.2. Message/news obsoleted 4.2. Message/news obsoleted
The Media Type "message/news", as previously registered with IANA, is The Media Type "message/news", as previously registered with IANA, is
hereby declared obsolete. It was never widely implemented, and its hereby declared obsolete. It was never widely implemented, and its
default treatment as "application/octet-stream" by agents that did default treatment as "application/octet-stream" by agents that did
not recognize it was counter productive. The Media Type not recognize it was counter productive. The Media Type
"message/rfc822" SHOULD be used in its place. "message/rfc822" SHOULD be used in its place.
5.3. Application/news-groupinfo 4.3. Application/news-groupinfo
The "application/news-groupinfo" is used in conjunction with the The "application/news-groupinfo" is used in conjunction with the
"newgroup" (6.2.1) and "mvgroup" (6.2.3) control messages. The "newgroup" (5.2.1) and "mvgroup" (5.2.3) control messages. The
<newsgroup-name> in the <newsgroups-line> MUST agree with the <newsgroup-name> in the <newsgroups-line> MUST agree with the
<newsgroup-name> in the "newgroup" or "mvgroup" control message. The <newsgroup-name> in the "newgroup" or "mvgroup" control message. The
Media Type "application/news-groupinfo" MUST NOT be used except as a Media Type "application/news-groupinfo" MUST NOT be used except as a
part of such control messages. part of such control messages.
The "application/news-groupinfo" body part contains brief information The "application/news-groupinfo" body part contains brief information
about a newsgroup, i.e. the group's name, it's <newsgroup- about a newsgroup, i.e. the group's name, it's <newsgroup-
description> and the <moderation-flag>. description> and the <moderation-flag>.
NOTE: The presence of the <newsgroups-tag> "For your newsgroups NOTE: The presence of the <newsgroups-tag> "For your newsgroups
skipping to change at page 13, line 47 skipping to change at page 12, line 4
MIME type name: application MIME type name: application
MIME subtype name: news-groupinfo MIME subtype name: news-groupinfo
Required parameters: none Required parameters: none
Disposition: by default, inline Disposition: by default, inline
Encoding considerations: "7bit" or "8bit" is sufficient and MUST be Encoding considerations: "7bit" or "8bit" is sufficient and MUST be
used to maintain compatibility. used to maintain compatibility.
Security considerations: this type MUST NOT be used except as part Security considerations: this type MUST NOT be used except as part
of a control message for the creation or of a control message for the creation or
modification of a Netnews newsgroup modification of a Netnews newsgroup
News Article Architecture and Protocols November 2006
Published specification: [USEPRO] Published specification: [USEPRO]
The content of the "application/news-groupinfo" body part is defined The content of the "application/news-groupinfo" body part is defined
as: as:
groupinfo-body = [ newsgroups-tag CRLF ] groupinfo-body = [ newsgroups-tag CRLF ]
newsgroups-line CRLF newsgroups-line CRLF
newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP
%x6E.65.77.73.67.72.6F.75.70.73 SP %x6E.65.77.73.67.72.6F.75.70.73 SP
%x66.69.6C.65.3A %x66.69.6C.65.3A
; case sensitive ; case sensitive
; "For your newsgroups file:" ; "For your newsgroups file:"
News Article Architecture and Protocols January 2006
newsgroups-line = newsgroup-name newsgroups-line = newsgroup-name
[ 1*HTAB newsgroup-description ] [ 1*HTAB newsgroup-description ]
[ 1*WSP moderation-flag ] [ 1*WSP moderation-flag ]
newsgroup-description newsgroup-description
= utext *( *WSP utext ) = utext *( *WSP utext )
moderation-flag = %x28.4D.6F.64.65.72.61.74.65.64.29 moderation-flag = %x28.4D.6F.64.65.72.61.74.65.64.29
; case sensitive "(Moderated)" ; case sensitive "(Moderated)"
The <newsgroup-description> MUST NOT contain any occurrence of the The <newsgroup-description> MUST NOT contain any occurrence of the
string "(Moderated)" within it. Although optional, the <newsgroups- string "(Moderated)" within it. Although optional, the <newsgroups-
skipping to change at page 14, line 33 skipping to change at page 12, line 45
NOTE: There is no provision for the use of charsets other than NOTE: There is no provision for the use of charsets other than
US-ASCII within a <newsgroup-description>. Such a facility may US-ASCII within a <newsgroup-description>. Such a facility may
be provided in a future extension to this standard. be provided in a future extension to this standard.
[That may seem harsh, but if we make any such provision now, it will [That may seem harsh, but if we make any such provision now, it will
make life more complicated and restrict our freedom when it comes to the make life more complicated and restrict our freedom when it comes to the
proposed I18N extension. Therefore I resisted the temptation to include proposed I18N extension. Therefore I resisted the temptation to include
any charset parameter with this Media Type. Note that this also applies any charset parameter with this Media Type. Note that this also applies
to the checkgroups message further on.] to the checkgroups message further on.]
5.4. Application/news-checkgroups 4.4. Application/news-checkgroups
The "application/news-checkgroups" Media Type is used in conjunction The "application/news-checkgroups" Media Type is used in conjunction
with the "checkgroups" control message (6.2.4). It MUST NOT be used with the "checkgroups" control message (5.2.4). It MUST NOT be used
except as a part of such control messages. except as a part of such control messages.
The "application/news-checkgroups" body part contains a complete list The "application/news-checkgroups" body part contains a complete list
of all the newsgroups in a (sub)hierarchy, their <newsgroup- of all the newsgroups in a (sub)hierarchy, their <newsgroup-
description>s and their moderation status. description>s and their moderation status.
The MIME Media Type definition of "application/news-checkgroups" is: The MIME Media Type definition of "application/news-checkgroups" is:
MIME type name: application MIME type name: application
MIME subtype name: news-checkgroups MIME subtype name: news-checkgroups
Required parameters: none Required parameters: none
News Article Architecture and Protocols November 2006
Disposition: by default, inline Disposition: by default, inline
Encoding considerations: "7bit" or "8bit" is sufficient and MUST be Encoding considerations: "7bit" or "8bit" is sufficient and MUST be
used to maintain compatibility. used to maintain compatibility.
Security considerations: this type MUST NOT be used except as part Security considerations: this type MUST NOT be used except as part
of a checkgroups control message of a checkgroups control message
Published specification: [USEPRO] Published specification: [USEPRO]
The content of the "application/news-checkgroups" body part is The content of the "application/news-checkgroups" body part is
defined as: defined as:
News Article Architecture and Protocols January 2006
checkgroups-body = *( valid-group CRLF ) checkgroups-body = *( valid-group CRLF )
valid-group = newsgroups-line ; see 5.3 valid-group = newsgroups-line ; see 4.3
6. Control Messages 5. Control Messages
The following sections document the control messages. "Message" is The following sections document the control messages. "Message" is
used herein as a synonym for "article" unless context indicates used herein as a synonym for "article" unless context indicates
otherwise. otherwise.
Each <control-command> comprises a <verb>, which indicates the action Each <control-command> comprises a <verb>, which indicates the action
to be taken, and <argument>(s), which supply the details (see F- to be taken, and <argument>(s), which supply the details (see F-
3.2.5). The following sections contain syntactic definitions for the 3.2.3). The following sections contain syntactic definitions for the
<verb>, <argument>s, and possibly the body, for each type of control <verb>, <argument>s, and possibly the body, for each type of control
message. message.
[The term <control-command> is now used to denote the syntactic object
within the Control header field, to distinguish it from "control
message", which refers to the whole article.]
The Newsgroups header field of each control message SHOULD include The Newsgroups header field of each control message SHOULD include
the <newsgroup-name>(s) for the group(s) affected (i.e. groups to be the <newsgroup-name>(s) for the group(s) affected (i.e. groups to be
created, modified or removed, or containing articles to be canceled). created, modified or removed, or containing articles to be canceled).
This is to ensure that the message propagates to all sites which This is to ensure that the message propagates to all sites which
receive (or would receive) that group(s). It MAY include other receive (or would receive) that group(s). It MAY include other
<newsgroup-name>s so as to improve propagation (but this practice may <newsgroup-name>s so as to improve propagation (but this practice may
cause the control message to propagate also to places where it is cause the control message to propagate also to places where it is
unwanted, or even cause it not to propagate where it should, so it unwanted, or even cause it not to propagate where it should, so it
should not be used without good reason). should not be used without good reason).
NOTE: Propagation is controlled by relaying agents, and it may NOTE: Propagation is controlled by relaying agents, and it may
be necessary for relaying agents to take special steps to ensure be necessary for relaying agents to take special steps to ensure
that control messages such as newgroup messages for not-yet- that control messages such as newgroup messages for not-yet-
existent newsgroups are propagated correctly (see 7.3). existent newsgroups are propagated correctly (see 6.3).
The presence of a Subject header field whose content starts with the The presence of a Subject header field whose content starts with the
string "cmsg " followed by a <control-command> was construed under string "cmsg " followed by a <control-command> was construed under
[RFC 1036] as a request to perform that control action (even if no [RFC 1036] as a request to perform that control action (even if no
genuine Control header field was present). Indeed, some genuine Control header field was present). Indeed, some
implementations went further and added the implied Control header implementations went further and added the implied Control header
field before injecting. Likewise, the presence of a <newsgroup-name> field before injecting. Likewise, the presence of a <newsgroup-name>
ending in ".ctl" in the Newsgroups header field caused the Subject ending in ".ctl" in the Newsgroups header field caused the Subject
header field content (not starting with "cmsg" in this case) to be header field content (not starting with "cmsg" in this case) to be
interpreted as a <control-command>. interpreted as a <control-command>.
All these practices, which have already largely fallen into disuse, All these practices, which have already largely fallen into disuse,
are now declared to be Obsolete, and Subject header fields MUST NOT are now declared to be Obsolete, and Subject header fields MUST NOT
now be interpreted as <control-command>s under any circumstances. now be interpreted as <control-command>s under any circumstances.
News Article Architecture and Protocols November 2006
[Possible addtional text:] [Possible addtional text:]
In order to prevent continuing interpretation of Subject header In order to prevent continuing interpretation of Subject header
fields in this way by existing agents, posting and injecting agents fields in this way by existing agents, posting and injecting agents
SHOULD detect and decline to post articles in which the Subject SHOULD detect and decline to post articles in which the Subject
header field starts with the word "cmsg" and in which there is no header field starts with the word "cmsg" and in which there is no
Control header field. Control header field.
News Article Architecture and Protocols January 2006
The descriptions below set out REQUIREMENTS to be followed by sites The descriptions below set out REQUIREMENTS to be followed by sites
that receive control messages and choose to honour them. However, that receive control messages and choose to honour them. However,
nothing in these descriptions should be taken as overriding the right nothing in these descriptions should be taken as overriding the right
of any such site, in accordance with its local policy, to refuse to of any such site, in accordance with its local policy, to refuse to
honour any particular control message, or to refer it to an honour any particular control message, or to refer it to an
administrator for approval (either as a class or on a case-by-case administrator for approval (either as a class or on a case-by-case
basis). basis).
6.1. Digital Signature of Header Fields 5.1. Digital Signature of Header Fields
It is most desirable that group control messages (6.2) in particular It is most desirable that group control messages (5.2) in particular
be authenticated by incorporating them within some digital signature be authenticated by incorporating them within some digital signature
scheme that encompasses other header fields closely associated with scheme that encompasses other header fields closely associated with
them (including at least Approved, Message-ID and Date). At the time them (including at least Approved, Message-ID and Date). At the time
of writing, this is usually done by means of a protocol known as of writing, this is usually done by means of a protocol known as
"PGPverify" ([PGPVERIFY]), and continued usage of this is encouraged "PGPverify" ([PGPVERIFY]), and continued usage of this is encouraged
at least as an interim measure. at least as an interim measure.
However, PGPverify is not considered suitable for standardization in However, PGPverify is not considered suitable for standardization in
its present form, for various technical reasons. It is therefore its present form, for various technical reasons. It is therefore
expected that an early extension to this standard will provide a expected that an early extension to this standard will provide a
robust and general purpose digital authentication mechanism with robust and general purpose digital authentication mechanism with
applicability to all situations requiring protection against applicability to all situations requiring protection against
malicious use of, or interference with, header fields. That malicious use of, or interference with, header fields. That
extension would also address other Netnews security issues. extension would also address other Netnews security issues.
6.2. Group Control Messages 5.2. Group Control Messages
"Group control messages" are the sub-class of control messages that "Group control messages" are the sub-class of control messages that
request some update to the configuration of the groups known to a request some update to the configuration of the groups known to a
serving agent, namely "newgroup", "rmgroup", "mvgroup" and storage agent, namely "newgroup", "rmgroup", "mvgroup" and
"checkgroups", plus any others created by extensions to this "checkgroups", plus any others created by extensions to this
standard. standard.
Group control messages that attempt to create groups with names that Group control messages that attempt to create groups with names that
are deprecated or reserved according to F-3.1.5 MUST NOT be issued, are deprecated or reserved according to F-3.1.4 MUST NOT be issued,
except by prior agreement within some cooperating subnet. Moreover, except by prior agreement within some cooperating subnet. Moreover,
sites receiving such control messages SHOULD check them for sites receiving such control messages SHOULD check them for
conformance before honouring them. conformance before honouring them.
All of the group control messages MUST have an Approved header field All of the group control messages MUST have an Approved header field
(F-3.2.9) which, in those hierarchies where appropriate (F-3.2.1) which, in those hierarchies where appropriate
administrative agencies exist (see 1.1), identifies the appropriate administrative agencies exist (see 1.1), identifies the appropriate
person or entity as authorized by those agencies. The authorized person or entity as authorized by those agencies. The authorized
person or entity SHOULD adhere to any conventions and restrictions on person or entity SHOULD adhere to any conventions and restrictions on
the format of <newsgroup-name>s established for those hierarchies the format of <newsgroup-name>s established for those hierarchies
News Article Architecture and Protocols November 2006
[USEAGE]. [USEAGE].
6.2.1. The 'newgroup' Control Message 5.2.1. The 'newgroup' Control Message
control-command =/ Newgroup-command control-command =/ Newgroup-command
Newgroup-command = "newgroup" Newgroup-arguments Newgroup-command = "newgroup" Newgroup-arguments
Newgroup-arguments = FWS newsgroup-name [ FWS newgroup-flag ] Newgroup-arguments = FWS newsgroup-name [ FWS newgroup-flag ]
newgroup-flag = "moderated" newgroup-flag = "moderated"
News Article Architecture and Protocols January 2006
The "newgroup" control message requests that the specified group be The "newgroup" control message requests that the specified group be
created or have its moderation status or <newsgroups-line> changed. created or have its moderation status or <newsgroups-line> changed.
When the request is honoured, if the <newgroup-flag> "moderated" is When the request is honoured, if the <newgroup-flag> "moderated" is
present then the status of the group SHOULD be marked as moderated, present then the status of the group SHOULD be marked as moderated,
and vice versa. "Moderated" is the only such flag defined by this and vice versa. "Moderated" is the only such flag defined by this
standard; other flags MAY be defined for use in cooperating subnets, standard; other flags MAY be defined for use in cooperating subnets,
but newgroup messages containing them MUST NOT be acted on outside of but newgroup messages containing them MUST NOT be acted on outside of
those subnets. those subnets.
NOTE: Specifically, some alternative flags such as "y" and "m", NOTE: Specifically, some alternative flags such as "y" and "m",
which are sent and recognized by some current software, are NOT which are sent and recognized by some current software, are NOT
part of this standard. Moreover, some existing implementations part of this standard. Moreover, some existing implementations
treat any flag other than "moderated" as indicating an treat any flag other than "moderated" as indicating an
unmoderated newsgroup. Both of these usages are contrary to this unmoderated newsgroup. Both of these usages are contrary to this
standard and control messages with such non-standard flags standard and control messages with such non-standard flags
should be ignored. should be ignored.
6.2.1.1. The Body of the 'newgroup' Control Message 5.2.1.1. The Body of the 'newgroup' Control Message
The body of the newgroup message contains the following subparts, The body of the newgroup message contains the following subparts,
preferably in the order shown: preferably in the order shown:
1. An "application/news-groupinfo" part (5.3) containing the name and 1. An "application/news-groupinfo" part (4.3) containing the name and
<newsgroups-line> (5.3) of the group. This part MUST be present <newsgroups-line> (4.3) of the group. This part MUST be present
and SHOULD be used to update any copy of the <newsgroups-line> and SHOULD be used to update any copy of the <newsgroups-line>
maintained by the serving agent. maintained by the storage agent.
2. Other parts containing useful information about the background of 2. Other parts containing useful information about the background of
the newgroup message (typically of type "text/plain"). the newgroup message (typically of type "text/plain").
3. Parts containing initial articles for the newsgroup. See section 3. Parts containing initial articles for the newsgroup. See section
6.2.1.2 for details. 5.2.1.2 for details.
In the event that there is only the single (i.e. application/news- In the event that there is only the single (i.e. application/news-
groupinfo) subpart present, it will suffice to include a "Content- groupinfo) subpart present, it will suffice to include a "Content-
Type: application/news-groupinfo" amongst the header fields of the Type: application/news-groupinfo" amongst the header fields of the
control message. Otherwise, a "Content-Type: multipart/mixed" header control message. Otherwise, a "Content-Type: multipart/mixed" header
field will be needed, and each separate part will then need its own field will be needed, and each separate part will then need its own
Content-Type header field. Content-Type header field.
6.2.1.2. Initial Articles 5.2.1.2. Initial Articles
Some subparts of a "newgroup" or "mvgroup" control message MAY Some subparts of a "newgroup" or "mvgroup" control message MAY
contain an initial set of articles to be posted to the affected contain an initial set of articles to be posted to the affected
newsgroup as soon as it has been created or modified. These parts are newsgroup as soon as it has been created or modified. These parts are
News Article Architecture and Protocols November 2006
identified by having the Media Type "application/news-transmission", identified by having the Media Type "application/news-transmission",
possibly with the parameter "usage=inject". The body of each such possibly with the parameter "usage=inject". The body of each such
part should be a complete proto-article, ready for posting. This part should be a complete proto-article, ready for posting. This
feature is intended for the posting of charters, initial FAQs and the feature is intended for the posting of charters, initial FAQs and the
like to the newly formed group. like to the newly formed group.
The Newsgroups header field of the proto-article MUST include the The Newsgroups header field of the proto-article MUST include the
<newsgroup-name> of the newly created or modified group. It MAY <newsgroup-name> of the newly created or modified group. It MAY
include other <newsgroup-name>s. If the proto-article includes a include other <newsgroup-name>s. If the proto-article includes a
News Article Architecture and Protocols January 2006
Message-ID header field, the message identifier in it MUST be Message-ID header field, the message identifier in it MUST be
different from that of any existing article and from that of the different from that of any existing article and from that of the
control message as a whole. Alternatively such a message identifier control message as a whole. Alternatively such a message identifier
MAY be derived by the injecting agent when the proto-article is MAY be derived by the injecting agent when the proto-article is
posted. The proto-article SHOULD include the header field posted. The proto-article SHOULD include the header field
"Distribution: local". "Distribution: local".
The proto-article SHOULD be injected at the serving agent that The proto-article SHOULD be injected at the storage agent that
processes the control message AFTER the newsgroup in question has processes the control message AFTER the newsgroup in question has
been created or modified. It MUST NOT be injected if the newsgroup been created or modified. It MUST NOT be injected if the newsgroup
is not, in fact, created (for whatever reason). It MUST NOT be is not, in fact, created (for whatever reason). It MUST NOT be
submitted to any relaying agent for transmission beyond the serving submitted to any relaying agent for transmission beyond the storage
agent(s) upon which the newsgroup creation has just been effected (in agent(s) upon which the newsgroup creation has just been effected (in
other words, it is to be treated as having a "Distribution: local" other words, it is to be treated as having a "Distribution: local"
header field, whether such a field is actually present or not). header field, whether such a field is actually present or not).
NOTE: It is not precluded that the proto-article is itself a NOTE: It is not precluded that the proto-article is itself a
control message or other type of special article, to be control message or other type of special article, to be
activated only upon creation of the new newsgroup. However, activated only upon creation of the new newsgroup. However,
except as might arise from that possibility, any except as might arise from that possibility, any
"application/news-transmission" within some nested "multipart/*" "application/news-transmission" within some nested "multipart/*"
structure within the proto-article is not to be activated. structure within the proto-article is not to be activated.
6.2.1.3. Example 5.2.1.3. Example
A "newgroup" with its charter: A "newgroup" with its charter:
From: "example.all Administrator" <admin@noc.example> From: "example.all Administrator" <admin@noc.example>
Newsgroups: example.admin.info,example.admin.announce Newsgroups: example.admin.info,example.admin.announce
Date: 27 Feb 2002 12:50:22 +0200 Date: 27 Feb 2006 12:50:22 +0200
Subject: cmsg newgroup example.admin.info moderated Subject: cmsg newgroup example.admin.info moderated
Approved: admin@noc.example Approved: admin@noc.example
Control: newgroup example.admin.info moderated Control: newgroup example.admin.info moderated
Message-ID: <ng-example.admin.info-20020227@noc.example> Message-ID: <ng-example.admin.info-20060227@noc.example>
MIME-Version: 1.0 MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="nxtprt" Content-Type: multipart/mixed; boundary="nxtprt"
Content-Transfer-Encoding: 8bit Content-Transfer-Encoding: 8bit
This is a MIME control message. This is a MIME control message.
--nxtprt --nxtprt
Content-Type: application/news-groupinfo Content-Type: application/news-groupinfo
For your newsgroups file: For your newsgroups file:
News Article Architecture and Protocols November 2006
example.admin.info About the example.* groups (Moderated) example.admin.info About the example.* groups (Moderated)
--nxtprt --nxtprt
Content-Type: application/news-transmission Content-Type: application/news-transmission
Newsgroups: example.admin.info Newsgroups: example.admin.info
From: "example.all Administrator" <admin@noc.example> From: "example.all Administrator" <admin@noc.example>
Subject: Charter for example.admin.info Subject: Charter for example.admin.info
Message-ID: <charter-example.admin.info-20020227@noc.example> Message-ID: <charter-example.admin.info-20020227@noc.example>
Distribution: local Distribution: local
News Article Architecture and Protocols January 2006
Content-Type: text/plain; charset=us-ascii Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit
The group example.admin.info contains regularly posted The group example.admin.info contains regularly posted
information on the example.* hierarchy. information on the example.* hierarchy.
--nxtprt-- --nxtprt--
6.2.2. The 'rmgroup' Control Message 5.2.2. The 'rmgroup' Control Message
control-command =/ Rmgroup-command control-command =/ Rmgroup-command
Rmgroup-command = "rmgroup" Rmgroup-arguments Rmgroup-command = "rmgroup" Rmgroup-arguments
Rmgroup-arguments = FWS newsgroup-name Rmgroup-arguments = FWS newsgroup-name
The "rmgroup" control message requests that the specified group be The "rmgroup" control message requests that the specified group be
removed from the list of valid groups. The Media Type of the body is removed from the list of valid groups. The Media Type of the body is
unspecified; it MAY contain anything, usually an explanatory text. unspecified; it MAY contain anything, usually an explanatory text.
NOTE: It is entirely proper for a serving agent to retain the NOTE: It is entirely proper for a storage agent to retain the
group until all the articles in it have expired, provided that group until all the articles in it have expired, provided that
it ceases to accept new articles. it ceases to accept new articles.
6.2.2.1. Example 5.2.2.1. Example
From: "example.all Administrator" <admin@noc.example> From: "example.all Administrator" <admin@noc.example>
Newsgroups: example.admin.obsolete, example.admin.announce Newsgroups: example.admin.obsolete, example.admin.announce
Date: 4 Apr 2002 22:04 -0900 (PST) Date: 4 Apr 2006 22:04 -0900 (PST)
Subject: cmsg rmgroup example.admin.obsolete Subject: cmsg rmgroup example.admin.obsolete
Message-ID: <rm-example.admin.obsolete-20020404@noc.example> Message-ID: <rm-example.admin.obsolete-20060404@noc.example>
Approved: admin@noc.example Approved: admin@noc.example
Control: rmgroup example.admin.obsolete Control: rmgroup example.admin.obsolete
The group example.admin.obsolete is obsolete. Please remove it The group example.admin.obsolete is obsolete. Please remove it
from your system. from your system.
6.2.3. The 'mvgroup' Control Message 5.2.3. The 'mvgroup' Control Message
control-command =/ Mvgroup-command control-command =/ Mvgroup-command
Mvgroup-command = "mvgroup" Mvgroup-arguments Mvgroup-command = "mvgroup" Mvgroup-arguments
Mvgroup-arguments = FWS newsgroup-name FWS newsgroup-name Mvgroup-arguments = FWS newsgroup-name FWS newsgroup-name
[ FWS newgroup-flag ] [ FWS newgroup-flag ]
News Article Architecture and Protocols November 2006
The "mvgroup" control message requests that the group specified by The "mvgroup" control message requests that the group specified by
the first <(old-)newsgroup-name> be moved to that specified by the the first <(old-)newsgroup-name> be moved to that specified by the
second <(new-)newsgroup-name>. Thus it is broadly equivalent to a second <(new-)newsgroup-name>. Thus it is broadly equivalent to a
"newgroup" control message for the second group followed by a "newgroup" control message for the second group followed by a
"rmgroup" control message for the first group. "rmgroup" control message for the first group.
The message body contains an "application/news-groupinfo" part (5.3) The message body contains an "application/news-groupinfo" part (4.3)
containing machine- and human-readable information about the new containing machine- and human-readable information about the new
group, and possibly other subparts as for a "newgroup" control group, and possibly other subparts as for a "newgroup" control
message. The information conveyed in the "application/news-groupinfo" message. The information conveyed in the "application/news-groupinfo"
body part, notably its <newsgroups-line> (5.3), is applied to the new body part, notably its <newsgroups-line> (4.3), is applied to the new
group. group.
News Article Architecture and Protocols January 2006
When this message is received, the new group is created (if it does When this message is received, the new group is created (if it does
not exist already) as for a "newgroup" control message, and SHOULD in not exist already) as for a "newgroup" control message, and SHOULD in
any case be made moderated if a <newgroup-flag> "moderated" is any case be made moderated if a <newgroup-flag> "moderated" is
present, and vice versa. At the same time, arrangements SHOULD be present, and vice versa. At the same time, arrangements SHOULD be
made to remove the old group (as with a "rmgroup" control message), made to remove the old group (as with a "rmgroup" control message),
but only after a suitable overlap period to allow the network to but only after a suitable overlap period to allow the network to
adjust to the new arrangement. adjust to the new arrangement.
At the same time as a serving agent acts upon this message, all At the same time as a storage agent acts upon this message, all
injecting agents associated with that serving agent SHOULD inhibit injecting agents associated with that storage agent SHOULD inhibit
the posting of new articles to the old group (preferably with some the posting of new articles to the old group (preferably with some
indication to the poster that the new group should have been used). indication to the poster that the new group should have been used).
Relaying agents, however, MUST continue to propagate such articles Relaying agents, however, MUST continue to propagate such articles
during the overlap period. during the overlap period.
NOTE: It is to be expected that different serving agents will NOTE: It is to be expected that different storage agents will
act on this message at different points of time, users of the act on this message at different points of time, users of the
old group will have to become accustomed to the new arrangement, old group will have to become accustomed to the new arrangement,
and followups to already established threads will likely and followups to already established threads will likely
continue under the old group. Therefore, there needs to be an continue under the old group. Therefore, there needs to be an
overlap period during which articles may continue to be accepted overlap period during which articles may continue to be accepted
by relaying and serving agents in either group. This standard by relaying and storage agents in either group. This standard
does not specify any standard period of overlap (though it would does not specify any standard period of overlap (though it would
be expected to be expressed in days rather than in months). The be expected to be expressed in days rather than in months). The
inhibition of injection of new articles to the old group may inhibition of injection of new articles to the old group may
seem draconian, but it is the surest way to prevent the seem draconian, but it is the surest way to prevent the
changeover from dragging on indefinitely. changeover from dragging on indefinitely.
Since the "mvgroup" control message is newly introduced in this Since the "mvgroup" control message is newly introduced in this
standard and may not be widely implemented initially, it SHOULD be standard and may not be widely implemented initially, it SHOULD be
followed shortly afterwards by a corresponding "newgroup" control followed shortly afterwards by a corresponding "newgroup" control
message; and again, after a reasonable overlap period, it MUST be message; and again, after a reasonable overlap period, it MUST be
followed by a "rmgroup" control message for the old group. followed by a "rmgroup" control message for the old group.
In order to facilitate a smooth changeover, serving agents MAY In order to facilitate a smooth changeover, storage agents MAY
arrange to service requests for access to the old group by providing arrange to service requests for access to the old group by providing
access to the new group, which would then contain, or appear to access to the new group, which would then contain, or appear to
contain, all articles posted to either group (including, ideally, the contain, all articles posted to either group (including, ideally, the
pre-changeover articles from the old one). Nevertheless, if this pre-changeover articles from the old one). Nevertheless, if this
feature is implemented, the articles themselves, as supplied to feature is implemented, the articles themselves, as supplied to
reading agents, MUST NOT be altered in any way (and, in particular, reading agents, MUST NOT be altered in any way (and, in particular,
News Article Architecture and Protocols November 2006
their Newsgroups header fields MUST contain exactly those newsgroups their Newsgroups header fields MUST contain exactly those newsgroups
present when they were injected). On the other hand, the Xref header present when they were injected). On the other hand, the Xref header
field (F-3.2.11) MAY contain entries for either group (or even both). field (F-3.2.14) MAY contain entries for either group (or even both).
NOTE: Some serving agents that use an "active" file permit an NOTE: Some storage agents that use an "active" file permit an
entry of the form "oldgroup xxx yyy =newgroup", which enables entry of the form "oldgroup xxx yyy =newgroup", which enables
any articles arriving for oldgroup to be diverted to newgroup, any articles arriving for oldgroup to be diverted to newgroup,
thus providing a simple implementation of this feature. However, thus providing a simple implementation of this feature. However,
it is known that not all current serving agents will find it is known that not all current storage agents will find
implementation so easy (especially in the short term) which is implementation so easy (especially in the short term) which is
why it is not mandated by this standard. Nevertheless, its why it is not mandated by this standard. Nevertheless, its
eventual implementation in all serving agents is to be eventual implementation in all storage agents is to be
considered highly desirable. considered highly desirable.
News Article Architecture and Protocols January 2006
On the other hand, it is recognized that this feature would On the other hand, it is recognized that this feature would
likely not be implementable if the new group was already in likely not be implementable if the new group was already in
existence with existing articles in it. This situation should existence with existing articles in it. This situation should
not normally arise except when there is already some confusion not normally arise except when there is already some confusion
as to which groups are, or are not, supposed to exist in that as to which groups are, or are not, supposed to exist in that
hierarchy. Note that the "mvgroup" control message is not really hierarchy. Note that the "mvgroup" control message is not really
intended to be used for merging two existing groups. intended to be used for merging two existing groups.
6.2.3.1. Example 5.2.3.1. Example
From: "example.all Administrator" <admin@noc.example> From: "example.all Administrator" <admin@noc.example>
Newsgroups: example.oldgroup,example.newgroup,example.admin.announce Newsgroups: example.oldgroup,example.newgroup,example.admin.announce
Date: 30 Apr 2002 22:04 -0500 (EST) Date: 30 Apr 2006 22:04 -0500 (EST)
Subject: cmsg mvgroup example.oldgroup example.newgroup moderated Subject: cmsg mvgroup example.oldgroup example.newgroup moderated
Message-ID: <mvgroup-example.oldgroup-20020430@noc.example> Message-ID: <mvgroup-example.oldgroup-20060430@noc.example>
Approved: admin@noc.example Approved: admin@noc.example
Control: mvgroup example.oldgroup example.newgroup moderated Control: mvgroup example.oldgroup example.newgroup moderated
MIME-Version: 1.0 MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=nxt Content-Type: multipart/mixed; boundary=nxt
--nxt --nxt
Content-Type: application/news-groupinfo Content-Type: application/news-groupinfo
For your newsgroups file: For your newsgroups file:
example.newgroup The new replacement group (Moderated) example.newgroup The new replacement group (Moderated)
skipping to change at page 21, line 47 skipping to change at page 20, line 4
if possible, arrange to file articles arriving for if possible, arrange to file articles arriving for
example.oldgroup as if they were in example.newgroup. example.oldgroup as if they were in example.newgroup.
--nxt --nxt
Content-Type: application/news-transmission Content-Type: application/news-transmission
Newsgroups: example.admin.info Newsgroups: example.admin.info
From: "example.all Administrator" <admin@noc.example> From: "example.all Administrator" <admin@noc.example>
Subject: Charter for example.newgroup Subject: Charter for example.newgroup
Message-ID: <mvgroup-example.newgroup-20020430@noc.example> Message-ID: <mvgroup-example.newgroup-20020430@noc.example>
News Article Architecture and Protocols November 2006
Distribution: local Distribution: local
Content-Type: text/plain; charset=us-ascii Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit Content-Transfer-Encoding: 7bit
This group (formerly known as example.oldgroup) is for the This group (formerly known as example.oldgroup) is for the
discussion of examples. discussion of examples.
--nxt-- --nxt--
6.2.4. The 'checkgroups' Control Message 5.2.4. The 'checkgroups' Control Message
The "checkgroups" control message contains a list of all the valid The "checkgroups" control message contains a list of all the valid
groups in a complete hierarchy. groups in a complete hierarchy.
News Article Architecture and Protocols January 2006
control-command =/ Checkgroup-command control-command =/ Checkgroup-command
Checkgroup-command = "checkgroups" Checkgroup-arguments Checkgroup-command = "checkgroups" Checkgroup-arguments
Checkgroup-arguments= [ chkscope ] [ chksernr ] Checkgroup-arguments= [ chkscope ] [ chksernr ]
chkscope = 1*( FWS ["!"] newsgroup-name ) chkscope = 1*( FWS ["!"] newsgroup-name )
chksernr = FWS "#" 1*DIGIT chksernr = FWS "#" 1*DIGIT
A "checkgroups" message applies to any (sub-)hierarchy with a prefix A "checkgroups" message applies to any (sub-)hierarchy with a prefix
listed in the <chkscope> argument, provided that the rightmost listed in the <chkscope> argument, provided that the rightmost
matching <newsgroup-name> in the list is not immediately preceded by matching <newsgroup-name> in the list is not immediately preceded by
a "!". If no <chkscope> argument is given, it applies to all a "!". If no <chkscope> argument is given, it applies to all
skipping to change at page 22, line 42 skipping to change at page 20, line 55
situations where the Date header field cannot be authenticated. situations where the Date header field cannot be authenticated.
Example: Example:
Control: checkgroups de !de.alt #248 Control: checkgroups de !de.alt #248
which includes the whole of the 'de.*' hierarchy, with the exception which includes the whole of the 'de.*' hierarchy, with the exception
of its 'de.alt.*' sub-hierarchy. of its 'de.alt.*' sub-hierarchy.
The body of the message has the Media Type "application/news- The body of the message has the Media Type "application/news-
checkgroups" (5.4). It asserts that the <valid-group>s it lists are checkgroups" (4.4). It asserts that the <valid-group>s it lists are
the only newsgroups in the specified hierarchies. the only newsgroups in the specified hierarchies.
News Article Architecture and Protocols November 2006
NOTE: The "checkgroups" message is intended to synchronize the NOTE: The "checkgroups" message is intended to synchronize the
list of newsgroups stored by a serving agent, and their list of newsgroups stored by a storage agent, and their
<newsgroup-description>s, with the lists stored by other serving <newsgroup-description>s, with the lists stored by other storage
agents throughout the network. However, it might be inadvisable agents throughout the network. However, it might be inadvisable
for the serving agent actually to create or delete any for the storage agent actually to create or delete any
newsgroups without first obtaining the approval of its newsgroups without first obtaining the approval of its
administrators for such proposed actions. administrators for such proposed actions.
NOTE: The possibility of removing a complete hierarchy by means NOTE: The possibility of removing a complete hierarchy by means
of an "invalidation" line beginning with a '!' in the of an "invalidation" line beginning with a '!' in the
checkgroups-body is no longer provided by this standard. The checkgroups-body is no longer provided by this standard. The
intent of the feature was widely misunderstood and it was intent of the feature was widely misunderstood and it was
misused more often than it was used correctly. The same effect, misused more often than it was used correctly. The same effect,
if required, can now be obtained by the use of an appropriate if required, can now be obtained by the use of an appropriate
<chkscope> argument in conjunction with an empty <checkgroups- <chkscope> argument in conjunction with an empty <checkgroups-
body>. body>.
News Article Architecture and Protocols January 2006 5.3. Cancel
6.3. Cancel
The "cancel" message requests that a target article be "canceled", The "cancel" message requests that a target article be "canceled",
i.e. be withdrawn from circulation or access. i.e. be withdrawn from circulation or access.
control-command =/ Cancel-command control-command =/ Cancel-command
Cancel-command = "cancel" Cancel-arguments Cancel-command = "cancel" Cancel-arguments
Cancel-arguments = FWS msg-id [FWS] Cancel-arguments = FWS msg-id [FWS]
The argument identifies the article to be cancelled by its message The argument identifies the article to be cancelled by its message
identifier. The body SHOULD contain an indication of why the identifier. The body SHOULD contain an indication of why the
cancellation was requested. The "cancel" message SHOULD be posted to cancellation was requested. The "cancel" message SHOULD be posted to
the same newsgroup(s), with the same distribution(s), as the article the same newsgroup(s), with the same distribution(s), as the article
it is attempting to cancel. it is attempting to cancel.
A serving agent that elects to honour a "cancel" message SHOULD make A storage agent that elects to honour a "cancel" message SHOULD make
the article unavailable for relaying or serving (perhaps by deleting the article unavailable for relaying or storage (perhaps by deleting
it completely). If the target article is unavailable, and the it completely). If the target article is unavailable, and the
acceptability of the "cancel" message cannot be established without acceptability of the "cancel" message cannot be established without
it, activation of the "cancel" message SHOULD be delayed until the it, activation of the "cancel" message SHOULD be delayed until the
target article has been seen. See also sections 7.3 and 7.4. target article has been seen. See also sections 6.3 and 6.4.
NOTE: It is expected that the security extension envisaged in NOTE: It is expected that the security extension envisaged in
section 6.1 will make more detailed provisions for establishing section 5.1 will make more detailed provisions for establishing
whether honouring a particular "cancel" message is in order. In whether honouring a particular "cancel" message is in order. In
particular, it is likely that there will be provision for the particular, it is likely that there will be provision for the
digital signature of 3rd party cancels (i.e. those issued other digital signature of 3rd party cancels (i.e. those issued other
than by the sender, the moderator, or the injector). than by the sender, the moderator, or the injector).
NOTE: A cancel submitted by the poster for an article in a NOTE: A cancel submitted by the poster for an article in a
moderated group will be forwarded to the moderator of that moderated group will be forwarded to the moderator of that
group, and it is up to that moderator to act upon it (7.8). group, and it is up to that moderator to act upon it (6.8).
NOTE: The former requirement [RFC 1036] that the From and/or NOTE: The former requirement [RFC 1036] that the From and/or
Sender header fields of the "cancel" message should match those Sender header fields of the "cancel" message should match those
of the original article has been removed from this standard, of the original article has been removed from this standard,
News Article Architecture and Protocols November 2006
since it only encouraged cancel issuers to conceal their true since it only encouraged cancel issuers to conceal their true
identity, and it was not usually checked or enforced by identity, and it was not usually checked or enforced by
canceling software. Therefore, both the From and/or Sender canceling software. Therefore, both the From and/or Sender
header fields and any Approved header field should now relate to header fields and any Approved header field should now relate to
the entity responsible for issuing the "cancel" message. the entity responsible for issuing the "cancel" message.
6.4. Ihave, sendme 5.4. Ihave, sendme
The "ihave" and "sendme" control messages implement a crude batched The "ihave" and "sendme" control messages implement a crude batched
predecessor of the NNTP [NNTP] protocol. They are largely obsolete on predecessor of the NNTP [RFC 3977] protocol. They are largely
the Internet, but still see use in conjunction with some transport obsolete on the Internet, but still see use in conjunction with some
protocols such as UUCP, especially for backup feeds that normally are transport protocols such as UUCP, especially for backup feeds that
active only when a primary feed path has failed. There is no normally are active only when a primary feed path has failed. There
requirement for relaying agents that do not support such transport is no requirement for relaying agents that do not support such
protocols to implement them. transport protocols to implement them.
News Article Architecture and Protocols January 2006
NOTE: The ihave and sendme messages defined here have ABSOLUTELY NOTE: The ihave and sendme messages defined here have ABSOLUTELY
NOTHING TO DO WITH NNTP, despite similarities of terminology. NOTHING TO DO WITH NNTP, despite similarities of terminology.
The two messages share the same syntax: The two messages share the same syntax:
control-command =/ Ihave-command control-command =/ Ihave-command
Ihave-command = "ihave" Ihave-argument Ihave-command = "ihave" Ihave-argument
Ihave-argument = relayer-name Ihave-argument = relayer-name
control-command =/ Sendme-command control-command =/ Sendme-command
Sendme-command = "sendme" Sendme-argument Sendme-command = "sendme" Sendme-argument
Sendme-argument = Ihave-argument Sendme-argument = Ihave-argument
relayer-name = path-identity ; see F-3.1.6 relayer-name = path-identity ; see F-3.1.5
ihave-body = *( msg-id CRLF ) ihave-body = *( msg-id CRLF )
sendme-body = ihave-body sendme-body = ihave-body
The body of the message consists of a list of <msg-id>s, one per The body of the message consists of a list of <msg-id>s, one per
line. [RFC 1036] also permitted the list of <msg-id>s to appear in line. [RFC 1036] also permitted the list of <msg-id>s to appear in
the <Ihave-> or <Sendme-argument> with the syntax the <Ihave-> or <Sendme-argument> with the syntax
Ihave-argument = [FWS] *( msg-id FWS ) [relayer-name] Ihave-argument = [FWS] *( msg-id FWS ) [relayer-name]
but this form SHOULD NOT now be used, though relaying agents MAY but this form SHOULD NOT now be used, though relaying agents MAY
recognize and process it for backward compatibility. recognize and process it for backward compatibility.
skipping to change at page 24, line 41 skipping to change at page 22, line 56
interest to the relaying agents receiving the ihave message. The interest to the relaying agents receiving the ihave message. The
"sendme" message requests that the agent receiving it send the "sendme" message requests that the agent receiving it send the
articles having the specified message identifiers to the named articles having the specified message identifiers to the named
relaying agent. relaying agent.
Upon receipt of the sendme message, the receiving agent sends the Upon receipt of the sendme message, the receiving agent sends the
article(s) requested, often (especially when the transport protocol article(s) requested, often (especially when the transport protocol
is UUCP) in the form of one or more batches, each containing several is UUCP) in the form of one or more batches, each containing several
articles. The usual form of a <batch> is defined by the following articles. The usual form of a <batch> is defined by the following
syntax (which is also used in the application/news transmission media syntax (which is also used in the application/news transmission media
type (5.1)). type (4.1)).
News Article Architecture and Protocols November 2006
batch = 1*( batch-header article ) batch = 1*( batch-header article )
batch-header = "#!" SP rnews SP article-size CRLF batch-header = "#!" SP rnews SP article-size CRLF
rnews = %x72.6E.65.77.73 ; case sensitive "rnews" rnews = %x72.6E.65.77.73 ; case sensitive "rnews"
article-size = 1*DIGIT article-size = 1*DIGIT
Thus a <batch> is a sequence of articles, each prefixed by a header Thus a <batch> is a sequence of articles, each prefixed by a header
line that includes its size. The <article-size> is a decimal count of line that includes its size. The <article-size> is a decimal count of
the octets in the article, counting each CRLF as one octet regardless the octets in the article, counting each CRLF as one octet regardless
of how it is actually represented. of how it is actually represented.
NOTE: Despite the similarity of this format to an executable NOTE: Despite the similarity of this format to an executable
UNIX script, it is EXTREMELY unwise to feed such a batch into a UNIX script, it is EXTREMELY unwise to feed such a batch into a
command interpreter in anticipation of it running a command command interpreter in anticipation of it running a command
named "rnews"; the security implications of so doing would be named "rnews"; the security implications of so doing would be
disastrous. disastrous.
News Article Architecture and Protocols January 2006
These control messages are normally sent essentially as point-to- These control messages are normally sent essentially as point-to-
point messages, by using <newsgroup-name>s in the Newsgroups header point messages, by using <newsgroup-name>s in the Newsgroups header
field of the form "to." followed by one (or possibly more) field of the form "to." followed by one (or possibly more)
<component>s in the form of a <relayer-name> (see section F-3.1.5 <component>s in the form of a <relayer-name> (see section F-3.1.4
which forbids "to" as the first <component> of a <newsgroup-name>). which forbids "to" as the first <component> of a <newsgroup-name>).
The control message SHOULD then be delivered ONLY to the relaying The control message SHOULD then be delivered ONLY to the relaying
agent(s) identified by that <relayer-name>, and any relaying agent agent(s) identified by that <relayer-name>, and any relaying agent
receiving such a message which includes its own <relayer-name> MUST receiving such a message which includes its own <relayer-name> MUST
NOT propagate it further. Each pair of relaying agent(s) sending and NOT propagate it further. Each pair of relaying agent(s) sending and
receiving these messages MUST be immediate neighbours, exchanging receiving these messages MUST be immediate neighbours, exchanging
news directly with each other. Each relaying agent advertises its new news directly with each other. Each relaying agent advertises its new
arrivals to the other using "ihave" messages, and each uses "sendme" arrivals to the other using "ihave" messages, and each uses "sendme"
messages to request the articles it lacks. messages to request the articles it lacks.
To reduce overhead, ihave and sendme messages SHOULD be sent To reduce overhead, ihave and sendme messages SHOULD be sent
relatively infrequently and SHOULD contain reasonable numbers of relatively infrequently and SHOULD contain reasonable numbers of
message identifiers. If ihave and sendme are being used to implement message identifiers. If ihave and sendme are being used to implement
a backup feed, it may be desirable to insert a delay between a backup feed, it may be desirable to insert a delay between
reception of an ihave and generation of a sendme, so that a slightly reception of an ihave and generation of a sendme, so that a slightly
slow primary feed will not cause large numbers of articles to be slow primary feed will not cause large numbers of articles to be
requested unnecessarily via sendme. requested unnecessarily via sendme.
6.5. Obsolete control messages. 5.5. Obsolete control messages.
The following control messages (as described in Appendix A) are The following control messages (as described in Appendix A) are
declared obsolete by this standard: declared obsolete by this standard:
sendsys sendsys
version version
whogets whogets
senduuname senduuname
7. Duties of Various Agents 6. Duties of Various Agents
The following section sets out the duties of various agents involved The following section sets out the duties of various agents involved
in the creation, relaying and serving of Netnews articles. Insofar as in the creation, relaying and storage of Netnews articles. Insofar as
these duties are described as sequences of steps to be followed, it these duties are described as sequences of steps to be followed, it
should be understood that it is the effect of these sequences that is should be understood that it is the effect of these sequences that is
News Article Architecture and Protocols November 2006
important, and implementations may use any method that gives rise to important, and implementations may use any method that gives rise to
that same effect. that same effect.
In this section, the word "trusted", as applied to the source of some In this section, the word "verified", as applied to the source of
article, means that an agent processing that article has verified, by some article, means that an agent processing that article has
some means, the identity of that source (which may be another agent established, by some means, the identity of that source (which may be
or a poster). another agent or a poster).
NOTE: In many implementations, a single agent may perform NOTE: In many implementations, a single agent may perform
various combinations of the injecting, relaying and serving various combinations of the injecting, relaying and storage
functions. Its duties are then the union of the various duties functions. Its duties are then the union of the various duties
concerned. concerned.
News Article Architecture and Protocols January 2006 6.1. General principles to be followed
7.1. General principles to be followed
There are two important principles that news implementors (and There are two important principles that news implementors (and
administrators) need to keep in mind. The first is the well-known administrators) need to keep in mind. The first is the well-known
Internet Robustness Principle: Internet Robustness Principle:
Be liberal in what you accept, and conservative in what you Be liberal in what you accept, and conservative in what you
send. send.
However, in the case of news there is an even more important However, in the case of news there is an even more important
principle, derived from a much older code of practice, the principle, derived from a much older code of practice, the
skipping to change at page 26, line 30 skipping to change at page 24, line 42
First, do no harm. First, do no harm.
It is VITAL to realize that decisions which might be merely It is VITAL to realize that decisions which might be merely
suboptimal in a smaller context can become devastating mistakes when suboptimal in a smaller context can become devastating mistakes when
amplified by the actions of thousands of hosts within a few minutes. amplified by the actions of thousands of hosts within a few minutes.
In the case of gateways, the primary corollary to this is: In the case of gateways, the primary corollary to this is:
Cause no loops. Cause no loops.
7.2. Duties of an Injecting Agent 6.2. Duties of an Injecting Agent
An Injecting Agent is responsible for taking a (proto-)article from a An Injecting Agent is responsible for taking a (proto-)article from a
posting (or other) agent and either forwarding it to a moderator or posting (or other) agent and either forwarding it to a moderator or
injecting it into the relaying system for access by readers. injecting it into the relaying system for access by readers.
As such, an injecting agent is considered responsible for ensuring As such, an injecting agent is considered responsible for ensuring
that any article it injects conforms with the rules of [USEFOR]. It that any article it injects conforms with the rules of [USEFOR]. It
is also expected to bear some responsibility towards the rest of the is also expected to bear some responsibility towards the rest of the
network for the behaviour of its posters. network for the behaviour of its posters.
In the normal course of events, an article that has already been In the normal course of events, an article that has already been
injected into a Netnews network will never pass through another injected into a Netnews network will never pass through another
injecting agent. So, if an injecting agent receives an otherwise injecting agent. So, if an injecting agent receives an otherwise
valid article that has already been injected (as evidenced by the valid article that has already been injected (as evidenced by the
presence of an Injection-Date header field, an Injection-Info header presence of an Injection-Date header field, an Injection-Info header
field, or more than one "POSTED" in a Path header field) it MAY field, or more than one occurrence of the <diag-keyword> "POSTED" in
choose to reject it, but otherwise SHOULD cause it to be relayed, as a Path header field) it MAY choose to reject it, but otherwise SHOULD
it stands, by a relaying agent (7.3). News Article Architecture and Protocols November 2006
cause it to be relayed, as it stands, by a relaying agent (6.3).
In exceptional circumstances (e.g. as part of some complex gatewaying In exceptional circumstances (e.g. as part of some complex gatewaying
process, or where a relaying agent considers it essential for process, or where a relaying agent considers it essential for
fulfilling its responsibility towards the rest of the network) an fulfilling its responsibility towards the rest of the network) an
already injected article MAY be "reinjected" into the network. This already injected article MAY be "reinjected" into the network. This
standard does not prescribe any such circumstance; rather this is a standard does not prescribe any such circumstance; rather this is a
matter of policy to be determined by the administrators of each matter of policy to be determined by the administrators of each
injecting agent, who have the responsibility to ensure that no harm injecting agent, who have the responsibility to ensure that no harm
arises. In all other circumstances, unintented reinjection is to be arises. In all other circumstances, unintented reinjection is to be
avoided (see 7.9). Nevertheless, in order to preserve the integrity avoided (see 6.9). Nevertheless, in order to preserve the integrity
of the network in these special cases, this standard does set out the of the network in these special cases, this standard does set out the
correct way to reinject (see special provisions in 7.2.2 Steps 3, 7 correct way to reinject (see special provisions in 6.2.2 Steps 3, 7
News Article Architecture and Protocols January 2006
and 9). and 9).
It is usual for an injecting agent to be closely associated with a It is usual for an injecting agent to be closely associated with a
serving agent, thus giving it access to the list (7.4) showing the storage agent, thus giving it access to the list (6.4) showing the
moderation status of the newsgroups it is likely to handle. In the moderation status of the newsgroups it is likely to handle. In the
event that it does not have such an associated serving agent, it MUST event that it does not have such an associated storage agent, it MUST
maintain that list itself. maintain that list itself.
7.2.1. Proto-articles 6.2.1. Proto-articles
A proto-article SHOULD NOT be propagated in that form to other than A proto-article SHOULD NOT be propagated in that form to other than
injecting agents. injecting agents.
A proto-article has the same format as a normal article except that A proto-article has the same format as a normal article except that
some of the following mandatory header fields MAY be omitted: some of the following mandatory header fields MAY be omitted:
Message-Id, Date, Path (and even From if the particular injecting Message-Id, Date, Path (and even From if the particular injecting
agent can derive that information from other sources). However, if agent can derive that information from other sources). However, if
it is intended to offer the proto-article to two or more injecting it is intended to offer the proto-article to two or more injecting
agents in parallel, then it is only the Path header field that MAY be agents in parallel, then it is only the Path header field that MAY be
omitted. The header fields that can be omitted MUST NOT contain omitted. The header fields that can be omitted MUST NOT contain
invalid values; they MUST either be correct or not present at all. invalid values; they MUST either be correct or not present at all.
[Maybe omit that last sentence.] [Maybe omit that last sentence.]
NOTE: An article that is offered for reinjection has, by NOTE: An article that is offered for reinjection has, by
definition, already been injected once, and is not therefore to definition, already been injected once, and is not therefore to
be considered as a proto-article. Hence a genuine proto-article be considered as a proto-article. Hence a genuine proto-article
will not contain any Injection-Date header field nor any will not contain any Injection-Date header field nor the <diag-
"POSTED" in its Path header field, though that header field MAY keyword> "POSTED" anywhere in its Path header field, though that
contain <path-identity>s corresponding to machines traversed header field MAY contain <path-identity>s corresponding to
between the posting agent and the injecting agent proper. machines traversed between the posting agent and the injecting
agent proper.
7.2.2. Procedure to be followed by Injecting Agents 6.2.2. Procedure to be followed by Injecting Agents
An injecting agent receives (proto-)articles from posting and An injecting agent receives (proto-)articles from posting and
followup agents. It verifies them, adds header fields where required, followup agents. It verifies them, adds header fields where required,
and then either forwards them to a moderator or injects them by and then either forwards them to a moderator or injects them by
passing them to serving or relaying agents. It MUST NOT forward an passing them to storage or relaying agents. It MUST NOT forward an
already injected article to a moderator. already injected article to a moderator.
News Article Architecture and Protocols November 2006
An injecting agent processes articles as follows: An injecting agent processes articles as follows:
1. It MUST remove any Injection-Info header field already present 1. It MUST remove any Injection-Info header field already present
(though it might be useful to copy it to a suitable "X-" header (though it might be useful to copy it to a suitable "X-" header
field). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace, field). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace,
or other non-standard tracing header field. or other non-standard tracing header field.
2. It SHOULD verify that the article is from a trusted source, and 2. It SHOULD ensure that the article is from a verified source, and
MAY reject articles in which header fields contain unverified MAY reject articles in which header fields contain unverified
email addresses, that is, addresses which are not known to be email addresses, that is, addresses which are not known to be
valid for the trusted source, though it would be perverse to valid for the verified source, though it would be perverse to
reject intentionally unverifiable addresses such as those ending reject intentionally unverifiable addresses such as those ending
in ".invalid" (7.5). in ".invalid" (6.5).
News Article Architecture and Protocols January 2006
3. It SHOULD reject any article whose Date header field (F-3.1.2) is 3. It SHOULD reject any article whose Date header field (F-3.1.1) is
more than 24 hours into the future (and MAY use a margin less than more than 24 hours into the future (and MAY use a margin less than
that 24 hours). It MUST (except when reinjecting) reject any that 24 hours). It MUST (except when reinjecting) reject any
article with an Injection-Date header field already present (and article with an Injection-Date header field already present (and
SHOULD do likewise with any NNTP-Posting-Date header field). When SHOULD do likewise with any NNTP-Posting-Date header field). When
reinjecting it MAY, in the absence of any Injection-Date header reinjecting it MAY, in the absence of any Injection-Date header
field, reject any article whose Date header field appears to be field, reject any article whose Date header field appears to be
stale (e.g. more than 72 hours into the past). stale (e.g. more than 72 hours into the past).
4. It MUST reject any article that does not have the proper mandatory 4. It MUST reject any article that does not have the proper mandatory
header fields for a proto-article or which contains any header header fields for a proto-article or which contains any header
field that does not have legal contents. It SHOULD reject any field that does not have legal contents. It SHOULD reject any
article which contains any header field deprecated for Netnews article which contains any header field deprecated for Netnews
(e.g. as in [RFC 2298]). It SHOULD reject any article whose (e.g. as in [RFC 2298]). It SHOULD reject any article whose
Newsgroups header field does not contain at least one <newsgroup- Newsgroups header field does not contain at least one <newsgroup-
name> for an existing group (as listed by its associated serving name> for an existing group (as listed by its associated storage
agent) and it MAY reject any <newsgroup-name> which violates one agent) and it MAY reject any <newsgroup-name> which violates one
of the restrictions in F-3.1.5 or which, although otherwise of the restrictions in F-3.1.4 or which, although otherwise
correct, violates a policy restriction established, for some correct, violates a policy restriction established, for some
(sub-)hierarchy, by an agency with the appropriate authority (sub-)hierarchy, by an agency with the appropriate authority
(1.2). Observe that crossposting to unknown newsgroups is not (1.2). Observe that crossposting to unknown newsgroups is not
precluded provided at least one of those in the Newsgroups header precluded provided at least one of those in the Newsgroups header
field is listed. field is listed.
NOTE: This ability to reject <newsgroup-name>s in breach of NOTE: This ability to reject <newsgroup-name>s in breach of
established policy does not extend to relaying agents, though it established policy does not extend to relaying agents, though it
might be reasonable for posting agents to do it. might be reasonable for posting agents to do it.
skipping to change at page 28, line 46 skipping to change at page 27, line 4
formatting errors or matters of site policy) the posting agent formatting errors or matters of site policy) the posting agent
SHOULD be informed (such as via an NNTP 44x response code) that SHOULD be informed (such as via an NNTP 44x response code) that
posting has failed and the article MUST NOT then be processed posting has failed and the article MUST NOT then be processed
further. further.
6. The Message-ID, Date and From header fields (with appropriate 6. The Message-ID, Date and From header fields (with appropriate
contents) MUST be added when not already present. A User-Agent contents) MUST be added when not already present. A User-Agent
header field MAY be added (or an already present User-Agent header header field MAY be added (or an already present User-Agent header
field MAY be augmented) so as to identify the software (e.g. field MAY be augmented) so as to identify the software (e.g.
"INN/1.7.2") used by the injecting agent. "INN/1.7.2") used by the injecting agent.
[That last sentence may need to be reconsidered (in which case see
consequential change in 7.3).] News Article Architecture and Protocols November 2006
[I think that last sentence needs to go (in which case see consequential
change in 7.3). Did we discuss this when we looked at User-Agent in
USEFOR?]
NOTE: The Message-ID, Date and From fields will already be NOTE: The Message-ID, Date and From fields will already be
present during reinjection. present during reinjection.
7. The injecting agent MUST NOT alter the body of the article in any 7. The injecting agent MUST NOT alter the body of the article in any
way (including any change of Content-Transfer-Encoding). It MAY way (including any change of Content-Transfer-Encoding). It MAY
(except when reinjecting) add other header fields not already (except when reinjecting) add other header fields not already
provided by the poster, but SHOULD NOT alter, delete, or reorder provided by the poster, but SHOULD NOT alter, delete, or reorder
any existing header field, with the specific exception of the any existing header field, with the specific exception of the
"tracing" header field Injection-Info, which is to be removed as "tracing" header field Injection-Info, which is to be removed as
already mentioned. already mentioned.
News Article Architecture and Protocols January 2006
8. If the Newsgroups header field contains one or more moderated 8. If the Newsgroups header field contains one or more moderated
groups and the article does NOT contain an Approved header field, groups and the article does NOT contain an Approved header field,
the injecting agent MUST forward it to a moderator as specified in the injecting agent MUST forward it to a moderator as specified in
section 7.2.3 below. section 6.2.3 below.
9. Otherwise, a Path header field with a <tail-entry> (F-3.1.6) MUST 9. Otherwise, a Path header field with a <tail-entry> (F-3.1.5) MUST
be correctly added if not already present. During reinjection, the be correctly added if not already present. During reinjection, the
existing Path header field SHOULD be retained. existing Path header field SHOULD be retained.
10.It MUST then prepend the <path-identity> of the injecting agent, 10.It MUST then prepend the <path-identity> of the injecting agent,
followed by a '!', the <path-keyword> "POSTED" and a further "!" followed by '!.' and the <diag-keyword> "POSTED", and then a
(or "!!" if appropriate) to the content of the Path header field; further "!", to the content of the Path header field; this header
this header field SHOULD then be folded if it would otherwise field SHOULD then be folded if it would otherwise result in a
result in a header line of excessive length. header line of excessive length.
[This may need further changes depending on the resolution of ticket
#1047.]
NOTE: This could result in more that one "POSTED" <path-keyword> NOTE: This could result in more that one "POSTED" <path-keyword>
in the case of reinjection. in the case of reinjection.
11.An Injection-Info header field (F-3.2.14) SHOULD be added, 11.An Injection-Info header field (F-3.2.8) SHOULD be added,
identifying the trusted source of the article and possibly an identifying the verified source of the article and possibly an
address for mailing complaints to. Each injecting agent SHOULD address for mailing complaints to. Each injecting agent SHOULD
use a consistent form of the Injection-Info header field for all use a consistent form of the Injection-Info header field for all
articles emanating from the same or similar origins. articles emanating from the same or similar origins.
NOTE: The step above is the only place in which an Injection- NOTE: The step above is the only place in which an Injection-
Info header field is to be created. It follows that this header Info header field is to be created. It follows that this header
field MUST NOT be created, replaced, changed or deleted by any field MUST NOT be created, replaced, changed or deleted by any
other agent (except during reinjection, in which case it will other agent (except during reinjection, in which case it will
always relate to the latest injection and is, to that extent, always relate to the latest injection and is, to that extent,
regarded as a variant header field). regarded as a variant header field).
12.The injecting agent MUST then add an Injection-Date header field 12.The injecting agent MUST then add an Injection-Date header field
(F-3.2.1) if one is not already present, but it MUST NOT alter, or (F-3.2.7) if one is not already present, but it MUST NOT alter, or
delete, an already present Injection-Date header field (and delete, an already present Injection-Date header field (and
likewise SHOULD NOT alter, or delete, an already present NNTP- likewise SHOULD NOT alter, or delete, an already present NNTP-
Posting-Date header field). Finally, it forwards the article to Posting-Date header field). Finally, it forwards the article to
one or more relaying or serving agents, and the injection process one or more relaying or storage agents, and the injection process
is to be considered complete. is to be considered complete.
News Article Architecture and Protocols November 2006
NOTE: The step above is the only place where an Injection-Date NOTE: The step above is the only place where an Injection-Date
header field is to be created It follows that it MUST NOT header field is to be created It follows that it MUST NOT
subsequently be replaced, changed or deleted by any other agent, subsequently be replaced, changed or deleted by any other agent,
even during reinjection. even during reinjection.
7.2.3. Procedure for Forwarding to a Moderator 6.2.3. Procedure for Forwarding to a Moderator
An injecting agent forwards an article to a moderator as follows: An injecting agent forwards an article to a moderator as follows:
1. It MUST forward it to the moderator of the first (leftmost) 1. It MUST forward it to the moderator of the first (leftmost)
moderated group listed in the Newsgroups header field, customarily moderated group listed in the Newsgroups header field, customarily
via email, (see 7.8 for how that moderator may forward it to via email, (see 6.8 for how that moderator may forward it to
further moderators). There are two possibilities for doing this: further moderators). There are two possibilities for doing this:
News Article Architecture and Protocols January 2006
(a) The complete article is encapsulated (header fields and all) (a) The complete article is encapsulated (header fields and all)
within the email, preferably using the Content-Type within the email, preferably using the Content-Type
"application/news-transmission" (5.1) with any usage "application/news-transmission" (4.1) with any usage
parameter set to "moderate". Moreover, there SHOULD NOT be parameter set to "moderate". Moreover, there SHOULD NOT be
more than one encapsulated article within the one email. more than one encapsulated article within the one email.
This method has the advantage of removing any possible This method has the advantage of removing any possible
conflict between Netnews and Email header fields, or of conflict between Netnews and Email header fields, or of
changes to those fields during transport through email. changes to those fields during transport through email.
(b) The article is sent as an email as it stands, with the (b) The article is sent as an email as it stands, with the
addition of such extra header fields (e.g. a To header field) addition of such extra header fields (e.g. a To header field)
as are necessary for an email. The existing Message-ID header as are necessary for an email. The existing Message-ID header
field SHOULD be retained. field SHOULD be retained.
skipping to change at page 30, line 41 skipping to change at page 28, line 55
which provide the service of forwarding to moderators, and the which provide the service of forwarding to moderators, and the
address to use with them is obtained as follows: address to use with them is obtained as follows:
(a) Each '.' in the <newsgroup-name> is replaced with a '-'. (a) Each '.' in the <newsgroup-name> is replaced with a '-'.
(b) The result of these operations is used as the <local-part> of (b) The result of these operations is used as the <local-part> of
the <mailbox> of the agent. For example, articles intended the <mailbox> of the agent. For example, articles intended
for "news.announce.important" would be emailed to "news- for "news.announce.important" would be emailed to "news-
announce-important@forwardingagent.example". announce-important@forwardingagent.example".
7.3. Duties of a Relaying Agent 6.3. Duties of a Relaying Agent
A Relaying Agent accepts injected articles from injecting and other A Relaying Agent accepts injected articles from injecting and other
relaying agents and passes them on to relaying or serving agents relaying agents and passes them on to relaying or storage agents
according to mutually agreed policy. Relaying agents SHOULD accept according to mutually agreed policy. Relaying agents SHOULD accept
articles ONLY from trusted agents. articles ONLY from verified sources.
News Article Architecture and Protocols November 2006
An article SHOULD NOT be relayed unless the sending agent has been An article SHOULD NOT be relayed unless the sending agent has been
configured to supply and the receiving agent to receive at least one configured to supply and the receiving agent to receive at least one
of the <newsgroup-name>s in its Newsgroups header field and at least of the <newsgroup-name>s in its Newsgroups header field and at least
one of the <dist-name>s in its Distribution header field, if any. one of the <dist-name>s in its Distribution header field, if any.
Exceptionally, ALL relaying agents are deemed willing to supply or Exceptionally, ALL relaying agents are deemed willing to supply or
accept the <dist-name> "world", and NO relaying agent should supply accept the <dist-name> "world", and NO relaying agent should supply
or accept the <dist-name> "local". or accept the <dist-name> "local".
However, if the particular implementation does not relay non-existent However, if the particular implementation does not relay non-existent
newsgroups, even when included in the Newsgroups header field and newsgroups, even when included in the Newsgroups header field and
implied (e.g. by some "wild card" notation) in the configuration implied (e.g. by some "wild card" notation) in the configuration
tables, then the agent MUST examine all group control messages (6.2) tables, then the agent MUST examine all group control messages (5.2)
in order to ensure that relaying of those messages proceeds normally. in order to ensure that relaying of those messages proceeds normally.
News Article Architecture and Protocols January 2006
NOTE: Although it would seem redundant to filter out unwanted NOTE: Although it would seem redundant to filter out unwanted
distributions at both ends of a relaying link (and it is clearly distributions at both ends of a relaying link (and it is clearly
more efficient to do so at the sending end), many sending sites more efficient to do so at the sending end), many sending sites
have been reluctant, historically speaking, to apply such have been reluctant, historically speaking, to apply such
filters (except to ensure that distributions local to their own filters (except to ensure that distributions local to their own
site or cooperating subnet did not escape); moreover they tended site or cooperating subnet did not escape); moreover they tended
to configure their filters on an "all but those listed" basis, to configure their filters on an "all but those listed" basis,
so that new and hitherto unheard of distributions would not be so that new and hitherto unheard of distributions would not be
caught. Indeed many "hub" sites actually wanted to receive all caught. Indeed many "hub" sites actually wanted to receive all
possible distributions so that they could feed on to their possible distributions so that they could feed on to their
skipping to change at page 31, line 32 skipping to change at page 29, line 46
simpler to do so locally than to inform each sending site of simpler to do so locally than to inform each sending site of
what is required, especially in the case of specialized what is required, especially in the case of specialized
distributions (for example for control messages, such as cancels distributions (for example for control messages, such as cancels
from certain issuers) which might need to be added at short from certain issuers) which might need to be added at short
notice. A similar possibility for reading agents to filter notice. A similar possibility for reading agents to filter
distributions is also suggested in [USEAGE]) for the same distributions is also suggested in [USEAGE]) for the same
reason. reason.
In order to avoid unnecessary relaying, an article SHOULD NOT be In order to avoid unnecessary relaying, an article SHOULD NOT be
relayed if the <path-identity> of the receiving agent (or some known relayed if the <path-identity> of the receiving agent (or some known
alias thereof) appears as a <path-identity> (excluding within the alias thereof) appears as a <path-identity> in its Path header field.
<tail-entry>) in its Path header field. But note that the <tail-entry> (which follows the last "!") is not a
<path-identity>, although not all current implementations observe
this distinction.
For this to work, each relaying agent needs to insert it own <path-
identity> (chosen according to 2.3) into the Path header field. It
MAY insert more than one <path-identity> for itself (in which case
the leftmost should be the one by which it is known to its immediate
neighbours), but where an article passes through several relaying
agents at the same site it MAY omit the <path-identity>s for some of
them (but NOT the one which finally relays it to an outside site).
News Article Architecture and Protocols November 2006
It MAY (and usually SHOULD) also add a <path-diagnostic> giving
additional information concerning the route taken by the article
through the network. A <path-diagnostic> consists of either the
special <diag-match> "!" (which effectively replaces the standard
delimiter "!" by "!!"), or it is composed from a <diag-keyword>
usually followed by a <diag-identity>. The following are the only
<diag-keyword>s defined by this standard:
o "POSTED" (already introduced in Step 10 of 6.2.2), which is never
followed by a <diag-identity>;
o "SEEN", whose following <diag-identity> indicates the verified
identity (see 6) of the agent from which the article was received
(but makes no claim as to whether it matched the <path-identity)
inserted by that agent);
o "MISMATCH", whose following <diag-identity> indicates the
verified identity of the agent from which the article was
received and asserts that it could not be reconciled with the
<path-identity> (supposedly) inserted by that agent.
Other <diag-keyword>s beginning with "X" MAY be used by a relaying
agent to make some assertion not envisaged here, but other (non-"X")
<diag-keyword>s MUST NOT be used unless defined by some extension to
this standard.
NOTE: The <diag-keyword> "MATCH", which might have indicated the
verified identity of the source agent with an assertion that it
agreed with the <path-identity> inserted by that agent, has NOT
been provided, since the special <diag-match> conveys exactly
that meaning for this commonly occurring case.
NOTE: Whilst <diag-keywords>s are case insensitive, it is
intended that they should normally be rendered in uppercase.
A relaying agent processes articles as follows: A relaying agent processes articles as follows:
1. It MUST establish the trusted identity of the source of the 1. It MUST/SHOULD establish the verified identity of the source of
article and compare it with the leftmost <path-identity> of the the article and compare it with the leftmost <path-identity> of
Path header field's content. If it matches it MUST then prepend the existing Path header field's content. Except possibly when
its own <path-identity> and a '!!' <path-delimiter> to that relaying to other hosts on the same site, It then MUST or SHOULD,
content. If it does not match then it prepends instead two entries as indicated, prepend to that content (from left to right) the
to that content; firstly the true established <path-identity> of following:
the source followed by a '!', the <path-keyword> "MISMATCH" and a o (MUST) its own <path-identity>;
further '!', and then, to the left of that, its own <path- o (MUST) a "!" delimiter;
identity> followed by a '!!' <path-delimiter> as usual. This o (MUST/SHOULD) if the verified and existing identities match,
prepending of two entries SHOULD NOT be done if the provided and a <diag-match> (effectively converting the "!" delimiter into
established identities match. This header field SHOULD then be "!!");
folded if it would otherwise result in a header line of excessive o (MUST/SHOULD) alternatively, where the identities do not
length. match (or have not been determied to match), a ".", the
[This may need further changes depending on the resolution of ticket <diag-keyword> "MISMATCH" (or "SEEN"), another ".", a <diag-
#1047.] identity> indicating the verified identity, and finally a
[It has been suggested that relaying agents should be permitted to further "!";
prepend more than the one or two entries permitted above.] o possibly further <path-identity>s etc. as above, identifying
[something like the following from Diablo might also be useful: itself.
>>> NOTE <<< you should grep through newly created spool directories This header field SHOULD then be folded if it would otherwise
every so often looking for .MISMATCH in the spool files to locate result in a header line of excessive length.
incoming feeds with improperly configured I found that four of my 80+
feeds were misconfigured. ] News Article Architecture and Protocols November 2006
News Article Architecture and Protocols January 2006
[The "MUST/SHOULD"s above were all "MUST" in the previous drafts.
Discussion is needed to resolve this.]
NOTE: Since each agent at one site can be assumed to be aware of
the identity of the others (and of itself), it would be most
unusual for their <path-identity>s to be separated other than by
"!!". Thus the presence of a single "!", unless followed by a
"." and some <diag-keyword>, can be taken as signifying an agent
that has not yet been upgraded to conform to this standard.
NOTE: Whilst the presence of a "MISMATCH" would normally
indicate that the existing Path was bogus in some sense, it
could also indicate that the ralaying agent was improperly
configured to recognise the identities or aliases used by its
neighbours. Administators of relaying agents should therefore
periodically monitor the <path-diagnostic> being inserted so as
to avoid this.
NOTE: In order to prevent overloading, relaying agents should NOTE: In order to prevent overloading, relaying agents should
not routinely query an external entity (such as a DNS-server) in not routinely query an external entity (such as a DNS-server) in
order to verify an article (though a local cache of the required order to determine a verified identity (though a local cache of
information might usefully be consulted). the required information might usefully be consulted).
2. It MUST examine the Injection-Date header field (or, if that is 2. It MUST examine the Injection-Date header field (or, if that is
absent, the Date header field) and reject the article as stale absent, the Date header field) and reject the article as stale
(F-3.2.1) if that predates the earliest articles of which it (F-3.2.7) if that predates the earliest articles of which it
normally keeps record, or if it is more than 24 hours into the normally keeps record, or if it is more than 24 hours into the
future (the margin MAY be less than that 24 hours). future (the margin MAY be less than that 24 hours).
3. It SHOULD reject any article that does not include all the 3. It SHOULD reject any article that does not include all the
mandatory header fields (section F-3.1). mandatory header fields (section F-3.1).
4. It MAY reject any article whose header fields do not have legal 4. It MAY reject any article whose header fields do not have legal
contents. contents.
5. It SHOULD reject any article that has already been sent to it (a 5. It SHOULD reject any article that has already been sent to it (a
database of message identifiers of recent messages is usually kept database of message identifiers of recent messages is usually kept
and matched against). and matched against).
NOTE: Even though commonly derived from the domain name of the NOTE: Even though commonly derived from the domain name of the
originating site (and domain names are case-insensitive), a originating site (and domain names are case-insensitive), a
message identifier MUST NOT be altered in any way during <msg-id-core> MUST NOT be altered in any way during transport,
transport, or when copied (as when forming a References header or when copied (as when forming a References header field), and
field), and thus a simple (case-sensitive) comparison of octets thus a simple (case-sensitive) comparison of octets will always
will always suffice to recognize that same message identifier suffice to recognize that same message identifier wherever it
wherever it subsequently reappears. subsequently reappears.
NOTE: These requirements are to be contrasted with those of the NOTE: These requirements are to be contrasted with those of the
un-normalized msg-ids defined by [RFC 2822], which may perfectly un-normalized msg-ids defined by [RFC 2822], which may perfectly
legitimately become normalized (or vice versa) during transport legitimately become normalized (or vice versa) during transport
or copying in email systems. or copying in email systems.
News Article Architecture and Protocols November 2006
6. It SHOULD reject any article that matches an already received 6. It SHOULD reject any article that matches an already received
cancel message (or an equivalent Supersedes header field) issued cancel message (or an equivalent Supersedes header field) issued
by its poster or by some other trusted entity. by its poster or by some other trusted entity.
7. It MAY reject any article without an Approved header field posted 7. It MAY reject any article without an Approved header field posted
to newsgroups known to be moderated (this practice is strongly to newsgroups known to be moderated (this practice is strongly
recommended, but the information necessary to do so may not be recommended, but the information necessary to do so may not be
available to all agents). available to all agents).
8. It MAY delete any Xref header field that is present. 8. It MAY delete any Xref header field that is present.
9. Finally, it passes the articles on to neighbouring relaying and 9. Finally, it passes the articles on to neighbouring relaying and
serving agents. storage agents.
If the article is rejected as being invalid, unwanted or unacceptable If the article is rejected as being invalid, unwanted or unacceptable
due to site policy, the agent that passed the article to the relaying due to site policy, the agent that passed the article to the relaying
agent SHOULD be informed (such as via an NNTP 43x response code) that agent SHOULD be informed (such as via an NNTP 43x response code) that
relaying failed. In order to prevent a large number of error messages relaying failed. In order to prevent a large number of error messages
being sent to one location, relaying agents MUST NOT inform any other being sent to one location, relaying agents MUST NOT inform any other
external entity that an article was not relayed UNLESS that external external entity that an article was not relayed UNLESS that external
News Article Architecture and Protocols January 2006
entity has explicitly requested that it be informed of such errors. entity has explicitly requested that it be informed of such errors.
Relaying agents MUST NOT alter, delete or rearrange any part of an Relaying agents MUST NOT alter, delete or rearrange any part of an
article except for header fields designated as variant (2.4). In article except for header fields designated as variant (2.4). In
particular particular
o they MUST NOT create or augment a User-Agent header field in o they MUST NOT create or augment a User-Agent header field in
order to identify themselves; order to identify themselves;
o they MUST NOT rewrite the Newsgroups header field in any way, o they MUST NOT rewrite the Newsgroups header field in any way,
even if some supposedly non-existent newsgroup is included; even if some supposedly non-existent newsgroup is included;
skipping to change at page 33, line 28 skipping to change at page 32, line 49
o they MUST NOT alter the Date header field or the Injection-Date o they MUST NOT alter the Date header field or the Injection-Date
header field; header field;
o they MUST NOT delete any unrecognized header field whose field o they MUST NOT delete any unrecognized header field whose field
name is syntactically correct (whether or not it is registered name is syntactically correct (whether or not it is registered
with IANA [RFC 3864]); with IANA [RFC 3864]);
o they MUST NOT change the Content-Transfer-Encoding of the body or o they MUST NOT change the Content-Transfer-Encoding of the body or
any body part; any body part;
o they MUST transmit lines of arbitrary length and articles of o they MUST transmit lines of arbitrary length and articles of
arbitrary size. arbitrary size.
7.3.1. Path Header Field Example 6.3.1. Path Header Field Example
Path: foo.isp.example!!foo-server!!bar.isp.example!MISMATCH! Path: bar.isp.example!.SEEN.news3.foo.isp.example!foo.isp.example
2001:DB8:0:0:8:800:200C:417A!!old.site.example!barbaz!! !!foo-server!.MISMATCH.2001:DB8:0:0:8:800:200C:417A
baz.isp.example!POSTED!!dialup123.baz.isp.example!not-for-mail !dubious.site.example!!old.site.example!barbaz!!baz.isp.example
!.POSTED!dialup123.baz.isp.example!not-for-mail
NOTE: That article was injected into the news stream by NOTE: That article was injected into the news stream by
baz.isp.example, as indicated by the <path-keyword> "POSTED" baz.isp.example, as indicated by the <diag-keyword> "POSTED"
(complaints may be addressed to abuse@baz.isp.example). The (complaints may be addressed to abuse@baz.isp.example). The
injector has chosen to record that it got it from injector has chosen to record that it got it from
dialup123.baz.isp.example. "not-for-mail" is a dummy <tail- dialup123.baz.isp.example. "not-for-mail" is a dummy <tail-
News Article Architecture and Protocols November 2006
entry>, though sometimes a real userid is put there. entry>, though sometimes a real userid is put there.
The article was relayed, perhaps by UUCP, to the machine known, The article was relayed, perhaps by UUCP, to the machine known,
at least to old.site.example, as "barbaz". at least to old.site.example, as "barbaz".
Barbaz relayed it to old.site.example, which does not yet Barbaz relayed it to old.site.example, which does not yet
conform to this standard (hence the '!' <path-delimiter). So one conform to this standard (hence the single '!' delimiter). So
cannot be sure that it really came from barbaz. one cannot be sure that it really came from barbaz.
Old.site.example relayed it to a site claiming to have the IPv6
address [2001:DB8:0:0:8:800:200C:417A], and claiming (by using
the '!!' <path-delimiter>) to have verified that it came from
old.site.example.
[2001:DB8:0:0:8:800:200C:417A] relayed it to "foo-server" which, Old.site.example relayed it to a site claiming to be
not being convinced that it truly came from dubious.site.example, and claiming (by using the '!!'
[2001:DB8:0:0:8:800:200C:417A], inserted the <path-keyword> delimiter) to have verified that it came from old.site.example.
"MISMATCH" and then did a reverse lookup on the actual source
and concluded it was known as bar.isp.example (that is not to
say that [2001:DB8:0:0:8:800:200C:417A] was not a correct IPv6
address for bar.isp.example, but simply that that connection
News Article Architecture and Protocols January 2006
could not be substantiated by foo-server). Observe that foo- Dubious.site.example relayed it to "foo-server" which, not being
server has now added two entries to the Path. convinced that it truly came from dubious.site.example, and
knowing that it in fact arrived from a host with the IPv6address
[2001:DB8:0:0:8:800:200C:417A], inserted the <path-diagnostic>
"!.MISMATCH.2001:DB8:0:0:8:800:200C:417A" (that is not to say
that [2001:DB8:0:0:8:800:200C:417A] was not a correct IPv6
address for dubious-site.example, but simply that that
connection could not be substantiated by foo-server).
"foo-server" is a locally significant name within the complex "foo-server" is a locally significant name within the complex
site of many machines run by foo.isp.example, so the latter site of many machines run by foo.isp.example, so the latter
should have no problem recognizing foo-server and using a '!!' should have no problem recognizing foo-server and using a '!!'
<path-delimiter>. Presumably foo.isp.example then delivered the delimiter. It was not strictly necessary to insert the <path-
article to its direct clients. identity> "foo-server" as well as "foo.isp.example" (but maybe
some site elsewhere had some reason to test for it).
[Please could Richard Clayton provide a more plausible reason why "foo-
server" might be a <path-nodot> here?]
It appears that foo-server and barbaz decided to fold the line, It then went to bar.isp.example which determined (by reverse
on the grounds that it seemed to be getting a little too long. DNS) that it had come from news3.foo.isp.example, but had taken
no steps to check whether that was a known alias for
foo.isp.example (which it probably was). Strictly, it SHOULD
have made such a check, and then inserted either a "!!" or a
"!MISMATCH..." according to the result.
7.4. Duties of a Serving Agent Presumably bar.isp.example then delivered the article to its
direct clients.
It appears that foo.isp.example, foo-server and baz.isp.example
decided to fold the line, on the grounds that it seemed to be
getting a little too long. Note that folding and whitespace is
permitted before (but not after) any "!" (but not within a
"!!"); hence continuation lines will always start with either
"!" or "!!".
6.4. Duties of a Serving Agent
A Serving Agent takes an article from a relaying or injecting agent A Serving Agent takes an article from a relaying or injecting agent
and files it in a "news database". It also provides an interface for and files it in a "news database". It also provides an interface for
reading agents to access the news database. This database is normally reading agents to access the news database. This database is normally
indexed by newsgroup with articles in each newsgroup identified by an indexed by newsgroup with articles in each newsgroup identified by an
News Article Architecture and Protocols November 2006
<article-locator> (usually in the form of a decimal number - see F- <article-locator> (usually in the form of a decimal number - see F-
3.2.11). 3.2.14).
A serving agent MUST maintain a list of the newsgroups it stores in A storage agent MUST maintain a list of the newsgroups it stores in
its news database showing the moderation status of each one (see its news database showing the moderation status of each one (see
6.2.1), and SHOULD include in that list all groups likely to be 5.2.1), and SHOULD include in that list all groups likely to be
crossposted to from those groups (e.g. all other groups in the same crossposted to from those groups (e.g. all other groups in the same
hierarchy(ies)). hierarchy(ies)).
NOTE: Since control messages are often of interest, but should NOTE: Since control messages are often of interest, but should
not be displayed as normal articles in regular newsgroups, it is not be displayed as normal articles in regular newsgroups, it is
common for serving agents to make them available in a pseudo- common for storage agents to make them available in a pseudo-
newsgroup named "control" or in a pseudo-newsgroup in a sub- newsgroup named "control" or in a pseudo-newsgroup in a sub-
hierarchy under "control." (e.g. "control.cancel"). hierarchy under "control." (e.g. "control.cancel").
A serving agent MAY decline to accept an article if the Path header A storage agent MAY decline to accept an article if the Path header
field contains some <path-identity> whose articles the serving agent field contains some <path-identity> whose articles the storage agent
does not want, as a matter of local policy. does not want, as a matter of local policy.
NOTE: This last facility is sometimes used to detect and decline NOTE: This last facility is sometimes used to detect and decline
control messages (notably cancel messages) which have been control messages (notably cancel messages) which have been
deliberately seeded with a <path-identity> to be "aliased out" deliberately seeded with a <path-identity> to be "aliased out"
by sites not wishing to act upon them. by sites not wishing to act upon them.
[INN at least does this. It might be argued that it is not necessary to [INN at least does this. It might be argued that it is not necessary to
mention it here.] mention it here.]
A serving agent processes articles as follows: A storage agent processes articles as follows:
1. It MUST establish the trusted identity of the source of the 1. It MUST/SHOULD establish the verified identity of the source of
article and modify the Path header field as for a relaying agent the article and modify the Path header field as for a relaying
(7.3). agent (6.3).
2. It MUST examine the Injection-Date header field (or, if that is 2. It MUST examine the Injection-Date header field (or, if that is
absent, the Date header field) and reject the article as stale absent, the Date header field) and reject the article as stale
(F-3.2.1) if that predates the earliest articles of which it (F-3.2.7) if that predates the earliest articles of which it
normally keeps record, or if it is more than 24 hours into the normally keeps record, or if it is more than 24 hours into the
News Article Architecture and Protocols January 2006
future (the margin MAY be less than that 24 hours). future (the margin MAY be less than that 24 hours).
3. It MUST reject any article that does not include all the mandatory 3. It MUST reject any article that does not include all the mandatory
header fields (section F-3.1), or which contains any header field header fields (section F-3.1), or which contains any header field
that does not have legal contents. that does not have legal contents.
4. It SHOULD reject any article that has already been sent to it (a 4. It SHOULD reject any article that has already been sent to it (a
database of message identifiers of recent articles is usually kept database of message identifiers of recent articles is usually kept
and matched against). and matched against).
5. It SHOULD reject any article that matches an already received 5. It SHOULD reject any article that matches an already received
cancel message (or an equivalent Supersedes header field) issued cancel message (or an equivalent Supersedes header field) issued
by its poster or by some other trusted entity. by its poster or by some other trusted entity.
Likewise, a newly received cancel message (or equivalent
Supersedes) should cause immediate deletion (or deactivation) of
the canceled article.
News Article Architecture and Protocols November 2006
NOTE: An article with a Supersedes header field is itself stored
normally.
6. It MUST reject any article without an Approved header field posted 6. It MUST reject any article without an Approved header field posted
to any newsgroup listed as moderated. to any newsgroup listed as moderated.
7. It MUST (exept when specially configured to preserve the 7. It MUST (exept when specially configured to preserve the
<article-locator>s set by the sending site) remove any Xref header <article-locator>s set by the sending site) remove any Xref header
field (F-3.2.11) from each article. It then MAY (and usually field (F-3.2.14) from each article. It then MAY (and usually
will) generate a fresh Xref header field. will) generate a fresh Xref header field.
8. Finally, it stores the article in its news database. 8. Finally, it stores the article in its news database.
Serving agents MUST NOT create new newsgroups simply because an Serving agents MUST NOT create new newsgroups simply because an
unrecognized <newsgroup-name> occurs in a Newsgroups header field unrecognized <newsgroup-name> occurs in a Newsgroups header field
(see 6.2.1 for the correct method of newsgroup creation). (see 5.2.1 for the correct method of newsgroup creation).
Serving agents MUST NOT alter, delete or rearrange any part of an Serving agents MUST NOT alter, delete or rearrange any part of an
article in any other way. The list of particular cases given for article in any other way. The list of particular cases given for
relaying agents (7.3) applies here also. relaying agents (6.3) applies here also.
7.5. Duties of a Posting Agent 6.5. Duties of a Posting Agent
A Posting Agent is used to assist the poster in creating a valid A Posting Agent is used to assist the poster in creating a valid
proto-article and forwarding it to an injecting agent. proto-article and forwarding it to an injecting agent.
Postings agents SHOULD ensure that proto-articles they create are Postings agents SHOULD ensure that proto-articles they create are
valid according to [USEFOR] and other applicable policies. In valid according to [USEFOR] and other applicable policies. In
particular, they MUST NOT create any Injection-Date or Injection-Info particular, they MUST NOT create any Injection-Date or Injection-Info
header field. header field.
Contrary to [RFC 2822], which implies that the mailbox(es) in the Contrary to [RFC 2822], which implies that the mailbox(es) in the
From header field should be that of the poster(s), a poster who does From header field should be that of the poster(s), a poster who does
not, for whatever reason, wish to use his own mailbox MAY use any not, for whatever reason, wish to use his own mailbox MAY use any
mailbox ending in the top level domain ".invalid" [RFC 2606]. mailbox ending in the top level domain ".invalid" [RFC 2606].
Posting agents meant for use by ordinary posters SHOULD reject any Posting agents meant for use by ordinary posters SHOULD reject any
attempt to post an article which cancels or Supersedes another attempt to post an article which cancels or Supersedes another
article of which the poster is not the author or sender. article of which the poster is not the author or sender.
News Article Architecture and Protocols January 2006 6.6. Duties of a Followup Agent
7.6. Duties of a Followup Agent
A Followup Agent is a special case of a posting agent, and as such is A Followup Agent is a special case of a posting agent, and as such is
bound by all the posting agent's requirements. Followup agents MUST bound by all the posting agent's requirements. Followup agents MUST
create valid followups and are subject to special requirements create valid followups and are subject to special requirements
involving the Newsgroups, Subject, Distribution and References header involving the Newsgroups, Subject, Distribution and References header
fields. Wherever in the following it is stated that, "by default", a fields. Wherever in the following it is stated that, "by default", a
header field is to be "inherited" from one of those header fields in header field is to be "inherited" from one of those header fields in
the precursor, it means that its initial (semantic) content is to be the precursor, it means that its initial (semantic) content is to be
a copy of the content of that precursor header field. However, a copy of the content of that precursor header field. However,
posters MAY then override that default before posting if they so posters MAY then override that default before posting if they so
wish. wish.
News Article Architecture and Protocols November 2006
NOTE: The Keywords header field is not inheritable, though some NOTE: The Keywords header field is not inheritable, though some
older newsreaders treated it as such. older user agents treated it as such.
1. The Newsgroups header field (F-3.1.5) SHOULD by default be 1. The Newsgroups header field (F-3.1.4) SHOULD by default be
inherited from the precursor's Followup-To header field if inherited from the precursor's Followup-To header field if
present, and otherwise from the precursor's Newsgroups header present, and otherwise from the precursor's Newsgroups header
field. However, if the content of that Followup-To header field field. However, if the content of that Followup-To header field
consists of "poster" (and the user does not override it), then the consists of "poster" (and the user does not override it), then the
followup MUST NOT be posted but, rather, is to be emailed to the followup MUST NOT be posted but, rather, is to be emailed to the
precursor's poster. precursor's poster.
2. The Subject header field SHOULD by default be inherited from that 2. The Subject header field SHOULD by default be inherited from that
of the precursor. The case sensitive string "Re: " MAY be of the precursor. The case sensitive string "Re: " MAY be
prepended to the content of its Subject header field, unless it prepended to the content of its Subject header field, unless it
already begins with that string. already begins with that string.
3. The Distribution header field (F-3.2.7) SHOULD by default be 3. The Distribution header field (F-3.2.4) SHOULD by default be
inherited from the precursor's Distribution header field, if any. inherited from the precursor's Distribution header field, if any.
4. The followup MUST (in accordance with the definition of that term) 4. The followup MUST (in accordance with the definition of that term
have a References header field referring to its precursor, - F-1.5) have a References header field referring to its
constructed in accordance with section 7.6.1 below. precursor, constructed in accordance with section 6.6.1 below.
NOTE: That "MUST" is to be contrasted with the weaker NOTE: That "MUST" is to be contrasted with the weaker
recomendation using "SHOULD" applied, in [RFC 2822], to the recomendation using "SHOULD" applied, in [RFC 2822], to the
generation of "replies" in email. Moreover, in Netnews, there is generation of "replies" in email. Moreover, in Netnews, there is
no expectation of any In-Reply-To header field in a followup. no expectation of any In-Reply-To header field in a followup.
7.6.1. Construction of the References header field 6.6.1. Construction of the References header field
The following procedure is to be used whenever some previous article The following procedure is to be used whenever some previous article
(the "parent") is to be referred to in the References header field (the "parent") is to be referred to in the References header field
(F-3.2.2) of a new article, whether in the course of generating a (F-3.2.10) of a new article, whether in the course of generating a
followup or for some other reason (e.g. the later parts of a followup or for some other reason (e.g. the later parts of a
multipart posting such as a FAQ, or the later parts of a multipart posting such as a FAQ, or the later parts of a
message/partial as suggested in [RFC 2046]). message/partial as suggested in [RFC 2046]).
The (semantic) content of the new article's References header field The (semantic) content (2.1) of the new article's References header
consists of the content of the Message-ID header field of the parent field consists of the content of the Message-ID header field of the
preceded, if the parent had a References header field, by the content parent preceded, if the parent had a References header field, by the
of that References header field and a SP (subject to trimming as content of that References header field and a SP (subject to trimming
News Article Architecture and Protocols January 2006 as described below).
described below).
If the resulting References header field would, after unfolding, If the resulting References header field would, after unfolding,
exceed 998 characters in length (including its field name but not the exceed 998 characters in length (including its field name but not the
final CRLF), it MUST be trimmed (and otherwise it MAY be trimmed). final CRLF), it MUST be trimmed (and otherwise it MAY be trimmed).
Trimming involves removing any number of message identifiers from its Trimming involves removing any number of message identifiers from its
content, except that the first message identifier and the last two content, except that the first message identifier and the last two
MUST NOT be removed. MUST NOT be removed.
NOTE: There is no provision in this standard for an article to NOTE: There is no provision in this standard for an article to
have more than one parent. The essential property of the have more than one parent. The essential property of the
References header field, guaranteed by the procedure above and References header field, guaranteed by the procedure above and
to be preserved in any future extension, is that no article can to be preserved in any future extension, is that no article can
News Article Architecture and Protocols November 2006
ever precede one of its own parents. ever precede one of its own parents.
7.7. Duties of a Reading Agent 6.7. Duties of a Reading Agent
A reading agent downloads articles from a serving agent, as directed A reading agent downloads articles from a storage agent, as directed
by the reader, and displays them to the reader (or processes them in by the reader, and displays them to the reader (or processes them in
some other manner). It SHOULD also have the capability to show the some other manner). It SHOULD also have the capability to show the
raw article exactly as received. raw article exactly as received.
It MAY present lists of articles available for display, and MAY It MAY present lists of articles available for display, and MAY
structure those lists so as to show the relationships between the structure those lists so as to show the relationships between the
articles, as determined by the References, Subject, Date and other articles, as determined by the References, Subject, Date and other
header fields (see [USEAGE] for some usual methods of doing this). header fields (see [USEAGE] for some usual methods of doing this).
[This whole section may yet get omitted] [This whole section may yet get omitted]
7.8. Duties of a Moderator 6.8. Duties of a Moderator
A Moderator receives news articles, customarily by email, decides A Moderator receives news articles, customarily by email, decides
whether to approve them and, if so, either injects them into the news whether to approve them and, if so, either injects them into the news
stream or forwards them to further moderators. stream or forwards them to further moderators.
Articles will be received by the moderator either encapsulated as an Articles will be received by the moderator either encapsulated as an
object of Content-Type application/news-transmission (or possibly object of Content-Type application/news-transmission (or possibly
encapsulated but without an explicit Content-Type header field), or encapsulated but without an explicit Content-Type header field), or
else directly as an email already containing all the header fields else directly as an email already containing all the header fields
appropriate for a Netnews article (see 7.2.2). Moderators SHOULD be appropriate for a Netnews article (see 6.2.2). Moderators SHOULD be
prepared to accept articles in either format. prepared to accept articles in either format.
A moderator processes an article, as submitted to any newsgroup that A moderator processes an article, as submitted to any newsgroup that
he moderates, as follows: he moderates, as follows:
1. He decides, on the basis of whatever moderation policy applies to 1. He decides, on the basis of whatever moderation policy applies to
his group, whether to approve or reject the article. He MAY do his group, whether to approve or reject the article. He MAY do
this manually, or else partially or wholly with the aid of this manually, or else partially or wholly with the aid of
appropriate software for whose operation he is then responsible. appropriate software for whose operation he is then responsible.
If the article is a cancel nessage (6.3) issued by the poster of If the article is a cancel nessage (5.3) issued by the poster of
an earlier article, then he is expected to cancel that earlier an earlier article, then he is expected to cancel that earlier
article (in which case there is no more to be done). He MAY article (in which case there is no more to be done). He MAY
modify the article if that is in accordance with the applicable modify the article if that is in accordance with the applicable
moderation policy (and in particular he MAY remove redundant moderation policy (and in particular he MAY remove redundant
header fields and add Comments and other informational header header fields and add Comments and other informational header
News Article Architecture and Protocols January 2006
fields). He also needs to be aware if any change he makes to the fields). He also needs to be aware if any change he makes to the
article will invalidate some authentication check provided by the article will invalidate some authentication check provided by the
poster or by an earlier moderator. poster or by an earlier moderator.
If the article is rejected, then it normally fails for all the If the article is rejected, then it normally fails for all the
newsgroups for which it was intended. If it is approved, the newsgroups for which it was intended. If it is approved, the
moderator proceeds with the following steps. moderator proceeds with the following steps.
2. If the Newsgroups header field contains further moderated 2. If the Newsgroups header field contains further moderated
newsgroups for which approval has not already been given, he adds newsgroups for which approval has not already been given, he adds
an indication (identifying both himself and the name of the group) an indication (identifying both himself and the name of the group)
that he approves the article, and then forwards it to the that he approves the article, and then forwards it to the
moderator of the leftmost unapproved group (which, if this moderator of the leftmost unapproved group (which, if this
News Article Architecture and Protocols November 2006
standard has been followed correctly, will generally be the next standard has been followed correctly, will generally be the next
moderated group to the right of his own). There are two ways to do moderated group to the right of his own). There are two ways to do
this: this:
(a) He emails it to the submission address of the next moderator (a) He emails it to the submission address of the next moderator
(see section 7.2.2 for the proper method of doing this), or (see section 6.2.2 for the proper method of doing this), or
(b) he rotates the <newsgroup-name>s in the Newsgroups header (b) he rotates the <newsgroup-name>s in the Newsgroups header
field to the left so that the targeted group is the leftmost field to the left so that the targeted group is the leftmost
moderated group in that field, and injects it again (thus moderated group in that field, and injects it again (thus
causing the injecting agent to forward it to the correct causing the injecting agent to forward it to the correct
moderator). However, he MUST first ensure that the article moderator). However, he MUST first ensure that the article
contains no Approved header field. contains no Approved header field.
NOTE: This standard does not prescribe how a moderator's NOTE: This standard does not prescribe how a moderator's
approval is to be indicated (though a future standard may do approval is to be indicated (though a future standard may do
so). Possible methods include adding an Approved header field so). Possible methods include adding an Approved header field
(or a similar but differently named header field if method (b) (or a similar but differently named header field if method (b)
is being used) listing all the approvals made so far, or adding is being used) listing all the approvals made so far, or adding
a separate header field for each individual approval (the header a separate header field for each individual approval (the header
field X-Auth is sometimes used for this purpose). The approval field X-Auth is sometimes used for this purpose). The approval
may also be confirmed with some form of digital signature (6.1). may also be confirmed with some form of digital signature (5.1).
3. If the Newsgroups header field contains no further unapproved 3. If the Newsgroups header field contains no further unapproved
moderated groups, he adds an Approved header field (F-3.2.9) moderated groups, he adds an Approved header field (F-3.2.1)
identifying himself and, insofar as is possible, all the other identifying himself and, insofar as is possible, all the other
moderators who have approved the article. He thus assumes moderators who have approved the article. He thus assumes
responsibility for having ensured that the article was approved by responsibility for having ensured that the article was approved by
the moderators of all the moderated groups involved. the moderators of all the moderated groups involved.
4. The Date header field SHOULD be retained. Any Injection-Date 4. The Date header field SHOULD be retained. Any Injection-Date
header field already present (though there should be none) MUST be header field already present (though there should be none) MUST be
removed. Exceptionally, if it is known that the injecting agent removed. Exceptionally, if it is known that the injecting agent
does not yet support the Injection-Date header field and the Date does not yet support the Injection-Date header field and the Date
header field appears to be stale (F-3.2.1) for reasons understood header field appears to be stale (F-3.2.7) for reasons understood
by the moderator (e.g. delays in the moderation process) he MAY by the moderator (e.g. delays in the moderation process) he MAY
substitute the current date. The Message-ID header field SHOULD substitute the current date. The Message-ID header field SHOULD
also be retained unless it is obviously non-compliant with this also be retained unless it is obviously non-compliant with
standard. [USEFOR].
News Article Architecture and Protocols January 2006
NOTE: A message identifier created by a conforming posting or NOTE: A <msg-id-core> created by a conforming posting or
injecting agent, or even by a mail user agent conforming to [RFC injecting agent, or even by a mail user agent conforming to [RFC
2822], may reasonably be supposed to be conformant (and will, in 2822], may reasonably be supposed to be conformant (and will, in
any case, be caught by the injecting agent if it is not). any case, be caught by the injecting agent if it is not).
5. Any variant header fields (2.4) MUST be removed, except that a 5. Any variant header fields (2.4) MUST be removed, except that a
Path header field MAY be truncated to only those entries following Path header field MAY be truncated to only those entries following
its "POSTED" <path-keyword>. Any Injection-Info header field (F- its "POSTED" <diag-keyword>. Any Injection-Info header field (F-
3.2.14) SHOULD be removed (and if not, the injecting agent will do 3.2.8) SHOULD be removed (and if not, the injecting agent will do
so, as required in 7.2.2). so, as required in 6.2.2).
6. He then causes the article to be injected, having first observed 6. He then causes the article to be injected, having first observed
all the duties of a posting agent. all the duties of a posting agent.
News Article Architecture and Protocols November 2006
NOTE: This standard does not prescribe how the moderator or NOTE: This standard does not prescribe how the moderator or
moderation policy for each newsgroup is established; rather it moderation policy for each newsgroup is established; rather it
assumes that whatever agencies are responsible for the relevant assumes that whatever agencies are responsible for the relevant
network or hierarchy (1.1) will have made appropriate network or hierarchy (1.1) will have made appropriate
arrangements in that regard. arrangements in that regard.
7.9. Duties of a Gateway 6.9. Duties of a Gateway
A Gateway transforms an article into the native message format of A Gateway transforms an article into the native message format of
another medium, or translates the messages of another medium into another medium, or translates the messages of another medium into
news articles. Encapsulation of a news article into a message of MIME news articles. Encapsulation of a news article into a message of MIME
type application/news-transmission, or the subsequent undoing of that type application/news-transmission, or the subsequent undoing of that
encapsulation, is not gatewaying, since it involves no transformation encapsulation, is not gatewaying, since it involves no transformation
of the article. of the article.
There are two basic types of gateway, the Outgoing Gateway that There are two basic types of gateway, the Outgoing Gateway that
transforms a news article into a different type of message, and the transforms a news article into a different type of message, and the
skipping to change at page 40, line 5 skipping to change at page 39, line 48
below. below.
If bidirectional gatewaying (both an incoming and an outgoing If bidirectional gatewaying (both an incoming and an outgoing
gateway) is being set up between Netnews and some other medium, the gateway) is being set up between Netnews and some other medium, the
incoming and outgoing gateways SHOULD be coordinated to avoid incoming and outgoing gateways SHOULD be coordinated to avoid
unintended reinjection of gated articles. Circular gatewaying unintended reinjection of gated articles. Circular gatewaying
(gatewaying a message into another medium and then back into Netnews) (gatewaying a message into another medium and then back into Netnews)
SHOULD NOT be done; encapsulation of the article SHOULD be used SHOULD NOT be done; encapsulation of the article SHOULD be used
instead where this is necessary. instead where this is necessary.
News Article Architecture and Protocols January 2006
A second general principal of gatewaying is that the transformations A second general principal of gatewaying is that the transformations
applied to the message SHOULD be as minimal as possible while still applied to the message SHOULD be as minimal as possible while still
accomplishing the gatewaying. Every change made by a gateway accomplishing the gatewaying. Every change made by a gateway
potentially breaks a property of one of the media or loses potentially breaks a property of one of the media or loses
information, and therefore only those transformations made necessary information, and therefore only those transformations made necessary
by the differences between the media should be applied. by the differences between the media should be applied.
It is worth noting that safe bidirectional gatewaying between a It is worth noting that safe bidirectional gatewaying between a
mailing list and a newsgroup is far easier if the newsgroup is mailing list and a newsgroup is far easier if the newsgroup is
moderated. Posts to the moderated group and submissions to the moderated. Posts to the moderated group and submissions to the
mailing list can then go through a single point that does the mailing list can then go through a single point that does the
necessary gatewaying and then sends the message out to both the necessary gatewaying and then sends the message out to both the
newsgroup and the mailing list at the same time, eliminating most of newsgroup and the mailing list at the same time, eliminating most of
News Article Architecture and Protocols November 2006
the possibility of loops. Bidirectional gatewaying between a mailing the possibility of loops. Bidirectional gatewaying between a mailing
list and an unmoderated newsgroup, in contrast, is difficult to do list and an unmoderated newsgroup, in contrast, is difficult to do
correctly and is far more fragile. correctly and is far more fragile.
Newsgroups intended to be bidirectionally gated to a mailing list Newsgroups intended to be bidirectionally gated to a mailing list
SHOULD therefore be moderated where possible, even if the moderator SHOULD therefore be moderated where possible, even if the moderator
is a simple gateway and injecting agent that correctly handles is a simple gateway and injecting agent that correctly handles
crossposting to other moderated groups and otherwise passes all crossposting to other moderated groups and otherwise passes all
traffic. traffic.
7.9.1. Duties of an Outgoing Gateway 6.9.1. Duties of an Outgoing Gateway
From the perspective of Netnews, an outgoing gateway is just a From the perspective of Netnews, an outgoing gateway is just a
special type of reading agent. The exact nature of what the outgoing special type of reading agent. The exact nature of what the outgoing
gateway will need to do to articles depends on the medium to which gateway will need to do to articles depends on the medium to which
the articles are being gated. The operation of the outgoing gateway the articles are being gated. The operation of the outgoing gateway
is subject to additional constraints due to the possibility of one or is subject to additional constraints due to the possibility of one or
more corresponding incoming gateways back from that medium to more corresponding incoming gateways back from that medium to
Netnews, since this opens the possibility of loops. Netnews, since this opens the possibility of loops.
In general, the following practices are recommended for all outgoing In general, the following practices are recommended for all outgoing
skipping to change at page 41, line 5 skipping to change at page 40, line 46
2. The Date and Injection-Date of the news article should also be 2. The Date and Injection-Date of the news article should also be
preserved if possible, for similar reasons. preserved if possible, for similar reasons.
3. The message should be tagged in some way so as to prevent its 3. The message should be tagged in some way so as to prevent its
reinjection into Netnews. This may be impossible to do without reinjection into Netnews. This may be impossible to do without
knowledge of potential incoming gateways, but it is better to try knowledge of potential incoming gateways, but it is better to try
to provide some indication even if not successful; at the least, a to provide some indication even if not successful; at the least, a
human-readable indication that the article should not be gated human-readable indication that the article should not be gated
back to Netnews can help locate a human problem. back to Netnews can help locate a human problem.
News Article Architecture and Protocols January 2006
4. Netnews control messages should not be gated to another medium 4. Netnews control messages should not be gated to another medium
unless they would somehow be meaningful in that medium. unless they would somehow be meaningful in that medium.
5. Changes MAY be made to the Content-Transfer-Encoding of some or 5. Changes MAY be made to the Content-Transfer-Encoding of some or
all parts of the body, and even to the charsets specified in all parts of the body, and even to the charsets specified in
<encoded-word>s or in Content-Type header fields, but such changes <encoded-word>s or in Content-Type header fields, but such changes
SHOULD NOT be made unless absolutely necessary. SHOULD NOT be made unless absolutely necessary.
7.9.2. Duties of an Incoming Gateway 6.9.2. Duties of an Incoming Gateway
The incoming gateway has the serious responsibility of ensuring that The incoming gateway has the serious responsibility of ensuring that
all of the requirements of this standard are met by the articles that all of the requirements of this standard are met by the articles that
it forms. In addition to its special duties as a gateway, it bears it forms. In addition to its special duties as a gateway, it bears
all of the duties and responsibilities of an injecting agent as well, all of the duties and responsibilities of an injecting agent as well,
News Article Architecture and Protocols November 2006
and additionally has the same responsibility of a relaying agent to and additionally has the same responsibility of a relaying agent to
reject articles that it has already gatewayed. reject articles that it has already gatewayed.
An incoming gateway MUST NOT gate the same message twice. It may not An incoming gateway MUST NOT gate the same message twice. It may not
be possible to ensure this in the face of mangling or modification of be possible to ensure this in the face of mangling or modification of
the message, but at the very least a gateway, when given a copy of a the message, but at the very least a gateway, when given a copy of a
message it has already gated identical except for trace header fields message it has already gated identical except for trace header fields
(like Received in Email or Path in Netnews) MUST NOT gate the message (like Received in Email or Path in Netnews) MUST NOT gate the message
again. An incoming gateway SHOULD take precautions against having again. An incoming gateway SHOULD take precautions against having
this rule bypassed by modifications of the message that can be this rule bypassed by modifications of the message that can be
skipping to change at page 42, line 4 skipping to change at page 41, line 46
Incoming gateways MUST NOT inject control messages other than Incoming gateways MUST NOT inject control messages other than
cancels. Encapsulation SHOULD be used instead of gatewaying, when cancels. Encapsulation SHOULD be used instead of gatewaying, when
direct posting is not possible or desirable. direct posting is not possible or desirable.
NOTE: It is not unheard of for mail-to-news gateways to be used NOTE: It is not unheard of for mail-to-news gateways to be used
to post control messages, but encapsulation should be used for to post control messages, but encapsulation should be used for
these cases instead. Gateways by their very nature are these cases instead. Gateways by their very nature are
particularly prone to loops. Spews of normal articles are bad particularly prone to loops. Spews of normal articles are bad
enough; spews of control messages with special significance to enough; spews of control messages with special significance to
the news system, possibly resulting in high processing load or the news system, possibly resulting in high processing load or
News Article Architecture and Protocols January 2006
even email sent for every message received, are catastrophic. It even email sent for every message received, are catastrophic. It
is far preferable to construct a system specifically for posting is far preferable to construct a system specifically for posting
control messages that can do appropriate consistency checks and control messages that can do appropriate consistency checks and
authentication of the originator of the message. authentication of the originator of the message.
If there is a message identifier that fills a role similar to that of If there is a message identifier that fills a role similar to that of
the Message-ID header field in news, it SHOULD be used in the the Message-ID header field in news, it SHOULD be used in the
formation of the message identifier of the news article, perhaps with formation of the message identifier of the news article, perhaps with
transformations required to meet the uniqueness requirement of transformations required to meet the uniqueness requirement of
Netnews and with the removal of any comments so as to comply with the Netnews and with the removal of any comments so as to comply with the
syntax in section F-3.1.3. Such transformations SHOULD be designed so syntax in section F-3.1.3. Such transformations SHOULD be designed so
that two messages with the same identifier generate the same that two messages with the same identifier generate the same
Message-ID header field. Message-ID header field.
News Article Architecture and Protocols November 2006
NOTE: Message identifiers play a central role in the prevention NOTE: Message identifiers play a central role in the prevention
of duplicates, and their correct use by gateways will do much to of duplicates, and their correct use by gateways will do much to
prevent loops. Netnews does, however, require that message prevent loops. Netnews does, however, require that message
identifiers be unique, and therefore message identifiers from identifiers be unique, and therefore message identifiers from
other media may not be suitable for use without modification. A other media may not be suitable for use without modification. A
balance must be struck by the gateway between preserving balance must be struck by the gateway between preserving
information used to prevent loops and generating unique message information used to prevent loops and generating unique message
identifiers. identifiers.
Exceptionally, if there are multiple incoming gateways for a Exceptionally, if there are multiple incoming gateways for a
skipping to change at page 43, line 4 skipping to change at page 42, line 47
information is available (e.g. date but not time), this SHOULD be information is available (e.g. date but not time), this SHOULD be
fleshed out to a full Date and/or Injection-Date header field by fleshed out to a full Date and/or Injection-Date header field by
adding default values rather than discarding this information. Only adding default values rather than discarding this information. Only
in very exceptional circumstances should Date information be in very exceptional circumstances should Date information be
discarded, as it plays an important role in preventing reinjection of discarded, as it plays an important role in preventing reinjection of
old messages. old messages.
An incoming gateway MUST add a Sender header field to the news An incoming gateway MUST add a Sender header field to the news
article it forms containing the <mailbox> of the administrator of the article it forms containing the <mailbox> of the administrator of the
gateway. Problems with the gateway may be reported to this gateway. Problems with the gateway may be reported to this
News Article Architecture and Protocols January 2006
<mailbox>. The <display-name> portion of this <mailbox> SHOULD <mailbox>. The <display-name> portion of this <mailbox> SHOULD
indicate that the entity responsible for injection of the message is indicate that the entity responsible for injection of the message is
a gateway. If the original message already had a Sender header field, a gateway. If the original message already had a Sender header field,
it SHOULD be renamed so that its contents can be preserved. it SHOULD be renamed so that its contents can be preserved.
7.9.3. Example 6.9.3. Example
To illustrate the type of precautions that should be taken against To illustrate the type of precautions that should be taken against
loops, here is an example of the measures taken by one particular loops, here is an example of the measures taken by one particular
combination of mail-to-news and news-to-mail gateways at Stanford combination of mail-to-news and news-to-mail gateways at Stanford
University designed to handle bidirectional gatewaying between University designed to handle bidirectional gatewaying between
mailing lists and unmoderated groups. mailing lists and unmoderated groups.
News Article Architecture and Protocols November 2006
1. The news-to-mail gateway preserves the message identifier of the 1. The news-to-mail gateway preserves the message identifier of the
news article in the generated email message. The mail-to-news news article in the generated email message. The mail-to-news
gateway likewise preserves the email message identifier provided gateway likewise preserves the email message identifier provided
that it is syntactically valid for Netnews. This allows the news that it is syntactically valid for Netnews. This allows the news
system's built-in suppression of duplicates to serve as the first system's built-in suppression of duplicates to serve as the first
line of defense against loops. line of defense against loops.
2. The news-to-mail gateway adds an X-Gateway header field to all 2. The news-to-mail gateway adds an X-Gateway header field to all
messages it generates. The mail-to-news gateway discards any messages it generates. The mail-to-news gateway discards any
incoming messages containing this header field. This is robust incoming messages containing this header field. This is robust
skipping to change at page 44, line 5 skipping to change at page 43, line 44
handled by the news administrators. handled by the news administrators.
These precautions have proven effective in practice at preventing These precautions have proven effective in practice at preventing
loops for this particular application (bidirectional gatewaying loops for this particular application (bidirectional gatewaying
between mailing lists and locally distributed newsgroups where both between mailing lists and locally distributed newsgroups where both
gateways can be designed together). General gatewaying to world-wide gateways can be designed together). General gatewaying to world-wide
newsgroups poses additional difficulties; one must be very wary of newsgroups poses additional difficulties; one must be very wary of
strange configurations, such as a newsgroup gated to a mailing list strange configurations, such as a newsgroup gated to a mailing list
which is in turn gated to a different newsgroup. which is in turn gated to a different newsgroup.
News Article Architecture and Protocols January 2006 7. Security and Related Considerations
8. Security and Related Considerations
There is no security. Don't fool yourself. Usenet is a prime example There is no security. Don't fool yourself. Usenet is a prime example
of an Internet Adhocratic-Anarchy; that is, an environment in which of an Internet Adhocratic-Anarchy; that is, an environment in which
trust forms the basis of all agreements. It works. trust forms the basis of all agreements. It works.
See also F-5 for further security considerations related to the See also F-5 for further security considerations related to the
format of articles. format of articles.
[And a similar pointer from there to here might be in order.]
8.1. Leakage 7.1. Leakage
Articles which are intended to have restricted distribution are Articles which are intended to have restricted distribution are
dependent on the goodwill of every site receiving them. The dependent on the goodwill of every site receiving them. The
"Archive: no" header field (F-3.2.12) is available as a signal to "Archive: no" header field (F-3.2.2) is available as a signal to
automated archivers not to file an article, but that cannot be automated archivers not to file an article, but that cannot be
guaranteed. guaranteed.
News Article Architecture and Protocols November 2006
The Distribution header field makes provision for articles which The Distribution header field makes provision for articles which
should not be propagated beyond a cooperating subnet. The key should not be propagated beyond a cooperating subnet. The key
security word here is "cooperating". When a machine is not configured security word here is "cooperating". When a machine is not configured
properly, it may become uncooperative and tend to distribute all properly, it may become uncooperative and tend to distribute all
articles. articles.
The flooding algorithm is extremely good at finding any path by which The flooding algorithm is extremely good at finding any path by which
articles can leave a subnet with supposedly restrictive boundaries, articles can leave a subnet with supposedly restrictive boundaries,
and substantial administrative effort is required to avoid this. and substantial administrative effort is required to avoid this.
Organizations wishing to control such leakage are strongly advised to Organizations wishing to control such leakage are strongly advised to
designate a small number of official gateways to handle all news designate a small number of official gateways to handle all news
exchange with the outside world (however, making such gateways too exchange with the outside world (however, making such gateways too
restrictive can also encourage the setting up of unofficial paths restrictive can also encourage the setting up of unofficial paths
which can be exceedingly hard to track down). which can be exceedingly hard to track down).
The sendme control message (6.4), insofar as it is still used, can be The sendme control message (5.4), insofar as it is still used, can be
used to request articles with a given message identifier, even one used to request articles with a given message identifier, even one
that is not supposed to be supplied to the requestor. that is not supposed to be supplied to the requestor.
8.2. Attacks 7.2. Attacks
8.2.1. Denial of Service 7.2.1. Denial of Service
The proper functioning of individual newsgroups can be disrupted by The proper functioning of individual newsgroups can be disrupted by
the massive posting of "noise" articles, by the repeated posting of the massive posting of "noise" articles, by the repeated posting of
identical or near identical articles, by posting followups unrelated identical or near identical articles, by posting followups unrelated
to their precursors, or which quote their precursors in full with the to their precursors, or which quote their precursors in full with the
addition of minimal extra material (especially if this process is addition of minimal extra material (especially if this process is
iterated), and by crossposting to, or setting followups to, totally iterated), and by crossposting to, or setting followups to, totally
unrelated newsgroups. unrelated newsgroups.
Many have argued that "spam", massively multiposted (and to a lesser Many have argued that "spam", massively multiposted (and to a lesser
extent massively crossposted) articles, usually for advertising extent massively crossposted) articles, usually for advertising
purposes, also constitutes a DoS attack in its own regard. This may purposes, also constitutes a DoS attack in its own regard. This may
be so. be so.
News Article Architecture and Protocols January 2006
Such articles intended to deny service, or other articles of an Such articles intended to deny service, or other articles of an
inflammatory nature, may also have their From or Reply-To addresses inflammatory nature, may also have their From or Reply-To addresses
set to valid but incorrect email addresses, thus causing large set to valid but incorrect email addresses, thus causing large
volumes of email to descend on the true owners of those addresses. volumes of email to descend on the true owners of those addresses.
Similar effects could be caused by any email header field which could Similar effects could be caused by any email header field which could
cause every reading agent receiving it to take some externally cause every reading agent receiving it to take some externally
visible action. For example, the Disposition-Notification-To header visible action. For example, the Disposition-Notification-To header
field defined in [RFC 2298] could cause huge numbers of field defined in [RFC 2298] could cause huge numbers of
acknowledgements to be emailed to an unsuspecting third party (for acknowledgements to be emailed to an unsuspecting third party (for
which reason [RFC 2298] declares that that header field SHOULD NOT be which reason [RFC 2298] declares that that header field SHOULD NOT be
used in Netnews). used in Netnews).
It is a violation of this standard for a poster to use as his address It is a violation of this standard for a poster to use as his address
a <mailbox> which he is not entitled to use. Even addresses with an a <mailbox> which he is not entitled to use. Even addresses with an
invalid <local-part> but a valid <domain> can cause disruption to the invalid <local-part> but a valid <domain> can cause disruption to the
administrators of such domains. Posters who wish to remain anonymous administrators of such domains. Posters who wish to remain anonymous
or to prevent automated harvesting of their addresses, but who do not or to prevent automated harvesting of their addresses, but who do not
News Article Architecture and Protocols November 2006
care to take the additional precautions of using more sophisticated care to take the additional precautions of using more sophisticated
anonymity measures, should avoid that violation by the use of anonymity measures, should avoid that violation by the use of
addresses ending in the ".invalid" top-level-domain (see 7.5). addresses ending in the ".invalid" top-level-domain (see 6.5).
A malicious poster may also prevent his article being seen at a A malicious poster may also prevent his article being seen at a
particular site by preloading that site into the Path header field particular site by preloading that site into the Path header field
(F-3.1.6) and may thus prevent the true owner of a forged From or (F-3.1.5) and may thus prevent the true owner of a forged From or
Reply-To address from ever seeing it. Reply-To address from ever seeing it.
A malicious complainer may submit a modified copy of an article (e.g. A malicious complainer may submit a modified copy of an article (e.g.
with an altered Injection-Info header field) to the administrator of with an altered Injection-Info header field) to the administrator of
an injecting agent in an attempt to discredit the author of that an injecting agent in an attempt to discredit the author of that
article and even to have his posting privileges removed. article and even to have his posting privileges removed.
Administrators should therefore obtain a genuine copy of the article Administrators should therefore obtain a genuine copy of the article
from their own serving agent before taking such precipitate action. from their own storage agent before taking such precipitate action.
Administrative agencies with responsibility for establishing policies Administrative agencies with responsibility for establishing policies
in particular hierarchies can and should set bounds upon the in particular hierarchies can and should set bounds upon the
behaviour that is considered acceptable within those hierarchies (for behaviour that is considered acceptable within those hierarchies (for
example by promulgating charters for individual newsgroups, and other example by promulgating charters for individual newsgroups, and other
codes of conduct). codes of conduct).
Whilst this standard places an onus upon injecting agents to bear Whilst this standard places an onus upon injecting agents to bear
responsibility for the misdemeanours of their posters (which includes responsibility for the misdemeanours of their posters (which includes
non-adherence to established policies of the relevant hierarchies as non-adherence to established policies of the relevant hierarchies as
provided in section 7.2), and to provide assistance to the rest of provided in section 6.2), and to provide assistance to the rest of
the network by making proper use of the Injection-Info (F-3.2.14) the network by making proper use of the Injection-Info (F-3.2.8)
header field, it makes no provision for enforcement, which may in header field, it makes no provision for enforcement, which may in
consequence be patchy. Nevertheless, injecting sites which consequence be patchy. Nevertheless, injecting sites which
persistently fail to honour their responsibilities or to comply with persistently fail to honour their responsibilities or to comply with
generally accepted standards of behaviour are likely to find generally accepted standards of behaviour are likely to find
themselves blacklisted, with their articles refused propagation and themselves blacklisted, with their articles refused propagation and
even subject to cancellation, and other relaying sites would be well even subject to cancellation, and other relaying sites would be well
advised to withdraw peering arrangements from them. advised to withdraw peering arrangements from them.
News Article Architecture and Protocols January 2006 7.2.2. Compromise of System Integrity
8.2.2. Compromise of System Integrity
The posting of unauthorized (as determined by the policies of the The posting of unauthorized (as determined by the policies of the
relevant hierarchy) control messages can cause unwanted newsgroups to relevant hierarchy) control messages can cause unwanted newsgroups to
be created, or wanted ones removed, from serving agents. be created, or wanted ones removed, from storage agents.
Administrators of such agents SHOULD therefore take steps to verify Administrators of such agents SHOULD therefore take steps to verify
the authenticity of such control messages, either by manual the authenticity of such control messages, either by manual
inspection (particularly of the Approved header field) or by checking inspection (particularly of the Approved header field) or by checking
any digital signatures that may be provided (see 6.1). In addition, any digital signatures that may be provided (see 5.1). In addition,
they SHOULD periodically compare the newsgroups carried against any they SHOULD periodically compare the newsgroups carried against any
regularly issued checkgroups messages, or against lists maintained by regularly issued checkgroups messages, or against lists maintained by
trusted servers and accessed by out-of-band protocols such as FTP or trusted servers and accessed by out-of-band protocols such as FTP or
HTTP. HTTP.
Malicious cancel messages (6.3) can cause valid articles to be Malicious cancel messages (5.3) can cause valid articles to be
removed from serving agents. Administrators of such agents SHOULD removed from storage agents. Administrators of such agents SHOULD
therefore take steps to verify that they originated from the therefore take steps to verify that they originated from the
(apparent) poster, the injector or the moderator of the article, or (apparent) poster, the injector or the moderator of the article, or
that in other cases they came from a place that is trusted to work that in other cases they came from a place that is trusted to work
News Article Architecture and Protocols November 2006
within established policies and customs. Such steps SHOULD include within established policies and customs. Such steps SHOULD include
the checking of any digital signatures, or other security devices, the checking of any digital signatures, or other security devices,
that may be provided (see 6.1). Articles containing Supersedes that may be provided (see 5.1). Articles containing Supersedes
header fields (F-3.2.6) are effectively cancel messages, and SHOULD header fields (F-3.2.12) are effectively cancel messages, and SHOULD
be subject to the same checks. Currently, many sites choose to be subject to the same checks. Currently, many sites choose to
ignore all cancel messages on account of the difficulty of conducting ignore all cancel messages on account of the difficulty of conducting
such checks. such checks.
Improperly configured serving agents can allow articles posted to Improperly configured injecting and relaying agents can allow
moderated groups onto the net without first being approved by the articles posted to moderated groups onto the net without first being
moderator. Injecting agents SHOULD verify that moderated articles approved by the moderator. Injecting agents SHOULD verify that
were received from one of the entities given in their Approved header moderated articles were received from one of the entities given in
fields and/or check any digital signatures that may be provided (see their Approved header fields and/or check any digital signatures that
6.1). may be provided (see 5.1).
There may be weaknesses in particular implementations that are There may be weaknesses in particular implementations that are
subject to malicious exploitation. In particular, it has not been subject to malicious exploitation. In particular, it has not been
unknown for complete shell scripts to be included within Control unknown for complete shell scripts to be included within Control
header fields. Implementors need to be aware of this. header fields. Implementors need to be aware of this.
Reading agents should be chary of acting automatically upon MIME Reading agents should be chary of acting automatically upon MIME
objects with an "application" Content-Type that could change the objects with an "application" Content-Type that could change the
state of that agent, except in contexts where such applications are state of that agent, except in contexts where such applications are
specifically expected (as in 5). Even the Content-Type "text/html" specifically expected (as in 4). Even the Content-Type "text/html"
could have unexpected side effects on account of embedded objects, could have unexpected side effects on account of embedded objects,
especially embedded executable code or URIs that invoke non-news especially embedded executable code or URIs that invoke non-news
protocols such as HTTP [RFC 2616]. It is therefore generally protocols such as HTTP [RFC 2616]. It is therefore generally
recommended that reading agents do not enable the execution of such recommended that reading agents do not enable the execution of such
code (since it is extremely unlikely to have a valid application code (since it is extremely unlikely to have a valid application
within Netnews) and that they only honour URIs referring to other within Netnews) and that they only honour URIs referring to other
parts of the same article. parts of the same article.
Non-printable characters embedded in article bodies may have Non-printable characters embedded in article bodies may have
surprising effects on printers or terminals, notably by reconfiguring surprising effects on printers or terminals, notably by reconfiguring
them in undesirable ways which may become apparent only after the them in undesirable ways which may become apparent only after the
News Article Architecture and Protocols January 2006
reading agent has terminated. reading agent has terminated.
8.3. Liability 7.3. Liability
[This whole section might be better removed to [USEAGE].]
There is a presumption that a poster who sends an article to Usenet There is a presumption that a poster who sends an article to Usenet
intends it to be stored on a multitude of serving agents, and has intends it to be stored on a multitude of storage agents, and has
therefore given permission for it to be copied to that extent. therefore given permission for it to be copied to that extent.
Nevertheless, Usenet is not exempt from the Copyright laws, and it Nevertheless, Usenet is not exempt from the Copyright laws, and it
should not be assumed that permission has been given for the article should not be assumed that permission has been given for the article
to be copied outside of Usenet, nor for its permanent archiving to be copied outside of Usenet, nor for its permanent archiving
contrary to any Archive header field that may be present. contrary to any Archive header field that may be present.
Posters also need to be aware that they are responsible if they Posters also need to be aware that they are responsible if they
breach Copyright, Libel, Harassment or other restrictions relating to breach Copyright, Libel, Harassment or other restrictions relating to
material that they post, and that they may possibly find themselves material that they post, and that they may possibly find themselves
liable for such breaches in jurisdictions far from their own. Serving liable for such breaches in jurisdictions far from their own. Serving
agents may also be liable in some jurisdictions, especially if the agents may also be liable in some jurisdictions, especially if the
News Article Architecture and Protocols November 2006
breach has been explicitly drawn to their attention. breach has been explicitly drawn to their attention.
Users who are concerned about such matters should seek advice from Users who are concerned about such matters should seek advice from
competent legal authorities. competent legal authorities.
9. IANA Considerations 8. IANA Considerations
IANA is requested to register the following media types, described IANA is requested to register the following media types, described
elsewhere in this standard, for use with the Content-Type header elsewhere in this standard, for use with the Content-Type header
field, in the IETF tree in accordance with the procedures set out in field, in the IETF tree in accordance with the procedures set out in
[RFC 2048]. [RFC 4288].
application/news-transmission (5.1) application/news-transmission (4.1)
application/news-groupinfo (5.3) application/news-groupinfo (4.3)
application/news-checkgroups (5.4) application/news-checkgroups (4.4)
IANA is also requested to change the status of the following media IANA is also requested to change the status of the following media
type to "OBSOLETE". type to "OBSOLETE".
message/news (5.2) message/news (4.2)
NOTE: "Application/news-transmission" is an update, with NOTE: "Application/news-transmission" is an update, with
clarification and additional optional parameters, to an existing clarification and additional optional parameters, to an existing
registration. "Message/rfc822" should now be used in place of registration. "Message/rfc822" should now be used in place of
the obsoleted "message/news". the obsoleted "message/news". nr H1 7
10. References 9. References
10.1. Normative References 9.1. Normative References
[ANSI X3.4] "American National Standard for Information Systems - [ANSI X3.4] "American National Standard for Information Systems -
Coded Character Sets - 7-Bit American National Standard Code for Coded Character Sets - 7-Bit American National Standard Code for
Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986. Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986.
News Article Architecture and Protocols January 2006
[RFC 2048] N. Freed, J. Klensin, and J. Postel, "Multipurpose
Internet Mail Extensions (MIME) Part Four: Registration
Procedures", RFC 2048, November 1996.
[RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate [RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, March 1997. Requirement Levels", RFC 2119, March 1997.
[RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names", [RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names",
RFC 2606, June 1999. RFC 2606, June 1999.
[RFC 2822] P. Resnick, "Internet Message Format", RFC 2822, April [RFC 2822] P. Resnick, "Internet Message Format", RFC 2822, April
2001. 2001.
[RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration [RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration
procedures for message header fields", RFC 3864. procedures for message header fields", RFC 3864.
[USEAGE] draft-ietf-usefor-useage-*.txt.
[USEFOR] K. Murchison et al, "News Article Format", draft-ietf- [USEFOR] K. Murchison et al, "News Article Format", draft-ietf-
usefor-usefor-*.txt. usefor-usefor-11.txt.
[USEPRO] This Standard. [USEPRO] This Standard.
10.2. Informative References News Article Architecture and Protocols November 2006
[NNTP] Clive D.W. Feather, "Network News Transport Protocol", draft- 9.2. Informative References
ietf-nntpext-base-*.txt.
[PGPVERIFY] David Lawrence, [PGPVERIFY] David Lawrence,
<ftp://ftp.isc.org/pub/pgpcontrol/README.html>. <ftp://ftp.isc.org/pub/pgpcontrol/README.html>.
[RFC 1036] M. Horton and R. Adams, "Standard for Interchange of [RFC 1036] M. Horton and R. Adams, "Standard for Interchange of
USENET Messages", RFC 1036, December 1987. USENET Messages", RFC 1036, December 1987.
[RFC 2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail [RFC 2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail
Extensions (MIME) Part One: Format of Internet Message Bodies", Extensions (MIME) Part One: Format of Internet Message Bodies",
RFC 2045, November 1996. RFC 2045, November 1996.
skipping to change at page 49, line 5 skipping to change at page 48, line 33
[RFC 2142] D. Crocker, "Mailbox Names for Common Services, Roles and [RFC 2142] D. Crocker, "Mailbox Names for Common Services, Roles and
Functions", RFC 2142, May 1997. Functions", RFC 2142, May 1997.
[RFC 2298] R. Fajman, "An Extensible Message Format for Message [RFC 2298] R. Fajman, "An Extensible Message Format for Message
Disposition Notifications", RFC 2298, March 1998. Disposition Notifications", RFC 2298, March 1998.
[RFC 2616] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, [RFC 2616] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter,
P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol -- P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol --
HTTP/1.1", RFC 2616, June 1999. HTTP/1.1", RFC 2616, June 1999.
News Article Architecture and Protocols January 2006
[RFC 2821] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer [RFC 2821] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer
Protocol", RFC 2821, April 2001. Protocol", RFC 2821, April 2001.
[RFC 3977] C. Feather, "Network News Transport Protocol (NNTP)", RFC
3977.
[RFC 4288] N. Freed and J. Klensin, "Media Type Specifications and
Registration Procedures", RFC 4288, December 2005.
[RFC 976] Mark R. Horton, "UUCP mail interchange format standard", [RFC 976] Mark R. Horton, "UUCP mail interchange format standard",
RFC 976, February 1986. RFC 976, February 1986.
[Son-of-1036] Henry Spencer, "News article format and transmission", [Son-of-1036] Henry Spencer, "News article format and transmission",
<ftp://ftp.zoo.toronto.edu/pub/news.txt.Z>, June 1994. <ftp://ftp.zoo.toronto.edu/pub/news.txt.Z>, June 1994.
11. Acknowledgements [USEAGE] draft-ietf-usefor-useage-*.txt.
As this document is the result of an eight year effort, the number of 10. Acknowledgements
As this document is the result of a ten year effort, the number of
people that have contributed to its content are too numerous to people that have contributed to its content are too numerous to
mention individually. Many thanks go out to all past and present mention individually. Many thanks go out to all past and present
members of the USEFOR Working Group of the Internet Engineering Task members of the USEFOR Working Group of the Internet Engineering Task
Force (IETF) and the accompanying mailing list. Force (IETF) and the accompanying mailing list.
12. Contact Address News Article Architecture and Protocols November 2006
11. Contact Address
Editor Editor
Charles. H. Lindsey Charles. H. Lindsey
5 Clerewood Avenue 5 Clerewood Avenue
Heald Green Heald Green
Cheadle Cheadle
Cheshire SK8 3JU Cheshire SK8 3JU
United Kingdom United Kingdom
Phone: +44 161 436 6131 Phone: +44 161 436 6131
skipping to change at page 49, line 53 skipping to change at page 49, line 36
] ]
Comments on this draft should preferably be sent to the mailing list Comments on this draft should preferably be sent to the mailing list
of the Usenet Format Working Group at of the Usenet Format Working Group at
ietf-usefor@imc.org. ietf-usefor@imc.org.
Appendix A - Obsolete Control Messages Appendix A - Obsolete Control Messages
This present standard obsoletes certain control messages defined in This present standard obsoletes certain control messages defined in
[RFC 1036] (see 6.5), all of which had the effect of requesting a [RFC 1036] (see 5.5), all of which had the effect of requesting a
description of a relaying or serving agent's software, or its peering description of a relaying or storage agent's software, or its peering
arrangements with neighbouring sites, to be emailed to the article's arrangements with neighbouring sites, to be emailed to the article's
reply address. Whilst of some utility when Usenet was much smaller reply address. Whilst of some utility when Usenet was much smaller
than it is now, they had become no more than a tool for the malicious than it is now, they had become no more than a tool for the malicious
sending of mailbombs. Moreover, many organizations now consider sending of mailbombs. Moreover, many organizations now consider
information about their internal connectivity to be confidential. information about their internal connectivity to be confidential.
News Article Architecture and Protocols January 2006
version version
sendsys sendsys
whogets whogets
senduuname senduuname
"Version" requested details of the transport software in use at a "Version" requested details of the transport software in use at a
site. "Sendsys" requested the full list of newsgroups taken, and the site. "Sendsys" requested the full list of newsgroups taken, and the
peering arrangements. "Whogets" was similar, but restricted to a peering arrangements. "Whogets" was similar, but restricted to a
named newsgroup. "Senduuname" resembled "sendsys" but restricted to named newsgroup. "Senduuname" resembled "sendsys" but restricted to
the list of peers connected by UUCP. the list of peers connected by UUCP.
Historically, a checkgroups body consisting of one or two lines, the Historically, a checkgroups body consisting of one or two lines, the
first of the form "-n newsgroup", caused check-groups to apply to first of the form "-n newsgroup", caused check-groups to apply to
only that single newsgroup. only that single newsgroup.
News Article Architecture and Protocols November 2006
Historically, an article posted to a newsgroup whose name had exactly Historically, an article posted to a newsgroup whose name had exactly
three components of which the third was "ctl" signified that article three components of which the third was "ctl" signified that article
was to be taken as a control message. The Subject header field was to be taken as a control message. The Subject header field
specified the actions, in the same way the Control header field does specified the actions, in the same way the Control header field does
now. now.
These forms are documented for archaeological purposes only; they These forms are documented for archaeological purposes only; they
MUST NO LONGER be used. MUST NO LONGER be used.
Appendix B - Notices Appendix B - Differences from the Protocols in RFC 1036 and its
derivatives
This apendix contains a list of changes that have been made to the
protocols of the earlier standards, specifically [RFC 1036]. See F-
Appendix B. for the related syntax changes.
o There is a new Control message 'mvgroup' to facilitate moving a
group to a different place (name) in a hierarchy.
o Certain Control messages (Appendix A) have been made obsolete,
and the special significance of "cmsg" when at the start of a
Subject header field has been removed (section 5).
o Additional media types are defined for better structuring of
control messages (4.3 and 4.4).
o Distributions are expected to be checked at the receiving end, as
well as the sending end, of a relaying link.
o There are numerous other small changes, clarifications and
enhancements.
Appendix C - Transitional Arrangements
It is the intention that the new features of [USEFOR] and of this
standard can be assimilated into Usenet as it presently operates
without major interruption to the service, though some of the new
features may not begin to show benefit until they become widely
implemented.
An important distinction must be made between news servers, which are
responsible for the distribution and storage of news articles, and
user agents, which are responsible for interactions with users. It is
important that the former should be upgraded to conform to this
standard as soon as possible to provide the benefit of the enhanced
facilities. Fortunately, the number of distinct implementations of
such servers is rather small, at least so far as the main "backbone"
of Usenet is concerned, and many of the new features are already
supported. Contrariwise, there are a great number of implementations
of user agents, installed on a vastly greater number of small sites.
Therefore, the new functionality has been designed so that existing
user agents may continue to be used, although the full benefits may
not be realised until a substantial proportion of them have been
upgraded.
In the list which follows, care has been taken to distinguish the
implications for both kinds of agent.
News Article Architecture and Protocols November 2006
o The introduction of whitespace and folding into the Newsgroups
and related header fields (F-3.1.4, F-3.2.4, F-3.2.6) and of
<comment>s into the References header field (F-3.2.10) will cause
problems for existing agents, and [USEFOR] provides that they
SHOULD NOT be generated at the present time.
o The requirement in [USEFOR] to support MIME reflects a practice
that is already widespread. Articles in strict compliance with
the previous standards (using strict US-ASCII) will be
unaffected.
o All the header fields newly introduced by [USEFOR] can safely be
ignored by existing software, albeit with loss of the new
functionality.
o The new style of Path header field, using a <diag-match> to allow
"!!" as a <delimiter> and introducing <path-diagnostic>s (which
can be distinguished from <path-identity>s by their leading ".")
is already consistent with the previous standards. However, the
intention is that relaying agents should eventually reject
articles in the old style, and so this possibility should be
offered as a configurable option in relaying agents. User agents
are unaffected.
o The new Control: mvgroup command will need to be implemented in
storage agents. For the benefit of older storage agents it is
therefore RECOMMENDED that it be followed shortly by a
corresponding newgroup command and it MUST always be followed by
a rmgroup command for the old group after a reasonable overlap
period. An implementation of the mvgroup command as an alias for
the newgroup command would thus be minimally conforming. User
agents are unaffected.
o Provision is made for relaying and storage agents to use the Date
header field in the case of articles injected through existing
agents which do not yet provide an Injection-Date header field.
Appendix D - Notices
Intellectual Property Intellectual Property
The IETF takes no position regarding the validity or scope of any The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79. found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr. http://www.ietf.org/ipr.
News Article Architecture and Protocols November 2006
The IETF invites any interested party to bring to its attention any The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf- this standard. Please address the information to the IETF at ietf-
ipr@ietf.org. ipr@ietf.org.
News Article Architecture and Protocols January 2006
Full Copyright Statement Full Copyright Statement
Copyright (C) The Internet Society (2006). This document is subject Copyright (C) The IETF Trust (2006). This document is subject to the
to the rights, licenses and restrictions contained in BCP 78, and rights, licenses and restrictions contained in BCP 78, and except as
except as set forth therein, the authors retain all their rights. set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."
Appendix C - Change Log Appendix E - Change Log
[This Appendix is to be removed prior to final publication.] [This Appendix is to be removed prior to final publication.]
For version 01 For version 01
1 Numerous texts describing protocol features related to 1 Numerous texts describing protocol features related to
particular header fields in parts of [ARTICLE] which were particular header fields in parts of [ARTICLE] which were
destined to become part of [USEFOR] have been moved to destined to become part of [USEFOR] have been moved to
appropriate locations within section 7 of this document. Such appropriate locations within section 7 of this document. Such
revised texts will be found in sections revised texts will be found in sections
skipping to change at page 51, line 47 skipping to change at page 52, line 53
7.9.1 Step 5. 7.9.1 Step 5.
2 A section on "Duties of a Reading Agent" (7.8) has been added. 2 A section on "Duties of a Reading Agent" (7.8) has been added.
3 Some demotions MUST -> SHOULD -> MAY, as noted in pseudo- 3 Some demotions MUST -> SHOULD -> MAY, as noted in pseudo-
comments, have been made or proposed in sections comments, have been made or proposed in sections
7.3 7.3
7.3 Step 4. 7.3 Step 4.
4 Part of the procedure for examining Path header fields by 4 Part of the procedure for examining Path header fields by
relaying agents has been moved to serving agents, as explained relaying agents has been moved to storage agents, as explained
in pseudo-comments in section 7.4. in pseudo-comments in section 7.4.
5 Some renumbering of sections and minor textual clarifications. 5 Some renumbering of sections and minor textual clarifications.
For version 02 For version 02
News Article Architecture and Protocols November 2006
1 2nd para. of a-7 temporarily reinstated in section 6. 1 2nd para. of a-7 temporarily reinstated in section 6.
2 Para. in section 6 relating to propagation of control messages 2 Para. in section 6 relating to propagation of control messages
and local policy removed to [USEAGE].] and local policy removed to [USEAGE].]
News Article Architecture and Protocols January 2006
3 Requirement for some relaying agents to examine control messages 3 Requirement for some relaying agents to examine control messages
for non-existent groups for non-existent groups
6 6
7.3 7.3
4 Text regarding "aliasing out" brought into line with actual 4 Text regarding "aliasing out" brought into line with actual
practice. practice.
7.3 7.3
skipping to change at page 52, line 52 skipping to change at page 54, line 4
removed. removed.
3.1 3.1
7.6 7.6
For version 03 For version 03
1 The term "inheritable header" is no longer defined. Instead, the 1 The term "inheritable header" is no longer defined. Instead, the
term "inherited' is used in place of "taken" when defining the term "inherited' is used in place of "taken" when defining the
actions of a followup agent. actions of a followup agent.
7.6 7.6
News Article Architecture and Protocols November 2006
2 Consequent changes to "variant header field", and also mention 2 Consequent changes to "variant header field", and also mention
of Injection-Info as sometimes variant. of Injection-Info as sometimes variant.
2.3 2.3
3 The term "reply address" is no longer defined. 3 The term "reply address" is no longer defined.
News Article Architecture and Protocols January 2006
4 References now made to sections within USEFOR using "F-..." 4 References now made to sections within USEFOR using "F-..."
notation. notation.
5 Cross-references to sections within USEFOR added. Consistent use 5 Cross-references to sections within USEFOR added. Consistent use
of <...> around all mentions of syntactic objects. All of <...> around all mentions of syntactic objects. All
occurrences of "Foobar-header" changed to "Foobar header". Many occurrences of "Foobar-header" changed to "Foobar header". Many
other minor textual changes. other minor textual changes.
6 <control-message> changed to <control-command>, to avoid 6 <control-message> changed to <control-command>, to avoid
confusion with "control message", which signifies the complete confusion with "control message", which signifies the complete
skipping to change at page 53, line 53 skipping to change at page 55, line 5
7 New section on Identification of news-servers describing 7 New section on Identification of news-servers describing
acceptable forms for <path-identity>s. acceptable forms for <path-identity>s.
2.3 2.3
8 Definition of "semantic content" of a header field. 8 Definition of "semantic content" of a header field.
2.1 2.1
9 Systematic replacement of "header" by "header field". 9 Systematic replacement of "header" by "header field".
News Article Architecture and Protocols November 2006
10 More stringent rules for checking <newsgroup-name>s in control 10 More stringent rules for checking <newsgroup-name>s in control
messages for compliance with USEFOR. messages for compliance with USEFOR.
6.2 6.2
For version 05 For version 05
News Article Architecture and Protocols January 2006
1 Historical Appendices A.1 and A.2 removed, in anticipation of 1 Historical Appendices A.1 and A.2 removed, in anticipation of
republication of Son-of-1036 as an Informational RFC. republication of Son-of-1036 as an Informational RFC.
1.3 1.3
2 Definitions of technical terms adopted from USEFOR rather than 2 Definitions of technical terms adopted from USEFOR rather than
defining them separately. defining them separately.
3 Discussion of <path-identity> rewritten to reflect recent 3 Discussion of <path-identity> rewritten to reflect recent
developments (but still awaiting further work in USEFOR). developments (but still awaiting further work in USEFOR).
2.3 2.3
4 Items now included in Appendix A of USEFOR have been removed 4 Items now included in Appendix A of USEFOR have been removed
from section 3, and the "Transitional Arrangements" (which still from section 3, and the "Transitional Arrangements" (which still
cover the USEFOR changes) have been modified to reflect this. cover the USEFOR changes) have been modified to reflect this.
For version 06
1 Procedures for constructing the Path header field updated to
conform with USEFOR.
2 s/serving agent/storage agent/ throughout.
3 s/trusted source/verified source/
4 Chapter 3 (Changes to the existing protocols) moved to
Appendices B & C, and subsequent chapters renumbered.
 End of changes. 309 change blocks. 
611 lines changed or deleted 680 lines changed or added

This html diff was produced by rfcdiff 1.33. The latest version is available from http://tools.ietf.org/tools/rfcdiff/