draft-ietf-usefor-usepro-06.txt   draft-ietf-usefor-usepro-07.txt 
INTERNET-DRAFT Charles H. Lindsey Usenet Format Working Group R. Allbery, Ed.
Usenet Format Working Group University of Manchester Internet-Draft Stanford University
November 2006 Obsoletes: 1036 (if approved) C. Lindsey
Intended status: Standards Track University of Manchester
Expires: July 7, 2007 January 3, 2007
News Article Architecture and Protocols Netnews Architecture and Protocols
<draft-ietf-usefor-usepro-06.txt> draft-ietf-usefor-usepro-07
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
.QP Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working groups.
Note that other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are working documents of the Internet Engineering
months and may be updated, replaced, or obsoleted by other Task Force (IETF), its areas, and its working groups. Note that
documents at any time. It is inappropriate to use Internet-Drafts other groups may also distribute working documents as Internet-
as reference material or to cite them other than as "work in Drafts.
progress."
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.html. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire in May 2007. This Internet-Draft will expire on July 7, 2007.
Abstract
This Draft, together with its companion draft [USEFOR], are
intended as standards track documents, together obsoleting RFC
1036, which itself dates from 1987.
This Standard defines the architecture of Netnews systems and Copyright Notice
specifies the requirements to be met by software which originates,
distributes, stores and displays Netnews articles.
Backward compatibility has been a major goal of this endeavour, but Copyright (C) The Internet Society (2007).
where this standard and earlier documents or practices conflict, this
standard should be followed. In most such cases, current practice is
already compatible with these changes.
A companion Best Current Practice document [USEAGE], addressing Abstract
requirements which are present for Social rather than Normative
reasons is in preparation.
[This is the latest in the line of USEPRO drafts. However, the USEFOR This document defines the architecture of Netnews systems and
Working Group is currently considering the possibility of a complete specifies the correct manipulation and interpretation of Netnews
rewrite of this document.] articles by software which originates, distributes, stores, and
News Article Architecture and Protocols November 2006 displays them. It also specifies the requirements that must be met
by any protocol used to transport and serve Netnews articles.
[The use of the words "this standard" within this document when Internet Draft Comments
referring to itself does not imply that this draft yet has pretensions
to be a standard, but rather indicates what will become the case if and
when it is accepted as an RFC with the status of a proposed or draft
standard.]
[Remarks enclosed in square brackets and aligned with the left margin, Comments are solicited and should be addressed to the Usenet Format
such as this one, are not part of this draft, but are editorial notes to Working Group at ietf-usefor@imc.org.
explain matters amongst ourselves, or to point out alternatives, or to
assist the RFC Editor.]
Table of Contents Table of Contents
1. Introduction .................................................. 4 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1. Basic Concepts ............................................ 4 1.1. Basic Concepts . . . . . . . . . . . . . . . . . . . . . . 5
1.2. Objectives ................................................ 4 1.2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3. Historical Outline ........................................ 5 1.3. Requirements Notation . . . . . . . . . . . . . . . . . . 5
2. Definitions, Notations and Conventions ........................ 5 1.4. Syntax Notation . . . . . . . . . . . . . . . . . . . . . 5
2.1. Definitions ............................................... 5 1.5. Definitions . . . . . . . . . . . . . . . . . . . . . . . 6
2.2. Defining the Architecture ................................. 5 2. Transport . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3. Identification of news servers ............................ 6 3. Duties of Agents . . . . . . . . . . . . . . . . . . . . . . . 8
2.4. Variant Header Fields ..................................... 8 3.1. General Principles . . . . . . . . . . . . . . . . . . . . 8
2.5. Textual Notations ......................................... 8 3.2. The Path Header Field . . . . . . . . . . . . . . . . . . 9
3. Transport ..................................................... 9 3.2.1. Constructing the Path Header Field . . . . . . . . . . 10
4. Definition of new Media Types ................................. 10 3.2.2. Path Header Field Example . . . . . . . . . . . . . . 11
4.1. Application/news-transmission ............................. 10 3.3. Duties of a Posting Agent . . . . . . . . . . . . . . . . 12
4.2. Message/news obsoleted .................................... 11 3.3.1. Proto-articles . . . . . . . . . . . . . . . . . . . . 12
4.3. Application/news-groupinfo ................................ 11 3.3.2. Reinjection of Articles . . . . . . . . . . . . . . . 13
4.4. Application/news-checkgroups .............................. 12 3.3.3. Followups . . . . . . . . . . . . . . . . . . . . . . 13
5. Control Messages .............................................. 13 3.3.4. Construction of the References Header Field . . . . . 14
5.1. Digital Signature of Header Fields ........................ 14 3.4. Duties of an Injecting Agent . . . . . . . . . . . . . . . 15
5.2. Group Control Messages .................................... 14 3.4.1. Forwarding Messages to a Moderator . . . . . . . . . . 17
5.2.1. The 'newgroup' Control Message ........................ 15 3.5. Duties of a Relaying Agent . . . . . . . . . . . . . . . . 18
5.2.1.1. The Body of the 'newgroup' Control Message ........ 15 3.6. Duties of a Serving Agent . . . . . . . . . . . . . . . . 19
5.2.1.2. Initial Articles .................................. 15 3.7. Duties of a Reading Agent . . . . . . . . . . . . . . . . 21
5.2.1.3. Example ........................................... 16 3.8. Duties of a Moderator . . . . . . . . . . . . . . . . . . 21
5.2.2. The 'rmgroup' Control Message ......................... 17 3.9. Duties of a Gateway . . . . . . . . . . . . . . . . . . . 22
5.2.2.1. Example ........................................... 17 3.9.1. Duties of an Outgoing Gateway . . . . . . . . . . . . 23
5.2.3. The 'mvgroup' Control Message ......................... 17 3.9.2. Duties of an Incoming Gateway . . . . . . . . . . . . 24
5.2.3.1. Example ........................................... 19 3.9.3. Gateway Example . . . . . . . . . . . . . . . . . . . 26
5.2.4. The 'checkgroups' Control Message ..................... 20 4. Media Types . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.3. Cancel .................................................... 21 4.1. application/news-transmission . . . . . . . . . . . . . . 28
5.4. Ihave, sendme ............................................. 22 4.2. application/news-groupinfo . . . . . . . . . . . . . . . . 29
5.5. Obsolete control messages. ............................... 23 4.3. application/news-checkgroups . . . . . . . . . . . . . . . 30
6. Duties of Various Agents ...................................... 23 5. Control Messages . . . . . . . . . . . . . . . . . . . . . . . 32
6.1. General principles to be followed ......................... 24 5.1. Authentication and Authorization . . . . . . . . . . . . . 32
6.2. Duties of an Injecting Agent .............................. 24 5.2. Group Control Messages . . . . . . . . . . . . . . . . . . 33
6.2.1. Proto-articles ........................................ 25 5.2.1. The newgroup Control Message . . . . . . . . . . . . . 33
6.2.2. Procedure to be followed by Injecting Agents .......... 25 5.2.2. The rmgroup Control Message . . . . . . . . . . . . . 35
6.2.3. Procedure for Forwarding to a Moderator ............... 28 5.2.3. The checkgroups Control Message . . . . . . . . . . . 35
6.3. Duties of a Relaying Agent ................................ 28 5.3. The cancel Control Message . . . . . . . . . . . . . . . . 37
6.3.1. Path Header Field Example ............................. 32 5.4. The Supersedes Header Field . . . . . . . . . . . . . . . 37
6.4. Duties of a Serving Agent ................................. 33 5.5. The ihave and sendme Control Messages . . . . . . . . . . 38
News Article Architecture and Protocols November 2006 5.6. Obsolete Control Messages . . . . . . . . . . . . . . . . 39
6. Security Considerations . . . . . . . . . . . . . . . . . . . 40
6.5. Duties of a Posting Agent ................................. 35 6.1. Compromise of System Integrity . . . . . . . . . . . . . . 40
6.6. Duties of a Followup Agent ................................ 35 6.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 41
6.6.1. Construction of the References header field ........... 36 6.3. Leakage . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.7. Duties of a Reading Agent ................................. 37 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 43
6.8. Duties of a Moderator ..................................... 37 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.9. Duties of a Gateway ....................................... 39 8.1. Normative References . . . . . . . . . . . . . . . . . . . 44
6.9.1. Duties of an Outgoing Gateway ......................... 40 8.2. Informative References . . . . . . . . . . . . . . . . . . 44
6.9.2. Duties of an Incoming Gateway ......................... 40 Appendix A. Changes to the Existing Protocols . . . . . . . . . . 46
6.9.3. Example ............................................... 42 Appendix B. Acknowledgements . . . . . . . . . . . . . . . . . . 47
7. Security and Related Considerations ........................... 43 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 48
7.1. Leakage ................................................... 43 Intellectual Property and Copyright Statements . . . . . . . . . . 49
7.2. Attacks ................................................... 44
7.2.1. Denial of Service ..................................... 44
7.2.2. Compromise of System Integrity ........................ 45
7.3. Liability ................................................. 46
8. IANA Considerations ........................................... 47
9. References .................................................... 47
9.1. Normative References ...................................... 47
9.2. Informative References .................................... 48
10. Acknowledgements ............................................. 48
11. Contact Address .............................................. 49
Appendix A - Obsolete Control Messages ............................ 49
Appendix B - Differences from the Protocols in RFC 1036 and its
derivatives ....................................................... 50
Appendix C - Transitional Arrangements ............................ 50
Appendix D - Notices .............................................. 51
Appendix E - Change Log ........................................... 52
News Article Architecture and Protocols November 2006
1. Introduction 1. Introduction
1.1. Basic Concepts 1.1. Basic Concepts
"Netnews" is a set of protocols for generating, storing and "Netnews" is a set of protocols for generating, storing and
retrieving news "articles", as introduced in F-1.1. retrieving news "articles" whose format is defined in [USEFOR], and
for exchanging them amongst a readership that is potentially widely
distributed. It is organized around "newsgroups", with the
expectation that each reader will be able to see all articles posted
to each newsgroup in which he participates. These protocols most
commonly use a flooding algorithm which propagates copies throughout
a network of participating servers. Typically, only one copy is
stored per server, and each server makes it available on demand to
readers able to access that server.
"Usenet" is a particular worldwide publicly accessible network based "Usenet" is a particular worldwide publicly accessible network based
upon the Netnews protocols, with the newsgroups being organized into on the Netnews protocols. It is only one such possible network;
recognized "hierarchies". Anybody can join (it is simply necessary there are deployments of the Netnews protocols other than Usenet
to negotiate an exchange of articles with one or more other (such as ones internal to particular organizations). This document
participating hosts). discusses the more general Netnews architecture and protocols.
An important characteristic of Usenet is the lack of any requirement
for a central administration or for the establishment of any
controlling host to manage the network. Nevertheless, administrative
agencies do exists with varying degrees of authority to establish
"policies" applicable to particular parts of Usenet.
A "policy" is a rule intended to facilitate the smooth operation of a
network by establishing parameters which restrict behaviour that,
whilst technically unexceptionable, would nevertheless contravene
some accepted standard of "Good Netkeeping". Since the ultimate
beneficiaries of a network are its human readers, who will be less
tolerant of poorly designed interfaces than mere computers, articles
in breach of established policy can cause considerable annoyance to
their recipients.
[Could omit that last sentence, and perhaps even the whole paragraph?]
1.2. Objectives
The purpose of this present standard is to define the overall
architecture and the protocols to be used for Netnews in general, and
for Usenet in particular, and to set standards to be followed by
software that implements those protocols. The companion standard
[USEFOR] sets out the canonical format of news articles exchanged
between the various agents comprising that architecture. In this
standard, references to individual sections in the companion [USEFOR]
are prefixed with "F-".
A set of hosts within a network which, by mutual arrangement,
operates some variant (whether more or less restrictive) of the
Netnews protocols is a "cooperating subnet".
It is NOT the purpose of this standard to settle matters of policy,
nor aspects of software behaviour which do not impinge upon the
generation, transmission, storage and reception of articles, nor how
the authority of various agencies to create such policies and to
exercise control or oversight of the various parts of Usenet is
established. For these purposes, a separate Best Current Practice
document [USEAGE] is being provided.
News Article Architecture and Protocols November 2006 1.2. Scope
Nevertheless, it is assumed that such agencies with the necessary This document defines the architecture of Netnews systems and
authority will exist, and tools are provided within the protocols for specifies the correct manipulation and interpretation of Netnews
their use. articles by software which originates, distributes, stores, and
displays them. It addresses protocol issues that are independent of
transport protocols such as NNTP [RFC3977] and specifies the
requirements Netnews places on those underlying transport protocols.
It also specifies the handling of control messages.
1.3. Historical Outline The format and syntax of Netnews articles are specified in [USEFOR],
which should be read in conjunction with this document.
Network news originated as the medium of communication for Usenet, 1.3. Requirements Notation
circa 1980. Since then, Usenet has grown explosively, and many
Internet and non-Internet sites participate in it. In addition, the
news technology is now in widespread use for other purposes, on the
Internet and elsewhere.
For an account of the earlier formats used in Netnews prior to [RFC The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
1036], see Henry Spencer's 1994 draft, popularly referred to as "Son "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
of 1036" [Son-of-1036], which has recently been republished as an document are to be interpreted as described in [RFC2119].
Informational RFC.
[That is a tentative statement, which may need revision.]
Although never adopted as a formal standard, [Son-of-1036] had a 1.4. Syntax Notation
considerable effect on the development of Netnews and hence on these
present standards, and it is hoped that we have followed its spirit
and intentions.
.nr H1 1
2. Definitions, Notations and Conventions Syntax defined in this document uses the Augmented Backus-Naur Form
(ABNF) notation (including the Core Rules) defined in [RFC4234] and
constructs defined in [USEFOR] and [RFC2822].
2.1. Definitions 1.5. Definitions
All the technical terms defined in F-1.5 are to be considered as Any term used in this document that is defined in Section 1.5 of
defined also, with the same meaning, in this standard. In addition, [USEFOR] is used with the definition given there. In addition, the
some further terms are defined here, and in the following section. following terms will be used:
A "hierarchy" is the set of all newsgroups whose names share a first A "hierarchy" is the set of all newsgroups whose names share a first
<component> (as defined in F-3.1.4). The term "sub-hierarchy" is <component> (as defined in Section 3.1.4 of [USEFOR]). A "sub-
also used where several initial components are shared. hierarchy" is the set of all newsgroups whose names share several
initial components.
The "semantic content" (often abbreviated to just "content" when the
context is clear) of a header field is its semantic interpretation;
i.e. what remains after unfolding it and removing its field name with
its colon and any leading and trailing whitespace and, in the case of
structured header fields only, ignoring comments and other
semantically invisible items and replacing white space by a single
SP. See 6.6.1 for the use of this term.
2.2. Defining the Architecture
A Netnews system is a distributed database composed of agents of
various types which, acting together according to the protocols
defined in section 6 of this standard, causes articles to be
propagated throughout the system and to be made available to its
readers. The protocols ensure that all copies of a given article,
wherever stored, are identical apart from those header fields defined
as variant (2.4). For explaining the working of the protocols, it is
convenient to define particular sub-categories of agent as follows:
News Article Architecture and Protocols November 2006
A "posting agent" is the software that assists posters to prepare
proto-articles in compliance with [USEFOR]. The proto-article is
then passed on to an "injecting agent" for final checking and
injection into the news stream. If the article is not compliant, or
is rejected by the injecting agent, then the posting agent informs
the poster with an explanation of the error.
A "reading agent" is software which presents articles to a reader.
A "followup agent" is a combination of reading agent and posting
agent that aids in the preparation and posting of a followup.
An "injecting agent" takes the finished article from the posting
agent (often via the NNTP "POST" command), performs some final checks
and passes it on to a "relaying agent" for general distribution.
A "relaying agent" is software which receives allegedly compliant A "news server" is further distinguished into the roles of "injecting
articles from injecting agents and/or other relaying agents, and agent", "relaying agent", and "serving agent". An "injecting agent"
possibly passes copies on to other relaying agents and "storage accepts a proto-article with the goal of distributing it to relaying
agents". and serving agents and hence to readers. A "relaying agent" accepts
articles from other relaying agents or injecting agents and
distributes them to other relaying agents or serving agents. A
"serving agent" receives an article from a relaying agent or
injecting agent and makes it available to readers.
A "storage agent" receives an article from a relaying agent and files A "user agent" is further distinguished into the roles of "posting
it in a "news database". It also provides an interface for reading agent" and "reading agent". A "posting agent" is software which
agents to access the news database. assists in the preparation of a proto-article and then passes it to
an injecting agent. A "reading agent" is software which retrieves
articles from a serving agent for presentation to a reader.
A "news database" is the set of articles and related structural "Injecting" an article is the processing of a proto-article by an
information stored by a storage agent and made available for access injecting agent. Normally this action is done once and only once for
by reading agents. a given article. "Reinjecting" an article is passing an already-
injected article to an injection agent.
A "gateway" is software which receives news articles and converts A "gateway" is software which receives news articles and converts
them to messages of some other kind (e.g. mail to a mailing list), or them to messages of some other kind (such as [RFC2822] mail
vice versa; in essence it is a translating relaying agent that messages), receives messages of some other kind and converts them to
straddles boundaries between different methods of message exchange. news articles, or conveys articles between two separate Netnews
The most common type of gateway connects newsgroup(s) to mailing networks.
list(s), either unidirectionally or bidirectionally, but there are
also gateways between news networks using the [USEFOR] news format
and those using other formats.
Posting, reading and followup agents (which are usually just
different services provided by the same piece of software) together
comprise the "user agents" defined in F-1.5.
Likewise, injecting, relaying and storage agents (which are often
just different services provided by the same piece of software)
together comprise the "news servers".
2.3. Identification of news servers
[There are two alternative texts which have been proposed:]
In order to record the passage of articles through the network, news
servers need to identify themselves by means of a <path-identity>
(F-3.1.5), which can appear in Path, Injection-Info and Xref header
fields. Whatever <path-identity> is used in the Path header field
News Article Architecture and Protocols November 2006
SHOULD be used also in any Injection-Info header field (and it would
be normal to use it in any Xref header field also).
[Maybe that last sentence moves elsewhere.]
NOTE: Such <path-identity>s may also be suitable for sending
email to news server administrators (see [USEAGE]).
[1st alternative]
<Path-identity>s can take the following forms (in decreasing order of
preference):
1. A fully qualified domain name (FQDN) that SHOULD be resolvable in
the DNS (whether via an A, AAAA or MX record or an equivalent
CNAME), thus guaranteeing a unique identity. Ideally, it will also
provide a means to contact the administrators by email (according
to [RFC 2142], the forms "usenet@server" and "news@server" are
common addresses for a news server administrator).
2. Some other (arbitrary) name in the form of a <path-nodot>, and
believed to be unique and registered at least with all other news
servers sending articles directly to the given one. This option
SHOULD NOT be used unless the earlier option is unavailable (e.g.
because the server in question is not connected to the Internet),
or unless it is of longstanding usage and cessation would be
unduly disruptive, or unless the earlier option is provided as
well.
[2nd alternative]
<Path-identity>s can take the following forms (in decreasing order of
preference):
1. A fully qualified domain name (FQDN) that can be resolved to an
email server via an MX, A or AAAA record according to the
procedures of [RFC 2821]; this guarantees that the name is unique,
and makes it easy to contact the administrators if needed.
2. A fully qualified domain name (FQDN) that is guaranteed to be
unique by the administrators of the domain; for instance, the
uniqueness of "server.example.org" could be guaranteed by the
administrator of "example.org" even if nothing is stored in the
DNS for that name.
3. Some other (arbitrary) name in the form of a <path-nodot>, and
believed to be unique and registered at least with all other
news-servers sending articles directly to the given one. This
option SHOULD NOT be used unless the earlier options are
unavailable, or unless the name is of longstanding usage and
cessation would be unduly disruptive, or unless one of the earlier
options is provided as well.
News Article Architecture and Protocols November 2006
According to [RFC 2142]], the forms "usenet@server" and "news@server"
are common addresses for a news server administrator.
[end of alternatives]
NOTE: Although domain names are case insensitive and it is
intended that <path-nodot>s should also be so, it is customary
to render them all in lowercase, since many implementations
compare them case sensitively for reasons of efficiency.
NOTE: A news server administrator who chooses a <path-nodot>
which turns out not to be unique (disregarding case) will have
to bear the consequences.
NOTE: An IP address is not permitted as a <path-identity>,
although it may still appear in a <diag-identity>. Since the
syntax permits a colon (":" which, prior to this standard, was
an alternative to the "!" delimiter) within any <diag-identity>
which takes the form of an <IPv6address>, it would be unwise to
choose as a <path-nodot> anything composed solely from four or
less hexadecimal digits.
2.4. Variant Header Fields
Header fields with the variant property may differ between (or even
be completely absent from) copies of the same article as stored or
relayed throughout a Netnews system. The manner of the difference (or
absence) MUST be as specified in this (or some future) standard.
Typically, these header fields are modified as articles are
propagated, or they reflect the status of the article on a particular
storage agent, or cooperating group of such agents. A variant header
field MAY be placed anywhere within the header fields (though placing
it first is recommended).
The following header fields are classified as "variant":
o Path (F-3.1.5) - augmented at each relaying agent that an article
passes through.
o Xref (F-3.2.14) - used to keep track of the <article-locator>s of
crossposted articles so that reading agents serviced by a
particular storage agent can mark such articles as read.
o Injection-Info (F-3.2.8) is also considered variant in some
special situations involving reinjection (6.2 and 6.2.2).
2.5. Textual Notations
This standard contains explanatory NOTEs using the following format.
These may be skipped by persons interested solely in the content of
the specification. The purpose of the notes is to explain why choices
were made, to place them in context, or to suggest possible
implementation techniques.
NOTE: While such explanatory notes may seem superfluous in
principle, they often help the less-than-omniscient reader grasp
the purpose of the specification and the constraints involved.
Given the limitations of natural language for descriptive
News Article Architecture and Protocols November 2006
purposes, this improves the probability that implementors and
users will understand the true intent of the specification in
cases where the wording is not entirely clear.
"US-ASCII" is short for "the ANSI X3.4 character set" [ANSI X3.4].
US-ASCII is a 7 bit character set. Please note that this standard
requires that all agents be 8 bit clean; that is, they must accept
and transmit data without changing or omitting the 8th bit.
Certain words, when capitalized, are used to define the significance
of individual requirements. The key words "MUST", "REQUIRED",
"SHOULD", "RECOMMENDED", "MAY" and "OPTIONAL", and any of those words
associated with the word "NOT", are to be interpreted as described in
[RFC 2119].
NOTE: A requirement imposed on a relaying or storage agent
regarding some particular article should be understood as
applying only if that article is actually accepted for
processing (since any agent may always reject any article
entirely, for reasons of site policy).
Wherever the context permits, use of the masculine includes the
feminine and use of the singular includes the plural, and vice versa.
Throughout this standard we will give examples of various
definitions, header fields and other specifications. It needs to be
remembered that these samples are for the aid of the reader only and
do NOT define any specification themselves. In order to prevent
possible conflict with "Real World" entities and people the top level
domain ".example" is used in all sample domains and addresses. The
hierarchy "example.*" is also used as a sample hierarchy.
Information on the ".example" top level domain is in [RFC 2606].
3. Transport
As in this standard's predecessors, the exact means used to transmit
articles from one host to another is not specified. NNTP [RFC 3977]
is the most common transmission method on the Internet, but much
transmission takes place entirely independent of the Internet. Other
methods in use include the UUCP protocol [RFC 976] extensively used
in the early days of Usenet, FTP, tunneling through email using
application news/transmission, downloading via satellite, tape
archives, and physically delivered magnetic and optical media.
Transmission paths for news articles MUST treat news articles as
uninterpreted sequences of octets, excluding the values 0 (US-ASCII
NUL) and 13 and 10 (US-ASCII CR and LF, which MUST ONLY appear in the
combination CRLF which denotes a line separator).
NOTE: this corresponds to the range of octets permitted for MIME
"8bit data" [RFC 2045]. Thus raw binary data cannot be
transmitted in an article body except by the use of a Content-
Transfer-Encoding such as base64.
News Article Architecture and Protocols November 2006
In particular, transmission paths MUST convey all header fields
(including body part header fields and header fields within
message/rfc822 objects) intact, even if they contain octets in the
range 128 to 255. Furthermore, relaying agents MUST, and other
agents SHOULD, convey lines even if they exceed 998 characters in
length, especially in article bodies. These requirements include the
transmissiom paths between posting agents, injecting agents, relaying
agents, storage agents and reading agents, but NOT the paths
traversed by Netnews articles that have been gatewayed into Email
(6.9.1).
[At some point it will be necessary for the IMAP standards to catch up
with these requirements.]
4. Definition of new Media Types
This standard defines (or redefines) several new Media Types, which
require to be registered with IANA as provided for in [RFC 4288].
4.1. Application/news-transmission
The Media Type "application/news-transmission" is intended for the
encapsulation of complete news articles where the intention is that
the recipient should then inject them into Netnews. This Application
type provides one of the methods for mailing articles to moderators
(see 6.2.2) and it is also the preferred method when sending to an
email-to-news gateway (see 6.9.2).
NOTE: The benefit of such encapsulation is that it removes
possible conflict between news and email header fields and it
provides a convenient way of "tunnelling" a news article through
a transport medium that does not support 8bit characters.
The MIME Media Type definition of "application/news-transmission" is:
MIME type name: application
MIME subtype name: news-transmission
Required parameters: none
Optional parameters: usage=moderate
usage=inject
usage=relay
Encoding considerations: A transfer-encoding (such as Quoted-
Printable or Base64) different from that of
the article transmitted MAY be supplied
(perhaps en route) to ensure correct
transmission over some 7bit transport
medium.
Security considerations: A news article may be a "control message",
which could have effects on the recipient
host's system beyond just storage of the
article. However, such control messages
also occur in normal news flow, so most
hosts will already be suitably defended
against undesired effects.
Published specification: [USEPRO]
News Article Architecture and Protocols November 2006
Body part: A complete article or proto-article, ready
for injection into Netnews, or a batch of
such articles in the batch format described
in section 5.4.
NOTE: It is likely that the recipient of an "application/news-
transmission" will be a specialized gateway (e.g. a moderator's
submission address) able to accept articles with only one of the
three usage parameters "moderate", "inject" and "relay", hence
the reason why they are optional, being redundant in most
situations. Nevertheless, they MAY be used to signify the
originator's intention with regard to the transmission, so
removing any possible doubt.
When the parameter "relay" is used, or implied, the body part MAY be
a batch of articles to be transmitted together, in which case the
batch format defined in section 5.4 MUST be used.
4.2. Message/news obsoleted
The Media Type "message/news", as previously registered with IANA, is
hereby declared obsolete. It was never widely implemented, and its
default treatment as "application/octet-stream" by agents that did
not recognize it was counter productive. The Media Type
"message/rfc822" SHOULD be used in its place.
4.3. Application/news-groupinfo
The "application/news-groupinfo" is used in conjunction with the
"newgroup" (5.2.1) and "mvgroup" (5.2.3) control messages. The
<newsgroup-name> in the <newsgroups-line> MUST agree with the
<newsgroup-name> in the "newgroup" or "mvgroup" control message. The
Media Type "application/news-groupinfo" MUST NOT be used except as a
part of such control messages.
The "application/news-groupinfo" body part contains brief information
about a newsgroup, i.e. the group's name, it's <newsgroup-
description> and the <moderation-flag>.
NOTE: The presence of the <newsgroups-tag> "For your newsgroups
file:" is intended to make the whole newgroup message compatible
with current practice as described in [Son-of-1036].
The MIME Media Type definition of "application/news-groupinfo" is:
MIME type name: application
MIME subtype name: news-groupinfo
Required parameters: none
Disposition: by default, inline
Encoding considerations: "7bit" or "8bit" is sufficient and MUST be
used to maintain compatibility.
Security considerations: this type MUST NOT be used except as part
of a control message for the creation or
modification of a Netnews newsgroup
News Article Architecture and Protocols November 2006
Published specification: [USEPRO]
The content of the "application/news-groupinfo" body part is defined
as:
groupinfo-body = [ newsgroups-tag CRLF ]
newsgroups-line CRLF
newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP
%x6E.65.77.73.67.72.6F.75.70.73 SP
%x66.69.6C.65.3A
; case sensitive
; "For your newsgroups file:"
newsgroups-line = newsgroup-name
[ 1*HTAB newsgroup-description ]
[ 1*WSP moderation-flag ]
newsgroup-description
= utext *( *WSP utext )
moderation-flag = %x28.4D.6F.64.65.72.61.74.65.64.29
; case sensitive "(Moderated)"
The <newsgroup-description> MUST NOT contain any occurrence of the
string "(Moderated)" within it. Although optional, the <newsgroups-
tag> SHOULD be included until such time as this standard has been
widely adopted, to ensure compatibility with present practice.
Moderated newsgroups MUST be marked by appending the case sensitive
text " (Moderated)" at the end. It is NOT recommended that the
moderator's email address be included in the <newsgroup-description>
as has sometimes been done.
NOTE: There is no provision for the use of charsets other than
US-ASCII within a <newsgroup-description>. Such a facility may
be provided in a future extension to this standard.
[That may seem harsh, but if we make any such provision now, it will
make life more complicated and restrict our freedom when it comes to the
proposed I18N extension. Therefore I resisted the temptation to include
any charset parameter with this Media Type. Note that this also applies
to the checkgroups message further on.]
4.4. Application/news-checkgroups
The "application/news-checkgroups" Media Type is used in conjunction
with the "checkgroups" control message (5.2.4). It MUST NOT be used
except as a part of such control messages.
The "application/news-checkgroups" body part contains a complete list
of all the newsgroups in a (sub)hierarchy, their <newsgroup-
description>s and their moderation status.
The MIME Media Type definition of "application/news-checkgroups" is:
MIME type name: application
MIME subtype name: news-checkgroups
Required parameters: none
News Article Architecture and Protocols November 2006
Disposition: by default, inline
Encoding considerations: "7bit" or "8bit" is sufficient and MUST be
used to maintain compatibility.
Security considerations: this type MUST NOT be used except as part
of a checkgroups control message
Published specification: [USEPRO]
The content of the "application/news-checkgroups" body part is
defined as:
checkgroups-body = *( valid-group CRLF )
valid-group = newsgroups-line ; see 4.3
5. Control Messages
The following sections document the control messages. "Message" is
used herein as a synonym for "article" unless context indicates
otherwise.
Each <control-command> comprises a <verb>, which indicates the action
to be taken, and <argument>(s), which supply the details (see F-
3.2.3). The following sections contain syntactic definitions for the
<verb>, <argument>s, and possibly the body, for each type of control
message.
The Newsgroups header field of each control message SHOULD include
the <newsgroup-name>(s) for the group(s) affected (i.e. groups to be
created, modified or removed, or containing articles to be canceled).
This is to ensure that the message propagates to all sites which
receive (or would receive) that group(s). It MAY include other
<newsgroup-name>s so as to improve propagation (but this practice may
cause the control message to propagate also to places where it is
unwanted, or even cause it not to propagate where it should, so it
should not be used without good reason).
NOTE: Propagation is controlled by relaying agents, and it may
be necessary for relaying agents to take special steps to ensure
that control messages such as newgroup messages for not-yet-
existent newsgroups are propagated correctly (see 6.3).
The presence of a Subject header field whose content starts with the
string "cmsg " followed by a <control-command> was construed under
[RFC 1036] as a request to perform that control action (even if no
genuine Control header field was present). Indeed, some
implementations went further and added the implied Control header
field before injecting. Likewise, the presence of a <newsgroup-name>
ending in ".ctl" in the Newsgroups header field caused the Subject
header field content (not starting with "cmsg" in this case) to be
interpreted as a <control-command>.
All these practices, which have already largely fallen into disuse,
are now declared to be Obsolete, and Subject header fields MUST NOT
now be interpreted as <control-command>s under any circumstances.
News Article Architecture and Protocols November 2006
[Possible addtional text:]
In order to prevent continuing interpretation of Subject header
fields in this way by existing agents, posting and injecting agents
SHOULD detect and decline to post articles in which the Subject
header field starts with the word "cmsg" and in which there is no
Control header field.
The descriptions below set out REQUIREMENTS to be followed by sites
that receive control messages and choose to honour them. However,
nothing in these descriptions should be taken as overriding the right
of any such site, in accordance with its local policy, to refuse to
honour any particular control message, or to refer it to an
administrator for approval (either as a class or on a case-by-case
basis).
5.1. Digital Signature of Header Fields
It is most desirable that group control messages (5.2) in particular
be authenticated by incorporating them within some digital signature
scheme that encompasses other header fields closely associated with
them (including at least Approved, Message-ID and Date). At the time
of writing, this is usually done by means of a protocol known as
"PGPverify" ([PGPVERIFY]), and continued usage of this is encouraged
at least as an interim measure.
However, PGPverify is not considered suitable for standardization in
its present form, for various technical reasons. It is therefore
expected that an early extension to this standard will provide a
robust and general purpose digital authentication mechanism with
applicability to all situations requiring protection against
malicious use of, or interference with, header fields. That
extension would also address other Netnews security issues.
5.2. Group Control Messages
"Group control messages" are the sub-class of control messages that
request some update to the configuration of the groups known to a
storage agent, namely "newgroup", "rmgroup", "mvgroup" and
"checkgroups", plus any others created by extensions to this
standard.
Group control messages that attempt to create groups with names that
are deprecated or reserved according to F-3.1.4 MUST NOT be issued,
except by prior agreement within some cooperating subnet. Moreover,
sites receiving such control messages SHOULD check them for
conformance before honouring them.
All of the group control messages MUST have an Approved header field
(F-3.2.1) which, in those hierarchies where appropriate
administrative agencies exist (see 1.1), identifies the appropriate
person or entity as authorized by those agencies. The authorized
person or entity SHOULD adhere to any conventions and restrictions on
the format of <newsgroup-name>s established for those hierarchies
News Article Architecture and Protocols November 2006
[USEAGE].
5.2.1. The 'newgroup' Control Message
control-command =/ Newgroup-command
Newgroup-command = "newgroup" Newgroup-arguments
Newgroup-arguments = FWS newsgroup-name [ FWS newgroup-flag ]
newgroup-flag = "moderated"
The "newgroup" control message requests that the specified group be
created or have its moderation status or <newsgroups-line> changed.
When the request is honoured, if the <newgroup-flag> "moderated" is
present then the status of the group SHOULD be marked as moderated,
and vice versa. "Moderated" is the only such flag defined by this
standard; other flags MAY be defined for use in cooperating subnets,
but newgroup messages containing them MUST NOT be acted on outside of
those subnets.
NOTE: Specifically, some alternative flags such as "y" and "m",
which are sent and recognized by some current software, are NOT
part of this standard. Moreover, some existing implementations
treat any flag other than "moderated" as indicating an
unmoderated newsgroup. Both of these usages are contrary to this
standard and control messages with such non-standard flags
should be ignored.
5.2.1.1. The Body of the 'newgroup' Control Message
The body of the newgroup message contains the following subparts,
preferably in the order shown:
1. An "application/news-groupinfo" part (4.3) containing the name and
<newsgroups-line> (4.3) of the group. This part MUST be present
and SHOULD be used to update any copy of the <newsgroups-line>
maintained by the storage agent.
2. Other parts containing useful information about the background of
the newgroup message (typically of type "text/plain").
3. Parts containing initial articles for the newsgroup. See section
5.2.1.2 for details.
In the event that there is only the single (i.e. application/news-
groupinfo) subpart present, it will suffice to include a "Content-
Type: application/news-groupinfo" amongst the header fields of the
control message. Otherwise, a "Content-Type: multipart/mixed" header
field will be needed, and each separate part will then need its own
Content-Type header field.
5.2.1.2. Initial Articles
Some subparts of a "newgroup" or "mvgroup" control message MAY
contain an initial set of articles to be posted to the affected
newsgroup as soon as it has been created or modified. These parts are
News Article Architecture and Protocols November 2006
identified by having the Media Type "application/news-transmission",
possibly with the parameter "usage=inject". The body of each such
part should be a complete proto-article, ready for posting. This
feature is intended for the posting of charters, initial FAQs and the
like to the newly formed group.
The Newsgroups header field of the proto-article MUST include the
<newsgroup-name> of the newly created or modified group. It MAY
include other <newsgroup-name>s. If the proto-article includes a
Message-ID header field, the message identifier in it MUST be
different from that of any existing article and from that of the
control message as a whole. Alternatively such a message identifier
MAY be derived by the injecting agent when the proto-article is
posted. The proto-article SHOULD include the header field
"Distribution: local".
The proto-article SHOULD be injected at the storage agent that
processes the control message AFTER the newsgroup in question has
been created or modified. It MUST NOT be injected if the newsgroup
is not, in fact, created (for whatever reason). It MUST NOT be
submitted to any relaying agent for transmission beyond the storage
agent(s) upon which the newsgroup creation has just been effected (in
other words, it is to be treated as having a "Distribution: local"
header field, whether such a field is actually present or not).
NOTE: It is not precluded that the proto-article is itself a
control message or other type of special article, to be
activated only upon creation of the new newsgroup. However,
except as might arise from that possibility, any
"application/news-transmission" within some nested "multipart/*"
structure within the proto-article is not to be activated.
5.2.1.3. Example
A "newgroup" with its charter:
From: "example.all Administrator" <admin@noc.example>
Newsgroups: example.admin.info,example.admin.announce
Date: 27 Feb 2006 12:50:22 +0200
Subject: cmsg newgroup example.admin.info moderated
Approved: admin@noc.example
Control: newgroup example.admin.info moderated
Message-ID: <ng-example.admin.info-20060227@noc.example>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="nxtprt"
Content-Transfer-Encoding: 8bit
This is a MIME control message.
--nxtprt
Content-Type: application/news-groupinfo
For your newsgroups file:
News Article Architecture and Protocols November 2006
example.admin.info About the example.* groups (Moderated)
--nxtprt
Content-Type: application/news-transmission
Newsgroups: example.admin.info
From: "example.all Administrator" <admin@noc.example>
Subject: Charter for example.admin.info
Message-ID: <charter-example.admin.info-20020227@noc.example>
Distribution: local
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
The group example.admin.info contains regularly posted
information on the example.* hierarchy.
--nxtprt--
5.2.2. The 'rmgroup' Control Message
control-command =/ Rmgroup-command
Rmgroup-command = "rmgroup" Rmgroup-arguments
Rmgroup-arguments = FWS newsgroup-name
The "rmgroup" control message requests that the specified group be
removed from the list of valid groups. The Media Type of the body is
unspecified; it MAY contain anything, usually an explanatory text.
NOTE: It is entirely proper for a storage agent to retain the
group until all the articles in it have expired, provided that
it ceases to accept new articles.
5.2.2.1. Example
From: "example.all Administrator" <admin@noc.example>
Newsgroups: example.admin.obsolete, example.admin.announce
Date: 4 Apr 2006 22:04 -0900 (PST)
Subject: cmsg rmgroup example.admin.obsolete
Message-ID: <rm-example.admin.obsolete-20060404@noc.example>
Approved: admin@noc.example
Control: rmgroup example.admin.obsolete
The group example.admin.obsolete is obsolete. Please remove it
from your system.
5.2.3. The 'mvgroup' Control Message
control-command =/ Mvgroup-command
Mvgroup-command = "mvgroup" Mvgroup-arguments
Mvgroup-arguments = FWS newsgroup-name FWS newsgroup-name
[ FWS newgroup-flag ]
News Article Architecture and Protocols November 2006
The "mvgroup" control message requests that the group specified by
the first <(old-)newsgroup-name> be moved to that specified by the
second <(new-)newsgroup-name>. Thus it is broadly equivalent to a
"newgroup" control message for the second group followed by a
"rmgroup" control message for the first group.
The message body contains an "application/news-groupinfo" part (4.3)
containing machine- and human-readable information about the new
group, and possibly other subparts as for a "newgroup" control
message. The information conveyed in the "application/news-groupinfo"
body part, notably its <newsgroups-line> (4.3), is applied to the new
group.
When this message is received, the new group is created (if it does
not exist already) as for a "newgroup" control message, and SHOULD in
any case be made moderated if a <newgroup-flag> "moderated" is
present, and vice versa. At the same time, arrangements SHOULD be
made to remove the old group (as with a "rmgroup" control message),
but only after a suitable overlap period to allow the network to
adjust to the new arrangement.
At the same time as a storage agent acts upon this message, all
injecting agents associated with that storage agent SHOULD inhibit
the posting of new articles to the old group (preferably with some
indication to the poster that the new group should have been used).
Relaying agents, however, MUST continue to propagate such articles
during the overlap period.
NOTE: It is to be expected that different storage agents will
act on this message at different points of time, users of the
old group will have to become accustomed to the new arrangement,
and followups to already established threads will likely
continue under the old group. Therefore, there needs to be an
overlap period during which articles may continue to be accepted
by relaying and storage agents in either group. This standard
does not specify any standard period of overlap (though it would
be expected to be expressed in days rather than in months). The
inhibition of injection of new articles to the old group may
seem draconian, but it is the surest way to prevent the
changeover from dragging on indefinitely.
Since the "mvgroup" control message is newly introduced in this
standard and may not be widely implemented initially, it SHOULD be
followed shortly afterwards by a corresponding "newgroup" control
message; and again, after a reasonable overlap period, it MUST be
followed by a "rmgroup" control message for the old group.
In order to facilitate a smooth changeover, storage agents MAY
arrange to service requests for access to the old group by providing
access to the new group, which would then contain, or appear to
contain, all articles posted to either group (including, ideally, the
pre-changeover articles from the old one). Nevertheless, if this
feature is implemented, the articles themselves, as supplied to
reading agents, MUST NOT be altered in any way (and, in particular,
News Article Architecture and Protocols November 2006
their Newsgroups header fields MUST contain exactly those newsgroups
present when they were injected). On the other hand, the Xref header
field (F-3.2.14) MAY contain entries for either group (or even both).
NOTE: Some storage agents that use an "active" file permit an
entry of the form "oldgroup xxx yyy =newgroup", which enables
any articles arriving for oldgroup to be diverted to newgroup,
thus providing a simple implementation of this feature. However,
it is known that not all current storage agents will find
implementation so easy (especially in the short term) which is
why it is not mandated by this standard. Nevertheless, its
eventual implementation in all storage agents is to be
considered highly desirable.
On the other hand, it is recognized that this feature would
likely not be implementable if the new group was already in
existence with existing articles in it. This situation should
not normally arise except when there is already some confusion
as to which groups are, or are not, supposed to exist in that
hierarchy. Note that the "mvgroup" control message is not really
intended to be used for merging two existing groups.
5.2.3.1. Example
From: "example.all Administrator" <admin@noc.example>
Newsgroups: example.oldgroup,example.newgroup,example.admin.announce
Date: 30 Apr 2006 22:04 -0500 (EST)
Subject: cmsg mvgroup example.oldgroup example.newgroup moderated
Message-ID: <mvgroup-example.oldgroup-20060430@noc.example>
Approved: admin@noc.example
Control: mvgroup example.oldgroup example.newgroup moderated
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=nxt
--nxt
Content-Type: application/news-groupinfo
For your newsgroups file:
example.newgroup The new replacement group (Moderated)
--nxt
The moderated group example.oldgroup is replaced by
example.newgroup. Please update your configuration, and please,
if possible, arrange to file articles arriving for
example.oldgroup as if they were in example.newgroup.
--nxt
Content-Type: application/news-transmission
Newsgroups: example.admin.info
From: "example.all Administrator" <admin@noc.example>
Subject: Charter for example.newgroup
Message-ID: <mvgroup-example.newgroup-20020430@noc.example>
News Article Architecture and Protocols November 2006
Distribution: local
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
This group (formerly known as example.oldgroup) is for the
discussion of examples.
--nxt--
5.2.4. The 'checkgroups' Control Message 2. Transport
The "checkgroups" control message contains a list of all the valid The exact means used to transmit articles from one agent to another
groups in a complete hierarchy. is not specified. NNTP [RFC3977] is the most common transport
mechanism for Netnews networks. Other methods in use include the
UUCP protocol [RFC0976] (extensively used in the early days of
Usenet) and physically delivered magnetic and optical media. Any
mechanism may be used in conjunction with this protocol provided that
it can meet the requirements specified here.
control-command =/ Checkgroup-command Transports for Netnews articles MUST treat news articles as
Checkgroup-command = "checkgroups" Checkgroup-arguments uninterpreted sequences of octets, excluding the values 0 (which may
Checkgroup-arguments= [ chkscope ] [ chksernr ] not occur in Netnews articles) and 13 and 10 (which MUST only appear
chkscope = 1*( FWS ["!"] newsgroup-name ) in Netnews articles as a pair in that order and which together denote
chksernr = FWS "#" 1*DIGIT a line separator). These octets are the US-ASCII [ASCII] characters
NUL, CR, and LF respectively.
A "checkgroups" message applies to any (sub-)hierarchy with a prefix NOTE: This corresponds to the range of octets permitted in MIME
listed in the <chkscope> argument, provided that the rightmost 8bit data [RFC2045]. Transports for Netnews are not required to
matching <newsgroup-name> in the list is not immediately preceded by support transmission of MIME binary data.
a "!". If no <chkscope> argument is given, it applies to all
hierarchies for which group statements appear in the body of the
message.
NOTE: Some existing software does not support the <chkscope> In particular, transports MUST convey all header fields (including
argument. Thus a "checkgroups" message SHOULD also contain the header fields within message/rfc822 objects in article bodies)
groups of other subhierarchies the sender is not responsible unmodified even if they contain octets in the range 128 to 255.
for. "New" software MUST ignore groups which do not fall within Furthermore, transports for relaying and serving agents MUST, and
the <chkscope> argument of the "checkgroups" message. transports for other agents SHOULD, convey lines even if they exceed
998 characters in length, especially in article bodies. (This
requirement is stricter than MIME 8bit data.) These requirements
include the transport paths between posting agents, injecting agents,
serving agents, and reading agents.
The <chksernr> argument is a serial number, which can be any positive 3. Duties of Agents
integer (e.g. just numbered or the date in YYYYMMDD). It SHOULD
increase by an arbitrary value with every change to the group list
and MUST NOT ever decrease.
NOTE: This was added to circumvent security problems in The following section specifies the duties of the agents involved in
situations where the Date header field cannot be authenticated. the creation, relaying, and serving of Netnews articles. This
protocol is described by following the life of a typical Usenet
article: it is prepared by a posting agent, given to an injecting
agent, transferred through one or more relaying agents, accepted by a
serving agent, and finally retrieved by a reading agent. Articles
submitted to moderated groups go through an additional process, which
is described separately. Finally, the additional duties and
requirements of a gateway are discussed.
Example: At each step, each agent has a set of checks and transformations of
the article that it is required to perform. These are described as
sequences of steps to be followed, but it should be understood that
it is the effect of these sequences that is important, and
implementations may use any method that gives rise to the same
effect.
Control: checkgroups de !de.alt #248 Many news servers combine the functions of injecting agent, relaying
agent, and serving agent in a single software package. For the
purposes of this specification, such combined agents should
conceptually be treated as an injecting agent which sends articles to
a serving agent and optionally a relaying agent. The requirements of
all three agents MUST be still met when the news server is performing
the functions of those agents.
which includes the whole of the 'de.*' hierarchy, with the exception Control messages may have additional effects than those described
of its 'de.alt.*' sub-hierarchy. below on news servers which accept them. Those effects are described
in Section 5.
The body of the message has the Media Type "application/news- 3.1. General Principles
checkgroups" (4.4). It asserts that the <valid-group>s it lists are
the only newsgroups in the specified hierarchies.
News Article Architecture and Protocols November 2006 There are two important principles that news implementors and
administrators need to keep in mind. The first is the well-known
Internet Robustness Principle:
NOTE: The "checkgroups" message is intended to synchronize the Be liberal in what you accept, and conservative in what you send.
list of newsgroups stored by a storage agent, and their
<newsgroup-description>s, with the lists stored by other storage
agents throughout the network. However, it might be inadvisable
for the storage agent actually to create or delete any
newsgroups without first obtaining the approval of its
administrators for such proposed actions.
NOTE: The possibility of removing a complete hierarchy by means As applied to Netnews, this primarily means that unwanted or non-
of an "invalidation" line beginning with a '!' in the compliant articles SHOULD be rejected as early as possible, but once
checkgroups-body is no longer provided by this standard. The they are in general circulation, relaying and serving agents may wish
intent of the feature was widely misunderstood and it was to accept them where possible rather than lose information. Posting
misused more often than it was used correctly. The same effect, agents and injecting agents SHOULD therefore be maximally strict in
if required, can now be obtained by the use of an appropriate their application of both this protocol and [USEFOR], and reading
<chkscope> argument in conjunction with an empty <checkgroups- agents SHOULD be robust in the presence of violations of the Netnews
body>. article format where possible.
5.3. Cancel In the case of Netnews, there is an even more important principle,
derived from a much older code of practice, the Hippocratic Oath (we
may thus call this the Hippocratic Principle):
The "cancel" message requests that a target article be "canceled", First, do no harm.
i.e. be withdrawn from circulation or access.
control-command =/ Cancel-command It is vital to realize that decisions which might be merely
Cancel-command = "cancel" Cancel-arguments suboptimal in a smaller context can become devastating mistakes when
Cancel-arguments = FWS msg-id [FWS] amplified by the actions of thousands of hosts within a few minutes.
The argument identifies the article to be cancelled by its message No Netnews agent is ever required to accept any article. It is
identifier. The body SHOULD contain an indication of why the common for injecting, relaying, and serving agents to reject well-
cancellation was requested. The "cancel" message SHOULD be posted to formed articles for reasons of local policy (such as not wishing to
the same newsgroup(s), with the same distribution(s), as the article carry a particular newsgroup or attempting to filter out unwanted
it is attempting to cancel. articles). This document specifies how articles are to be treated if
they are accepted and specifies some cases where they must be
rejected, but an agent MAY always reject any article for other
reasons than those stated here.
A storage agent that elects to honour a "cancel" message SHOULD make A primary goal of the Netnews protocol is to ensure that all readers
the article unavailable for relaying or storage (perhaps by deleting receiving a particular article (as uniquely identified by the content
it completely). If the target article is unavailable, and the of its Message-ID header field) see the identical article, apart from
acceptability of the "cancel" message cannot be established without allowable divergence in trace headers and local metadata.
it, activation of the "cancel" message SHOULD be delayed until the Accordingly, agents (other than moderators) MUST NOT modify articles
target article has been seen. See also sections 6.3 and 6.4. in ways other than described here. Unacceptable articles MUST be
rejected rather than corrected.
NOTE: It is expected that the security extension envisaged in 3.2. The Path Header Field
section 5.1 will make more detailed provisions for establishing
whether honouring a particular "cancel" message is in order. In
particular, it is likely that there will be provision for the
digital signature of 3rd party cancels (i.e. those issued other
than by the sender, the moderator, or the injector).
NOTE: A cancel submitted by the poster for an article in a All news server components (injecting agents, relaying agents, and
moderated group will be forwarded to the moderator of that serving agents) MUST identify themselves, when processing an article,
group, and it is up to that moderator to act upon it (6.8). by prepending their <path-identity> (defined in Section 3.1.5 of
[USEFOR]) to the Path header field. Injecting agents MUST also use
the same identity in Injection-Info header fields they add, and
serving and relaying agents SHOULD use the same identity in any Xref
header fields they add.
NOTE: The former requirement [RFC 1036] that the From and/or The <path-identity> used by an agent may be chosen via one of the
Sender header fields of the "cancel" message should match those following methods (in decreasing order of preference):
of the original article has been removed from this standard,
News Article Architecture and Protocols November 2006
since it only encouraged cancel issuers to conceal their true 1. The fully-qualified domain name (FQDN) of the system on which the
identity, and it was not usually checked or enforced by agent is running.
canceling software. Therefore, both the From and/or Sender
header fields and any Approved header field should now relate to
the entity responsible for issuing the "cancel" message.
5.4. Ihave, sendme 2. A fully-qualified domain name (FQDN) within a domain affiliated
with the administrators of the agent and guaranteed to be unique
by the administrators of that domain. For example, the
uniqueness of server.example.org could be guaranteed by the
administrator of example.org even if there is no DNS record for
server.example.org itself.
The "ihave" and "sendme" control messages implement a crude batched 3. Some other (arbitrary) name in the form <path-nodot> believed to
predecessor of the NNTP [RFC 3977] protocol. They are largely be unique and registered at least with all the other news servers
obsolete on the Internet, but still see use in conjunction with some to which that relaying agent or injecting agent sends articles.
transport protocols such as UUCP, especially for backup feeds that This option SHOULD NOT be used unless the earlier options are
normally are active only when a primary feed path has failed. There unavailable or unless the name is of longstanding usage.
is no requirement for relaying agents that do not support such
transport protocols to implement them.
NOTE: The ihave and sendme messages defined here have ABSOLUTELY Some existing implementations treat <path-identity> as case-
NOTHING TO DO WITH NNTP, despite similarities of terminology. sensitive, some case-insensitive. The <path-identity> therefore
SHOULD be all lowercase and implementations SHOULD compare identities
case-insensitively.
The two messages share the same syntax: 3.2.1. Constructing the Path Header Field
control-command =/ Ihave-command If a relaying or serving agent receives an article from an injecting
Ihave-command = "ihave" Ihave-argument or serving agent that is part of the same news server, it MAY leave
Ihave-argument = relayer-name the Path header field of the article unchanged. Otherwise, every
control-command =/ Sendme-command injecting, relaying, or serving agent that accepts an article MUST
Sendme-command = "sendme" Sendme-argument update the Path header field as follows. Note that the Path header
Sendme-argument = Ihave-argument field content is constructed from right to left by prepending
relayer-name = path-identity ; see F-3.1.5 elements.
ihave-body = *( msg-id CRLF )
sendme-body = ihave-body
The body of the message consists of a list of <msg-id>s, one per 1. The agent MUST prepend "!" to the Path header field content.
line. [RFC 1036] also permitted the list of <msg-id>s to appear in
the <Ihave-> or <Sendme-argument> with the syntax
Ihave-argument = [FWS] *( msg-id FWS ) [relayer-name]
but this form SHOULD NOT now be used, though relaying agents MAY
recognize and process it for backward compatibility.
The "ihave" message states that the named relaying agent has received 2. An injecting agent SHOULD prepend the <path-diagnostic>
articles with the specified message identifiers, which may be of "!.POSTED", optionally followed by "." and the FQDN or IP address
interest to the relaying agents receiving the ihave message. The of the source, to the Path header field content.
"sendme" message requests that the agent receiving it send the
articles having the specified message identifiers to the named
relaying agent.
Upon receipt of the sendme message, the receiving agent sends the 3. A relaying or serving agent SHOULD prepend a <path-diagnostic> to
article(s) requested, often (especially when the transport protocol the Path header field content, where the <path-diagnostic> is
is UUCP) in the form of one or more batches, each containing several chosen as follows:
articles. The usual form of a <batch> is defined by the following
syntax (which is also used in the application/news transmission media
type (4.1)).
News Article Architecture and Protocols November 2006 * If the expected <path-identity> of the source of the article
matches the leftmost <path-identity> of the Path header
field's content, use "!" (<diag-match>).
batch = 1*( batch-header article ) * If the expected <path-identity> of the source of the article
batch-header = "#!" SP rnews SP article-size CRLF does not match, use "!.MISMATCH." followed by the expected
rnews = %x72.6E.65.77.73 ; case sensitive "rnews" <path-identity> of the source or its IP address.
article-size = 1*DIGIT
Thus a <batch> is a sequence of articles, each prefixed by a header * If the relaying or serving agent is not willing or able to
line that includes its size. The <article-size> is a decimal count of check the <path-identity>, use "!.SEEN." followed by the FQDN,
the octets in the article, counting each CRLF as one octet regardless IP address, or expected <path-identity> of the source.
of how it is actually represented.
NOTE: Despite the similarity of this format to an executable 4. The agent MUST then prepend its own <path-identity> to the Path
UNIX script, it is EXTREMELY unwise to feed such a batch into a header field content.
command interpreter in anticipation of it running a command
named "rnews"; the security implications of so doing would be
disastrous.
These control messages are normally sent essentially as point-to- 5. The agent MAY then prepend additional <path-identity>s for itself
point messages, by using <newsgroup-name>s in the Newsgroups header to the Path header field content, following each <path-identity>
field of the form "to." followed by one (or possibly more) so added with either "!!" or "!". This is permitted for agents
<component>s in the form of a <relayer-name> (see section F-3.1.4 that have multiple <path-identity>s (such as during a transition
which forbids "to" as the first <component> of a <newsgroup-name>). from one to another). Each of these <path-identity>s MUST meet
The control message SHOULD then be delivered ONLY to the relaying the requirements set out in Section 3.2.
agent(s) identified by that <relayer-name>, and any relaying agent
receiving such a message which includes its own <relayer-name> MUST
NOT propagate it further. Each pair of relaying agent(s) sending and
receiving these messages MUST be immediate neighbours, exchanging
news directly with each other. Each relaying agent advertises its new
arrivals to the other using "ihave" messages, and each uses "sendme"
messages to request the articles it lacks.
To reduce overhead, ihave and sendme messages SHOULD be sent Any agent which modifies the Path header field MAY fold it by
relatively infrequently and SHOULD contain reasonable numbers of inserting FWS immediately after any <path-identity> or <diag-other>
message identifiers. If ihave and sendme are being used to implement it added (see section 3.1.5 of [USEFOR] for allowable locations for
a backup feed, it may be desirable to insert a delay between FWS).
reception of an ihave and generation of a sendme, so that a slightly
slow primary feed will not cause large numbers of articles to be
requested unnecessarily via sendme.
5.5. Obsolete control messages. 3.2.2. Path Header Field Example
The following control messages (as described in Appendix A) are Here is an example of a Path header field created following the rules
declared obsolete by this standard: for injecting and relaying agents.
sendsys Path: foo.isp.example!.SEEN.isp.example!!foo-news
version !.MISMATCH.2001:DB:0:0:8:800:200C:417A!bar.isp.example
whogets !!old.site.example!barbaz!!baz.isp.example
senduuname !.POSTED.dialup123.baz.isp.example!not-for-mail
6. Duties of Various Agents This article was injected by baz.isp.example as indicated by the
<diag-keyword> "POSTED". The injector has recorded that it received
the article from dialup123.baz.isp.example. "not-for-mail is a common
<tail-entry>.
The following section sets out the duties of various agents involved The article was relayed to the relaying agent known, at least to
in the creation, relaying and storage of Netnews articles. Insofar as old.site.example, as "barbaz".
these duties are described as sequences of steps to be followed, it
should be understood that it is the effect of these sequences that is
News Article Architecture and Protocols November 2006
important, and implementations may use any method that gives rise to barbaz relayed it to old.site.example, which does not support <diag-
that same effect. keyword> and therefore used the old "!" delimiter. This indicates
that the identity of "barbaz" was not verified and may have been
forged.
In this section, the word "verified", as applied to the source of old.site.example relayed it to a news server using the <path-
some article, means that an agent processing that article has identity> of bar.isp.example and claiming (by using the "!!" <path-
established, by some means, the identity of that source (which may be delimiter>) to have verified that it came from old.site.example.
another agent or a poster).
NOTE: In many implementations, a single agent may perform bar.isp.example relayed it to foo-news which, not being convinced
various combinations of the injecting, relaying and storage that it truly came from bar.isp.example, inserted the <diag-keyword>
functions. Its duties are then the union of the various duties "MISMATCH" and then stated that it received the article from the IPv6
concerned. address [2001:DB8:0:0:8:800:200C:417A]. (This is not to say that
bar.isp.example was not a correct <path-identity> for that source but
simply that that identity did not match the expectations of foo-news.
6.1. General principles to be followed foo-news then passed the article to foo.isp.example, which declined
to validate its <path-identity> and instead appended the <diag-
keyword> "SEEN" to indicate it knows the source of the article as
isp.example. This may be either an expected <path-identity> or the
FQDN of the system from which it received the article. Presumably
foo.isp.example is a serving agent that then delivered the article to
a reading agent.
There are two important principles that news implementors (and baz.isp.example, bar.isp.example, and foo-news folded the Path header
administrators) need to keep in mind. The first is the well-known field.
Internet Robustness Principle:
Be liberal in what you accept, and conservative in what you 3.3. Duties of a Posting Agent
send.
However, in the case of news there is an even more important A posting agent is the component of a user agent that assists a
principle, derived from a much older code of practice, the poster in creating a valid proto-article and forwarding it to an
Hippocratic Oath (we may thus call this the Hippocratic Principle): injecting agent.
First, do no harm. Posting agents SHOULD ensure that proto-articles they create are
valid according to [USEFOR] and any other applicable policies. They
MUST NOT create any Injection-Date or Injection-Info header fields;
these headers will be added by the injecting agent.
It is VITAL to realize that decisions which might be merely Contrary to [RFC2822], which implies that the mailbox or mailboxes in
suboptimal in a smaller context can become devastating mistakes when the From header field should be that of the poster or posters, a
amplified by the actions of thousands of hosts within a few minutes. poster who does not, for whatever reason, wish to use his own mailbox
MAY use any mailbox ending in the top level domain ".invalid"
[RFC2606].
In the case of gateways, the primary corollary to this is: Posting agents meant for use by ordinary posters SHOULD reject any
attempt to post an article which cancels or Supersedes another
article of which the poster is not the author or sender.
Cause no loops. 3.3.1. Proto-articles
6.2. Duties of an Injecting Agent A proto-article is an article in the format used by a posting agent
for offering to an injecting agent. It may omit certain header
fields which can be better-supplied by the injecting agent and will
not contain header fields that are added by the injecting agent. A
proto-article is only for transmission to an injecting agent and
SHOULD NOT be transmitted to any other agent.
An Injecting Agent is responsible for taking a (proto-)article from a A proto-article has the same format as a normal article except that
posting (or other) agent and either forwarding it to a moderator or the Injection-Date, Injection-Info, and Xref header fields MUST NOT
injecting it into the relaying system for access by readers. be present; the Path header field MUST NOT contain a "POSTED" <diag-
keyword>; and any of the following mandatory header fields MAY be
omitted: Message-ID, Date, and Path. In all other respects, a proto-
article MUST be a valid Netnews article. In particular, the header
fields which may be omitted MUST NOT be present with invalid content.
As such, an injecting agent is considered responsible for ensuring If a posting agent intends to offer the same proto-article to
that any article it injects conforms with the rules of [USEFOR]. It multiple injecting agents, the header fields Message-ID and Date MUST
is also expected to bear some responsibility towards the rest of the be present and identical in all copies of the proto-article.
network for the behaviour of its posters.
In the normal course of events, an article that has already been 3.3.2. Reinjection of Articles
injected into a Netnews network will never pass through another
injecting agent. So, if an injecting agent receives an otherwise
valid article that has already been injected (as evidenced by the
presence of an Injection-Date header field, an Injection-Info header
field, or more than one occurrence of the <diag-keyword> "POSTED" in
a Path header field) it MAY choose to reject it, but otherwise SHOULD
News Article Architecture and Protocols November 2006
cause it to be relayed, as it stands, by a relaying agent (6.3). A given article SHOULD be processed by an injecting agent once and
only once. The Injection-Date or Injection-Info header fields are
added by an injecting agent and are not permitted in a proto-article.
Their presence (or the presence of other unstandardized or obsolete
trace headers such as NNTP-Posting-Host, NNTP-Posting-Date, or
X-Trace) indicates that the proto-article is instead an article and
has already been processed by an injecting agent. A posting agent
SHOULD normally reject such articles.
In exceptional circumstances (e.g. as part of some complex gatewaying In the exceptional case that an article needs to be reinjected for
process, or where a relaying agent considers it essential for some reason (such as transferring an article from one Netnews to
fulfilling its responsibility towards the rest of the network) an another where those networks have no relaying agreement), the posting
already injected article MAY be "reinjected" into the network. This agent doing the reinjection MUST convert the article back into a
standard does not prescribe any such circumstance; rather this is a proto-article before passing it to an injecting agent (such as by
matter of policy to be determined by the administrators of each renaming the Injection-Info and Injection-Date header fields and
injecting agent, who have the responsibility to ensure that no harm removing any Xref header field) and MUST perform the date checks on
arises. In all other circumstances, unintented reinjection is to be the existing Injection-Date or Date header fields that would
avoided (see 6.9). Nevertheless, in order to preserve the integrity otherwise be done by the injecting agent.
of the network in these special cases, this standard does set out the
correct way to reinject (see special provisions in 6.2.2 Steps 3, 7
and 9).
It is usual for an injecting agent to be closely associated with a Reinjecting articles may cause loops, loss of trace information, and
storage agent, thus giving it access to the list (6.4) showing the other problems and should only be done with care and when there is no
moderation status of the newsgroups it is likely to handle. In the available alternative. A posting agent that does reinjection is a
event that it does not have such an associated storage agent, it MUST limited type of gateway and as such is subject to all of the
maintain that list itself. requirements of an incoming gateway in addition to the requirements
of a posting agent.
6.2.1. Proto-articles 3.3.3. Followups
A proto-article SHOULD NOT be propagated in that form to other than A followup is an article that contains a response to the contents of
injecting agents. an earlier article, its precursor. In addition to its normal duties,
a posting agent preparing a followup is also subject to the following
requirements. Wherever in the following it is stated that by default
a header field is said to be inherited from one of those header
fields in the precursor, it means that its initial content is to be a
copy of the content of that precursor header field (with changes in
folding permitted). However, posters MAY then override that default
before posting.
A proto-article has the same format as a normal article except that Despite the historic practice of some posting agents, the Keywords
some of the following mandatory header fields MAY be omitted: header field SHOULD NOT be inherited by default from the precursor
Message-Id, Date, Path (and even From if the particular injecting article.
agent can derive that information from other sources). However, if
it is intended to offer the proto-article to two or more injecting
agents in parallel, then it is only the Path header field that MAY be
omitted. The header fields that can be omitted MUST NOT contain
invalid values; they MUST either be correct or not present at all.
[Maybe omit that last sentence.]
NOTE: An article that is offered for reinjection has, by 1. If the Followup-To header field of the precursor article consists
definition, already been injected once, and is not therefore to of "poster", the followup MUST NOT be posted by default but by
be considered as a proto-article. Hence a genuine proto-article default is to be emailed to the address given in the precursor's
will not contain any Injection-Date header field nor the <diag- Reply-To or From header field following the rules for an email
keyword> "POSTED" anywhere in its Path header field, though that reply [RFC2822]. This action MAY be overridden by the poster, in
header field MAY contain <path-identity>s corresponding to which case the posting agent should continue as if the
machines traversed between the posting agent and the injecting Followup-To header field in the precursor did not exist.
agent proper.
6.2.2. Procedure to be followed by Injecting Agents 2. The Newsgroups header field SHOULD by default be inherited from
the precursor's Followup-To header field if present, and
otherwise from the precursor's Newsgroups header field.
An injecting agent receives (proto-)articles from posting and 3. The Subject header field SHOULD by default be inherited from that
followup agents. It verifies them, adds header fields where required, of the precursor. The case-sensitive string "Re: " (including
and then either forwards them to a moderator or injects them by the space after the colon) MAY be prepended to the content of its
passing them to storage or relaying agents. It MUST NOT forward an Subject header field unless it already begins with that string.
already injected article to a moderator.
News Article Architecture and Protocols November 2006 NOTE: Prepending "Re: " serves no protocol function and hence
is not required, but it is widely expected and not doing so
would be surprising.
An injecting agent processes articles as follows: 4. The Distribution header field SHOULD by default be inherited from
the precursor's Distribution header field, if present.
1. It MUST remove any Injection-Info header field already present 5. The followup MUST have a References header field referring to its
(though it might be useful to copy it to a suitable "X-" header precursor constructed in accordance with Section 3.3.4.
field). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace,
or other non-standard tracing header field.
2. It SHOULD ensure that the article is from a verified source, and 3.3.4. Construction of the References Header Field
MAY reject articles in which header fields contain unverified
email addresses, that is, addresses which are not known to be
valid for the verified source, though it would be perverse to
reject intentionally unverifiable addresses such as those ending
in ".invalid" (6.5).
3. It SHOULD reject any article whose Date header field (F-3.1.1) is The following procedure is to be used whenever some previous article
more than 24 hours into the future (and MAY use a margin less than (the "parent") is to be referred to in the References header field of
that 24 hours). It MUST (except when reinjecting) reject any a new article, whether because the new article is a followup and the
article with an Injection-Date header field already present (and parent is its precursor or for some other reason.
SHOULD do likewise with any NNTP-Posting-Date header field). When
reinjecting it MAY, in the absence of any Injection-Date header
field, reject any article whose Date header field appears to be
stale (e.g. more than 72 hours into the past).
4. It MUST reject any article that does not have the proper mandatory The content of the new article's References header field MUST be
header fields for a proto-article or which contains any header formed from the content of the parent's References header field if
field that does not have legal contents. It SHOULD reject any present and the content of the Message-ID header field of the parent.
article which contains any header field deprecated for Netnews If the parent had a References header, FWS as defined in [USEFOR]
(e.g. as in [RFC 2298]). It SHOULD reject any article whose MUST be added between its content and the Message-ID header field
Newsgroups header field does not contain at least one <newsgroup- content.
name> for an existing group (as listed by its associated storage
agent) and it MAY reject any <newsgroup-name> which violates one
of the restrictions in F-3.1.4 or which, although otherwise
correct, violates a policy restriction established, for some
(sub-)hierarchy, by an agency with the appropriate authority
(1.2). Observe that crossposting to unknown newsgroups is not
precluded provided at least one of those in the Newsgroups header
field is listed.
NOTE: This ability to reject <newsgroup-name>s in breach of If the resulting References header field would, after unfolding,
established policy does not extend to relaying agents, though it exceed 998 characters in length (including its field name but not the
might be reasonable for posting agents to do it. final CRLF), it MUST be trimmed (and otherwise MAY be trimmed).
Trimming means removing any number of message identifiers from its
content, except that the first message identifier and the last two
MUST NOT be removed.
5. If the article is rejected (for reasons given above, or for other An essential property of the References header field, guaranteed by
formatting errors or matters of site policy) the posting agent the above procedure and REQUIRED to be maintained by any extensions
SHOULD be informed (such as via an NNTP 44x response code) that to this protocol, is that an article MUST NOT precede one of its
posting has failed and the article MUST NOT then be processed parents.
further.
6. The Message-ID, Date and From header fields (with appropriate 3.4. Duties of an Injecting Agent
contents) MUST be added when not already present. A User-Agent
header field MAY be added (or an already present User-Agent header
field MAY be augmented) so as to identify the software (e.g.
"INN/1.7.2") used by the injecting agent.
News Article Architecture and Protocols November 2006 An injecting agent takes a proto-article from a posting agent and
either forwards it to a moderator or passes it to a relaying or
serving agent or agents. An injecting agent bears the primary
responsibility for ensuring that any article it injects conforms with
the rules of the Netnews standards. The administrator of an
injecting agent is also expected to bear some responsibility towards
the rest of the Netnews network to which it is connected for the
articles the injecting agent accepts.
[I think that last sentence needs to go (in which case see consequential Injecting agents, when rejecting articles, are encouraged to
change in 7.3). Did we discuss this when we looked at User-Agent in communicate the reason for rejection to the posting agent using
USEFOR?] whatever facility is provided by the underlying transport. The
injecting agent is in a unique position to communicate the reason for
rejection; relaying agents and serving agents normally have to reject
messages silently. The injecting agent therefore bears much of the
burden of diagnosing broken posting agents or communicating policy
violations to posters.
NOTE: The Message-ID, Date and From fields will already be An injecting agent MUST have available a list (possibly empty) of
present during reinjection. moderated groups for which it accepts articles and the corresponding
submission addresses. It SHOULD have available a list of valid
newsgroups to catch articles not posted to a valid newsgroup and
therefore likely to be silently discarded by relaying and serving
agents. Usually, an injecting agent is deployed in conjunction with
a serving agent and maintains these lists based on control messages
received by the serving agent.
7. The injecting agent MUST NOT alter the body of the article in any An injecting agent processes proto-articles as follows:
way (including any change of Content-Transfer-Encoding). It MAY
(except when reinjecting) add other header fields not already
provided by the poster, but SHOULD NOT alter, delete, or reorder
any existing header field, with the specific exception of the
"tracing" header field Injection-Info, which is to be removed as
already mentioned.
8. If the Newsgroups header field contains one or more moderated 1. It SHOULD verify that the article is from a trusted source (by,
groups and the article does NOT contain an Approved header field, for example, relying on the authorization capability of the
the injecting agent MUST forward it to a moderator as specified in underlying transport used to talk to the posting agent).
section 6.2.3 below.
9. Otherwise, a Path header field with a <tail-entry> (F-3.1.5) MUST 2. It MUST reject any proto-article that does not have the proper
be correctly added if not already present. During reinjection, the mandatory header fields for a proto-article; that has Injection-
existing Path header field SHOULD be retained. Date, Injection-Info, or Xref header fields; that has a Path
header field containing the "POSTED" <diag-keyword>; or that is
not syntactically valid as defined by [USEFOR]. It SHOULD
reject any proto-article which contains a header field
deprecated for Netnews. It MAY reject any proto-article that
contains trace header fields indicating that it was already
injected by an injecting agent that did not add Injection-Info
or Injection-Date.
10.It MUST then prepend the <path-identity> of the injecting agent, 3. It SHOULD reject any article whose Date header field is more
followed by '!.' and the <diag-keyword> "POSTED", and then a than 24 hours into the future (and MAY use a margin less than 24
further "!", to the content of the Path header field; this header hours). It SHOULD reject any article whose Date header appears
field SHOULD then be folded if it would otherwise result in a to be stale (more than 72 hours into the past, for example, or
header line of excessive length. too old to still be recorded in the database of a relaying agent
the injecting agent will be using) since not all news servers
support Injection-Date.
NOTE: This could result in more that one "POSTED" <path-keyword> 4. It SHOULD reject any proto-article whose Newsgroups header field
in the case of reinjection. does not contain at least one <newsgroup-name> for a valid
group, or containing a <newsgroup-name> reserved for specific
purposes by Section 3.1.4 of [USEFOR] unless that specific
purpose or local agreement applies to the proto-article being
processed. Crossposting to unknown newsgroups is not precluded
provided that at least one of the newsgroups in the Newsgroups
header is valid.
11.An Injection-Info header field (F-3.2.8) SHOULD be added, 5. The Message-ID and Date header fields with appropriate contents
identifying the verified source of the article and possibly an MUST be added when not present in the proto-article.
address for mailing complaints to. Each injecting agent SHOULD
use a consistent form of the Injection-Info header field for all
articles emanating from the same or similar origins.
NOTE: The step above is the only place in which an Injection- 6. The injecting agent MUST NOT alter the body of the article in
Info header field is to be created. It follows that this header any way (including any change of Content-Transfer-Encoding). It
field MUST NOT be created, replaced, changed or deleted by any MAY add other header fields not already provided by the poster,
other agent (except during reinjection, in which case it will but injecting agents are encouraged to use the Injection-Info
always relate to the latest injection and is, to that extent, header for such information and to minimize the addition of
regarded as a variant header field). other headers. It SHOULD NOT alter, delete, or reorder any
existing header field except the Path header.
12.The injecting agent MUST then add an Injection-Date header field 7. If the Newsgroups header contains one or more moderated groups
(F-3.2.7) if one is not already present, but it MUST NOT alter, or and the proto-article does not contain an Approved header field,
delete, an already present Injection-Date header field (and the injecting agent MUST either forward it to a moderator as
likewise SHOULD NOT alter, or delete, an already present NNTP- specified in Section 3.4.1 or, if that is not possible, reject
Posting-Date header field). Finally, it forwards the article to it. This forwarding MUST be done after adding the Message-ID
one or more relaying or storage agents, and the injection process and Date headers if required, and before adding the Injection-
is to be considered complete. Info and Injection-Date headers.
News Article Architecture and Protocols November 2006 8. Otherwise, a Path header field with a <tail-entry> MUST be added
if not already present.
NOTE: The step above is the only place where an Injection-Date 9. The injecting agent MUST then update the Path header field as
header field is to be created It follows that it MUST NOT described in Section 3.2.1.
subsequently be replaced, changed or deleted by any other agent,
even during reinjection.
6.2.3. Procedure for Forwarding to a Moderator 10. An Injection-Info header field SHOULD be added identifying the
source of the article and possibly other trace information as
described in Section 3.2.8 of [USEFOR].
An injecting agent forwards an article to a moderator as follows: 11. The injecting agent MUST then add an Injection-Date header field
containing the current date and time.
1. It MUST forward it to the moderator of the first (leftmost) 12. Finally, the injecting agent forwards the article to one or more
moderated group listed in the Newsgroups header field, customarily relaying agents, and the injection process is complete.
via email, (see 6.8 for how that moderator may forward it to
further moderators). There are two possibilities for doing this:
(a) The complete article is encapsulated (header fields and all) 3.4.1. Forwarding Messages to a Moderator
within the email, preferably using the Content-Type
"application/news-transmission" (4.1) with any usage
parameter set to "moderate". Moreover, there SHOULD NOT be
more than one encapsulated article within the one email.
This method has the advantage of removing any possible
conflict between Netnews and Email header fields, or of
changes to those fields during transport through email.
(b) The article is sent as an email as it stands, with the An injecting agent MUST forward the proto-article to the moderator of
addition of such extra header fields (e.g. a To header field) the leftmost moderated group listed in the Newsgroups header field,
as are necessary for an email. The existing Message-ID header customarily via email. There are two standard ways in which it may
field SHOULD be retained. do this:
Although both of these methods have seen use in the past, the 1. The complete proto-article is encapsulated, header fields and
preponderance of current usage on Usenet has been for method (b) all, within the email. This SHOULD be done by creating an email
and many moderators are ill-prepared to deal with method (a). message with a Content-Type of application/news-transmission with
Therefore, method (a) SHOULD NOT be used until such time as the the usage parameter set to "moderate". The body SHOULD NOT
majority of moderators are able to accept it. contain any content other than the message. This method has the
advantage of removing any possible conflict between Netnews and
email header fields and any changes to those fields during
transport through email.
2. This standard does not prescribe how the email address of the 2. The proto-article is sent as an email with the addition of any
moderator is to be determined, that being a matter of policy to be header fields (such as a To header field) required for an email.
arranged by the agency responsible for the oversight of each The existing Message-ID header field SHOULD be retained.
hierarchy. Nevertheless, there do exist various agents worldwide
which provide the service of forwarding to moderators, and the
address to use with them is obtained as follows:
(a) Each '.' in the <newsgroup-name> is replaced with a '-'. Although both of these methods have been used in the past and the
first has clear technical advantages, the second is in more common
use and many moderators are not prepared to deal with messages in the
first format. Accordingly, the first method SHOULD NOT be used
unless the moderator to which it is being forwarded is known to be
able to handle this method.
(b) The result of these operations is used as the <local-part> of NOTE: Deriving the email address of the moderator of a group is
the <mailbox> of the agent. For example, articles intended outside the scope of this document. It is worth mentioning,
for "news.announce.important" would be emailed to "news- however, that a common method is to use a forwarding service that
announce-important@forwardingagent.example". handles submissions for many moderated groups. For maximum
compatibility with existing news servers, such forwarding services
generally form the submission address for a moderated group by
replacing each "." in the <newsgroup-name> with "-" and then using
that value as the <local-part> of a <mailbox> formed by appending
a set domain.
6.3. Duties of a Relaying Agent Forwarding proto-articles to moderators via email is the most general
method and most common in large Netnews networks such as Usenet, but
any means of forwarding the article that preserves it without
injecting it MAY be used. For example, if the injecting agent has
access to a database used by the moderator to store proto-articles
awaiting processing, it may place the proto-article directly into
that database. Such methods may be more appropriate for smaller
Netnews networks.
A Relaying Agent accepts injected articles from injecting and other 3.5. Duties of a Relaying Agent
relaying agents and passes them on to relaying or storage agents
according to mutually agreed policy. Relaying agents SHOULD accept
articles ONLY from verified sources.
News Article Architecture and Protocols November 2006 A relaying agent accepts injected articles from injecting and other
relaying agents and passes them on to relaying or serving agents. To
avoid bypass of injecting agent policies and forgery of Path and
Injector-Info headers, relaying agents SHOULD accept articles only
from trusted agents.
An article SHOULD NOT be relayed unless the sending agent has been An article SHOULD NOT be relayed unless the sending agent has been
configured to supply and the receiving agent to receive at least one configured to supply and the receiving agent to receive at least one
of the <newsgroup-name>s in its Newsgroups header field and at least of the <newsgroup-name>s in its Newsgroups header field and at least
one of the <dist-name>s in its Distribution header field, if any. one of the <dist-name>s in its Distribution header field (if
Exceptionally, ALL relaying agents are deemed willing to supply or present). Exceptionally, control messages creating or removing
accept the <dist-name> "world", and NO relaying agent should supply newsgroups (newgroup or rmgroup control messages, for example) SHOULD
or accept the <dist-name> "local". be relayed if the affected group appears in its Newsgroups header
field and and the sending agent and receiving relaying agents are
However, if the particular implementation does not relay non-existent both configured to relay a newsgroup of that name (whether or not
newsgroups, even when included in the Newsgroups header field and such a newsgroup exists).
implied (e.g. by some "wild card" notation) in the configuration
tables, then the agent MUST examine all group control messages (5.2)
in order to ensure that relaying of those messages proceeds normally.
NOTE: Although it would seem redundant to filter out unwanted
distributions at both ends of a relaying link (and it is clearly
more efficient to do so at the sending end), many sending sites
have been reluctant, historically speaking, to apply such
filters (except to ensure that distributions local to their own
site or cooperating subnet did not escape); moreover they tended
to configure their filters on an "all but those listed" basis,
so that new and hitherto unheard of distributions would not be
caught. Indeed many "hub" sites actually wanted to receive all
possible distributions so that they could feed on to their
clients in all possible geographical (or organizational)
regions.
Therefore, it is desirable to provide facilities for rejecting
unwanted distributions at the receiving end. Indeed, it may be
simpler to do so locally than to inform each sending site of
what is required, especially in the case of specialized
distributions (for example for control messages, such as cancels
from certain issuers) which might need to be added at short
notice. A similar possibility for reading agents to filter
distributions is also suggested in [USEAGE]) for the same
reason.
In order to avoid unnecessary relaying, an article SHOULD NOT be
relayed if the <path-identity> of the receiving agent (or some known
alias thereof) appears as a <path-identity> in its Path header field.
But note that the <tail-entry> (which follows the last "!") is not a
<path-identity>, although not all current implementations observe
this distinction.
For this to work, each relaying agent needs to insert it own <path-
identity> (chosen according to 2.3) into the Path header field. It
MAY insert more than one <path-identity> for itself (in which case
the leftmost should be the one by which it is known to its immediate
neighbours), but where an article passes through several relaying
agents at the same site it MAY omit the <path-identity>s for some of
them (but NOT the one which finally relays it to an outside site).
News Article Architecture and Protocols November 2006
It MAY (and usually SHOULD) also add a <path-diagnostic> giving
additional information concerning the route taken by the article
through the network. A <path-diagnostic> consists of either the
special <diag-match> "!" (which effectively replaces the standard
delimiter "!" by "!!"), or it is composed from a <diag-keyword>
usually followed by a <diag-identity>. The following are the only
<diag-keyword>s defined by this standard:
o "POSTED" (already introduced in Step 10 of 6.2.2), which is never
followed by a <diag-identity>;
o "SEEN", whose following <diag-identity> indicates the verified
identity (see 6) of the agent from which the article was received
(but makes no claim as to whether it matched the <path-identity)
inserted by that agent);
o "MISMATCH", whose following <diag-identity> indicates the
verified identity of the agent from which the article was
received and asserts that it could not be reconciled with the
<path-identity> (supposedly) inserted by that agent.
Other <diag-keyword>s beginning with "X" MAY be used by a relaying
agent to make some assertion not envisaged here, but other (non-"X")
<diag-keyword>s MUST NOT be used unless defined by some extension to
this standard.
NOTE: The <diag-keyword> "MATCH", which might have indicated the
verified identity of the source agent with an assertion that it
agreed with the <path-identity> inserted by that agent, has NOT
been provided, since the special <diag-match> conveys exactly
that meaning for this commonly occurring case.
NOTE: Whilst <diag-keywords>s are case insensitive, it is
intended that they should normally be rendered in uppercase.
A relaying agent processes articles as follows:
1. It MUST/SHOULD establish the verified identity of the source of
the article and compare it with the leftmost <path-identity> of
the existing Path header field's content. Except possibly when
relaying to other hosts on the same site, It then MUST or SHOULD,
as indicated, prepend to that content (from left to right) the
following:
o (MUST) its own <path-identity>;
o (MUST) a "!" delimiter;
o (MUST/SHOULD) if the verified and existing identities match,
a <diag-match> (effectively converting the "!" delimiter into
"!!");
o (MUST/SHOULD) alternatively, where the identities do not
match (or have not been determied to match), a ".", the
<diag-keyword> "MISMATCH" (or "SEEN"), another ".", a <diag-
identity> indicating the verified identity, and finally a
further "!";
o possibly further <path-identity>s etc. as above, identifying
itself.
This header field SHOULD then be folded if it would otherwise
result in a header line of excessive length.
News Article Architecture and Protocols November 2006
[The "MUST/SHOULD"s above were all "MUST" in the previous drafts.
Discussion is needed to resolve this.]
NOTE: Since each agent at one site can be assumed to be aware of
the identity of the others (and of itself), it would be most
unusual for their <path-identity>s to be separated other than by
"!!". Thus the presence of a single "!", unless followed by a
"." and some <diag-keyword>, can be taken as signifying an agent
that has not yet been upgraded to conform to this standard.
NOTE: Whilst the presence of a "MISMATCH" would normally
indicate that the existing Path was bogus in some sense, it
could also indicate that the ralaying agent was improperly
configured to recognise the identities or aliases used by its
neighbours. Administators of relaying agents should therefore
periodically monitor the <path-diagnostic> being inserted so as
to avoid this.
NOTE: In order to prevent overloading, relaying agents should
not routinely query an external entity (such as a DNS-server) in
order to determine a verified identity (though a local cache of
the required information might usefully be consulted).
2. It MUST examine the Injection-Date header field (or, if that is In order to avoid unnecessary relaying attempts, an article SHOULD
absent, the Date header field) and reject the article as stale NOT be relayed if the <path-identity> of the receiving agent (or some
(F-3.2.7) if that predates the earliest articles of which it known alias thereof) appears as a <path-identity> (excluding within
normally keeps record, or if it is more than 24 hours into the the <tail-entry> or following a "POSTED" <diag-keyword>) in its Path
future (the margin MAY be less than that 24 hours). header field.
3. It SHOULD reject any article that does not include all the A relaying agent processes an article as follows:
mandatory header fields (section F-3.1).
4. It MAY reject any article whose header fields do not have legal 1. It MUST reject any article without a Newsgroups header field or
contents. Message-ID header field, or without either an Injection-Date or
Date header field.
5. It SHOULD reject any article that has already been sent to it (a 2. It MUST reject any article that has already been successfully
database of message identifiers of recent messages is usually kept sent to it, based on the Message-ID header field of the article.
and matched against). To satisfy this requirement, a relaying agent normally keeps a
database of message identifiers it has already accepted.
NOTE: Even though commonly derived from the domain name of the 3. It MUST examine the Injection-Date header field or, if absent,
originating site (and domain names are case-insensitive), a the Date header field, and reject the article if that date
<msg-id-core> MUST NOT be altered in any way during transport, predates the earliest articles of which it keeps record or if
or when copied (as when forming a References header field), and that date is more than 24 hours into the future. It MAY reject
thus a simple (case-sensitive) comparison of octets will always articles with dates in the future with a smaller margin than 24
suffice to recognize that same message identifier wherever it hours.
subsequently reappears.
NOTE: These requirements are to be contrasted with those of the 4. It SHOULD reject any article that does not include all the
un-normalized msg-ids defined by [RFC 2822], which may perfectly mandatory header fields. It MAY reject any article that contains
legitimately become normalized (or vice versa) during transport header fields that do not have valid contents.
or copying in email systems.
News Article Architecture and Protocols November 2006 5. It SHOULD reject any article that matches an already-received
cancel control message or the contents of the Supersedes header
field of an accepted article, provided that the relaying agent
chose (on the basis of local site policy) to honor that cancel
control message or Supersedes header field.
6. It SHOULD reject any article that matches an already received 6. It MAY reject any article without an Approved header field posted
cancel message (or an equivalent Supersedes header field) issued to a newsgroup known to be moderated. This practice is strongly
by its poster or by some other trusted entity. encouraged but the information necessary to do so is not required
to be maintained by a relaying agent.
7. It MAY reject any article without an Approved header field posted 7. It MUST update the Path header field as described in
to newsgroups known to be moderated (this practice is strongly Section 3.2.1.
recommended, but the information necessary to do so may not be
available to all agents).
8. It MAY delete any Xref header field that is present. 8. It MAY delete any Xref header field present and MAY add a new
Xref header field with any valid content. The Xref header field
is not used by relaying agents, but relaying agents that are also
serving agents may generate Xref header fields for their own
internal purposes.
9. Finally, it passes the articles on to neighbouring relaying and 9. Finally, it passes the article on to other relaying and serving
storage agents. agents to which it is configured to send articles.
If the article is rejected as being invalid, unwanted or unacceptable Relaying agents SHOULD, where possible in the underlying transport,
due to site policy, the agent that passed the article to the relaying inform the agent that passed the article to the relaying agent if the
agent SHOULD be informed (such as via an NNTP 43x response code) that article was rejected. Relaying agents MUST NOT inform any other
relaying failed. In order to prevent a large number of error messages external entity of the rejection of an article unless that external
being sent to one location, relaying agents MUST NOT inform any other
external entity that an article was not relayed UNLESS that external
entity has explicitly requested that it be informed of such errors. entity has explicitly requested that it be informed of such errors.
Relaying agents MUST NOT alter, delete or rearrange any part of an Relaying agents MUST NOT alter, delete, or rearrange any part of an
article except for header fields designated as variant (2.4). In article except for the Path and Xref header fields. They MUST NOT
particular modify the body of articles in any way. If an article is not
acceptable as-is, the article MUST be rejected rather than modified.
o they MUST NOT create or augment a User-Agent header field in
order to identify themselves;
o they MUST NOT rewrite the Newsgroups header field in any way,
even if some supposedly non-existent newsgroup is included;
o they MUST NOT refold any header field (i.e. they must pass on the
folding as received);
o they MUST NOT alter the Date header field or the Injection-Date
header field;
o they MUST NOT delete any unrecognized header field whose field
name is syntactically correct (whether or not it is registered
with IANA [RFC 3864]);
o they MUST NOT change the Content-Transfer-Encoding of the body or
any body part;
o they MUST transmit lines of arbitrary length and articles of
arbitrary size.
6.3.1. Path Header Field Example
Path: bar.isp.example!.SEEN.news3.foo.isp.example!foo.isp.example
!!foo-server!.MISMATCH.2001:DB8:0:0:8:800:200C:417A
!dubious.site.example!!old.site.example!barbaz!!baz.isp.example
!.POSTED!dialup123.baz.isp.example!not-for-mail
NOTE: That article was injected into the news stream by
baz.isp.example, as indicated by the <diag-keyword> "POSTED"
(complaints may be addressed to abuse@baz.isp.example). The
injector has chosen to record that it got it from
dialup123.baz.isp.example. "not-for-mail" is a dummy <tail-
News Article Architecture and Protocols November 2006
entry>, though sometimes a real userid is put there.
The article was relayed, perhaps by UUCP, to the machine known,
at least to old.site.example, as "barbaz".
Barbaz relayed it to old.site.example, which does not yet
conform to this standard (hence the single '!' delimiter). So
one cannot be sure that it really came from barbaz.
Old.site.example relayed it to a site claiming to be
dubious.site.example, and claiming (by using the '!!'
delimiter) to have verified that it came from old.site.example.
Dubious.site.example relayed it to "foo-server" which, not being
convinced that it truly came from dubious.site.example, and
knowing that it in fact arrived from a host with the IPv6address
[2001:DB8:0:0:8:800:200C:417A], inserted the <path-diagnostic>
"!.MISMATCH.2001:DB8:0:0:8:800:200C:417A" (that is not to say
that [2001:DB8:0:0:8:800:200C:417A] was not a correct IPv6
address for dubious-site.example, but simply that that
connection could not be substantiated by foo-server).
"foo-server" is a locally significant name within the complex
site of many machines run by foo.isp.example, so the latter
should have no problem recognizing foo-server and using a '!!'
delimiter. It was not strictly necessary to insert the <path-
identity> "foo-server" as well as "foo.isp.example" (but maybe
some site elsewhere had some reason to test for it).
[Please could Richard Clayton provide a more plausible reason why "foo-
server" might be a <path-nodot> here?]
It then went to bar.isp.example which determined (by reverse
DNS) that it had come from news3.foo.isp.example, but had taken
no steps to check whether that was a known alias for
foo.isp.example (which it probably was). Strictly, it SHOULD
have made such a check, and then inserted either a "!!" or a
"!MISMATCH..." according to the result.
Presumably bar.isp.example then delivered the article to its
direct clients.
It appears that foo.isp.example, foo-server and baz.isp.example
decided to fold the line, on the grounds that it seemed to be
getting a little too long. Note that folding and whitespace is
permitted before (but not after) any "!" (but not within a
"!!"); hence continuation lines will always start with either
"!" or "!!".
6.4. Duties of a Serving Agent
A Serving Agent takes an article from a relaying or injecting agent
and files it in a "news database". It also provides an interface for
reading agents to access the news database. This database is normally
indexed by newsgroup with articles in each newsgroup identified by an
News Article Architecture and Protocols November 2006
<article-locator> (usually in the form of a decimal number - see F-
3.2.14).
A storage agent MUST maintain a list of the newsgroups it stores in
its news database showing the moderation status of each one (see
5.2.1), and SHOULD include in that list all groups likely to be
crossposted to from those groups (e.g. all other groups in the same
hierarchy(ies)).
NOTE: Since control messages are often of interest, but should
not be displayed as normal articles in regular newsgroups, it is
common for storage agents to make them available in a pseudo-
newsgroup named "control" or in a pseudo-newsgroup in a sub-
hierarchy under "control." (e.g. "control.cancel").
A storage agent MAY decline to accept an article if the Path header
field contains some <path-identity> whose articles the storage agent
does not want, as a matter of local policy.
NOTE: This last facility is sometimes used to detect and decline 3.6. Duties of a Serving Agent
control messages (notably cancel messages) which have been
deliberately seeded with a <path-identity> to be "aliased out"
by sites not wishing to act upon them.
[INN at least does this. It might be argued that it is not necessary to
mention it here.]
A storage agent processes articles as follows: A serving agent accepts articles from a relaying agent or injecting
agent, stores it, and makes it available to reading agents. Articles
are normally indexed by newsgroup and <article-locator> (Section
3.2.14 of [USEFOR]), usually in the form of a decimal number.
1. It MUST/SHOULD establish the verified identity of the source of If the serving agent stores articles by newsgroup, control messages
the article and modify the Path header field as for a relaying SHOULD NOT be stored in the newsgroups listed in the control
agent (6.3). message's Newsgroups header field. Instead, they SHOULD be stored in
a newsgroup in the hierarchy "control", which is reserved for this
purpose. Conventionally, control messages are stored in newsgroups
named for the type of control message (such as "control.cancel" for
cancel control messages).
2. It MUST examine the Injection-Date header field (or, if that is A serving agent MUST have available a list (possibly empty) of
absent, the Date header field) and reject the article as stale moderated groups for which it accepts articles so that it can reject
(F-3.2.7) if that predates the earliest articles of which it unapproved articles posted to moderated groups. Frequently a serving
normally keeps record, or if it is more than 24 hours into the agent is deployed in combination with an injecting agent and can use
future (the margin MAY be less than that 24 hours). the same list as the injecting agent.
3. It MUST reject any article that does not include all the mandatory A serving agent processes articles as follows:
header fields (section F-3.1), or which contains any header field
that does not have legal contents.
4. It SHOULD reject any article that has already been sent to it (a 1. It MUST reject any article that does not include all the
database of message identifiers of recent articles is usually kept mandatory header fields or any article which contains header
and matched against). fields that do not have valid contents.
5. It SHOULD reject any article that matches an already received 2. It MUST examine the Injection-Date header field or, if absent,
cancel message (or an equivalent Supersedes header field) issued the Date header field, and reject the article if that date
by its poster or by some other trusted entity. predates the earliest articles of which it keeps record or if
that date is more than 24 hours into the future. It MAY reject
articles with dates in the future with a smaller margin than 24
hours.
Likewise, a newly received cancel message (or equivalent 3. It MUST reject any article that has already been successfully
Supersedes) should cause immediate deletion (or deactivation) of sent to it, based on the Message-ID header field of the article.
the canceled article. To satisfy this requirement, a relaying agent normally keeps a
database of message identifiers it has already accepted.
News Article Architecture and Protocols November 2006 4. It SHOULD reject any article that matches an already-received and
honored cancel message or Supersedes header field following the
same rules as a relaying agent (Section 3.5).
NOTE: An article with a Supersedes header field is itself stored 5. It MUST reject any article without an Approved header field
normally. posted to any newsgroup listed as moderated.
6. It MUST reject any article without an Approved header field posted 6. It MUST update the Path header field as described in
to any newsgroup listed as moderated. Section 3.2.1.
7. It MUST (exept when specially configured to preserve the 7. It MUST (except when specially configured to preserve the
<article-locator>s set by the sending site) remove any Xref header <article-locator>s set by the sending site) remove any Xref
field (F-3.2.14) from each article. It then MAY (and usually header field from each article. It then MAY (and usually will)
will) generate a fresh Xref header field. generate a fresh Xref header field.
8. Finally, it stores the article in its news database. 8. Finally, it stores the article and makes it available for reading
agents.
Serving agents MUST NOT create new newsgroups simply because an Serving agents MUST NOT create new newsgroups simply because an
unrecognized <newsgroup-name> occurs in a Newsgroups header field unrecognized <newsgroup-name> occurs in a Newsgroups header field.
(see 5.2.1 for the correct method of newsgroup creation). Newsgroups are normally created via control messages (Section 5.2.1).
Serving agents MUST NOT alter, delete or rearrange any part of an
article in any other way. The list of particular cases given for
relaying agents (6.3) applies here also.
6.5. Duties of a Posting Agent
A Posting Agent is used to assist the poster in creating a valid
proto-article and forwarding it to an injecting agent.
Postings agents SHOULD ensure that proto-articles they create are
valid according to [USEFOR] and other applicable policies. In
particular, they MUST NOT create any Injection-Date or Injection-Info
header field.
Contrary to [RFC 2822], which implies that the mailbox(es) in the
From header field should be that of the poster(s), a poster who does
not, for whatever reason, wish to use his own mailbox MAY use any
mailbox ending in the top level domain ".invalid" [RFC 2606].
Posting agents meant for use by ordinary posters SHOULD reject any
attempt to post an article which cancels or Supersedes another
article of which the poster is not the author or sender.
6.6. Duties of a Followup Agent
A Followup Agent is a special case of a posting agent, and as such is
bound by all the posting agent's requirements. Followup agents MUST
create valid followups and are subject to special requirements
involving the Newsgroups, Subject, Distribution and References header
fields. Wherever in the following it is stated that, "by default", a
header field is to be "inherited" from one of those header fields in
the precursor, it means that its initial (semantic) content is to be
a copy of the content of that precursor header field. However,
posters MAY then override that default before posting if they so
wish.
News Article Architecture and Protocols November 2006
NOTE: The Keywords header field is not inheritable, though some
older user agents treated it as such.
1. The Newsgroups header field (F-3.1.4) SHOULD by default be
inherited from the precursor's Followup-To header field if
present, and otherwise from the precursor's Newsgroups header
field. However, if the content of that Followup-To header field
consists of "poster" (and the user does not override it), then the
followup MUST NOT be posted but, rather, is to be emailed to the
precursor's poster.
2. The Subject header field SHOULD by default be inherited from that
of the precursor. The case sensitive string "Re: " MAY be
prepended to the content of its Subject header field, unless it
already begins with that string.
3. The Distribution header field (F-3.2.4) SHOULD by default be
inherited from the precursor's Distribution header field, if any.
4. The followup MUST (in accordance with the definition of that term
- F-1.5) have a References header field referring to its
precursor, constructed in accordance with section 6.6.1 below.
NOTE: That "MUST" is to be contrasted with the weaker
recomendation using "SHOULD" applied, in [RFC 2822], to the
generation of "replies" in email. Moreover, in Netnews, there is
no expectation of any In-Reply-To header field in a followup.
6.6.1. Construction of the References header field
The following procedure is to be used whenever some previous article
(the "parent") is to be referred to in the References header field
(F-3.2.10) of a new article, whether in the course of generating a
followup or for some other reason (e.g. the later parts of a
multipart posting such as a FAQ, or the later parts of a
message/partial as suggested in [RFC 2046]).
The (semantic) content (2.1) of the new article's References header
field consists of the content of the Message-ID header field of the
parent preceded, if the parent had a References header field, by the
content of that References header field and a SP (subject to trimming
as described below).
If the resulting References header field would, after unfolding,
exceed 998 characters in length (including its field name but not the
final CRLF), it MUST be trimmed (and otherwise it MAY be trimmed).
Trimming involves removing any number of message identifiers from its
content, except that the first message identifier and the last two
MUST NOT be removed.
NOTE: There is no provision in this standard for an article to
have more than one parent. The essential property of the
References header field, guaranteed by the procedure above and
to be preserved in any future extension, is that no article can
News Article Architecture and Protocols November 2006
ever precede one of its own parents.
6.7. Duties of a Reading Agent Serving agents MUST NOT alter, delete, or rearrange any part of an
article except for the Path and Xref header fields. They MUST NOT
modify the body of the articles in any way. If an article is not
acceptable as-is, the article MUST be rejected rather than modified.
A reading agent downloads articles from a storage agent, as directed 3.7. Duties of a Reading Agent
by the reader, and displays them to the reader (or processes them in
some other manner). It SHOULD also have the capability to show the
raw article exactly as received.
It MAY present lists of articles available for display, and MAY Since a reading agent is only a passive participant in a Netnews
structure those lists so as to show the relationships between the network, there are no specific protocol requirements placed on it.
articles, as determined by the References, Subject, Date and other See [USEAGE] for best-practice recommendations.
header fields (see [USEAGE] for some usual methods of doing this).
[This whole section may yet get omitted]
6.8. Duties of a Moderator 3.8. Duties of a Moderator
A Moderator receives news articles, customarily by email, decides A moderator receives news articles, customarily by email, decides
whether to approve them and, if so, either injects them into the news whether to approve them and, if so, either passes them to an
stream or forwards them to further moderators. injecting agent or forwards them to further moderators.
Articles will be received by the moderator either encapsulated as an Articles are normally received by the moderator in email either
object of Content-Type application/news-transmission (or possibly encapsulated as an object of Content-Type application/
encapsulated but without an explicit Content-Type header field), or news-transmission (or possibly encapsulated but without an explicit
else directly as an email already containing all the header fields Content-Type header field), or else directly as an email already
appropriate for a Netnews article (see 6.2.2). Moderators SHOULD be containing all the header fields appropriate for a Netnews article
prepared to accept articles in either format. (see Section 3.4.1). Moderators who may receive articles via email
SHOULD be prepared to accept articles in either format.
A moderator processes an article, as submitted to any newsgroup that There are no protocol restrictions on what criteria are used for
he moderates, as follows: accepting or rejecting messages or on what modifications a moderator
may make to a message (both header fields and body) before injecting
it. Recommended best practice, however, is to make the minimal
required changes. Moderators need to be aware that modifications
made to articles may invalidate signatures created by the poster or
previous moderators. See [USEAGE] for further best-practice
recommendations.
1. He decides, on the basis of whatever moderation policy applies to Moderators process articles as follows:
his group, whether to approve or reject the article. He MAY do
this manually, or else partially or wholly with the aid of
appropriate software for whose operation he is then responsible.
If the article is a cancel nessage (5.3) issued by the poster of
an earlier article, then he is expected to cancel that earlier
article (in which case there is no more to be done). He MAY
modify the article if that is in accordance with the applicable
moderation policy (and in particular he MAY remove redundant
header fields and add Comments and other informational header
fields). He also needs to be aware if any change he makes to the
article will invalidate some authentication check provided by the
poster or by an earlier moderator.
If the article is rejected, then it normally fails for all the 1. They decide whether to approve or reject a proto-article, and if
newsgroups for which it was intended. If it is approved, the approved, prepare the proto-article for injection. If the proto-
moderator proceeds with the following steps. article was received as an unencapsulated email message, this
will require converting it back to a valid Netnews proto-article.
If the article is rejected, it is normally rejected for all
newsgroups to which it was posted and nothing further is done.
If it is approved, the moderator proceeds with the following
steps.
2. If the Newsgroups header field contains further moderated 2. If the Newsgroups header field contains further moderated
newsgroups for which approval has not already been given, he adds newsgroups for which approval has not already been given, they
an indication (identifying both himself and the name of the group) may either reach some agreement with the other moderators on the
that he approves the article, and then forwards it to the disposition of the article or, more generally, add an indication
moderator of the leftmost unapproved group (which, if this (identifying both the moderator and the name of the newsgroup)
News Article Architecture and Protocols November 2006 that they approve the article and then forward it to the
moderator of the leftmost unapproved newsgroup. This forwarding
standard has been followed correctly, will generally be the next SHOULD be done following the procedure in Section 3.4.1 and MAY
moderated group to the right of his own). There are two ways to do be done by rotating the <newsgroup-name>s in the Newsgroups
this: header field so that the leftmost unapproved newsgroup is is the
leftmost moderated newsgroup in that field and then posting it,
(a) He emails it to the submission address of the next moderator letting the injecting agent do the forwarding. However, if using
(see section 6.2.2 for the proper method of doing this), or this mechanism, they MUST first ensure that the article contains
no Approved header field.
(b) he rotates the <newsgroup-name>s in the Newsgroups header
field to the left so that the targeted group is the leftmost
moderated group in that field, and injects it again (thus
causing the injecting agent to forward it to the correct
moderator). However, he MUST first ensure that the article
contains no Approved header field.
NOTE: This standard does not prescribe how a moderator's
approval is to be indicated (though a future standard may do
so). Possible methods include adding an Approved header field
(or a similar but differently named header field if method (b)
is being used) listing all the approvals made so far, or adding
a separate header field for each individual approval (the header
field X-Auth is sometimes used for this purpose). The approval
may also be confirmed with some form of digital signature (5.1).
3. If the Newsgroups header field contains no further unapproved 3. If the Newsgroups header field contains no further unapproved
moderated groups, he adds an Approved header field (F-3.2.1) moderated groups, they add an Approved header field (see Section
identifying himself and, insofar as is possible, all the other 3.2.1 of [USEFOR]) identifying the moderator and, insofar as is
moderators who have approved the article. He thus assumes possible, all the other moderators who have approved the article.
responsibility for having ensured that the article was approved by The moderator who takes this step assumes responsibility for
the moderators of all the moderated groups involved. ensuring that the article was approved by the moderators of all
moderated newsgroups to which it was posted.
4. The Date header field SHOULD be retained. Any Injection-Date
header field already present (though there should be none) MUST be
removed. Exceptionally, if it is known that the injecting agent
does not yet support the Injection-Date header field and the Date
header field appears to be stale (F-3.2.7) for reasons understood
by the moderator (e.g. delays in the moderation process) he MAY
substitute the current date. The Message-ID header field SHOULD
also be retained unless it is obviously non-compliant with
[USEFOR].
NOTE: A <msg-id-core> created by a conforming posting or
injecting agent, or even by a mail user agent conforming to [RFC
2822], may reasonably be supposed to be conformant (and will, in
any case, be caught by the injecting agent if it is not).
5. Any variant header fields (2.4) MUST be removed, except that a
Path header field MAY be truncated to only those entries following
its "POSTED" <diag-keyword>. Any Injection-Info header field (F-
3.2.8) SHOULD be removed (and if not, the injecting agent will do
so, as required in 6.2.2).
6. He then causes the article to be injected, having first observed 4. Moderators are encouraged to retain the Message-ID header field
all the duties of a posting agent. if it is valid, and also retain the Date header field unless it
appears to be stale (72 hours or more in the past) for reasons
understood by the moderator (such as delays in the moderation
process) in which case they MAY substitute the current date. Any
Injection-Date, Injection-Info, or Xref header fields already
present (though there should be none) MUST be removed.
News Article Architecture and Protocols November 2006 5. Any Path header field MUST either be removed or truncated to only
those entries following its "POSTED" <diag-keyword>, if any.
NOTE: This standard does not prescribe how the moderator or 6. The moderator then passes the article to an injecting agent,
moderation policy for each newsgroup is established; rather it having first observed all the duties of a posting agent.
assumes that whatever agencies are responsible for the relevant
network or hierarchy (1.1) will have made appropriate
arrangements in that regard.
6.9. Duties of a Gateway 3.9. Duties of a Gateway
A Gateway transforms an article into the native message format of A gateway transforms an article into the native message format of
another medium, or translates the messages of another medium into another medium, or translates the messages of another medium into
news articles. Encapsulation of a news article into a message of MIME news articles, or transforms articles into proto-articles for
type application/news-transmission, or the subsequent undoing of that injection into a separate Netnews network. Encapsulation of a news
encapsulation, is not gatewaying, since it involves no transformation article into a message of MIME type application/news-transmission, or
of the article. the subsequent undoing of that encapsulation, is not gatewaying,
since it involves no transformation of the article.
There are two basic types of gateway, the Outgoing Gateway that There are two basic types of gateway, the outgoing gateway that
transforms a news article into a different type of message, and the transforms a news article into a different type of message, and the
Incoming Gateway that transforms a message from another medium into a incoming gateway that transforms a message from another network into
news article and injects it into a news system. These are handled a news proto-article and injects it into a Netnews network. These
separately below. are handled separately below.
The primary diktat for a gateway is:
Above all, prevent loops.
Transformation of an article into another medium stands a very high Transformation of an article into another medium stands a very high
chance of discarding or interfering with the protection inherent in chance of discarding or interfering with the protection inherent in
the news system against duplicate articles. The most common problem the news system against duplicate articles. The most common problem
caused by gateways is "spews", gateway loops that cause previously caused by gateways is loops that repeatedly reinject previously
posted articles to be reinjected repeatedly into Usenet. To prevent posted articles. To prevent this, a gateway MUST take precautions
this, a gateway MUST take precautions against loops, as detailed against loops, as detailed below.
below.
The transformations applied to the message SHOULD be as minimal as
possible while still accomplishing the gatewaying. Every change made
by a gateway potentially breaks a property of one of the media or
loses information, and therefore only those transformations made
necessary by the differences between the media should be applied.
If bidirectional gatewaying (both an incoming and an outgoing If bidirectional gatewaying (both an incoming and an outgoing
gateway) is being set up between Netnews and some other medium, the gateway) is being set up between Netnews and some other medium, the
incoming and outgoing gateways SHOULD be coordinated to avoid incoming and outgoing gateways SHOULD be coordinated to avoid
unintended reinjection of gated articles. Circular gatewaying unintended reinjection of gated articles. Circular gatewaying
(gatewaying a message into another medium and then back into Netnews) (gatewaying a message into another medium and then back into Netnews)
SHOULD NOT be done; encapsulation of the article SHOULD be used SHOULD NOT be done; encapsulation of the article SHOULD be used
instead where this is necessary. instead where this is necessary.
A second general principal of gatewaying is that the transformations Safe bidirectional gatewaying between a mailing list and a newsgroup
applied to the message SHOULD be as minimal as possible while still is far easier if the newsgroup is moderated. Posts to the moderated
accomplishing the gatewaying. Every change made by a gateway group and submissions to the mailing list can then go through a
potentially breaks a property of one of the media or loses single point that does the necessary gatewaying and then sends the
information, and therefore only those transformations made necessary message out to both the newsgroup and the mailing list at the same
by the differences between the media should be applied. time, eliminating most of the possibility of loops. Bidirectional
gatewaying between a mailing list and an unmoderated newsgroup, in
It is worth noting that safe bidirectional gatewaying between a contrast, is difficult to do correctly and is far more fragile.
mailing list and a newsgroup is far easier if the newsgroup is
moderated. Posts to the moderated group and submissions to the
mailing list can then go through a single point that does the
necessary gatewaying and then sends the message out to both the
newsgroup and the mailing list at the same time, eliminating most of
News Article Architecture and Protocols November 2006
the possibility of loops. Bidirectional gatewaying between a mailing
list and an unmoderated newsgroup, in contrast, is difficult to do
correctly and is far more fragile.
Newsgroups intended to be bidirectionally gated to a mailing list Newsgroups intended to be bidirectionally gated to a mailing list
SHOULD therefore be moderated where possible, even if the moderator SHOULD therefore be moderated where possible, even if the moderator
is a simple gateway and injecting agent that correctly handles is a simple gateway and injecting agent that correctly handles
crossposting to other moderated groups and otherwise passes all crossposting to other moderated groups and otherwise passes all
traffic. traffic.
6.9.1. Duties of an Outgoing Gateway 3.9.1. Duties of an Outgoing Gateway
From the perspective of Netnews, an outgoing gateway is just a From the perspective of Netnews, an outgoing gateway is just a
special type of reading agent. The exact nature of what the outgoing special type of reading agent. The exact nature of what the outgoing
gateway will need to do to articles depends on the medium to which gateway will need to do to articles depends on the medium to which
the articles are being gated. The operation of the outgoing gateway the articles are being gated. The operation of the outgoing gateway
is subject to additional constraints due to the possibility of one or is subject to additional constraints due to the possibility of one or
more corresponding incoming gateways back from that medium to more corresponding incoming gateways back from that medium to
Netnews, since this opens the possibility of loops. Netnews, since this raises the danger of loops.
In general, the following practices are recommended for all outgoing The following practices are encouraged for all outgoing gateways,
gateways, regardless of whether there is known to be a related regardless of whether there is known to be a related incoming
incoming gateway, both as a precautionary measure and as a guideline gateway, both as precautionary measures and as a guideline to quality
to quality of implementation. of implementation:
1. The message identifier of the news article should be preserved if 1. The message identifier of the news article should be preserved if
at all possible, preferably as or within the corresponding unique at all possible, preferably as or within the corresponding unique
identifier of the other medium, but if not at least as a comment identifier of the other medium, but if not at least as a comment
in the message. This helps greatly with preventing loops. in the message. This helps greatly with preventing loops.
2. The Date and Injection-Date of the news article should also be 2. The Date and Injection-Date of the news article should also be
preserved if possible, for similar reasons. preserved if possible, for similar reasons.
3. The message should be tagged in some way so as to prevent its 3. The message should be tagged in some way so as to prevent its
reinjection into Netnews. This may be impossible to do without reinjection into Netnews. This may be impossible to do without
knowledge of potential incoming gateways, but it is better to try knowledge of potential incoming gateways, but it is better to try
to provide some indication even if not successful; at the least, a to provide some indication even if not successful; at the least,
human-readable indication that the article should not be gated a human-readable indication that the article should not be gated
back to Netnews can help locate a human problem. back to Netnews can help locate a human problem.
4. Netnews control messages should not be gated to another medium 4. Netnews control messages should not be gated to another medium
unless they would somehow be meaningful in that medium. unless they would somehow be meaningful in that medium.
5. Changes MAY be made to the Content-Transfer-Encoding of some or 3.9.2. Duties of an Incoming Gateway
all parts of the body, and even to the charsets specified in
<encoded-word>s or in Content-Type header fields, but such changes
SHOULD NOT be made unless absolutely necessary.
6.9.2. Duties of an Incoming Gateway
The incoming gateway has the serious responsibility of ensuring that
all of the requirements of this standard are met by the articles that
it forms. In addition to its special duties as a gateway, it bears
all of the duties and responsibilities of an injecting agent as well,
News Article Architecture and Protocols November 2006
and additionally has the same responsibility of a relaying agent to The incoming gateway has the responsibility of ensuring that all of
the requirements of this protocol are met by the articles that it
forms. In addition to its special duties as a gateway, it bears all
of the duties and responsibilities of a posting agent as well, and
additionally has the same responsibility of a relaying agent to
reject articles that it has already gatewayed. reject articles that it has already gatewayed.
An incoming gateway MUST NOT gate the same message twice. It may not An incoming gateway MUST NOT gate the same message twice. It may not
be possible to ensure this in the face of mangling or modification of be possible to ensure this in the face of mangling or modification of
the message, but at the very least a gateway, when given a copy of a the message, but at the very least a gateway, when given a copy of a
message it has already gated identical except for trace header fields message it has already gated identical except for trace header fields
(like Received in Email or Path in Netnews) MUST NOT gate the message (like Received in Email or Path in Netnews) MUST NOT gate the message
again. An incoming gateway SHOULD take precautions against having again. An incoming gateway SHOULD take precautions against having
this rule bypassed by modifications of the message that can be this rule bypassed by modifications of the message that can be
anticipated. anticipated.
News articles prepared by gateways MUST be legal news articles. In News articles prepared by gateways MUST be valid news proto-articles
particular, they MUST include all of the mandatory header fields, (see Section 3.3.1). This often requires the gateway to synthesize a
MUST fully conform to the restrictions on those fields, and SHOULD conforming article from non-conforming input. The gateway MUST then
exclude any deprecated header fields (e.g. as in [RFC 2298]). This pass the article to an injecting agent, not directly to a relaying
often requires that a gateway function not only as a relaying agent, agent.
but also partly as a posting agent, aiding in the synthesis of a
conforming article from non-conforming input.
Incoming gateways MUST NOT pass control messages (articles containing Incoming gateways MUST NOT pass control messages (articles containing
a Control or Supersedes header field) without removing or renaming a Control or Supersedes header field) without removing or renaming
that header field. Gateways MAY, however, generate their own cancel that header field. Gateways MAY, however, generate cancel control
messages, under the general allowance for injecting agents to cancel messages for messages they have gatewayed. If a gateway receives a
their own messages ([USEAGE]). If a gateway receives a message that message that it can determine is a valid equivalent of a cancel
it can determine is a valid equivalent of a cancel message in the control message in the medium it is gatewaying, it SHOULD discard
medium it is gatewaying, it SHOULD discard that message without that message without gatewaying it, generate a corresponding cancel
gatewaying it, generate a corresponding cancel message of its own, control message of its own, and inject that cancel control message.
and inject that cancel message.
Incoming gateways MUST NOT inject control messages other than
cancels. Encapsulation SHOULD be used instead of gatewaying, when
direct posting is not possible or desirable.
NOTE: It is not unheard of for mail-to-news gateways to be used NOTE: It is not unheard of for mail-to-news gateways to be used to
to post control messages, but encapsulation should be used for post control messages, but encapsulation should be used for these
these cases instead. Gateways by their very nature are cases instead. Gateways by their very nature are particularly
particularly prone to loops. Spews of normal articles are bad prone to loops. Spews of normal articles are bad enough; spews of
enough; spews of control messages with special significance to control messages with special significance to the news system,
the news system, possibly resulting in high processing load or possibly resulting in high processing load or even email sent for
even email sent for every message received, are catastrophic. It every message received, are catastrophic. It is far preferable to
is far preferable to construct a system specifically for posting construct a system specifically for posting control messages that
control messages that can do appropriate consistency checks and can do appropriate consistency checks and authentication of the
authentication of the originator of the message. originator of the message.
If there is a message identifier that fills a role similar to that of If there is a message identifier that fills a role similar to that of
the Message-ID header field in news, it SHOULD be used in the the Message-ID header field in news, it SHOULD be used in the
formation of the message identifier of the news article, perhaps with formation of the message identifier of the news article, perhaps with
transformations required to meet the uniqueness requirement of transformations required to meet the uniqueness requirement of
Netnews and with the removal of any comments so as to comply with the Netnews and with the removal of any comments so as to comply with the
syntax in section F-3.1.3. Such transformations SHOULD be designed so syntax in Section 3.1.3 of [USEFOR]. Such transformations SHOULD be
that two messages with the same identifier generate the same designed so that two messages with the same identifier generate the
Message-ID header field. same Message-ID header field.
News Article Architecture and Protocols November 2006
NOTE: Message identifiers play a central role in the prevention NOTE: Message identifiers play a central role in the prevention of
of duplicates, and their correct use by gateways will do much to duplicates, and their correct use by gateways will do much to
prevent loops. Netnews does, however, require that message prevent loops. Netnews does, however, require that message
identifiers be unique, and therefore message identifiers from identifiers be unique, and therefore message identifiers from
other media may not be suitable for use without modification. A other media may not be suitable for use without modification. A
balance must be struck by the gateway between preserving balance must be struck by the gateway between preserving
information used to prevent loops and generating unique message information used to prevent loops and generating unique message
identifiers. identifiers.
Exceptionally, if there are multiple incoming gateways for a Exceptionally, if there are multiple incoming gateways for a
particular set of messages, each to a different newsgroup(s), each particular set of messages, each to a different newsgroup(s), each
one SHOULD generate a message identifier unique to that gateway. Each one SHOULD generate a message identifier unique to that gateway.
incoming gateway nonetheless MUST ensure that it does not gate the Each incoming gateway nonetheless MUST ensure that it does not gate
same message twice. the same message twice.
NOTE: Consider the example of two gateways of a given mailing NOTE: Consider the example of two gateways of a given mailing list
list into the world-wide Usenet newsgroups, both of which into two separate Usenet newsgroups, both of which preserve the
preserve the email message identifier. Each newsgroup may then email message identifier. Each newsgroup may then receive a
receive a portion of the messages (different sites seeing portion of the messages (different sites seeing different
different portions). In these cases, where there is no one portions). In these cases, where there is no one "official"
"official" gateway, some other method of generating message gateway, some other method of generating message identifiers has
identifiers has to be used to avoid collisions. It would to be used to avoid collisions. It would obviously be preferable
obviously be preferable for there to be only one gateway which for there to be only one gateway which crossposts, but this may
crossposts, but this may not be possible to coordinate. not be possible to coordinate.
If no date information is available, the gateway MAY supply a Date If no date information is available, the gateway MAY supply a Date
header field with the gateway's current date. If no injection-date header field with the gateway's current date. If only partial
information is available, the gateway MUST supply an Injection-Date information is available (such as date but not time), this SHOULD be
header field with whatever date information is available, and fleshed out to a full Date by adding default values rather than
otherwise with the gateway's current date. If only partial discarding this information. Only in very exceptional circumstances
information is available (e.g. date but not time), this SHOULD be should Date information be discarded, as it plays an important role
fleshed out to a full Date and/or Injection-Date header field by in preventing reinjection of old messages.
adding default values rather than discarding this information. Only
in very exceptional circumstances should Date information be
discarded, as it plays an important role in preventing reinjection of
old messages.
An incoming gateway MUST add a Sender header field to the news An incoming gateway MUST add a Sender header field to the news
article it forms containing the <mailbox> of the administrator of the article it forms containing the <mailbox> of the administrator of the
gateway. Problems with the gateway may be reported to this gateway. Problems with the gateway may be reported to this
<mailbox>. The <display-name> portion of this <mailbox> SHOULD <mailbox>. The <display-name> portion of this <mailbox> SHOULD
indicate that the entity responsible for injection of the message is indicate that the entity responsible for injection of the message is
a gateway. If the original message already had a Sender header field, a gateway. If the original message already had a Sender header
it SHOULD be renamed so that its contents can be preserved. field, it SHOULD be renamed so that its contents can be preserved.
6.9.3. Example 3.9.3. Gateway Example
To illustrate the type of precautions that should be taken against To illustrate the type of precautions that should be taken against
loops, here is an example of the measures taken by one particular loops, here is an example of the measures taken by one particular
combination of mail-to-news and news-to-mail gateways at Stanford combination of mail-to-news and news-to-mail gateways designed to
University designed to handle bidirectional gatewaying between handle bidirectional gatewaying between mailing lists and unmoderated
mailing lists and unmoderated groups. groups:
News Article Architecture and Protocols November 2006
1. The news-to-mail gateway preserves the message identifier of the 1. The news-to-mail gateway preserves the message identifier of the
news article in the generated email message. The mail-to-news news article in the generated email message. The mail-to-news
gateway likewise preserves the email message identifier provided gateway likewise preserves the email message identifier provided
that it is syntactically valid for Netnews. This allows the news that it is syntactically valid for Netnews. This allows the news
system's built-in suppression of duplicates to serve as the first system's built-in suppression of duplicates to serve as the first
line of defense against loops. line of defense against loops.
2. The news-to-mail gateway adds an X-Gateway header field to all 2. The news-to-mail gateway adds an X-Gateway header field to all
messages it generates. The mail-to-news gateway discards any messages it generates. The mail-to-news gateway discards any
incoming messages containing this header field. This is robust incoming messages containing this header field. This is robust
against mailing list managers that replace the message identifier, against mailing list managers that replace the message
and against any number of email hops, provided that the other identifier, and against any number of email hops, provided that
message header fields are preserved. the other message header fields are preserved.
3. The mail-to-news gateway prepends the host name from which it 3. The mail-to-news gateway prepends the host name from which it
received the email message to the content of the Path header received the email message to the content of the Path header
field. The news-to-mail gateway refuses to gateway any message field. The news-to-mail gateway refuses to gateway any message
that contains the list server name in its Path header field. This that contains the list server name in its Path header field
is robust against any amount of munging of the message header (including in the tail section). This is robust against any
fields by the mailing list, provided that the email only goes amount of munging of the message header fields by the mailing
through one hop. list, provided that the email only goes through one hop.
4. The mail-to-news gateway is designed never to generate bounces to 4. The mail-to-news gateway is designed never to generate bounces to
the envelope sender. Instead, articles that are rejected by the the envelope sender. Instead, articles that are rejected by the
news server (for reasons not warranting silent discarding of the news server (for reasons not warranting silent discarding of the
message) result in a bounce message sent to an errors address message) result in a bounce message sent to an errors address
known not to forward to any mailing lists, so that they can be known not to forward to any mailing lists, so that they can be
handled by the news administrators. handled by the news administrators.
These precautions have proven effective in practice at preventing These precautions have proven effective in practice at preventing
loops for this particular application (bidirectional gatewaying loops for this particular application (bidirectional gatewaying
between mailing lists and locally distributed newsgroups where both between mailing lists and locally distributed newsgroups where both
gateways can be designed together). General gatewaying to world-wide gateways can be designed together). General gatewaying to world-wide
newsgroups poses additional difficulties; one must be very wary of newsgroups poses additional difficulties; one must be very wary of
strange configurations, such as a newsgroup gated to a mailing list strange configurations, such as a newsgroup gated to a mailing list
which is in turn gated to a different newsgroup. which is in turn gated to a different newsgroup.
7. Security and Related Considerations 4. Media Types
There is no security. Don't fool yourself. Usenet is a prime example This document defines several media types, which shall be registered
of an Internet Adhocratic-Anarchy; that is, an environment in which with IANA as provided for in [RFC4288].
trust forms the basis of all agreements. It works.
See also F-5 for further security considerations related to the The media type message/news, as previously registered with IANA, is
format of articles. hereby declared obsolete. It was never widely implemented, and its
default treatment as application/octet-stream by agents that did not
recognize it was counter-productive. The media type message/rfc822
SHOULD be used in its place.
7.1. Leakage 4.1. application/news-transmission
Articles which are intended to have restricted distribution are The media type application/news-transmission is intended for the
dependent on the goodwill of every site receiving them. The encapsulation of complete news articles where the intention is that
"Archive: no" header field (F-3.2.2) is available as a signal to the recipient should then inject them into Netnews. This application
automated archivers not to file an article, but that cannot be type provides one of the methods for mailing articles to moderators
guaranteed. (see Section 3.4.1) and may be used to convey messages to an
injecting agent. This encapsulation removes the need to transform an
email message into a Netnews proto-article and provides a way to send
a Netnews article using MIME through a transport medium that does not
support 8bit data.
News Article Architecture and Protocols November 2006 The MIME media type definition of application/news-transmission is:
The Distribution header field makes provision for articles which MIME type name: application
should not be propagated beyond a cooperating subnet. The key MIME subtype name: news-transmission
security word here is "cooperating". When a machine is not configured Required parameters: none
properly, it may become uncooperative and tend to distribute all Optional parameters: One and only one of "usage=moderate",
articles. "usage=inject", or "usage=relay".
Encoding considerations: A transfer-encoding different from that of
the article transmitted MAY be supplied to
ensure correct transmission over some 7bit
transport medium.
Security considerations: A news article may be a control message,
which if processed could have effects on
the recipient host's system beyond just
storage of the article.
Published specification: This specification.
Body part: A complete proto-article ready for
injection into Netnews or an article being
relayed to another agent
The flooding algorithm is extremely good at finding any path by which usage=moderate indicates the article is intended for a moderator,
articles can leave a subnet with supposedly restrictive boundaries, usage=inject for an injecting agent, and usage=relay for a relaying
and substantial administrative effort is required to avoid this. agent. The entity receiving the article may only implement one type
Organizations wishing to control such leakage are strongly advised to of agent, in which case the parameter MAY be omitted.
designate a small number of official gateways to handle all news
exchange with the outside world (however, making such gateways too
restrictive can also encourage the setting up of unofficial paths
which can be exceedingly hard to track down).
The sendme control message (5.4), insofar as it is still used, can be Contrary to the prior registration of this media type, article
used to request articles with a given message identifier, even one batches are not permitted as a body part. Multiple messages or a
that is not supposed to be supplied to the requestor. message with multiple application/news-transmission parts may be used
instead.
7.2. Attacks 4.2. application/news-groupinfo
7.2.1. Denial of Service The application/news-groupinfo media type is used in conjunction with
the newgroup control message (see Section 5.2.1). Its body part
contains brief information about a newsgroup: the newsgroup name, its
description, and its moderation status.
The proper functioning of individual newsgroups can be disrupted by The MIME media type definition of application/news-groupinfo is:
the massive posting of "noise" articles, by the repeated posting of
identical or near identical articles, by posting followups unrelated
to their precursors, or which quote their precursors in full with the
addition of minimal extra material (especially if this process is
iterated), and by crossposting to, or setting followups to, totally
unrelated newsgroups.
Many have argued that "spam", massively multiposted (and to a lesser MIME type name: application
extent massively crossposted) articles, usually for advertising MIME subtype name: news-groupinfo
purposes, also constitutes a DoS attack in its own regard. This may Required parameters: none
be so. Optional parameters: charset, which MUST be a charset registered
for use with MIME text types and has the
same syntax as the parameter defined for
text/plain [RFC2046]. Specifies the
charset of the body part. If not given,
the charset defaults to US-ASCII [ASCII].
Disposition: by default, inline
Encoding considerations: 7bit or 8bit MUST be used to maintain
compatibility.
Security considerations: None.
Published specification: This specification.
Such articles intended to deny service, or other articles of an The content of the application/news-groupinfo body part is defined
inflammatory nature, may also have their From or Reply-To addresses as:
set to valid but incorrect email addresses, thus causing large
volumes of email to descend on the true owners of those addresses.
Similar effects could be caused by any email header field which could groupinfo-body = [ newsgroups-tag CRLF ]
cause every reading agent receiving it to take some externally newsgroups-line CRLF
visible action. For example, the Disposition-Notification-To header newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP
field defined in [RFC 2298] could cause huge numbers of %x6E.65.77.73.67.72.6F.75.70.73 SP
acknowledgements to be emailed to an unsuspecting third party (for %x66.69.6C.65.3A
which reason [RFC 2298] declares that that header field SHOULD NOT be ; case sensitive
used in Netnews). ; "For your newsgroups file:"
newsgroups-line = newsgroup-name
[ 1*HTAB newsgroup-description ]
[ 1*WSP moderation-flag ]
newsgroup-description
= eightbit-utext *( *WSP eightbit-utext )
moderation-flag = %x28.4D.6F.64.65.72.61.74.65.64.29
; case sensitive "(Moderated)"
eightbit-utext = utext / %d127-255
This unusual format is backward-compatible with the scanning of the
body of newgroup control messages for descriptions done by Netnews
implementations that predate this specification. Although optional,
the <newsgroups-tag> SHOULD be included for backward compatibility.
It is a violation of this standard for a poster to use as his address The <newsgroup-description> MUST NOT contain any occurrence of the
a <mailbox> which he is not entitled to use. Even addresses with an string "(Moderated)" within it. Moderated newsgroups MUST be marked
invalid <local-part> but a valid <domain> can cause disruption to the by appending the case sensitive text " (Moderated)" at the end.
administrators of such domains. Posters who wish to remain anonymous
or to prevent automated harvesting of their addresses, but who do not
News Article Architecture and Protocols November 2006
care to take the additional precautions of using more sophisticated While a charset parameter is defined for this media type, most
anonymity measures, should avoid that violation by the use of existing software does not understand MIME header fields or correctly
addresses ending in the ".invalid" top-level-domain (see 6.5). handle descriptions in a variety of charsets. Using a charset of US-
ASCII where possible is therefore RECOMMENDED; if not possible, UTF-8
[RFC3629] SHOULD be used. Regardless of the charset used, the
constraints of the above grammar MUST be met and the <newsgroup-name>
MUST be represented using the same octets as would be used with a
charset of US-ASCII.
A malicious poster may also prevent his article being seen at a 4.3. application/news-checkgroups
particular site by preloading that site into the Path header field
(F-3.1.5) and may thus prevent the true owner of a forged From or
Reply-To address from ever seeing it.
A malicious complainer may submit a modified copy of an article (e.g. The application/news-checkgroups media type contains a list of
with an altered Injection-Info header field) to the administrator of newsgroups within a hierarchy or hierarchies, including their
an injecting agent in an attempt to discredit the author of that descriptions and moderation status. It is primarily for use with the
article and even to have his posting privileges removed. checkgroups control message (see Section 5.2.3).
Administrators should therefore obtain a genuine copy of the article
from their own storage agent before taking such precipitate action.
Administrative agencies with responsibility for establishing policies The MIME media type definition of application/news-checkgroups is:
in particular hierarchies can and should set bounds upon the
behaviour that is considered acceptable within those hierarchies (for
example by promulgating charters for individual newsgroups, and other
codes of conduct).
Whilst this standard places an onus upon injecting agents to bear MIME type name: application
responsibility for the misdemeanours of their posters (which includes MIME subtype name: news-checkgroups
non-adherence to established policies of the relevant hierarchies as Required parameters: none
provided in section 6.2), and to provide assistance to the rest of Optional parameters: charset, which MUST be a charset registered
the network by making proper use of the Injection-Info (F-3.2.8) for use with MIME text types and has the
header field, it makes no provision for enforcement, which may in same syntax as the parameter defined for
consequence be patchy. Nevertheless, injecting sites which text/plain [RFC2046]. Specifies the
persistently fail to honour their responsibilities or to comply with charset of the body part. If not given,
generally accepted standards of behaviour are likely to find the charset defaults to US-ASCII [ASCII].
themselves blacklisted, with their articles refused propagation and Disposition: by default, inline
even subject to cancellation, and other relaying sites would be well Encoding considerations: 7bit or 8bit MUST be used to maintain
advised to withdraw peering arrangements from them. compatibility.
Security considerations: This media type provides only a means for
conveying a list of newsgroups and does not
provide any information indicating whether
the sender is authorized to state which
newsgroups should exist within a hierarchy.
Such authorization must be accomplished by
other means.
Published specification: This specification.
7.2.2. Compromise of System Integrity The content of the application/news-groupinfo body part is defined
as:
The posting of unauthorized (as determined by the policies of the checkgroups-body = *( valid-group CRLF )
relevant hierarchy) control messages can cause unwanted newsgroups to valid-group = newsgroups-line ; see 4.2
be created, or wanted ones removed, from storage agents.
Administrators of such agents SHOULD therefore take steps to verify
the authenticity of such control messages, either by manual
inspection (particularly of the Approved header field) or by checking
any digital signatures that may be provided (see 5.1). In addition,
they SHOULD periodically compare the newsgroups carried against any
regularly issued checkgroups messages, or against lists maintained by
trusted servers and accessed by out-of-band protocols such as FTP or
HTTP.
Malicious cancel messages (5.3) can cause valid articles to be The same restrictions on charset, <newsgroup-name>, and <newsgroup-
removed from storage agents. Administrators of such agents SHOULD description> apply for this media type as for application/
therefore take steps to verify that they originated from the news-groupinfo.
(apparent) poster, the injector or the moderator of the article, or
that in other cases they came from a place that is trusted to work
News Article Architecture and Protocols November 2006
within established policies and customs. Such steps SHOULD include One application/news-checkgroups message may contain information for
the checking of any digital signatures, or other security devices, one or more hierarchies and is considered complete for any hierarchy
that may be provided (see 5.1). Articles containing Supersedes for which it contains a <valid-group>. In other words, an
header fields (F-3.2.12) are effectively cancel messages, and SHOULD application/news-checkgroups body part consisting of:
be subject to the same checks. Currently, many sites choose to
ignore all cancel messages on account of the difficulty of conducting
such checks.
Improperly configured injecting and relaying agents can allow example.moderated A moderated newsgroup (Moderated)
articles posted to moderated groups onto the net without first being example.test An unmoderated test group
approved by the moderator. Injecting agents SHOULD verify that
moderated articles were received from one of the entities given in
their Approved header fields and/or check any digital signatures that
may be provided (see 5.1).
There may be weaknesses in particular implementations that are is a statement that the example.* hierarchy contains two newsgroups,
subject to malicious exploitation. In particular, it has not been example.moderated and example.test, and no others. This media type
unknown for complete shell scripts to be included within Control therefore MUST NOT be used for conveying partial information about a
header fields. Implementors need to be aware of this. hierarchy; if a group from a given hierarchy is present, all groups
that exist in that hierarchy MUST be listed.
Reading agents should be chary of acting automatically upon MIME 5. Control Messages
objects with an "application" Content-Type that could change the
state of that agent, except in contexts where such applications are
specifically expected (as in 4). Even the Content-Type "text/html"
could have unexpected side effects on account of embedded objects,
especially embedded executable code or URIs that invoke non-news
protocols such as HTTP [RFC 2616]. It is therefore generally
recommended that reading agents do not enable the execution of such
code (since it is extremely unlikely to have a valid application
within Netnews) and that they only honour URIs referring to other
parts of the same article.
Non-printable characters embedded in article bodies may have A control message is an article which contains a Control header field
surprising effects on printers or terminals, notably by reconfiguring and thereby indicates that some action should be taken by an agent
them in undesirable ways which may become apparent only after the other than distribution and display. Any article containing a
reading agent has terminated. Control header field (defined in Section 3.2.3 of [USEFOR]) is a
control message. Additionally, the action of an article containing a
Supersedes header field is described here; while such an article is
not a control message, it specifies an action similar to the cancel
control message.
7.3. Liability The <control-command> of a Control header field comprises a <verb>,
which indicates the action to be taken, and one or more <argument>
values, which supply the details. For some control messages, the
body of the article is also significant. Each recognized <verb> (the
control message type) is described in a separate section below.
Agents MAY accept other control message types than those specified
below, and MUST either ignore or reject control messages with
unrecognized types. Syntactic definitions of valid <argument> values
and restrictions on control message bodies are given in the section
for each control message type.
[This whole section might be better removed to [USEAGE].] Contrary to [RFC1036], the presence of a Subject header field
starting with the string "cmsg " MUST NOT cause an article to
interpreted as a control message. Agents MAY reject an article with
no Control header field and such a Subject header field as ambiguous.
Likewise, the presence of a <newsgroup-name> ending in ".ctl" in the
Newsgroups header field or the presence of an Also-Control header
field MUST NOT cause the article to be interpreted as a control
message.
There is a presumption that a poster who sends an article to Usenet 5.1. Authentication and Authorization
intends it to be stored on a multitude of storage agents, and has
therefore given permission for it to be copied to that extent.
Nevertheless, Usenet is not exempt from the Copyright laws, and it
should not be assumed that permission has been given for the article
to be copied outside of Usenet, nor for its permanent archiving
contrary to any Archive header field that may be present.
Posters also need to be aware that they are responsible if they Control messages specify actions above and beyond the normal
breach Copyright, Libel, Harassment or other restrictions relating to processing of an article and are therefore potential vectors of abuse
material that they post, and that they may possibly find themselves and unauthorized action. There is, at present, no standardized means
liable for such breaches in jurisdictions far from their own. Serving of authenticating the sender of a control message or verifying that
agents may also be liable in some jurisdictions, especially if the the contents of a control message were sent by the claimed sender.
News Article Architecture and Protocols November 2006 There are, however, some unstandardized authentication mechanisms in
common use.
breach has been explicitly drawn to their attention. Agents acting on control messages SHOULD take steps to authenticate
control messages before acting on them, as determined by local
authorization policy. Whether this is done via the use of an
unstandardized authentication protocol, by comparison with
information obtained through another protocol, by human review, or by
some other means is left unspecified by this document. Further
extensions or revisions of this protocol are expected to standardize
a digital signature mechanism.
Users who are concerned about such matters should seek advice from Agents are expected to have their own local authorization policies
competent legal authorities. for which control messages will be honored. No Netnews agent is ever
required to act on any control message. The following descriptions
specify the actions that a control message requests, but an agent MAY
always decline to act on any given control message.
8. IANA Considerations 5.2. Group Control Messages
IANA is requested to register the following media types, described A group control message is any control message type that requests
elsewhere in this standard, for use with the Content-Type header some update to the list of newsgroups known to a news server. The
field, in the IETF tree in accordance with the procedures set out in standard group control message types are "newgroup", "rmgroup", and
[RFC 4288]. "checkgroups".
application/news-transmission (4.1) Before honoring any group control message, an agent MUST check the
application/news-groupinfo (4.3) newsgroup or newsgroups affected by that control message and decline
application/news-checkgroups (4.4) to create any newsgroups not in conformance with the restrictions in
Section 3.1.4 of [USEFOR].
IANA is also requested to change the status of the following media All of the group control messages MUST have an Approved header field
type to "OBSOLETE". (Section 3.2.1 of [USEFOR]). Group control messages without an
Approved header field SHOULD NOT be honored.
message/news (4.2) Group control messages affecting specific groups (newgroup and
rmgroup control messages, for example) SHOULD include the <newsgroup-
name> for the group or groups affected in their Newsgroups header
field. Other newsgroups MAY be included in the Newsgroups header
field so that the control message will reach more news servers, but
due to the special relaying rules for group control messages (see
Section 3.5) this is normally unnecessary and may be excessive.
NOTE: "Application/news-transmission" is an update, with 5.2.1. The newgroup Control Message
clarification and additional optional parameters, to an existing
registration. "Message/rfc822" should now be used in place of
the obsoleted "message/news". nr H1 7
9. References The newgroup control message requests the specified group be created
or, if already existing, have its moderation status or description
changed. The syntax of its Control header field is:
9.1. Normative References control-command =/ Newgroup-command
Newgroup-command = "newgroup" Newgroup-arguments
Newgroup-arguments = 1*WSP newsgroup-name [ 1*WSP newgroup-flag ]
newgroup-flag = "moderated"
[ANSI X3.4] "American National Standard for Information Systems - If the request is honored, the moderation status of the group SHOULD
Coded Character Sets - 7-Bit American National Standard Code for be set in accordance with the presence or absence of the <newgroup-
Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986. flag> "moderated". "moderated" is the only flag defined by this
protocol. Other flags MAY be defined by extensions to this protocol
and accepted by agents. If an agent does not recognize the
<newgroup-flag> of a newgroup control message, it SHOULD ignore that
control message.
[RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate The body of a newgroup message SHOULD contain an entity of type
Requirement Levels", RFC 2119, March 1997. application/news-groupinfo specifying the description of the
newsgroup, either as the entire body or as an entity within a
multipart/mixed object [RFC2046]. If such an entity is present, the
moderation status specified therein MUST match the moderation status
specified by the <newgroup-flag>. The body of a newgroup message MAY
contain other entities (encapsulated in multipart/mixed) that provide
additional information about the newsgroup or the circumstances of
the control message.
[RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names", In the absence of an application/news-groupinfo entity, a news server
RFC 2606, June 1999. MAY search the body of the message for the line "For your newsgroups
file:" and take the following line as a <newsgroups-line>. Prior to
the standardization of application/news-groupinfo, this was the
convention for providing a newsgroup description.
[RFC 2822] P. Resnick, "Internet Message Format", RFC 2822, April If the request is honored and contains a newsgroup description, and
2001. if the news server honoring it stores newsgroup descriptions, the
stored newsgroup description SHOULD be updated to the description
specified in the control message, even if no other property of the
group has changed.
[RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration 5.2.1.1. newgroup Control Message Example
procedures for message header fields", RFC 3864.
[USEFOR] K. Murchison et al, "News Article Format", draft-ietf- A newgroup control message requesting creation of the moderated
usefor-usefor-11.txt. newsgroup example.admin.info.
[USEPRO] This Standard. From: "example.* Administrator" <admin@noc.example>
Newsgroups: example.admin.info
Date: 27 Feb 2002 12:50:22 +0200
Subject: cmsg newgroup example.admin.info moderated
Approved: admin@noc.example
Control: newgroup example.admin.info moderated
Message-ID: <ng-example.admin.info-20020227@noc.example>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="nxtprt"
Content-Transfer-Encoding: 8bit
News Article Architecture and Protocols November 2006 This is a MIME control message.
--nxtprt
Content-Type: application/news-groupinfo
9.2. Informative References For your newsgroups file:
example.admin.info About the example.* groups (Moderated)
[PGPVERIFY] David Lawrence, --nxtprt
<ftp://ftp.isc.org/pub/pgpcontrol/README.html>. Content-Type: text/plain
[RFC 1036] M. Horton and R. Adams, "Standard for Interchange of A moderated newsgroup for announcements about new newsgroups in
USENET Messages", RFC 1036, December 1987. the example.* hierarchy.
[RFC 2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail --nxtprt--
Extensions (MIME) Part One: Format of Internet Message Bodies",
RFC 2045, November 1996.
[RFC 2046] N. Freed and N. Borenstein, "Multipurpose Internet Mail 5.2.2. The rmgroup Control Message
Extensions (MIME) Part Two: Media Types", RFC 2046, November
1996.
[RFC 2142] D. Crocker, "Mailbox Names for Common Services, Roles and The rmgroup control message requests the specified group be removed
Functions", RFC 2142, May 1997. from a news server's list of valid groups. The syntax of its Control
header field is:
[RFC 2298] R. Fajman, "An Extensible Message Format for Message control-command =/ Rmgroup-command
Disposition Notifications", RFC 2298, March 1998. Rmgroup-command = "rmgroup" Rmgroup-arguments
Rmgroup-arguments = 1*WSP newsgroup-name
[RFC 2616] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, The body of the control message MAY contain anything, usually an
P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol -- explanatory text.
HTTP/1.1", RFC 2616, June 1999.
[RFC 2821] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer 5.2.3. The checkgroups Control Message
Protocol", RFC 2821, April 2001.
[RFC 3977] C. Feather, "Network News Transport Protocol (NNTP)", RFC The checkgroups control message contains a list of all the valid
3977. groups in a hierarchy with descriptions and moderation status. It
requests a news server update its valid newsgroup list for that
hierarchy to include the groups specified, remove any groups not
specified, and update group descriptions to match those given in the
checkgroups control message. The syntax of its Control header field
is:
[RFC 4288] N. Freed and J. Klensin, "Media Type Specifications and control-command =/ Checkgroup-command
Registration Procedures", RFC 4288, December 2005. Checkgroup-command = "checkgroups" Checkgroup-arguments
Checkgroup-arguments= [ chkscope ] [ chksernr ]
chkscope = 1*( 1*WSP ["!"] newsgroup-name )
chksernr = 1*WSP "#" 1*DIGIT
[RFC 976] Mark R. Horton, "UUCP mail interchange format standard", A checkgroups message is interpreted as an exhaustive list of the
RFC 976, February 1986. valid groups in all hierarchies or sub-hierarchies with a prefix
listed in the <chkscope> argument, excluding any sub-hierarchy where
the <chkscope> argument is prefixed by "!". If no <chkscope>
argument is given, it applies to all hierarchies for which group
statements appear in the body of the message. Since much existing
software does not honor the <chkscope> argument, the body of the
checkgroups control message MUST NOT contain group statements for
newsgroups outside the intended scope and SHOULD contain a correct
newsgroup list even for sub-hierarchies excluded with "!" <chkscope>
terms. News servers, however, MUST honor <chkscope> as specified
here.
[Son-of-1036] Henry Spencer, "News article format and transmission", The <chksernr> argument may be any positive integer. If present, it
<ftp://ftp.zoo.toronto.edu/pub/news.txt.Z>, June 1994. MUST increase with every change to the newsgroup list and MUST NOT
ever decrease. If provided, news servers SHOULD remember the
<chksernr> value of the previous checkgroups control message honored
for a particular hierarchy or sub-hierarchy and decline to honor any
subsequent checkgroups control message for the same hierarchy or sub-
hierarchy with a smaller <chksernr> value.
[USEAGE] draft-ietf-usefor-useage-*.txt. For example, the following Control header field
10. Acknowledgements Control: checkgroups de !de.alt #248
As this document is the result of a ten year effort, the number of indicates the body of the message will list every newsgroup in the
people that have contributed to its content are too numerous to de.* hierarchy, excepting the de.alt.* sub-hierarchy, and should not
mention individually. Many thanks go out to all past and present be honored if a checkgroups control message with a serial number
members of the USEFOR Working Group of the Internet Engineering Task greater than 248 was previously honored.
Force (IETF) and the accompanying mailing list.
News Article Architecture and Protocols November 2006 The body of the message is an entity of type application/
news-checkgroups. It SHOULD be declared as such with appropriate
MIME headers, but news servers SHOULD interpret checkgroups messages
that lack the appropriate MIME headers as if the body were of type
application/news-checkgroups for backward compatibility.
11. Contact Address 5.3. The cancel Control Message
Editor The cancel control message requests that a target article be
withdrawn from circulation and access. The syntax of its Control
header field is:
Charles. H. Lindsey control-command =/ Cancel-command
5 Clerewood Avenue Cancel-command = "cancel" Cancel-arguments
Heald Green Cancel-arguments = 1*WSP msg-id
Cheadle
Cheshire SK8 3JU
United Kingdom
Phone: +44 161 436 6131
Email: chl@clerew.man.ac.uk
[ The argument identifies the article to be cancelled by its message
identifier. The body of the control message MAY contain anything,
usually an explanatory text.
Working group chairs A serving agent that elects to honor a cancel message SHOULD make the
article unavailable to reading agents (perhaps by deleting it
completely). If the cancel control message arrives before the
article it targets, news servers choosing to honor it SHOULD remember
the message identifier that was cancelled and reject the cancelled
article when it arrives.
Alexey Melnikov <alexey.melnikov-usefor@isode.com> To best ensure that it will be relayed to the same news servers as
Harald Tveit Alvestrand <harald@alvestrand.no> the original message, a cancel control message SHOULD have the same
] Newsgroups header field as the message it is cancelling.
Comments on this draft should preferably be sent to the mailing list Cancel control messages listing moderated newsgroups in their
of the Usenet Format Working Group at Newsgroups header field MUST contain an Approved header field like
any other article in a moderated newsgroup. This means that cancels
posted to a moderated newsgroup will normally be sent to the
moderator first for approval. Outside of moderated newsgroups,
cancel messages are not required to contain an Approved header field.
ietf-usefor@imc.org. Contrary to [RFC1036], cancel control messages are not required to
contain From and Sender header fields matching the target message.
This requirement only encouraged cancel issuers to conceal their
identity and provided no security.
Appendix A - Obsolete Control Messages 5.4. The Supersedes Header Field
This present standard obsoletes certain control messages defined in The presence of a Supersedes header field in an article requests that
[RFC 1036] (see 5.5), all of which had the effect of requesting a the message identifier given in that header field be withdrawn in
description of a relaying or storage agent's software, or its peering exactly the same manner as if it were the target of a cancel control
arrangements with neighbouring sites, to be emailed to the article's message. Accordingly, news servers SHOULD use the same
reply address. Whilst of some utility when Usenet was much smaller authentication and authorization checks for deciding whether to honor
than it is now, they had become no more than a tool for the malicious a Supersedes header field as they use for cancel control messages.
sending of mailbombs. Moreover, many organizations now consider If the Supersedes header field is honored, the news server SHOULD
information about their internal connectivity to be confidential. take the same actions as it would take when honoring a cancel control
message for the given target article.
5.5. The ihave and sendme Control Messages
The ihave and sendme control messages implement a predecessor of the
NNTP [RFC3977] protocol. They are largely obsolete on the Internet
but still see use in conjunction with some transport protocols such
as UUCP. News servers are not required to support them.
ihave and sendme control messages share similar syntax for their
Control header fields and bodies:
control-command =/ Ihave-command
Ihave-command = "ihave" Ihave-arguments
Ihave-arguments = 1*WSP relayer-name
/ 1*( 1*WSP msg-id ) [ 1*WSP relayer-name ]
control-command =/ Sendme-command
Sendme-command = "sendme" Sendme-arguments
Sendme-arguments = Ihave-arguments
relayer-name = path-identity ; see 3.1.5 of [USEFOR]
ihave-body = *( msg-id CRLF )
sendme-body = ihave-body
The body of the article consists of a list of <msg-id>s, one per
line. The message identifiers SHOULD be put in the body of the
article, not in the Control header field, but news servers MAY
recognize and process message identifiers in the Control header field
for backward compatibility. Message identifiers MUST NOT be put in
the Control header field if they are present in the body of the
control message.
The ihave message states that the named relaying agent has received
articles with the specified message identifiers, which may be of
interest to the relaying agents receiving the ihave message. The
sendme message requests that the agent receiving it send the articles
having the specified message identifiers to the named relaying agent.
If <relayer-name> is not given, it is determined from the origin of
the control message.
Upon receipt of the sendme message (and a decision to honor it), the
receiving agent sends the article or articles requested. The
mechanism for this transmission is unspecified by this document and
is arranged between the sites involved.
These control messages are normally sent as point-to-point articles
between two sites and not then sent on to other sites. Newsgroups
beginning with "to." are reserved for such point-to-point
communications and are formed by prepending "to." to a <relayer-name>
to form a <newsgroup-name>. Articles with such a group in their
Newsgroups header fields SHOULD NOT be sent to any news server other
than the one identified by <relayer-name>.
5.6. Obsolete Control Messages
The following control message types are declared obsolete by this
document and SHOULD NOT be sent or honored:
version
sendsys sendsys
version
whogets whogets
senduuname senduuname
"Version" requested details of the transport software in use at a 6. Security Considerations
site. "Sendsys" requested the full list of newsgroups taken, and the
peering arrangements. "Whogets" was similar, but restricted to a
named newsgroup. "Senduuname" resembled "sendsys" but restricted to
the list of peers connected by UUCP.
Historically, a checkgroups body consisting of one or two lines, the
first of the form "-n newsgroup", caused check-groups to apply to
only that single newsgroup.
News Article Architecture and Protocols November 2006
Historically, an article posted to a newsgroup whose name had exactly Netnews is designed for broad dissemination of public messages and
three components of which the third was "ctl" signified that article offers little in the way of security. What protection Netnews has
was to be taken as a control message. The Subject header field against abuse and impersonation is provided primarily by the
specified the actions, in the same way the Control header field does underlying transport layer. In large Netnews networks where news
now. servers cannot be relied upon to enforce authentication and
authorization requirements at the transport layer, articles may be
trivially forged and widely read, and the identities of article
senders and privacy of articles cannot be assured.
These forms are documented for archaeological purposes only; they See Section 5 of [USEFOR] for further security considerations related
MUST NO LONGER be used. to the format of articles.
Appendix B - Differences from the Protocols in RFC 1036 and its 6.1. Compromise of System Integrity
derivatives
This apendix contains a list of changes that have been made to the Control messages pose a particular security concern since acting on
protocols of the earlier standards, specifically [RFC 1036]. See F- unauthorized control messages may cause newsgroups to be removed,
Appendix B. for the related syntax changes. articles to be deleted, and unwanted newsgroups to be created.
Administrators of news servers SHOULD therefore take steps to verify
the authenticity of control messages as discussed in Section 5.1.
Articles containing Supersedes header fields are effectively cancel
control messages and SHOULD be subject to the same checks as
discussed in Section 5.4. Currently, many sites are ignoring all
cancel control messages and Supersedes header fields due to the
difficulty of authenticating them and their widespread abuse.
o There is a new Control message 'mvgroup' to facilitate moving a Cancel control messages are not required to have the same Newsgroups
group to a different place (name) in a hierarchy. header field as the messages they are cancelling and, since they are
o Certain Control messages (Appendix A) have been made obsolete, sometimes processed before the original message is received, it may
and the special significance of "cmsg" when at the start of a not be possible to check that they do. This allows a malicious
Subject header field has been removed (section 5). poster to inject a cancel control message for an article in a
o Additional media types are defined for better structuring of moderated newsgroup without adding an Approved header field to the
control messages (4.3 and 4.4). control message, and to hide malicious cancel control messages from
o Distributions are expected to be checked at the receiving end, as some reading agents by using a different Newsgroups header field so
well as the sending end, of a relaying link. that the cancel control message is not accepted by all news servers
o There are numerous other small changes, clarifications and that accepted the original message.
enhancements.
Appendix C - Transitional Arrangements All agents should be aware that all article content, most notably
including the content of the Control header field, is potentially
untrusted and malicious. Articles may be constructed in
syntactically invalid ways to attempt to overflow internal buffers,
violate hidden assumptions, or exploit implementation weaknesses.
For example, some news server implementations have been successfully
attacked via inclusion of Unix shell code in the Control header
field. All article contents, and particularly control message
contents, SHOULD be handled with care and rigorously verified before
any action is taken on the basis of the contents of the article.
It is the intention that the new features of [USEFOR] and of this A malicious poster may add an Approved header field to bypass the
standard can be assimilated into Usenet as it presently operates moderation process of a moderated newsgroup. Injecting agents SHOULD
without major interruption to the service, though some of the new verify that messages approved for a moderated newsgroup are being
features may not begin to show benefit until they become widely injected by the moderator using authentication information from the
implemented. underlying transport or some other authentication mechanism arranged
with the moderator.
An important distinction must be made between news servers, which are A malicious news server participating in a Netnews network may bypass
responsible for the distribution and storage of news articles, and checks performed by injecting agents, forge Path header fields and
user agents, which are responsible for interactions with users. It is other trace information (such as Injection-Info header fields), and
important that the former should be upgraded to conform to this otherwise compromise the authorization requirements of a Netnews
standard as soon as possible to provide the benefit of the enhanced network. News servers SHOULD use the facilities of the underlying
facilities. Fortunately, the number of distinct implementations of transport to authenticate their peers and reject articles from
such servers is rather small, at least so far as the main "backbone" injecting and relaying agents that do not follow the requirements of
of Usenet is concerned, and many of the new features are already this protocol or the Netnews network.
supported. Contrariwise, there are a great number of implementations
of user agents, installed on a vastly greater number of small sites.
Therefore, the new functionality has been designed so that existing
user agents may continue to be used, although the full benefits may
not be realised until a substantial proportion of them have been
upgraded.
In the list which follows, care has been taken to distinguish the 6.2. Denial of Service
implications for both kinds of agent.
News Article Architecture and Protocols November 2006 The proper functioning of individual newsgroups can be disrupted by
the excessive posting of unwanted articles; by the repeated posting
of identical or near identical articles; by posting followups
unrelated to their precursors or which quote their precursors in full
with the addition of minimal extra material (especially if this
process is iterated); by crossposting to, or requesting followups to,
totally unrelated newsgroups; and by abusing control messages and the
Supersedes header field to delete articles or newsgroups.
o The introduction of whitespace and folding into the Newsgroups Such articles intended to deny service, or other articles of an
and related header fields (F-3.1.4, F-3.2.4, F-3.2.6) and of inflammatory nature, may also have their From or Reply-To addresses
<comment>s into the References header field (F-3.2.10) will cause set to valid but incorrect email addresses, thus causing large
problems for existing agents, and [USEFOR] provides that they volumes of email to descend on the true owners of those addresses.
SHOULD NOT be generated at the present time. Users and agents should always be aware that identifying information
o The requirement in [USEFOR] to support MIME reflects a practice in articles may be forged.
that is already widespread. Articles in strict compliance with
the previous standards (using strict US-ASCII) will be
unaffected.
o All the header fields newly introduced by [USEFOR] can safely be
ignored by existing software, albeit with loss of the new
functionality.
o The new style of Path header field, using a <diag-match> to allow
"!!" as a <delimiter> and introducing <path-diagnostic>s (which
can be distinguished from <path-identity>s by their leading ".")
is already consistent with the previous standards. However, the
intention is that relaying agents should eventually reject
articles in the old style, and so this possibility should be
offered as a configurable option in relaying agents. User agents
are unaffected.
o The new Control: mvgroup command will need to be implemented in
storage agents. For the benefit of older storage agents it is
therefore RECOMMENDED that it be followed shortly by a
corresponding newgroup command and it MUST always be followed by
a rmgroup command for the old group after a reasonable overlap
period. An implementation of the mvgroup command as an alias for
the newgroup command would thus be minimally conforming. User
agents are unaffected.
o Provision is made for relaying and storage agents to use the Date
header field in the case of articles injected through existing
agents which do not yet provide an Injection-Date header field.
Appendix D - Notices A malicious poster may prevent an article from being seen at a
particular site by including in the Path header field of the proto-
article the <path-identity> of that site. Use of the <diag-keyword>
"POSTED" by injecting agents to mark the point of injection can
prevent this attack.
Intellectual Property Primary responsibility for preventing such attacks lies with
injecting agents, which can apply authentication and authorization
checks via the underlying transport and prevent those attempting
denial of service attacks from posting messages. If specific
injecting agents fail to live up to their responsibilities, they may
be excluded from the Netnews network by configuring relaying agents
to reject articles originating from them.
The IETF takes no position regarding the validity or scope of any A malicious complainer may submit a modified copy of an article (with
Intellectual Property Rights or other rights that might be claimed to an altered Injection-Info header field, for instance) to the
pertain to the implementation or use of the technology described in administrator of an injecting agent in an attempt to discredit the
this document or the extent to which any license under such rights author of that article and even to have his posting privileges
might or might not be available; nor does it represent that it has removed. Administrators SHOULD therefore obtain a genuine copy of
made any independent effort to identify any such rights. Information the article from their own serving agent before taking action in
on the procedures with respect to rights in RFC documents can be response to such a complaint.
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any 6.3. Leakage
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
News Article Architecture and Protocols November 2006 Articles which are intended to have restricted distribution are
dependent on the goodwill of every site receiving them. Restrictions
on dissemination and retention of articles may be requested via the
Archive and Distribution header fields, but such requests cannot be
enforced by the protocol.
The IETF invites any interested party to bring to its attention any The flooding algorithm used by Netnews transports such as NNTP
copyrights, patents or patent applications, or other proprietary [RFC3977] is extremely good at finding any path by which articles can
rights that may cover technology that may be required to implement leave a subnet with supposedly restrictive boundaries, and
this standard. Please address the information to the IETF at ietf- substantial administrative effort is required to avoid this.
ipr@ietf.org. Organizations wishing to control such leakage are advised to
designate a small number of gateways to handle all news exchange with
the outside world.
Full Copyright Statement The sendme control message Section 5.5, insofar as it is still used,
can be used to request articles the requester should not have access
to.
Copyright (C) The IETF Trust (2006). This document is subject to the 7. IANA Considerations
rights, licenses and restrictions contained in BCP 78, and except as
set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an IANA is requested to register the following media types, described
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS elsewhere in this document, for use with the Content-Type header
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND field, in the IETF tree in accordance with the procedures set out in
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS [RFC4288].
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."
Appendix E - Change Log application/news-transmission (4.1)
application/news-groupinfo (4.2)
application/news-checkgroups (4.3)
[This Appendix is to be removed prior to final publication.] application/news-transmission is a change to a previous registration.
For version 01 IANA is also requested to change the status of the message/news media
type to "OBSOLETE". message/rfc822 should be used instead.
1 Numerous texts describing protocol features related to 8. References
particular header fields in parts of [ARTICLE] which were
destined to become part of [USEFOR] have been moved to
appropriate locations within section 7 of this document. Such
revised texts will be found in sections
7.2.2 Steps 4, 6, 7, 10, 11, 12;
7.2.3 Step 1(b);
7.3 introductory paragraphs, Steps 1, 4, 8, 9, and some final
paragraphs;
7.4 introductory and final paragraphs;
7.9.1 Step 5.
2 A section on "Duties of a Reading Agent" (7.8) has been added. 8.1. Normative References
3 Some demotions MUST -> SHOULD -> MAY, as noted in pseudo- [ASCII] "American National Standard for Information Systems -
comments, have been made or proposed in sections Coded Character Sets - 7-Bit American National Standard
7.3 Code for Information Interchange (7-Bit ASCII)",
7.3 Step 4. ANSI X3.4, 1986.
4 Part of the procedure for examining Path header fields by [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
relaying agents has been moved to storage agents, as explained Extensions (MIME) Part Two: Media Types", RFC 2046,
in pseudo-comments in section 7.4. November 1996.
5 Some renumbering of sections and minor textual clarifications. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
For version 02 [RFC2822] Resnick, P., "Internet Message Format", RFC 2822,
News Article Architecture and Protocols November 2006 April 2001.
1 2nd para. of a-7 temporarily reinstated in section 6. [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO
10646", STD 63, RFC 3629, November 2003.
2 Para. in section 6 relating to propagation of control messages [RFC4234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
and local policy removed to [USEAGE].] Specifications: ABNF", RFC 4234, October 2005.
3 Requirement for some relaying agents to examine control messages [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and
for non-existent groups Registration Procedures", BCP 13, RFC 4288, December 2005.
6
7.3
4 Text regarding "aliasing out" brought into line with actual [USEFOR] Murchison, K., Ed., Lindsey, C., and D. Kohn, "Netnews
practice. Article Format".
7.3
5 More realistic wording regarding the expectations of reading 8.2. Informative References
agents
7.7
7.4
6 "Precursor" is now defined for all cases in which a References [RFC0976] Horton, M., "UUCP mail interchange format standard",
header field may be used (even though "followup" is not always RFC 976, February 1986.
defined under Alternative-1).
2.1
7 Provision is made for a poster to use a mailbox ending in [RFC1036] Horton, M. and R. Adams, "Standard for interchange of
".invalid" in a From header field (formerly in a-5.2). USENET messages", RFC 1036, December 1987.
7.5
8 "Inheritable" and "Variant" header fields defined (formerly in [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
a-4.2.5). Extensions (MIME) Part One: Format of Internet Message
2.3 Bodies", RFC 2045, November 1996.
9 Additional wording regarding function of verb/arguments/body in [RFC2606] Eastlake, D. and A. Panitz, "Reserved Top Level DNS
control messages (formerly in a-6.13). Names", BCP 32, RFC 2606, June 1999.
6
10 NOTE regarding not altering message indentifiers during [RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)",
transport or copying added (formerly in a-5.3). RFC 3977, October 2006.
7.3
11 All mention of MIME-style parameters and extension-parameters [USEAGE] Lindsey, C., "Usenet Best Practice".
removed.
3.1
7.6
For version 03 Appendix A. Changes to the Existing Protocols
1 The term "inheritable header" is no longer defined. Instead, the This document prescribes many changes, clarifications, and new
term "inherited' is used in place of "taken" when defining the features since the protocol described in [RFC1036]. Most notably:
actions of a followup agent.
7.6
News Article Architecture and Protocols November 2006
2 Consequent changes to "variant header field", and also mention o A new, backward-compatible Path header field format that permits
of Injection-Info as sometimes variant. standardized embedding of additional trace and authentication
2.3 information is now RECOMMENDED. See Section 3.4 and Section 3.5.
Folding of the Path header is permitted.
3 The term "reply address" is no longer defined. o Trimming of the References header field is REQUIRED and a
mechanism for doing so is defined.
4 References now made to sections within USEFOR using "F-..." o Addition of the new Injection-Date header field is required for
notation. injecting agents (Section 3.4) and MUST be used by news servers
for date checks (Section 3.5). Injecting agents are strongly
encouraged to put all local trace information in the new
Injection-Info header field.
5 Cross-references to sections within USEFOR added. Consistent use o A new media type is defined for transmitting Netnews articles
of <...> around all mentions of syntactic objects. All through other media (Section 4.1), and moderators SHOULD prepare
occurrences of "Foobar-header" changed to "Foobar header". Many to receive submissions in that format (Section 3.4.1).
other minor textual changes.
6 <control-message> changed to <control-command>, to avoid o Certain control messages (Section 5.6) are declared obsolete, and
confusion with "control message", which signifies the complete the special significance of "cmsg" at the start of a Subject
article containing the <control-command>. header field is removed.
7 <ihave-arguments> has been changed to <ihave-argument> (since o Additional media types are defined for improved structuring,
the earlier practice of multiple arguments is now deprecated). specification, and automated processing of control messages
Likewise <sendme-argument>. (Section 4.2 and Section 4.3).
For version 04 o Two new optional parameters are added to the checkgroups control
message.
1 References divided into Normative and Informational. o The message/news media type is declared obsolete.
2 All mention of the Mail-Copies-To, Posted-And-Mailed and o Cancel control messages are no longer required to have From and
Complaints-To header fields removed. Sender header fields matching those of the target message, as this
requirement added no real security.
3 NOTE added to contrast MUST for References header field with In addition, many protocol steps and article verification
SHOULD in RFC 2822. requirements unmentioned or left ambiguous by [RFC1036] but widely
7.6 implemented by Netnews servers have been standardized and specified
in detail.
4 Changes arising from the new syntax of <path-delimiter>s and Appendix B. Acknowledgements
<path-keyword>s.
7.3
5 Changes to clarify the construction of the References header This document is the result of a ten year effort and the number of
field. people that have contributed to its content are too numerous to
7.6.1 mention individually. Many thanks go out to all past and present
members of the USEFOR Working Group of the Internet Engineering Task
Force (IETF) and the accompanying mailing list.
6 Changes due to removal of <comment>s from further header fields. Special thanks are due to Henry Spencer, whose Son-of-1036 draft
served as the initial basis for this document.
7 New section on Identification of news-servers describing Authors' Addresses
acceptable forms for <path-identity>s.
2.3
8 Definition of "semantic content" of a header field. Russ Allbery (editor)
2.1 Stanford University
P.O. Box 20066
Stanford, CA 94309
US
9 Systematic replacement of "header" by "header field". Email: rra@stanford.edu
URI: http://www.eyrie.org/~eagle/
News Article Architecture and Protocols November 2006 Charles H. Lindsey
University of Manchester
5 Clerewood Avenue
Heald Green
Cheadle
Cheshire SK8 3JU
United Kingdom
10 More stringent rules for checking <newsgroup-name>s in control Phone: +44 161 436 6131
messages for compliance with USEFOR. Email: chl@clerew.man.ac.uk
6.2
For version 05 Full Copyright Statement
1 Historical Appendices A.1 and A.2 removed, in anticipation of Copyright (C) The Internet Society (2007).
republication of Son-of-1036 as an Informational RFC.
1.3
2 Definitions of technical terms adopted from USEFOR rather than This document is subject to the rights, licenses and restrictions
defining them separately. contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
3 Discussion of <path-identity> rewritten to reflect recent This document and the information contained herein are provided on an
developments (but still awaiting further work in USEFOR). "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
2.3 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
4 Items now included in Appendix A of USEFOR have been removed Intellectual Property
from section 3, and the "Transitional Arrangements" (which still
cover the USEFOR changes) have been modified to reflect this.
For version 06 The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
1 Procedures for constructing the Path header field updated to Copies of IPR disclosures made to the IETF Secretariat and any
conform with USEFOR. assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
2 s/serving agent/storage agent/ throughout. The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
3 s/trusted source/verified source/ Acknowledgment
4 Chapter 3 (Changes to the existing protocols) moved to Funding for the RFC Editor function is provided by the IETF
Appendices B & C, and subsequent chapters renumbered. Administrative Support Activity (IASA).
 End of changes. 348 change blocks. 
2517 lines changed or deleted 1483 lines changed or added

This html diff was produced by rfcdiff 1.33. The latest version is available from http://tools.ietf.org/tools/rfcdiff/