[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: (draft-allbery-usefor-usepro) 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 RFC 5537

INTERNET-DRAFT                               Charles H. Lindsey
Usenet Format Working Group                  University of Manchester
                                             February 2005

                News Article Architecture and Protocols
                   <draft-ietf-usefor-usepro-03.txt>

Status of this Memo

   By submitting this Internet-Draft, I certify that any applicable
   patent or other IPR claims of which I am aware have been
   disclosed, or will be disclosed, and any of which I become aware
   will be disclosed, in accordance with RFC 3668.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time. It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.html.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire in August 2005.

Abstract

   This Draft, together with its companion draft [USEFOR], are
   intended as standards track documents, together obsoleting RFC
   1036, which itself dates from 1987.

   This Standard defines the architecture of Netnews systems and
   specifies the requirements to be met by software which originates,
   distributes, stores and displays Netnews articles.

   Since the 1980s, Usenet has grown explosively, and many Internet and
   non-Internet sites now participate. In addition, the Netnews
   technology is now in widespread use for other purposes.

   Backward compatibility has been a major goal of this endeavour, but
   where this standard and earlier documents or practices conflict, this
   standard should be followed. In most such cases, current practice is
   already compatible with these changes.

   A companion Current Best Practice document [USEAGE], addressing
   requirements which are present for Social rather than Normative

C. H. Lindsey                                                   [Page 1]


                News Article Architecture and Protocols    February 2005

   reasons is in preparation.

[The use of the words "this standard" within this document when
referring to itself does not imply that this draft yet has pretensions
to be a standard, but rather indicates what will become the case if and
when it is accepted as an RFC with the status of a proposed or draft
standard.]

[Remarks enclosed in square brackets and aligned with the left margin,
such as this one, are not part of this draft, but are editorial notes to
explain matters amongst ourselves, or to point out alternatives, or to
assist the RFC Editor.]

[In this draft, references to [NNTP] are to be replaced by references to
the RFC arising from the series of drafts draft-ietf-nntpext-base-*.txt,
which has now passed its IETF last call.]

                           Table of Contents

1.  Introduction ..................................................    4
  1.1.  Basic Concepts ............................................    4
  1.2.  Objectives ................................................    5
  1.3.  Historical Outline ........................................    5
2.  Definitions, Notations and Conventions ........................    6
  2.1.  Definitions ...............................................    6
  2.2.  Defining the Architecture .................................    7
  2.3.  Variant Headers ...........................................    8
  2.4.  Textual Notations .........................................    9
3.  Changes to the existing protocols .............................    9
  3.1.  Principal Changes .........................................   10
  3.2.  Transitional Arrangements .................................   10
4.  Transport .....................................................   11
5.  Definition of new Media Types .................................   12
  5.1.  Application/news-transmission .............................   12
  5.2.  Message/news obsoleted ....................................   13
  5.3.  Application/news-groupinfo ................................   13
  5.4.  Application/news-checkgroups ..............................   14
6.  Control Messages ..............................................   15
  6.1.  Digital Signature of Headers ..............................   16
  6.2.  Group Control Messages ....................................   16
    6.2.1.  The 'newgroup' Control Message ........................   16
      6.2.1.1.  The Body of the 'newgroup' Control Message ........   17
      6.2.1.2.  Initial Articles ..................................   17
      6.2.1.3.  Example ...........................................   18
    6.2.2.  The 'rmgroup' Control Message .........................   19
      6.2.2.1.  Example ...........................................   19
    6.2.3.  The 'mvgroup' Control Message .........................   19
      6.2.3.1.  Example ...........................................   21
    6.2.4.  The 'checkgroups' Control Message .....................   21
  6.3.  Cancel ....................................................   23
  6.4.  Ihave, sendme .............................................   23
  6.5.  Obsolete control messages.  ...............................   25
7.  Duties of Various Agents ......................................   25

C. H. Lindsey                                                   [Page 2]


                News Article Architecture and Protocols    February 2005

  7.1.  General principles to be followed .........................   26
  7.2.  Duties of an Injecting Agent ..............................   26
    7.2.1.  Proto-articles ........................................   27
    7.2.2.  Procedure to be followed by Injecting Agents ..........   27
    7.2.3.  Procedure for Forwarding to a Moderator ...............   30
  7.3.  Duties of a Relaying Agent ................................   30
    7.3.1.  Path-Header Example ...................................   33
  7.4.  Duties of a Serving Agent .................................   34
  7.5.  Duties of a Posting Agent .................................   35
  7.6.  Duties of a Followup Agent ................................   36
  7.7.  Duties of a Reading Agent .................................   37
  7.8.  Duties of a Moderator .....................................   37
  7.9.  Duties of a Gateway .......................................   39
    7.9.1.  Duties of an Outgoing Gateway .........................   40
    7.9.2.  Duties of an Incoming Gateway .........................   41
    7.9.3.  Example ...............................................   43
8.  Security and Related Considerations ...........................   44
  8.1.  Leakage ...................................................   44
  8.2.  Attacks ...................................................   44
    8.2.1.  Denial of Service .....................................   44
    8.2.2.  Compromise of System Integrity ........................   46
  8.3.  Liability .................................................   47
9.  IANA Considerations ...........................................   47
10.  References ...................................................   48
11.  Acknowledgements .............................................   49
12.  Contact Address ..............................................   49
Appendix A.1 - A-News Article Format ..............................   49
Appendix A.2 - Early B-News Article Format ........................   50
Appendix A.3 - Obsolete Control Messages ..........................   51
Appendix B - Notices ..............................................   51
Appendix C - Change Log ...........................................   52












C. H. Lindsey                                                   [Page 3]


                News Article Architecture and Protocols    February 2005

[This draft [USEPRO] and its partner [USEFOR] are an interim stage in
the splitting into two parts of the earlier draft [ARTICLE].  There is a
certain amount of material - basic concepts, definitions, etc - which
ultimately need occur in only one of the documents, and further such
material which may not be needed at all (e.g. terms currently defined
which in the event may not get used). For the moment, all such material
has been retained in the present draft (it being, in any case, easier to
take unwanted stuff out than to put new stuff in). It has also to be
decided, for such material which is needed by both documents, which one
(the "Primary") should contain it and which one should incorporate it by
reference (essentially, this draft is written so that it could be the
Primary).]

1.  Introduction

1.1.  Basic Concepts

   "Netnews" is a set of protocols for generating, storing and
   retrieving news "articles" (which resemble email messages) and for
   exchanging them amongst a readership which is potentially widely
   distributed. It is organized around "newsgroups", with the
   expectation that each reader will be able to see all articles posted
   to each newsgroup in which he participates. These protocols most
   commonly use a flooding algorithm which propagates copies throughout
   a network of participating servers.  Typically, only one copy is
   stored per server, and each server makes it available on demand to
   readers able to access that server.

   An important characteristic of Netnews is the lack of any requirement
   for a central administration or for the establishment of any
   controlling host to manage the network. A set of hosts within a
   network which, by mutual arrangement, operates some variant (whether
   more or less restrictive) of the Netnews protocols is a "cooperating
   subnet".

   "Usenet" is a particular worldwide publicly accessible network based
   upon the Netnews protocols, with the newsgroups being organized into
   recognized "hierarchies".  Anybody can join (it is simply necessary
   to negotiate an exchange of articles with one or more other
   participating hosts).

   A "policy" is a rule intended to facilitate the smooth operation of a
   network by establishing parameters which restrict behaviour that,
   whilst technically unexceptionable, would nevertheless contravene
   some accepted standard of "Good Netkeeping". Since the ultimate
   beneficiaries of a network are its human readers, who will be less
   tolerant of poorly designed interfaces than mere computers, articles
   in breach of established policy can cause considerable annoyance to
   their recipients.



C. H. Lindsey                                                   [Page 4]


                News Article Architecture and Protocols    February 2005

1.2.  Objectives

   The purpose of this present standard is to define the overall
   architecture and the protocols to be used for Netnews in general, and
   for Usenet in particular, and to set standards to be followed by
   software that implements those protocols. A companion standard
   [USEFOR] sets out the canonical format of news articles exchanged
   between the various agents comprising that architecture. In this
   standard, references to sections in the companion [USEFOR] are
   prefixed with "F-".

   It is NOT the purpose of this standard to settle matters of policy,
   nor aspects of software behaviour which do not impinge upon the
   generation, transmission, storage and reception of articles, nor how
   the authority of various agencies to create such policies and to
   exercise control or oversight of the various parts of Usenet is
   established. For these purposes, a separate Best Current Practice
   document [USEAGE] is being provided.

   Nevertheless, it is assumed that such agencies with the necessary
   authority will exist, and tools are provided within the protocols for
   their use.

1.3.  Historical Outline

   Network news originated as the medium of communication for Usenet,
   circa 1980.  Since then, Usenet has grown explosively, and many
   Internet and non-Internet sites participate in it.  In addition, the
   news technology is now in widespread use for other purposes, on the
   Internet and elsewhere.

   The earliest news interchange used the so-called "A News" article
   format.  Shortly thereafter, an article format vaguely resembling
   Internet Mail was devised and used briefly.  Both of those formats
   are completely obsolete; they are documented in Appendix A.1 and
   Appendix A.2 for historical reasons only.  With publication of [RFC
   850] in 1983, news articles came to closely resemble Internet Mail
   messages, with some restrictions and some additional headers. [RFC
   1036] in 1987 updated [RFC 850] without making major changes.

   A Draft popularly referred to as "Son of 1036" [Son-of-1036] was
   written in 1994 by Henry Spencer.  Much is taken directly from Son of
   1036, and it is hoped that we have followed its spirit and
   intentions.

[It is anticipated that [Son-of-1036] will shortly be published as an
informational RFC (for purposes of historical documentation only), in
which case most historical information can be removed from this draft,
including the whole of Appendix A.1 and Appendix A.2.]



C. H. Lindsey                                                   [Page 5]


                News Article Architecture and Protocols    February 2005

2.  Definitions, Notations and Conventions

2.1.  Definitions

   An "article" is the unit of news, synonymous with an [RFC 2822]
   "message".  A "proto-article" (7.2.1) is one that has been created by
   a "posting agent" but has not yet been injected into a Netnews system
   system by an "injecting agent". It may lack some otherwise mandatory
   headers.

   A "message identifier" (F-3.1.3) is a unique identifier for an
   article, usually supplied by the posting agent which posted it or,
   failing that, by the injecting agent. It distinguishes the article
   from every other article ever posted anywhere. Articles with the same
   message identifier are treated as if they are the same article
   regardless of any differences in the body or headers.

   A "newsgroup" is a single news forum, a logical bulletin board,
   having a name and nominally intended for articles on a specific
   topic. An article is "posted to" a single newsgroup or several
   newsgroups. When an article is posted to more than one newsgroup, it
   is said to be "crossposted"; note that this differs from posting the
   same text as part of each of several articles, one per newsgroup.

   A newsgroup may be "moderated", in which case submissions are not
   posted directly, but mailed to a "moderator" for consideration and
   possible posting.  Moderators are typically human but may be
   implemented partially or entirely in software.

   A "hierarchy" is the set of all newsgroups whose names share a first
   component (as defined in F-3.1.5).  The term "sub-hierarchy" is also
   used where several initial components are shared.

   A "poster" is the person or software that composes a possibly
   compliant article for submission to a posting agent. The poster is
   synonymous with [RFC 2822]'s author.

   A "reader" is the person or software reading news articles.

[Alternative-1.]

   A "followup" is an article containing a response to the contents of
   an earlier article.

   An article is a "precursor" of some later article which is a followup
   to it, or which is otherwise intended to be grouped with it for
   purposes of display (e.g. as a part of a multipart posting such as a
   FAQ).

[Alternative-2.]

   A "followup" to an earlier article (its "precursor") is one intended
   to be grouped with that article for purposes of display (e.g. because
   it is a response to its contents or is a part of a multipart posting

C. H. Lindsey                                                   [Page 6]


                News Article Architecture and Protocols    February 2005

   such as a FAQ).
[End of alternatives]

   An (email) "address" is the mailbox [RFC 2822] (or more particularly
   the addr-spec within that mailbox) which directs the delivery of an
   email to its intended recipient, who is said to "own" that address.

   A "sender" is the person or software (usually, but not always, the
   same as the poster) responsible for the operation of the posting
   agent or, which amounts to the same thing, for passing the article to
   the injecting agent.
[Is the definition in RFC 2822 sufficient?]

   A "control message" is an article which is marked as containing
   control information; a "serving agent" (and in some cases a "relaying
   agent") receiving such an article may (subject to the policies
   observed at that site) take actions beyond just filing and passing on
   the article.

2.2.  Defining the Architecture

   A Netnews system is a distributed database composed of "agents" of
   various types which, acting together according to the protocols
   defined in section 7 of this standard, causes articles to be
   propagated throughout the system and to be made available to its
   readers. The protocols ensure that all copies of a given article,
   wherever stored, are identical apart from those headers defined as
   variant (2.3).

   A "posting agent" is the software that assists posters to prepare
   proto-articles in compliance with [USEFOR].  The proto-article is
   then passed on to an "injecting agent" for final checking and
   injection into the news stream. If the article is not compliant, or
   is rejected by the injecting agent, then the posting agent informs
   the poster with an explanation of the error.

   A "reading agent" is software which presents articles to a reader.

[Alternative-1.]

   A "followup agent" is a combination of reading agent and posting
   agent that aids in the preparation and posting of a followup.

[Alternative-2.]

   A "followup agent" is a combination of reading agent and posting
   agent that aids in the preparation and posting of an article which is
   a response to some precursor.
[End of alternatives]

   An "injecting agent" takes the finished article from the posting
   agent (often via the NNTP "POST" command), performs some final checks
   and passes it on to a "relaying agent" for general distribution. It
   is expected to bear some responsibility towards the rest of the

C. H. Lindsey                                                   [Page 7]


                News Article Architecture and Protocols    February 2005

   network for the behaviour of its posters (and provision is therefore
   made for it to be easily contactable by email).
[That provision is expected to go into USEFOR.]

   A "relaying agent" is software which receives allegedly compliant
   articles from injecting agents and/or other relaying agents, and
   possibly passes copies on to other relaying agents and "serving
   agents".

   A "serving agent" receives an article from a relaying agent and files
   it in a "news database". It also provides an interface for reading
   agents to access the news database.

   A "news database" is the set of articles and related structural
   information stored by a serving agent and made available for access
   by reading agents.

   A "gateway" is software which receives news articles and converts
   them to messages of some other kind (e.g. mail to a mailing list), or
   vice versa; in essence it is a translating relaying agent that
   straddles boundaries between different methods of message exchange.
   The most common type of gateway connects newsgroup(s) to mailing
   list(s), either unidirectionally or bidirectionally, but there are
   also gateways between news networks using the [USEFOR] news format
   and those using other formats.

   Posting, reading and followup agents (which are usually just
   different services provided by the same piece of software) are known
   collectively as "user agents".

2.3.  Variant Headers

   Headers with the variant property may differ between (or even be
   completely absent from) copies of the same article as stored or
   relayed throughout a Netnews system. The manner of the difference (or
   absence) MUST be as specified in this (or some future) standard.
   Typically, these headers are modified as articles are propagated, or
   they reflect the status of the article on a particular serving agent,
   or cooperating group of such agents. A variant header MAY be placed
   anywhere within the headers (though placing it first is recommended).

   The following headers are classified as "variant":
     o Path (F-3.1.6) - augmented at each relaying agent that an article
       passes through.
     o Xref (F-3.2.10) - used to keep track of the <article-locator>s of
       crossposted articles so that reading agents serviced by a
       particular serving agent can mark such articles as read.
     o Injection-Info (F-3.2.13) is also considered variant in some
       special situations involving reinjection (7.2 and 7.2.2).



C. H. Lindsey                                                   [Page 8]


                News Article Architecture and Protocols    February 2005

2.4.  Textual Notations

   This standard contains explanatory NOTEs using the following format.
   These may be skipped by persons interested solely in the content of
   the specification. The purpose of the notes is to explain why choices
   were made, to place them in context, or to suggest possible
   implementation techniques.

        NOTE: While such explanatory notes may seem superfluous in
        principle, they often help the less-than-omniscient reader grasp
        the purpose of the specification and the constraints involved.
        Given the limitations of natural language for descriptive
        purposes, this improves the probability that implementors and
        users will understand the true intent of the specification in
        cases where the wording is not entirely clear.

   "US-ASCII" is short for "the ANSI X3.4 character set" [ANSI X3.4].
   US-ASCII is a 7 bit character set. Please note that this standard
   requires that all agents be 8 bit clean; that is, they must accept
   and transmit data without changing or omitting the 8th bit.

   Certain words, when capitalized, are used to define the significance
   of individual requirements. The key words "MUST", "REQUIRED",
   "SHOULD", "RECOMMENDED", "MAY" and "OPTIONAL", and any of those words
   associated with the word "NOT", are to be interpreted as described in
   [RFC 2119].

        NOTE: The use of "MUST" or "SHOULD" implies a requirement that
        would or could lead to interoperability problems if not
        followed.

        NOTE: A requirement imposed on a relaying or serving agent
        regarding some particular article should be understood as
        applying only if that article is actually accepted for
        processing (since any agent may always reject any article
        entirely, for reasons of site policy).

   Wherever the context permits, use of the masculine includes the
   feminine and use of the singular includes the plural, and vice versa.

   Throughout this standard we will give examples of various
   definitions, headers and other specifications. It needs to be
   remembered that these samples are for the aid of the reader only and
   do NOT define any specification themselves.  In order to prevent
   possible conflict with "Real World" entities and people the top level
   domain ".example" is used in all sample domains and addresses. The
   hierarchy "example.*" is also used as a sample hierarchy.
   Information on the ".example" top level domain is in [RFC 2606].

3.  Changes to the existing protocols

   This standard prescribes many changes, clarifications and new
   features since the protocols described in [RFC 1036] and [Son-of-
   1036].  It is the intention that they can be assimilated into Usenet

C. H. Lindsey                                                   [Page 9]


                News Article Architecture and Protocols    February 2005

   as it presently operates without major interruption to the service,
   though some of the new features may not begin to show benefit until
   they become widely implemented. This section summarizes the main
   changes, and comments on some features of the transition.

3.1.  Principal Changes

     o The [RFC 2822] conventions for parenthesis-enclosed <comment>s in
       headers are supported.
     o Whitespace is permitted in Newsgroups headers, permitting folding
       of such headers. Indeed, all headers can now be folded.
     o An enhanced syntax for the Path header enables the injection
       point of and the route taken by an article to be determined with
       certainty.
     o MIME is recognized as an integral part of Netnews.
     o There is a new Control message 'mvgroup' to facilitate moving a
       group to a different place (name) in a hierarchy.
     o There is a new mandatory Injection-Date header to facilitate the
       rejection of stale articles.
     o There are several new optional headers defined, notably Archive,
       Complaints-To, Injection-Info, Mail-Copies-To, Posted-And-Mailed
       and User-Agent, leading to increased functionality.
     o Certain headers and Control messages (F-3.3 and Appendix A.3)
       have been made obsolete.
     o Distributions are expected to be checked at the receiving end, as
       well as the sending end, of a relaying link.
     o There are numerous other small changes, clarifications and
       enhancements.

3.2.  Transitional Arrangements

   An important distinction must be made between serving and relaying
   agents, which are responsible for the distribution and storage of
   news articles, and user agents, which are responsible for
   interactions with users. It is important that the former should be
   upgraded to conform to this standard as soon as possible to provide
   the benefit of the enhanced facilities.  Fortunately, the number of
   distinct implementations of such agents is rather small, at least so
   far as the main "backbone" of Usenet is concerned, and many of the
   new features are already supported. Contrariwise, there are a great
   number of implementations of user agents, installed on a vastly
   greater number of small sites. Therefore, the new functionality has
   been designed so that existing user agents may continue to be used,
   although the full benefits may not be realised until a substantial
   proportion of them have been upgraded.

   In the list which follows, care has been taken to distinguish the
   implications for both kinds of agent.

     o [RFC 2822] style <comment>s in headers do not affect serving and
       relaying agents (note that the Message-ID, Newsgroups,
       Distribution and Path headers do not contain them). They are
       unlikely to hinder their proper display in existing reading
       agents except in the case of the References header in agents

C. H. Lindsey                                                  [Page 10]


                News Article Architecture and Protocols    February 2005

       which thread articles. Therefore, it is provided that they SHOULD
       NOT be generated except where permitted by the previous
       standards.
     o Because of its importance to all serving agents, the extension
       permitting whitespace and folding in Newsgroups headers SHOULD
       NOT be used until it has been widely deployed amongst relaying
       agents. User agents are unaffected.
     o The new style of Path header is already consistent with the
       previous standards. However, the intention is that relaying
       agents should eventually reject articles in the old style, and so
       this possibility should be offered as a configurable option in
       relaying agents. User agents are unaffected.
     o The introduction of MIME reflects a practice that is already
       widespread.  Articles in strict compliance with the previous
       standards (using strict US-ASCII) will be unaffected. Many user
       agents already support it, at least to the extent of widely used
       charsets such as ISO-8859-1. Users expecting to read articles
       using other charsets will need to acquire suitable reading
       agents. It is not intended, in general, that any single user
       agent will be able to display every charset known to IANA, but
       all such agents MUST support US-ASCII. Serving and relaying
       agents are not affected.
     o The new Control: mvgroup command will need to be implemented in
       serving agents. For the benefit of older serving agents it is
       therefore RECOMMENDED that it be followed shortly by a
       corresponding newgroup command and it MUST always be followed by
       a rmgroup command for the old group after a reasonable overlap
       period. An implementation of the mvgroup command as an alias for
       the newgroup command would thus be minimally conforming. User
       agents are unaffected.
     o Provision is made for relaying and serving agents to use the Date
       header in the case of articles injected through existing agents
       which do not provide an Injection-Date header.
     o All the headers newly introduced by this standard can safely be
       ignored by existing software, albeit with loss of the new
       functionality.

4.  Transport

   As in this standard's predecessors, the exact means used to transmit
   articles from one host to another is not specified. NNTP [NNTP] is
   the most common transmission method on the Internet, but much
   transmission takes place entirely independent of the Internet. Other
   methods in use include the UUCP protocol [RFC 976] extensively used
   in the early days of Usenet, FTP, downloading via satellite, tape
   archives, and physically delivered magnetic and optical media.

   Transmission paths for news articles MUST treat news articles as
   uninterpreted sequences of octets, excluding the values 0 (US-ASCII
   NUL) and 13 and 10 (US-ASCII CR and LF, which MUST ONLY appear in the
   combination CRLF which denotes a line separator).


C. H. Lindsey                                                  [Page 11]


                News Article Architecture and Protocols    February 2005

        NOTE: this corresponds to the range of octets permitted for MIME
        "8bit data" [RFC 2045].  Thus raw binary data cannot be
        transmitted in an article body except by the use of a Content-
        Transfer-Encoding such as base64.

   In particular, transmission paths MUST convey all headers (including
   body part headers and headers within message/rfc822 objects) intact,
   even if they contain octets in the range 128 to 255.  These
   requirements include the transmissiom paths between posting agents,
   injecting agents, relaying agents, serving agents and reading agents,
   but NOT the paths traversed by Netnews articles that have been
   gatewayed into Email (7.9.1).
[At some point it will be necessary for the IMAP standards to catch up
with these requirements.]

5.  Definition of new Media Types

   This standard defines (or redefines) several new Media Types, which
   require to be registered with IANA as provided for in [RFC 2048].

5.1.  Application/news-transmission

   The Media Type "application/news-transmission" is intended for the
   encapsulation of complete news articles where the intention is that
   the recipient should then inject them into Netnews. This Application
   type provides one of the methods for mailing articles to moderators
   (see 7.2.2) and it is also the preferred method when sending to an
   email-to-news gateway (see 7.9.2).

        NOTE: The benefit of such encapsulation is that it removes
        possible conflict between news and email headers and it provides
        a convenient way of "tunnelling" a news article through a
        transport medium that does not support 8bit characters.

   The MIME Media Type definition of "application/news-transmission" is:

   MIME type name:           application
   MIME subtype name:        news-transmission
   Required parameters:      none
   Optional parameters:      usage=moderate
                             usage=inject
                             usage=relay
   Encoding considerations:  A transfer-encoding (such as Quoted-
                             Printable or Base64) different from that of
                             the article transmitted MAY be supplied
                             (perhaps en route) to ensure correct
                             transmission over some 7bit transport
                             medium.
   Security considerations:  A news article may be a "control message",
                             which could have effects on the recipient
                             host's system beyond just storage of the
                             article. However, such control messages
                             also occur in normal news flow, so most
                             hosts will already be suitably defended

C. H. Lindsey                                                  [Page 12]


                News Article Architecture and Protocols    February 2005

                             against undesired effects.
   Published specification:  [USEPRO]
   Body part:                A complete article or proto-article, ready
                             for injection into Netnews, or a batch of
                             such articles in the batch format described
                             in section 6.4.

        NOTE: It is likely that the recipient of an "application/news-
        transmission" will be a specialized gateway (e.g. a moderator's
        submission address) able to accept articles with only one of the
        three usage parameters "moderate", "inject" and "relay", hence
        the reason why they are optional, being redundant in most
        situations. Nevertheless, they MAY be used to signify the
        originator's intention with regard to the transmission, so
        removing any possible doubt.

   When the parameter "relay" is used, or implied, the body part MAY be
   a batch of articles to be transmitted together, in which case the
   batch format defined in section 6.4 MUST be used.

5.2.  Message/news obsoleted

   The Media Type "message/news", as previously registered with IANA, is
   hereby declared obsolete. It was never widely implemented, and its
   default treatment as "application/octet-stream" by agents that did
   not recognize it was counter productive. The Media Type
   "message/rfc822" SHOULD be used in its place.

5.3.  Application/news-groupinfo

   The "application/news-groupinfo" is used in conjunction with the
   "newgroup" (6.2.1) and "mvgroup" (6.2.3) control messages.  The
   <newsgroup-name> in the <newsgroups-line> MUST agree with the
   <newsgroup-name> in the "newgroup" or "mvgroup" control message.  The
   Media Type "application/news-groupinfo" MUST NOT be used except as a
   part of such control messages.

   The "application/news-groupinfo" body part contains brief information
   about a newsgroup, i.e. the group's name, it's <newsgroup-
   description> and the <moderation-flag>.

        NOTE: The presence of the <newsgroups-tag> "For your newsgroups
        file:" is intended to make the whole newgroup message compatible
        with current practice as described in [Son-of-1036].

   The MIME Media Type definition of "application/news-groupinfo" is:

   MIME type name:           application
   MIME subtype name:        news-groupinfo
   Required parameters:      none
   Disposition:              by default, inline
   Encoding considerations:  "7bit" or "8bit" is sufficient and MUST be
                             used to maintain compatibility.
   Security considerations:  this type MUST NOT be used except as part

C. H. Lindsey                                                  [Page 13]


                News Article Architecture and Protocols    February 2005

                             of a control message for the creation or
                             modification of a Netnews newsgroup
   Published specification:  [USEPRO]

   The content of the "application/news-groupinfo" body part is defined
   as:

      groupinfo-body      = [ newsgroups-tag CRLF ]
                               newsgroups-line CRLF
      newsgroups-tag      = %x46.6F.72 SP %x79.6F.75.72 SP
                               %x6E.65.77.73.67.72.6F.75.70.73 SP
                               %x66.69.6C.65.3A
                               ; case sensitive
                               ; "For your newsgroups file:"
      newsgroups-line     = newsgroup-name
                               [ 1*HTAB newsgroup-description ]
                               [ 1*WSP moderation-flag ]
      newsgroup-description
                          = utext *( *WSP utext )
      moderation-flag     = %x28.4D.6F.64.65.72.61.74.65.64.29
                               ; case sensitive "(Moderated)"

   The <newsgroup-description> MUST NOT contain any occurrence of the
   string "(Moderated)" within it.  Although optional, the <newsgroups-
   tag> SHOULD be included until such time as this standard has been
   widely adopted, to ensure compatibility with present practice.

   Moderated newsgroups MUST be marked by appending the case sensitive
   text " (Moderated)" at the end. It is NOT recommended that the
   moderator's email address be included in the <newsgroup-description>
   as has sometimes been done.

        NOTE: There is no provision for the use of charsets other than
        US-ASCII within a <newsgroup-description>. Such a facility may
        be provided in a future extension to this standard.
[That may seem harsh, but if we make any such provision now, it will
make life more complicated and restrict our freedom when it comes to the
proposed I18N extension. Therefore I resisted the temptation to include
any charset parameter with this Media Type. Note that this also applies
to the checkgroups message further on.]

5.4.  Application/news-checkgroups

   The "application/news-checkgroups" Media Type is used in conjunction
   with the "checkgroups" control message (6.2.4).  It MUST NOT be used
   except as a part of such control messages.

   The "application/news-checkgroups" body part contains a complete list
   of all the newsgroups in a (sub)hierarchy, their <newsgroup-
   description>s and their moderation status.

   The MIME Media Type definition of "application/news-checkgroups" is:

   MIME type name:           application

C. H. Lindsey                                                  [Page 14]


                News Article Architecture and Protocols    February 2005

   MIME subtype name:        news-checkgroups
   Required parameters:      none
   Disposition:              by default, inline
   Encoding considerations:  "7bit" or "8bit" is sufficient and MUST be
                             used to maintain compatibility.
   Security considerations:  this type MUST NOT be used except as part
                             of a checkgroups control message
   Published specification:  [USEPRO]

   The content of the "application/news-checkgroups" body part is
   defined as:

      checkgroups-body    = *( valid-group CRLF )
      valid-group         = newsgroups-line ; see 5.3

6.  Control Messages

   The following sections document the control messages.  "Message" is
   used herein as a synonym for "article" unless context indicates
   otherwise.

   Each <control-command> comprises a <verb>, which indicates the action
   to be taken, and <argument>(s), which supply the details (see F-
   3.2.4).  The following sections contain syntactic definitions for the
   <verb>, <argument>s, and possibly the body, for each type of control
   message.
[The term <control-command> is now used to denote the syntactic object
within the Control header, to distinguish it from "control message",
which refers to the whole article.]

   The Newsgroups header of each control message SHOULD include the
   <newsgroup-name>(s) for the group(s) affected (i.e. groups to be
   created, modified or removed, or containing articles to be canceled).
   This is to ensure that the message propagates to all sites which
   receive (or would receive) that group(s). It MAY include other
   <newsgroup-name>s so as to improve propagation (but this practice may
   cause the control message to propagate also to places where it is
   unwanted, or even cause it not to propagate where it should, so it
   should not be used without good reason).

        NOTE: Propagation is controlled by relaying agents, and it may
        be necessary for relaying agents to take special steps to ensure
        that control messages such as newgroup messages for not-yet-
        existent newsgroups are propagated correctly (see 7.3).

   The presence of a Subject header whose content starts with the string
   "cmsg " followed by a <control-command> was construed under [RFC
   1036] as a request to perform that control action (even if no genuine
   Control header was present). Indeed, some implementations went
   further and added the implied Control header before injecting.
   Likewise, the presence of a <newsgroup-name> ending in ".ctl" in the
   Newsgroups header caused the Subject header content (not starting
   with "cmsg" in this case) to be interpreted as a <control-command>.

C. H. Lindsey                                                  [Page 15]


                News Article Architecture and Protocols    February 2005

   All these practices are now declared to be Obsolete, and Subject
   headers MUST NOT now be interpreted as <control-command>s under any
   circumstances.

   The descriptions below set out REQUIREMENTS to be followed by sites
   that receive control messages and choose to honour them. However,
   nothing in these descriptions should be taken as overriding the right
   of any such site, in accordance with its local policy, to refuse to
   honour any particular control message, or to refer it to an
   administrator for approval (either as a class or on a case-by-case
   basis).

6.1.  Digital Signature of Headers

   It is most desirable that group control messages (6.2) in particular
   be authenticated by incorporating them within some digital signature
   scheme that encompasses other headers closely associated with them
   (including at least the Approved, Message-ID and Date headers). At
   the time of writing, this is usually done by means of a protocol
   known as "PGPverify" ([PGPVERIFY]), and continued usage of this is
   encouraged at least as an interim measure.

   However, PGPverify is not considered suitable for standardization in
   its present form, for various technical reasons. It is therefore
   expected that an early extension to this standard will provide a
   robust and general purpose digital authentication mechanism with
   applicability to all situations requiring protection against
   malicious use of, or interference with, headers.  That extension
   would also address other Netnews security issues.

6.2.  Group Control Messages

   "Group control messages" are the sub-class of control messages that
   request some update to the configuration of the groups known to a
   serving agent, namely "newgroup", "rmgroup", "mvgroup" and
   "checkgroups", plus any others created by extensions to this
   standard.

   All of the group control messages MUST have an Approved header
   (F-3.2.8) which, in those hierarchies where appropriate
   administrative agencies exist (see 1.1), identifies the appropriate
   person or entity as authorized by those agencies.  The authorized
   person or entity SHOULD adhere to the conventions and restrictions on
   the format of <newsgroup-name>s established for those hierarchies
   (see F-3.1.5).
[But that reference presumes some changes in the current USEFOR.]

6.2.1.  The 'newgroup' Control Message

      control-command     =/ Newgroup-command
      Newgroup-command    = "newgroup" Newgroup-arguments
      Newgroup-arguments  = CFWS newsgroup-name [ CFWS newgroup-flag ]
      newgroup-flag       = "moderated"

C. H. Lindsey                                                  [Page 16]


                News Article Architecture and Protocols    February 2005

   The "newgroup" control message requests that the specified group be
   created or have its moderation status or <newsgroups-line> changed.
   When the request is honoured, if the <newgroup-flag> "moderated" is
   present then the status of the group SHOULD be marked as moderated,
   and vice versa.  "Moderated" is the only such flag defined by this
   standard; other flags MAY be defined for use in cooperating subnets,
   but newgroup messages containing them MUST NOT be acted on outside of
   those subnets.

        NOTE: Specifically, some alternative flags such as "y" and "m",
        which are sent and recognized by some current software, are NOT
        part of this standard.  Moreover, some existing implementations
        treat any flag other than "moderated" as indicating an
        unmoderated newsgroup. Both of these usages are contrary to this
        standard and control messages with such non-standard flags
        should be ignored.

6.2.1.1.  The Body of the 'newgroup' Control Message

   The body of the newgroup message contains the following subparts,
   preferably in the order shown:

   1. An "application/news-groupinfo" part (5.3) containing the name and
      <newsgroups-line> (5.3) of the group. This part MUST be present
      and SHOULD be used to update any copy of the <newsgroups-line>
      maintained by the serving agent.

   2. Other parts containing useful information about the background of
      the newgroup message (typically of type "text/plain").

   3. Parts containing initial articles for the newsgroup. See section
      6.2.1.2 for details.

   In the event that there is only the single (i.e. application/news-
   groupinfo) subpart present, it will suffice to include a "Content-
   Type:  application/news-groupinfo" amongst the headers of the control
   message.  Otherwise, a "Content-Type: multipart/mixed" header will be
   needed, and each separate part will then need its own Content-Type
   header.

6.2.1.2.  Initial Articles

   Some subparts of a "newgroup" or "mvgroup" control message MAY
   contain an initial set of articles to be posted to the affected
   newsgroup as soon as it has been created or modified. These parts are
   identified by having the Media Type "application/news-transmission",
   possibly with the parameter "usage=inject".  The body of each such
   part should be a complete proto-article, ready for posting. This
   feature is intended for the posting of charters, initial FAQs and the
   like to the newly formed group.

   The Newsgroups header of the proto-article MUST include the
   <newsgroup-name> of the newly created or modified group. It MAY
   include other <newsgroup-name>s. If the proto-article includes a

C. H. Lindsey                                                  [Page 17]


                News Article Architecture and Protocols    February 2005

   Message-ID header, the message identifier in it MUST be different
   from that of any existing article and from that of the control
   message as a whole.  Alternatively such a message identifier MAY be
   derived by the injecting agent when the proto-article is posted. The
   proto-article SHOULD include the header "Distribution: local".

   The proto-article SHOULD be injected at the serving agent that
   processes the control message AFTER the newsgroup in question has
   been created or modified.  It MUST NOT be injected if the newsgroup
   is not, in fact, created (for whatever reason). It MUST NOT be
   submitted to any relaying agent for transmission beyond the serving
   agent(s) upon which the newsgroup creation has just been effected (in
   other words, it is to be treated as having a "Distribution:  local"
   header, whether such a header is actually present or not).

        NOTE: It is not precluded that the proto-article is itself a
        control message or other type of special article, to be
        activated only upon creation of the new newsgroup. However,
        except as might arise from that possibility, any
        "application/news-transmission" within some nested "multipart/*"
        structure within the proto-article is not to be activated.

6.2.1.3.  Example

   A "newgroup" with its charter:

      From: "example.all Administrator" <admin@noc.example>
      Newsgroups: example.admin.info,example.admin.announce
      Date: 27 Feb 2002 12:50:22 +0200
      Subject: cmsg newgroup example.admin.info moderated
      Approved: admin@noc.example
      Control: newgroup example.admin.info moderated
      Message-ID: <ng-example.admin.info-20020227@noc.example>
      MIME-Version: 1.0
      Content-Type: multipart/mixed; boundary="nxtprt"
      Content-Transfer-Encoding: 8bit

      This is a MIME control message.
      --nxtprt
      Content-Type: application/news-groupinfo

      For your newsgroups file:
      example.admin.info      About the example.* groups (Moderated)

      --nxtprt
      Content-Type: application/news-transmission

      Newsgroups: example.admin.info
      From: "example.all Administrator" <admin@noc.example>
      Subject: Charter for example.admin.info
      Message-ID: <charter-example.admin.info-20020227@noc.example>
      Distribution: local


C. H. Lindsey                                                  [Page 18]


                News Article Architecture and Protocols    February 2005

      Content-Type: text/plain; charset=us-ascii
      Content-Transfer-Encoding: 7bit

      The group example.admin.info contains regularly posted
      information on the example.* hierarchy.

      --nxtprt--

6.2.2.  The 'rmgroup' Control Message

      control-command     =/ Rmgroup-command
      Rmgroup-command     = "rmgroup" Rmgroup-arguments
      Rmgroup-arguments   = CFWS newsgroup-name

   The "rmgroup" control message requests that the specified group be
   removed from the list of valid groups. The Media Type of the body is
   unspecified; it MAY contain anything, usually an explanatory text.

        NOTE: It is entirely proper for a serving agent to retain the
        group until all the articles in it have expired, provided that
        it ceases to accept new articles.

6.2.2.1.  Example

      From: "example.all Administrator" <admin@noc.example>
      Newsgroups: example.admin.obsolete, example.admin.announce
      Date: 4 Apr 2002 22:04 -0900 (PST)
      Subject: cmsg rmgroup example.admin.obsolete
      Message-ID: <rm-example.admin.obsolete-20020404@noc.example>
      Approved: admin@noc.example
      Control: rmgroup example.admin.obsolete

      The group example.admin.obsolete is obsolete. Please remove it
      from your system.

6.2.3.  The 'mvgroup' Control Message

      control-command   =/ Mvgroup-command
      Mvgroup-command   = "mvgroup" Mvgroup-arguments
      Mvgroup-arguments = CFWS newsgroup-name CFWS newsgroup-name
                             [ CFWS newgroup-flag ]

   The "mvgroup" control message requests that the group specified by
   the first (old-)newsgroup-name be moved to that specified by the
   second (new-)newsgroup-name. Thus it is broadly equivalent to a
   "newgroup" control message for the second group followed by a
   "rmgroup" control message for the first group.

   The message body contains an "application/news-groupinfo" part (5.3)
   containing machine- and human-readable information about the new
   group, and possibly other subparts as for a "newgroup" control
   message. The information conveyed in the "application/news-groupinfo"
   body part, notably its <newsgroups-line> (5.3), is applied to the new
   group.

C. H. Lindsey                                                  [Page 19]


                News Article Architecture and Protocols    February 2005

   When this message is received, the new group is created (if it does
   not exist already) as for a "newgroup" control message, and SHOULD in
   any case be made moderated if a <newgroup-flag> "moderated" is
   present, and vice versa. At the same time, arrangements SHOULD be
   made to remove the old group (as with a "rmgroup" control message),
   but only after a suitable overlap period to allow the network to
   adjust to the new arrangement.

   At the same time as a serving agent acts upon this message, all
   injecting agents associated with that serving agent SHOULD inhibit
   the posting of new articles to the old group (preferably with some
   indication to the poster that the new group should have been used).
   Relaying agents, however, MUST continue to propagate such articles
   during the overlap period.

        NOTE: It is to be expected that different serving agents will
        act on this message at different points of time, users of the
        old group will have to become accustomed to the new arrangement,
        and followups to already established threads will likely
        continue under the old group. Therefore, there needs to be an
        overlap period during which articles may continue to be accepted
        by relaying and serving agents in either group. This standard
        does not specify any standard period of overlap (though it would
        be expected to be expressed in days rather than in months). The
        inhibition of injection of new articles to the old group may
        seem draconian, but it is the surest way to prevent the
        changeover from dragging on indefinitely.

   Since the "mvgroup" control message is newly introduced in this
   standard and may not be widely implemented initially, it SHOULD be
   followed shortly afterwards by a corresponding "newgroup" control
   message; and again, after a reasonable overlap period, it MUST be
   followed by a "rmgroup" control message for the old group.

   In order to facilitate a smooth changeover, serving agents MAY
   arrange to service requests for access to the old group by providing
   access to the new group, which would then contain, or appear to
   contain, all articles posted to either group (including, ideally, the
   pre-changeover articles from the old one). Nevertheless, if this
   feature is implemented, the articles themselves, as supplied to
   reading agents, MUST NOT be altered in any way (and, in particular,
   their Newsgroups headers MUST contain exactly those newsgroups
   present when they were injected). On the other hand, the Xref header
   (F-3.2.10) MAY contain entries for either group (or even both).

        NOTE: Some serving agents that use an "active" file permit an
        entry of the form "oldgroup xxx yyy =newgroup", which enables
        any articles arriving for oldgroup to be diverted to newgroup,
        thus providing a simple implementation of this feature. However,
        it is known that not all current serving agents will find
        implementation so easy (especially in the short term) which is
        why it is not mandated by this standard. Nevertheless, its
        eventual implementation in all serving agents is to be
        considered highly desirable.

C. H. Lindsey                                                  [Page 20]


                News Article Architecture and Protocols    February 2005

        On the other hand, it is recognized that this feature would
        likely not be implementable if the new group was already in
        existence with existing articles in it. This situation should
        not normally arise except when there is already some confusion
        as to which groups are, or are not, supposed to exist in that
        hierarchy. Note that the "mvgroup" control message is not really
        intended to be used for merging two existing groups.

6.2.3.1.  Example

      From: "example.all Administrator" <admin@noc.example>
      Newsgroups: example.oldgroup,example.newgroup,example.admin.announce
      Date: 30 Apr 2002 22:04 -0500 (EST)
      Subject: cmsg mvgroup example.oldgroup example.newgroup moderated
      Message-ID: <mvgroup-example.oldgroup-20020430@noc.example>
      Approved: admin@noc.example
      Control: mvgroup example.oldgroup example.newgroup moderated
      MIME-Version: 1.0
      Content-Type: multipart/mixed; boundary=nxt

      --nxt
      Content-Type: application/news-groupinfo

      For your newsgroups file:
      example.newgroup        The new replacement group (Moderated)

      --nxt

      The moderated group example.oldgroup is replaced by
      example.newgroup. Please update your configuration, and please,
      if possible, arrange to file articles arriving for
      example.oldgroup as if they were in example.newgroup.

      --nxt
      Content-Type: application/news-transmission

      Newsgroups: example.admin.info
      From: "example.all Administrator" <admin@noc.example>
      Subject: Charter for example.newgroup
      Message-ID: <mvgroup-example.newgroup-20020430@noc.example>
      Distribution: local
      Content-Type: text/plain; charset=us-ascii
      Content-Transfer-Encoding: 7bit

      This group (formerly known as example.oldgroup) is for the
      discussion of examples.

      --nxt--

6.2.4.  The 'checkgroups' Control Message

   The "checkgroups" control message contains a list of all the valid
   groups in a complete hierarchy.

C. H. Lindsey                                                  [Page 21]


                News Article Architecture and Protocols    February 2005

      control-command     =/ Checkgroup-command
      Checkgroup-command  = "checkgroups" Checkgroup-arguments
      Checkgroup-arguments= [ chkscope ] [ chksernr ]
      chkscope            = 1*( CFWS ["!"] newsgroup-name )
      chksernr            = CFWS "#" 1*DIGIT

   A "checkgroups" message applies to any (sub-)hierarchy with a prefix
   listed in the <chkscope> argument, provided that the rightmost
   matching <newsgroup-name> in the list is not immediately preceded by
   a "!".  If no <chkscope> argument is given, it applies to all
   hierarchies for which group statements appear in the body of the
   message.

        NOTE: Some existing software does not support the <chkscope>
        argument.  Thus a "checkgroups" message SHOULD also contain the
        groups of other subhierarchies the sender is not responsible
        for. "New" software MUST ignore groups which do not fall within
        the <chkscope> argument of the "checkgroups" message.

   The <chksernr> argument is a serial number, which can be any positive
   integer (e.g. just numbered or the date in YYYYMMDD).  It SHOULD
   increase by an arbitrary value with every change to the group list
   and MUST NOT ever decrease.

        NOTE: This was added to circumvent security problems in
        situations where the Date header cannot be authenticated.

   Example:

      Control: checkgroups de !de.alt #248

   which includes the whole of the 'de.*' hierarchy, with the exception
   of its 'de.alt.*' sub-hierarchy.

   The body of the message has the Media Type "application/news-
   checkgroups" (5.4).  It asserts that the <valid-group>s it lists are
   the only newsgroups in the specified hierarchies.

        NOTE: The "checkgroups" message is intended to synchronize the
        list of newsgroups stored by a serving agent, and their
        <newsgroup-description>s, with the lists stored by other serving
        agents throughout the network. However, it might be inadvisable
        for the serving agent actually to create or delete any
        newsgroups without first obtaining the approval of its
        administrators for such proposed actions.

        NOTE: The possibility of removing a complete hierarchy by means
        of an "invalidation" line beginning with a '!' in the
        checkgroups-body is no longer provided by this standard. The
        intent of the feature was widely misunderstood and it was
        misused more often than it was used correctly. The same effect,
        if required, can now be obtained by the use of an appropriate
        <chkscope> argument in conjunction with an empty <checkgroups-
        body>.

C. H. Lindsey                                                  [Page 22]


                News Article Architecture and Protocols    February 2005

6.3.  Cancel

   The "cancel" message requests that a target article be "canceled",
   i.e. be withdrawn from circulation or access.

      control-command     =/ Cancel-command
      Cancel-command      = "cancel" Cancel-arguments
      Cancel-arguments    = CFWS msg-id [CFWS]

   The argument identifies the article to be cancelled by its message
   identifier.  The body SHOULD contain an indication of why the
   cancellation was requested. The "cancel" message SHOULD be posted to
   the same newsgroup(s), with the same distribution(s), as the article
   it is attempting to cancel.

   A serving agent that elects to honour a "cancel" message SHOULD make
   the article unavailable for relaying or serving (perhaps by deleting
   it completely). If the target article is unavailable, and the
   acceptability of the "cancel" message cannot be established without
   it, activation of the "cancel" message SHOULD be delayed until the
   target article has been seen.  See also sections 7.3 and 7.4.

        NOTE: It is expected that the security extension envisaged in
        section 6.1 will make more detailed provisions for establishing
        whether honouring a particular "cancel" message is in order. In
        particular, it is likely that there will be provision for the
        digital signature of 3rd party cancels (i.e. those issued other
        than by the sender, the moderator, or the injector).

        NOTE: A cancel submitted by the poster for an article in a
        moderated group will be forwarded to the moderator of that
        group, and it is up to that moderator to act upon it (7.8).

        NOTE: The former requirement [RFC 1036] that the From and/or
        Sender headers of the "cancel" message should match those of the
        original article has been removed from this standard, since it
        only encouraged cancel issuers to conceal their true identity,
        and it was not usually checked or enforced by canceling
        software.  Therefore, both the From and/or Sender headers and
        any Approved header should now relate to the entity responsible
        for issuing the "cancel" message.

6.4.  Ihave, sendme

   The "ihave" and "sendme" control messages implement a crude batched
   predecessor of the NNTP [NNTP] protocol. They are largely obsolete on
   the Internet, but still see use in conjunction with some transport
   protocols such as UUCP, especially for backup feeds that normally are
   active only when a primary feed path has failed. There is no
   requirement for relaying agents that do not support such transport
   protocols to implement them.


C. H. Lindsey                                                  [Page 23]


                News Article Architecture and Protocols    February 2005

        NOTE: The ihave and sendme messages defined here have ABSOLUTELY
        NOTHING TO DO WITH NNTP, despite similarities of terminology.

   The two messages share the same syntax:

      control-command     =/ Ihave-command
      Ihave-command       = "ihave" Ihave-argument
      Ihave-argument      = relayer-name
      control-command     =/ Sendme-command
      Sendme-command      = "sendme" Sendme-argument
      Sendme-argument     = Ihave-argument
      relayer-name        = path-identity  ; see a-5.6.1
      ihave-body          = *( msg-id CRLF )
      sendme-body         = ihave-body

   The body of the message consists of a list of <msg-id>s, one per
   line. [RFC 1036] also permitted the list of <msg-id>s to appear in
   the Ihave- or Sendme-argument with the syntax
      Ihave-argument      = [FWS] *( msg-id FWS ) [relayer-name]
   but this form SHOULD NOT now be used, though relaying agents MAY
   recognize and process it for backward compatibility.

   The "ihave" message states that the named relaying agent has received
   articles with the specified message identifiers, which may be of
   interest to the relaying agents receiving the ihave message.  The
   "sendme" message requests that the agent receiving it send the
   articles having the specified message identifiers to the named
   relaying agent.

   Upon receipt of the sendme message, the receiving agent sends the
   article(s) requested, often (especially when the transport protocol
   is UUCP) in the form of one or more batches, each containing several
   articles. The usual form of a <batch> is defined by the following
   syntax (which is also used in the application/news transmission media
   type (5.1)).

      batch             = 1*( batch-header article )
      batch-header      = "#!" SP rnews SP article-size CRLF
      rnews             = %x72.6E.65.77.73 ; case sensitive "rnews"
      article-size      = 1*DIGIT

   Thus a <batch> is a sequence of articles, each prefixed by a header
   line that includes its size. The <article-size> is a decimal count of
   the octets in the article, counting each CRLF as one octet regardless
   of how it is actually represented.

        NOTE: Despite the similarity of this format to an executable
        UNIX script, it is EXTREMELY unwise to feed such a batch into a
        command interpreter in anticipation of it running a command
        named "rnews"; the security implications of so doing would be
        disastrous.


C. H. Lindsey                                                  [Page 24]


                News Article Architecture and Protocols    February 2005

   These control messages are normally sent essentially as point-to-
   point messages, by using <newsgroup-name>s in the Newsgroups header
   of the form "to."  followed by one (or possibly more) <component>s in
   the form of a <relayer-name> (see section a-5.5.1 which forbids "to"
   as the first <component> of a <newsgroup-name>). The control message
   SHOULD then be delivered ONLY to the relaying agent(s) identified by
   that <relayer-name>, and any relaying agent receiving such a message
   which includes its own <relayer-name> MUST NOT propagate it further.
   Each pair of relaying agent(s) sending and receiving these messages
   MUST be immediate neighbours, exchanging news directly with each
   other. Each relaying agent advertises its new arrivals to the other
   using "ihave" messages, and each uses "sendme" messages to request
   the articles it lacks.
[Assumes forbidden newsgroup-names will be in USEFOR.]

   To reduce overhead, ihave and sendme messages SHOULD be sent
   relatively infrequently and SHOULD contain reasonable numbers of
   message identifiers. If ihave and sendme are being used to implement
   a backup feed, it may be desirable to insert a delay between
   reception of an ihave and generation of a sendme, so that a slightly
   slow primary feed will not cause large numbers of articles to be
   requested unnecessarily via sendme.

6.5.  Obsolete control messages.

   The following control messages (as described in Appendix A.3) are
   declared obsolete by this standard:

        sendsys
        version
        whogets
        senduuname

7.  Duties of Various Agents

   The following section sets out the duties of various agents involved
   in the creation, relaying and serving of Netnews articles. Insofar as
   these duties are described as sequences of steps to be followed, it
   should be understood that it is the effect of these sequences that is
   important, and implementations may use any method that gives rise to
   that same effect.

   In this section, the word "trusted", as applied to the source of some
   article, means that an agent processing that article has verified, by
   some means, the identity of that source (which may be another agent
   or a poster).

        NOTE: In many implementations, a single agent may perform
        various combinations of the injecting, relaying and serving
        functions. Its duties are then the union of the various duties
        concerned.


C. H. Lindsey                                                  [Page 25]


                News Article Architecture and Protocols    February 2005

7.1.  General principles to be followed

   There are two important principles that news implementors (and
   administrators) need to keep in mind. The first is the well-known
   Internet Robustness Principle:

        Be liberal in what you accept, and conservative in what you
        send.

   However, in the case of news there is an even more important
   principle, derived from a much older code of practice, the
   Hippocratic Oath (we may thus call this the Hippocratic Principle):

        First, do no harm.

   It is VITAL to realize that decisions which might be merely
   suboptimal in a smaller context can become devastating mistakes when
   amplified by the actions of thousands of hosts within a few minutes.

   In the case of gateways, the primary corollary to this is:

        Cause no loops.

7.2.  Duties of an Injecting Agent

   An Injecting Agent is responsible for taking a (proto-)article from a
   posting (or other) agent and either forwarding it to a moderator or
   injecting it into the relaying system for access by readers.

   As such, an injecting agent is considered responsible for ensuring
   that any article it injects conforms with the rules of [USEFOR].

   In the normal course of events, an article that has already been
   injected into a Netnews network will never pass through another
   injecting agent.  So, if an injecting agent receives an otherwise
   valid article that has already been injected (as evidenced by the
   presence of an Injection-Date header, an Injection-Info header, or
   more than one '%' <path-delimiter> in a Path header) it MAY choose to
   reject it, but otherwise SHOULD cause it to be relayed, as it stands,
   by a relaying agent (7.3).

   In exceptional circumstances (e.g. as part of some complex gatewaying
   process, or where a relaying agent considers it essential for
   fulfilling its responsibility towards the rest of the network) an
   already injected article MAY be "reinjected" into the network.  This
   standard does not prescribe any such circumstance; rather this is a
   matter of policy to be determined by the administrators of each
   injecting agent, who have the responsibility to ensure that no harm
   arises. In all other circumstances, unintented reinjection is to be
   avoided (see 7.9).  Nevertheless, in order to preserve the integrity
   of the network in these special cases, this standard does set out the
   correct way to reinject (see special provisions in 7.2.2 Steps 3, 4,
   7 and 9).

C. H. Lindsey                                                  [Page 26]


                News Article Architecture and Protocols    February 2005

   It is usual for an injecting agent to be closely associated with a
   serving agent, thus giving it access to the list (7.4) showing the
   moderation status of the newsgroups it is likely to handle. In the
   event that it does not have such an associated serving agent, it MUST
   maintain that list itself.

7.2.1.  Proto-articles

   A proto-article SHOULD NOT be propagated in that form to other than
   injecting agents.

   A proto-article has the same format as a normal article except that
   some of the following mandatory headers MAY be omitted:  Message-Id,
   Date, Path (and even From if the particular injecting agent can
   derive that information from other sources).  However, if it is
   intended to offer the proto-article to two or more injecting agents
   in parallel, then it is only the Path header that MAY be omitted.
   The headers that can be omitted MUST NOT contain invalid values; they
   MUST either be correct or not present at all.
[Maybe omit that last sentence.]

        NOTE: An article that is offered for reinjection has, by
        definition, already been injected once, and is not therefore to
        be considered as a proto-article.  Hence a genuine proto-article
        will not contain any Injection-Date header nor any '%' <path-
        delimiter> in its Path header. It MAY contain <path-identity>s
        with other <path-delimiter>s in the pre-injection portion of its
        Path header (a-5.6.3).
[Assuming relevant mention of <path-delimiter> conventions in USEFOR.]

7.2.2.  Procedure to be followed by Injecting Agents

   An injecting agent receives (proto-)articles from posting and
   followup agents. It verifies them, adds headers where required, and
   then either forwards them to a moderator or injects them by passing
   them to serving or relaying agents.  It MUST NOT forward an already
   injected article to a moderator.

   An injecting agent processes articles as follows:

   1. It MUST remove any Injection-Info or Complaints-To header already
      present (though it might be useful to copy them to suitable X-
      headers). It SHOULD likewise remove any NNTP-Posting-Host, X-
      Trace, or other non-standard tracing header.

   2. It SHOULD verify that the article is from a trusted source, and
      MAY reject articles in which headers contain unverified email
      addresses, that is, addresses which are not known to be valid for
      the trusted source, though it would be perverse to reject
      intentionally unverifiable addresses such as those ending in
      ".invalid" (7.5).


C. H. Lindsey                                                  [Page 27]


                News Article Architecture and Protocols    February 2005

   3. It SHOULD reject any article whose Date header (F-3.1.2) is more
      than 24 hours into the future (and MAY use a margin less than that
      24 hours).  It MUST (except when reinjecting) reject any article
      with an Injection-Date header already present (and SHOULD do
      likewise with any NNTP-Posting-Date header). When reinjecting it
      MAY, in the absence of any Injection-Date header, reject any
      article whose Date header appears to be stale (e.g. more than 72
      hours into the past).

   4. It MUST reject any article that does not have the proper mandatory
      headers for a proto-article (except, when reinjecting, for the
      Injection-Date header), or which contains any header that does not
      have legal contents.  It SHOULD reject any article which contains
      any header deprecated for Netnews (F-3).  It SHOULD reject any
      article whose Newsgroups header does not contain at least one
      <newsgroup-name> for an existing group (as listed by its
      associated serving agent) and it MAY reject any <newsgroup-name>
      which, although syntactically correct, violates a policy
      restriction established, for some (sub-)hierarchy, by an agency
      with the appropriate authority (1.2).  Observe that crossposting
      to unknown newsgroups is not precluded provided at least one of
      those in the Newsgroups header is listed.
[The reference to F-3 presupposes some mention of deprecated headers in
there.]

        NOTE: This ability to reject <newsgroup-name>s in breach of
        established policy does not extend to relaying agents, though it
        might be reasonable for posting agents to do it.

   5. If the article is rejected (for reasons given above, or for other
      formatting errors or matters of site policy) the posting agent
      SHOULD be informed (such as via an NNTP 44x response code) that
      posting has failed and the article MUST NOT then be processed
      further.

   6. The Message-ID, Date and From headers (and their contents) MUST be
      added when not already present.  A User-Agent header MAY be added
      (or an already present User-Agent header MAY be augmented) so as
      to identify the software (e.g. "INN/1.7.2") used by the injecting
      agent.

        NOTE: The Message-ID, Date and From headers will already be
        present during reinjection.

   7. The injecting agent MUST NOT alter the body of the article in any
      way (including any change of Content-Transfer-Encoding). It MAY
      (except when reinjecting) add other headers not already provided
      by the poster, but SHOULD NOT alter, delete, or reorder any
      existing header, with the specific exception of "tracing" headers
      such as Injection-Info and Complaints-To, which are to be removed
      as already mentioned.  It MAY also, as an interim measure pending
      widespread adoption of the newly introduced (F-3.1.5) folding
      whitespace, reformat the Newsgroups and any Followup-To header by
      removing any such whitespace inserted by the posting agent.

C. H. Lindsey                                                  [Page 28]


                News Article Architecture and Protocols    February 2005

[Again, presupposes some mention in F-3.1.5.]

   8. If the Newsgroups header contains one or more moderated groups and
      the article does NOT contain an Approved header, the injecting
      agent MUST forward it to a moderator as specified in section 7.2.3
      below.

   9. Otherwise, a Path header with a <tail-entry> (F-3.1.6) MUST be
      correctly added if not already present. During reinjection, the
      existing Path header SHOULD be retained.

   10.It MUST then prepend the <path-identity> of the injecting agent
      and a '%' <path-delimiter> (which serves to separate the pre-
      injection and post-injection regions of the Path header) to the
      content of the Path header; this SHOULD then be followed by CRLF
      and WSP if it would otherwise result in a line longer than 79
      characters.  The prepended <path-identity> MUST be an FQDN
      mailable address (a-5.6.2).  See a-5.6.4 for the significance of
      the various <path-delimiter>s.

      NOTE: This could result in more that one '%' <path-delimiter> in
      the case of reinjection.
[There are unresolved issues with regard to the Path header; hence the
continued reference to [ARTICLE] above.]

   11.  An Injection-Info header (F-3.2.13) SHOULD be added, identifying
        the trusted source of the article, and a suitable Complaints-To
        header (a-6.20) MAY be added.  Each injecting agent SHOULD use a
        consistent form of the Injection-Info header for all articles
        emanating from the same or similar origins.
[There are unresolved issues concerning the Complaints-To header, so the
above step may yet be reworded in terms of a parameter of the
Injection-Info header.]

        NOTE: The step above is the only place in which an Injection-
        Info or Complaints-To header is to be created. It follows that
        these headers MUST NOT be created, replaced, changed or deleted
        by any other agent (except during reinjection, in which case
        they will always relate to the latest injection and are, to that
        extent, regarded as variant headers).

   12.The injecting agent MUST then add an Injection-Date header (F-
      3.1.7) if one is not already present, but it MUST NOT alter, or
      delete, an already present Injection-Date header (and likewise
      SHOULD NOT alter, or delete, an already present NNTP-Posting-Date
      header).  Finally, it forwards the article to one or more relaying
      or serving agents, and the injection process is to be considered
      complete.

        NOTE: The step above is the only place where an Injection-Date
        header is to be created It follows that it MUST NOT subsequently
        be replaced, changed or deleted by any other agent, even during
        reinjection.

C. H. Lindsey                                                  [Page 29]


                News Article Architecture and Protocols    February 2005

7.2.3.  Procedure for Forwarding to a Moderator

   An injecting agent forwards ar article to a moderator as follows:

   1. It MUST forward it to the moderator of the first (leftmost)
      moderated group listed in the Newsgroups header, customarily via
      email, (see 7.8 for how that moderator may forward it to further
      moderators). There are two possibilities for doing this:

      (a)  The complete article is encapsulated (headers and all) within
           the email, preferably using the Content-Type
           "application/news-transmission" (5.1) with any usage
           parameter set to "moderate". Moreover, there SHOULD NOT be
           more than one encapsulated article within the one email.
           This method has the advantage of removing any possible
           conflict between Netnews and Email headers, or of changes to
           those headers during transport through email.

      (b)  The article is sent as an email as it stands, with the
           addition of such extra headers (e.g. a To header) as are
           necessary for an email. The existing Message-ID header SHOULD
           be retained.

      Although both of these methods have seen use in the past, the
      preponderance of current usage on Usenet has been for method (b)
      and many moderators are ill-prepared to deal with method (a).
      Therefore, method (a) SHOULD NOT be used until such time as the
      majority of moderators are able to accept it.

   2. This standard does not prescribe how the email address of the
      moderator is to be determined, that being a matter of policy to be
      arranged by the agency responsible for the oversight of each
      hierarchy. Nevertheless, there do exist various agents worldwide
      which provide the service of forwarding to moderators, and the
      address to use with them is obtained as follows:

      (a)  Each '.' in the <newsgroup-name> is replaced with a '-'.

      (b)  The result of these operations is used as the <local-part> of
           the <mailbox> of the agent. For example, articles intended
           for "news.announce.important" would be emailed to "news-
           announce-important@forwardingagent.example".

7.3.  Duties of a Relaying Agent

   A Relaying Agent accepts injected articles from injecting and other
   relaying agents and passes them on to relaying or serving agents
   according to mutually agreed policy. Relaying agents SHOULD accept
   articles ONLY from trusted agents.

   An article SHOULD NOT be relayed unless the sending agent has been
   configured to supply and the receiving agent to receive at least one
   of the <newsgroup-name>s in its Newsgroups header and at least one of
   the <dist-name>s in its Distribution header, if any.  Exceptionally,

C. H. Lindsey                                                  [Page 30]


                News Article Architecture and Protocols    February 2005

   ALL relaying agents are deemed willing to supply or accept the
   <dist-name> "world", and NO relaying agent should supply or accept
   the <dist-name> "local".

   However, if the particular implementation does not relay non-existent
   newsgroups, even when included in the Newsgroups header and implied
   (e.g. by some "wild card" notation) in the configuration tables, then
   the agent MUST examine all group control messages (6.2) in order to
   ensure that relaying of those messages proceeds normally.

        NOTE: Although it would seem redundant to filter out unwanted
        distributions at both ends of a relaying link (and it is clearly
        more efficient to do so at the sending end), many sending sites
        have been reluctant, historically speaking, to apply such
        filters (except to ensure that distributions local to their own
        site or cooperating subnet did not escape); moreover they tended
        to configure their filters on an "all but those listed" basis,
        so that new and hitherto unheard of distributions would not be
        caught. Indeed many "hub" sites actually wanted to receive all
        possible distributions so that they could feed on to their
        clients in all possible geographical (or organizational)
        regions.

        Therefore, it is desirable to provide facilities for rejecting
        unwanted distributions at the receiving end. Indeed, it may be
        simpler to do so locally than to inform each sending site of
        what is required, especially in the case of specialized
        distributions (for example for control messages, such as cancels
        from certain issuers) which might need to be added at short
        notice.  A similar possibility for reading agents to filter
        distributions is also suggested in [USEAGE]) for the same
        reason.

   In order to avoid unnecessary relaying, an article SHOULD NOT be
   relayed if the <path-identity> of the receiving agent (or some known
   alias thereof) appears in its Path header.

   A relaying agent processes articles as follows:

   1. It MUST establish the trusted identity of the source of the
      article and compare it with the leftmost <path-identity> of the
      Path header's content. If it matches it MUST then prepend its own
      <path-identity> and a '/' <path-delimiter> to that content; this
      SHOULD then be followed by CRLF and WSP if it would otherwise
      result in a line longer than 79 characters.  If it does not match
      then it prepends instead two entries to that content; firstly the
      true established <path-identity> of the source followed by a '?'
      <path-delimiter>, and then, to the left of that, its own <path-
      identity> followed by a '/' <path-delimiter> as usual. This
      prepending of two entries SHOULD NOT be done if the provided and
      established identities match.  See a-5.6.4 for the significance of
      the various <path-delimiter>s.


C. H. Lindsey                                                  [Page 31]


                News Article Architecture and Protocols    February 2005

[All subject to unresolved issues concerning the Path header.]

        NOTE: In order to prevent overloading, relaying agents should
        not routinely query an external entity (such as a DNS-server) in
        order to verify an article (though a local cache of the required
        information might usefully be consulted).

   2. It MUST examine the Injection-Date header (or, if that is absent,
      the Date header) and reject the article as stale (F-3.1.7) if that
      predates the earliest articles of which it normally keeps record,
      or if it is more than 24 hours into the future (the margin MAY be
      less than that 24 hours).

   3. It SHOULD reject any article that does not include all the
      mandatory headers (section F-3.1).

   4. It MAY reject any article whose headers do not have legal
      contents.

   5. It SHOULD reject any article that has already been sent to it (a
      database of message identifiers of recent messages is usually kept
      and matched against).

        NOTE: Even though commonly derived from the domain name of the
        originating site (and domain names are case-insensitive), a
        message identifier MUST NOT be altered in any way during
        transport, or when copied (as into a References header), and
        thus a simple (case-sensitive) comparison of octets will always
        suffice to recognize that same message identifier wherever it
        subsequently reappears.

        NOTE: These requirements are to be contrasted with those of the
        un-normalized msg-ids defined by [RFC 2822], which may perfectly
        legitimately become normalized (or vice versa) during transport
        or copying in email systems.

        NOTE: Some old software may treat message identifiers that
        differ only in case within their <id-right> part as equivalent,
        and implementors of agents that generate message identifiers
        should be aware of this.

   6. It SHOULD reject any article that matches an already received
      cancel message (or an equivalent Supersedes header) issued by its
      poster or by some other trusted entity.

   7. It MAY reject any article without an Approved header posted to
      newsgroups known to be moderated (this practice is strongly
      recommended, but the information necessary to do so may not be
      available to all agents).

   8. It MAY delete any Xref header that is present.


C. H. Lindsey                                                  [Page 32]


                News Article Architecture and Protocols    February 2005

   9. Finally, it passes the articles on to neighbouring relaying and
      serving agents.

   If the article is rejected as being invalid, unwanted or unacceptable
   due to site policy, the agent that passed the article to the relaying
   agent SHOULD be informed (such as via an NNTP 43x response code) that
   relaying failed. In order to prevent a large number of error messages
   being sent to one location, relaying agents MUST NOT inform any other
   external entity that an article was not relayed UNLESS that external
   entity has explicitly requested that it be informed of such errors.

   Relaying agents MUST NOT alter, delete or rearrange any part of an
   article except for headers designated as variant (2.3).  In
   particular

     o they MUST NOT create or augment a User-Agent header in order to
       identify themselves;
     o they MUST NOT rewrite the Newsgroups header in any way, even if
       some supposedly non-existent newsgroup is included;
     o they MUST NOT refold any header (i.e. they must pass on the
       folding as received), even to remove FWS from a Newsgroups
       header;
     o they MUST NOT alter the Date header or the Injection-Date header;
     o they MUST NOT delete any unrecognized header whose header-name is
       syntactically correct (whether or not it is registered with IANA
       [RFC 3864]);
     o they MUST NOT change the Content-Transfer-Encoding of the body or
       any body part.

7.3.1.  Path-Header Example

      Path: foo.isp.example/
         foo-server/bar.isp.example?10.123.12.2/old.site.example!
         barbaz/baz.isp.example%dialup123.baz.isp.example!not-for-mail

        NOTE: That article was injected into the news stream by
        baz.isp.example (complaints may be addressed to
        abuse@baz.isp.example). The injector has taken care to record
        that it got it from dialup123.baz.isp.example. "not-for-mail" is
        a dummy <tail-entry>, though sometimes a real userid is put
        there.

        The article was relayed, perhaps by UUCP, to the machine known,
        at least to old.site.example, as "barbaz".

        Barbaz relayed it to old.site.example, which does not yet
        conform to this standard (hence the '!' <path-delimiter). So one
        cannot be sure that it really came from barbaz.

        Old.site.example relayed it to a site claiming to have the IP
        address [10.123.12.2], and claiming (by using the '/' <path-
        delimiter>) to have verified that it came from old.site.example.


C. H. Lindsey                                                  [Page 33]


                News Article Architecture and Protocols    February 2005

        [10.123.12.2] relayed it to "foo-server" which, not being
        convinced that it truly came from [10.123.12.2], did a reverse
        lookup on the actual source and concluded it was known as
        bar.isp.example (that is not to say that [10.123.12.2] was not a
        correct IP address for bar.isp.example, but simply that that
        connection could not be substantiated by foo-server).  Observe
        that foo-server has now added two entries to the Path.

        "foo-server" is a locally significant name within the complex
        site of many machines run by foo.isp.example, so the latter
        should have no problem recognizing foo-server and using a '/'
        <path-delimiter>.  Presumably foo.isp.example then delivered the
        article to its direct clients.

        It appears that foo.isp.example and old.site.example decided to
        fold the line, on the grounds that it seemed to be getting a
        little too long.

7.4.  Duties of a Serving Agent

   A Serving Agent takes an article from a relaying or injecting agent
   and files it in a "news database". It also provides an interface for
   reading agents to access the news database. This database is normally
   indexed by newsgroup with articles in each newsgroup identified by an
   <article-locator> (usually in the form of a decimal number - see F-
   3.2.10).

   A serving agent MUST maintain a list of the newsgroups it stores in
   its news database showing the moderation status of each one (see
   6.2.1), and SHOULD include in that list all groups likely to be
   crossposted to from those groups (e.g. all other groups in the same
   hierarchy(ies)).

        NOTE: Since control messages are often of interest, but should
        not be displayed as normal articles in regular newsgroups, it is
        common for serving agents to make them available in a pseudo-
        newsgroup named "control" or in a pseudo-newsgroup in a sub-
        hierarchy under "control." (e.g. "control.cancel").

   A serving agent MAY decline to accept an article if the Path header
   contains some <path-identity> whose articles the serving agent does
   not want, as a matter of local policy.

        NOTE: This last facility is sometimes used to detect and decline
        control messages (notably cancel messages) which have been
        deliberately seeded with a <path-identity> to be "aliased out"
        by sites not wishing to act upon them.
[INN at least does this. It might be argued that it is not necessary to
mention it here.]

   A serving agent processes articles as follows:


C. H. Lindsey                                                  [Page 34]


                News Article Architecture and Protocols    February 2005

   1. It MUST establish the trusted identity of the source of the
      article and modify the Path header as for a relaying agent (7.3).

   2. It MUST examine the Injection-Date header (or, if that is absent,
      the Date header) and reject the article as stale (F-3.1.7) if that
      predates the earliest articles of which it normally keeps record,
      or if it is more than 24 hours into the future (the margin MAY be
      less than that 24 hours).

   3. It MUST reject any article that does not include all the mandatory
      headers (section F-3.1), or which contains any header that does
      not have legal contents.

   4. It SHOULD reject any article that has already been sent to it (a
      database of message identifiers of recent articles is usually kept
      and matched against).

   5. It SHOULD reject any article that matches an already received
      cancel message (or an equivalent Supersedes header) issued by its
      poster or by some other trusted entity.

   6. It MUST reject any article without an Approved header posted to
      any newsgroup listed as moderated.

   7. It MUST remove any Xref header (F-3.2.10) from each article.  It
      then MAY (and usually will) generate a fresh Xref header.

   8. Finally, it stores the article in its news database.

   Serving agents MUST NOT create new newsgroups simply because an
   unrecognized <newsgroup-name> occurs in a Newsgroups header (see
   6.2.1 for the correct method of newsgroup creation).

   Serving agents MUST NOT alter, delete or rearrange any part of an
   article in any other way. The list of particular cases given for
   relaying agents (7.3) applies here also.

7.5.  Duties of a Posting Agent

   A Posting Agent is used to assist the poster in creating a valid
   proto-article and forwarding it to an injecting agent.

   Postings agents SHOULD ensure that proto-articles they create are
   valid according to [USEFOR] and other applicable policies. In
   particular, they MUST NOT create any Injection-Date, Injection-Info
   or Complaints-To header.

   Contrary to [RFC 2822], which implies that the mailbox(es) in the
   From header should be that of the poster(s), a poster who does not,
   for whatever reason, wish to use his own mailbox MAY use any mailbox
   ending in the top level domain ".invalid" [RFC 2606].


C. H. Lindsey                                                  [Page 35]


                News Article Architecture and Protocols    February 2005

   Posting agents meant for use by ordinary posters SHOULD reject any
   attempt to post an article which cancels or Supersedes another
   article of which the poster is not the author or sender.

7.6.  Duties of a Followup Agent

   A Followup Agent is a special case of a posting agent, and as such is
   bound by all the posting agent's requirements. Followup agents MUST
   create valid followups and are subject to special requirements
   involving the Newsgroups, Subject, Distribution and References
   headers.  Wherever in the following it is stated that, "by default",
   a header is to be "inherited" from one of those headers in the
   precursor, it means that its initial content is to be copied from the
   content of that precursor header.  However, posters MAY then override
   that default before posting if they so wish.

        NOTE: The Keywords header is not inheritable, though some older
        newsreaders treated it as such.

        NOTE: There is no provision in this standard for a followup to
        have more than one precursor (though it might be permitted in
        some future extension).

   1. The Newsgroups header (F-3.1.5) SHOULD by default be inherited
      from the precursor's Followup-To header if present, and otherwise
      from the precursor's Newsgroups header. However, if the content of
      that Followup-To header consists of "poster" (and the user does
      not override it), then the followup MUST NOT be posted but,
      rather, is to be emailed to the precursor's poster.

   2. The Subject header SHOULD by default be inherited from that of the
      precursor. The case sensitive string "Re: " MAY be prepended to
      its Subject-Content unless it already begins with that string.

   3. The Distribution header (F-3.2.6) SHOULD by default be inherited
      from the precursor's Distribution header, if any.

   4. If the precursor did not have a References header (F-3.2.1), the
      content of the followup's References header MUST be inherited from
      that of the Message-ID header of the precursor. A followup to an
      article which already had a References header MUST have a
      References header comprising the precursor's References header
      (subject to trimming as described below) followed by CFWS and the
      Message-ID header content of the precursor.

      If the resulting References header is excessively long, it MAY be
      trimmed, but the first and the last two message identifiers MUST
      NOT be removed.

   5. If the precursor contains a Mail-Copies-To header (a-6.8), the
      actions to be taken, in accordance with the content of the header,
      (and subject to manual override by the poster) are as follows:


C. H. Lindsey                                                  [Page 36]


                News Article Architecture and Protocols    February 2005

      "nobody" (or when the header is absent)
            The followup agent SHOULD NOT email a copy of a posted
            followup to the poster of the precursor.

      "poster"
            The followup agent SHOULD (if it has the necessary
            capability) email a copy of a posted followup, which MUST
            then be sent to the address(es) in the Reply-To header, and
            in the absence of that to the address(es) in the From
            header.

      a <copy-addr>
            The followup agent SHOULD likewise email a copy of a posted
            followup, which SHOULD then be sent to the <copy-addr>.
[There are still unresolved issues concerning the Mail-Copies-To header,
hence the continuing reference to [ARTICLE].]

      When emailing a copy, the followup agent SHOULD also include a
      "Posted-And-Mailed: yes" header (a-6.9).
[There are still unresolved issues concerning the Posted-And-Mailed
header, hence the continuing reference to [ARTICLE].]

      Followup agents SHOULD NOT attempt to send email to any address
      ending in ".invalid".

7.7.  Duties of a Reading Agent

   A reading agent downloads articles from a serving agent, as directed
   by the reader, and displays them to the reader (or processes them in
   some other manner). It SHOULD also have the capability to show the
   raw article exactly as received.

   It MAY present lists of articles available for display, and MAY
   structure those lists so as to show the relationships between the
   articles, as determined by the References, Subject, Date and other
   headers (see [USEAGE] for some usual methods of doing this).
[This whole section may yet get omitted]

7.8.  Duties of a Moderator

   A Moderator receives news articles, customarily by email, decides
   whether to approve them and, if so, either injects them into the news
   stream or forwards them to further moderators.

   Articles will be received by the moderator either encapsulated as an
   object of Content-Type application/news-transmission (or possibly
   encapsulated but without an explicit Content-Type header), or else
   directly as an email already containing all the headers appropriate
   for a Netnews article (see 7.2.2).  Moderators SHOULD be prepared to
   accept articles in either format.

   A moderator processes an article, as submitted to any newsgroup that
   he moderates, as follows:

C. H. Lindsey                                                  [Page 37]


                News Article Architecture and Protocols    February 2005

   1. He decides, on the basis of whatever moderation policy applies to
      his group, whether to approve or reject the article. He MAY do
      this manually, or else partially or wholly with the aid of
      appropriate software for whose operation he is then responsible.
      If the article is a cancel nessage (6.3) issued by the poster of
      an earlier article, then he is expected to cancel that earlier
      article (in which case there is no more to be done).  He MAY
      modify the article if that is in accordance with the applicable
      moderation policy (and in particular he MAY remove redundant
      headers and add Comments and other informational headers).  He
      also needs to be aware if any change he makes to the article will
      invalidate some authentication check provided by the poster or by
      an earlier moderator.

      If the article is rejected, then it normally fails for all the
      newsgroups for which it was intended. If it is approved, the
      moderator proceeds with the following steps.

   2. If the Newsgroups header contains further moderated newsgroups for
      which approval has not already been given, he adds an indication
      (identifying both himself and the name of the group) that he
      approves the article, and then forwards it to the moderator of the
      leftmost unapproved group (which, if this standard has been
      followed correctly, will generally be the next moderated group to
      the right of his own). There are two ways to do this:

      (a)  He emails it to the submission address of the next moderator
           (see section 7.2.2 for the proper method of doing this), or

      (b)  he rotates the <newsgroup-name>s in the Newsgroups header to
           the left so that the targeted group is the leftmost moderated
           group in that header, and injects it again (thus causing the
           injecting agent to forward it to the correct moderator).
           However, he MUST first ensure that the article contains no
           Approved header.

        NOTE: This standard does not prescribe how a moderator's
        approval is to be indicated (though a future standard may do
        so).  Possible methods include adding an Approved header (or a
        similar but differently named header if method (b) is being
        used) listing all the approvals made so far, or adding a
        separate header for each individual approval (the header X-Auth
        is sometimes used for this purpose).  The approval may also be
        confirmed with some form of digital signature (6.1).

   3. If the Newsgroups header contains no further unapproved moderated
      groups, he adds an Approved header (F-3.2.8) identifying himself
      and, insofar as is possible, all the other moderators who have
      approved the article. He thus assumes responsibility for having
      ensured that the article was approved by the moderators of all the
      moderated groups involved.


C. H. Lindsey                                                  [Page 38]


                News Article Architecture and Protocols    February 2005

   4. The Date header SHOULD be retained. Any Injection-Date header
      already present (though there should be none) MUST be removed.
      Exceptionally, if it is known that the injecting agent does not
      yet support the Injection-Date header and the Date header appears
      to be stale (F-3.1.7) for reasons understood by the moderator
      (e.g. delays in the moderation process) he MAY substitute the
      current date. The Message-ID header SHOULD also be retained unless
      it is obviously non-compliant with this standard.

        NOTE: A message identifier created by a conforming posting or
        injecting agent, or even by a mail user agent conforming to [RFC
        2822], may reasonably be supposed to be conformant (and will, in
        any case, be caught by the injecting agent if it is not).

   5. Any variant headers (2.3) MUST be removed, except that a Path
      header MAY be truncated to only its pre-injection region (a-
      5.6.3).  Any Injection-Info header (F-3.2.13) or Complaints-To
      header (a-6.20) SHOULD be removed (and if they are not, the
      injecting agent will do so, as required in 7.2.2).

   6. He then causes the article to be injected, having first observed
      all the duties of a posting agent.

        NOTE: This standard does not prescribe how the moderator or
        moderation policy for each newsgroup is established; rather it
        assumes that whatever agencies are responsible for the relevant
        network or hierarchy (1.1) will have made appropriate
        arrangements in that regard.

7.9.  Duties of a Gateway

   A Gateway transforms an article into the native message format of
   another medium, or translates the messages of another medium into
   news articles. Encapsulation of a news article into a message of MIME
   type application/news-transmission, or the subsequent undoing of that
   encapsulation, is not gatewaying, since it involves no transformation
   of the article.

   There are two basic types of gateway, the Outgoing Gateway that
   transforms a news article into a different type of message, and the
   Incoming Gateway that transforms a message from another medium into a
   news article and injects it into a news system. These are handled
   separately below.

   The primary dictat for a gateway is:

        Above all, prevent loops.

   Transformation of an article into another medium stands a very high
   chance of discarding or interfering with the protection inherent in
   the news system against duplicate articles. The most common problem
   caused by gateways is "spews", gateway loops that cause previously
   posted articles to be reinjected repeatedly into Usenet. To prevent
   this, a gateway MUST take precautions against loops, as detailed

C. H. Lindsey                                                  [Page 39]


                News Article Architecture and Protocols    February 2005

   below.

   If bidirectional gatewaying (both an incoming and an outgoing
   gateway) is being set up between Netnews and some other medium, the
   incoming and outgoing gateways SHOULD be coordinated to avoid
   unintended reinjection of gated articles. Circular gatewaying
   (gatewaying a message into another medium and then back into Netnews)
   SHOULD NOT be done; encapsulation of the article SHOULD be used
   instead where this is necessary.

   A second general principal of gatewaying is that the transformations
   applied to the message SHOULD be as minimal as possible while still
   accomplishing the gatewaying. Every change made by a gateway
   potentially breaks a property of one of the media or loses
   information, and therefore only those transformations made necessary
   by the differences between the media should be applied.

   It is worth noting that safe bidirectional gatewaying between a
   mailing list and a newsgroup is far easier if the newsgroup is
   moderated. Posts to the moderated group and submissions to the
   mailing list can then go through a single point that does the
   necessary gatewaying and then sends the message out to both the
   newsgroup and the mailing list at the same time, eliminating most of
   the possibility of loops. Bidirectional gatewaying between a mailing
   list and an unmoderated newsgroup, in contrast, is difficult to do
   correctly and is far more fragile.

   Newsgroups intended to be bidirectionally gated to a mailing list
   SHOULD therefore be moderated where possible, even if the moderator
   is a simple gateway and injecting agent that correctly handles
   crossposting to other moderated groups and otherwise passes all
   traffic.

7.9.1.  Duties of an Outgoing Gateway

   From the perspective of Netnews, an outgoing gateway is just a
   special type of reading agent. The exact nature of what the outgoing
   gateway will need to do to articles depends on the medium to which
   the articles are being gated. The operation of the outgoing gateway
   is subject to additional constraints due to the possibility of one or
   more corresponding incoming gateways back from that medium to
   Netnews, since this opens the possibility of loops.

   In general, the following practices are recommended for all outgoing
   gateways, regardless of whether there is known to be a related
   incoming gateway, both as a precautionary measure and as a guideline
   to quality of implementation.

   1. The message identifier of the news article should be preserved if
      at all possible, preferably as or within the corresponding unique
      identifier of the other medium, but if not at least as a comment
      in the message. This helps greatly with preventing loops.


C. H. Lindsey                                                  [Page 40]


                News Article Architecture and Protocols    February 2005

   2. The Date and Injection-Date of the news article should also be
      preserved if possible, for similar reasons.

   3. The message should be tagged in some way so as to prevent its
      reinjection into Netnews. This may be impossible to do without
      knowledge of potential incoming gateways, but it is better to try
      to provide some indication even if not successful; at the least, a
      human-readable indication that the article should not be gated
      back to Netnews can help locate a human problem.

   4. Netnews control messages should not be gated to another medium
      unless they would somehow be meaningful in that medium.

   5. Changes MAY be made to the Content-Transfer-Encoding of some or
      all parts of the body, and even to the charsets specified in
      <encoded-word>s or in Content-Type headers, but such changes
      SHOULD NOT be made unless absolutely necessary.

7.9.2.  Duties of an Incoming Gateway

   The incoming gateway has the serious responsibility of ensuring that
   all of the requirements of this standard are met by the articles that
   it forms. In addition to its special duties as a gateway, it bears
   all of the duties and responsibilities of an injecting agent as well,
   and additionally has the same responsibility of a relaying agent to
   reject articles that it has already gatewayed.

   An incoming gateway MUST NOT gate the same message twice. It may not
   be possible to ensure this in the face of mangling or modification of
   the message, but at the very least a gateway, when given a copy of a
   message it has already gated identical except for trace headers (like
   Received in Email or Path in Netnews) MUST NOT gate the message
   again.  An incoming gateway SHOULD take precautions against having
   this rule bypassed by modifications of the message that can be
   anticipated.

   News articles prepared by gateways MUST be legal news articles. In
   particular, they MUST include all of the mandatory headers, MUST
   fully conform to the restrictions on said headers, and SHOULD exclude
   any deprecated headers (F-3).  This often requires that a gateway
   function not only as a relaying agent, but also partly as a posting
   agent, aiding in the synthesis of a conforming article from non-
   conforming input.
[Presupposes suitable mention of deprecated headers in USEFOR.]

   Incoming gateways MUST NOT pass control messages (articles containing
   a Control or Supersedes header) without removing or renaming that
   header. Gateways MAY, however, generate their own cancel messages,
   under the general allowance for injecting agents to cancel their own
   messages ([USEAGE]).  If a gateway receives a message that it can
   determine is a valid equivalent of a cancel message in the medium it
   is gatewaying, it SHOULD discard that message without gatewaying it,
   generate a corresponding cancel message of its own, and inject that
   cancel message.

C. H. Lindsey                                                  [Page 41]


                News Article Architecture and Protocols    February 2005

   Incoming gateways MUST NOT inject control messages other than
   cancels.  Encapsulation SHOULD be used instead of gatewaying, when
   direct posting is not possible or desirable.

        NOTE: It is not unheard of for mail-to-news gateways to be used
        to post control messages, but encapsulation should be used for
        these cases instead. Gateways by their very nature are
        particularly prone to loops. Spews of normal articles are bad
        enough; spews of control messages with special significance to
        the news system, possibly resulting in high processing load or
        even email sent for every message received, are catastrophic. It
        is far preferable to construct a system specifically for posting
        control messages that can do appropriate consistency checks and
        authentication of the originator of the message.

   If there is a message identifier that fills a role similar to that of
   the Message-ID header in news, it SHOULD be used in the formation of
   the message identifier of the news article, perhaps with
   transformations required to meet the uniqueness requirement of
   Netnews and with the removal of any comments so as to comply with the
   syntax in section F-3.1.3. Such transformations SHOULD be designed so
   that two messages with the same identifier generate the same
   Message-ID header.

        NOTE: Message identifiers play a central role in the prevention
        of duplicates, and their correct use by gateways will do much to
        prevent loops. Netnews does, however, require that message
        identifiers be unique, and therefore message identifiers from
        other media may not be suitable for use without modification. A
        balance must be struck by the gateway between preserving
        information used to prevent loops and generating unique message
        identifiers.

   Exceptionally, if there are multiple incoming gateways for a
   particular set of messages, each to a different newsgroup(s), each
   one SHOULD generate a message identifier unique to that gateway. Each
   incoming gateway nonetheless MUST ensure that it does not gate the
   same message twice.

        NOTE: Consider the example of two gateways of a given mailing
        list into the world-wide Usenet newsgroups, both of which
        preserve the email message identifier. Each newsgroup may then
        receive a portion of the messages (different sites seeing
        different portions).  In these cases, where there is no one
        "official" gateway, some other method of generating message
        identifiers has to be used to avoid collisions. It would
        obviously be preferable for there to be only one gateway which
        crossposts, but this may not be possible to coordinate.

   If no date information is available, the gateway MAY supply a Date
   header with the gateway's current date.  If no injection-date
   information is available, the gateway MUST supply an Injection-Date
   header with whatever date information is available, and otherwise
   with the gateway's current date.  If only partial information is

C. H. Lindsey                                                  [Page 42]


                News Article Architecture and Protocols    February 2005

   available (e.g.  date but not time), this SHOULD be fleshed out to a
   full Date and/or Injection-Date header by adding default values
   rather than discarding this information. Only in very exceptional
   circumstances should Date information be discarded, as it plays an
   important role in preventing reinjection of old messages.

   An incoming gateway MUST add a Sender header to the news article it
   forms containing the <mailbox> of the administrator of the gateway.
   Problems with the gateway may be reported to this <mailbox>. The
   <display-name> portion of this <mailbox> SHOULD indicate that the
   entity responsible for injection of the message is a gateway. If the
   original message already had a Sender header, it SHOULD be renamed so
   that its contents can be preserved.

7.9.3.  Example

   To illustrate the type of precautions that should be taken against
   loops, here is an example of the measures taken by one particular
   combination of mail-to-news and news-to-mail gateways at Stanford
   University designed to handle bidirectional gatewaying between
   mailing lists and unmoderated groups.

   1. The news-to-mail gateway preserves the message identifier of the
      news article in the generated email message. The mail-to-news
      gateway likewise preserves the email message identifier provided
      that it is syntactically valid for Netnews.  This allows the news
      system's built-in suppression of duplicates to serve as the first
      line of defense against loops.

   2. The news-to-mail gateway adds an X-Gateway header to all messages
      it generates. The mail-to-news gateway discards any incoming
      messages containing this header. This is robust against mailing
      list managers that replace the message identifier, and against any
      number of email hops, provided that the other message headers are
      preserved.

   3. The mail-to-news gateway inserts the host name from which it
      received the email message in the pre-injection region of the Path
      (a-5.6.3).  The news-to-mail gateway refuses to gateway any
      message that contains the list server name in the pre-injection
      region of its Path header. This is robust against any amount of
      munging of the message headers by the mailing list, provided that
      the email only goes through one hop.

   4. The mail-to-news gateway is designed never to generate bounces to
      the envelope sender. Instead, articles that are rejected by the
      news server (for reasons not warranting silent discarding of the
      message) result in a bounce message sent to an errors address
      known not to forward to any mailing lists, so that they can be
      handled by the news administrators.

   These precautions have proven effective in practice at preventing
   loops for this particular application (bidirectional gatewaying
   between mailing lists and locally distributed newsgroups where both

C. H. Lindsey                                                  [Page 43]


                News Article Architecture and Protocols    February 2005

   gateways can be designed together). General gatewaying to world-wide
   newsgroups poses additional difficulties; one must be very wary of
   strange configurations, such as a newsgroup gated to a mailing list
   which is in turn gated to a different newsgroup.

8.  Security and Related Considerations

   There is no security. Don't fool yourself. Usenet is a prime example
   of an Internet Adhocratic-Anarchy; that is, an environment in which
   trust forms the basis of all agreements.  It works.

8.1.  Leakage

   Articles which are intended to have restricted distribution are
   dependent on the goodwill of every site receiving them.  The
   "Archive: no" header (F-3.2.11) is available as a signal to automated
   archivers not to file an article, but that cannot be guaranteed.

   The Distribution header makes provision for articles which should not
   be propagated beyond a cooperating subnet. The key security word here
   is "cooperating". When a machine is not configured properly, it may
   become uncooperative and tend to distribute all articles.

   The flooding algorithm is extremely good at finding any path by which
   articles can leave a subnet with supposedly restrictive boundaries,
   and substantial administrative effort is required to avoid this.
   Organizations wishing to control such leakage are strongly advised to
   designate a small number of official gateways to handle all news
   exchange with the outside world (however, making such gateways too
   restrictive can also encourage the setting up of unofficial paths
   which can be exceedingly hard to track down).

   The sendme control message (6.4), insofar as it is still used, can be
   used to request articles with a given message identifier, even one
   that is not supposed to be supplied to the requestor.

8.2.  Attacks

8.2.1.  Denial of Service

   The proper functioning of individual newsgroups can be disrupted by
   the massive posting of "noise" articles, by the repeated posting of
   identical or near identical articles, by posting followups unrelated
   to their precursors, or which quote their precursors in full with the
   addition of minimal extra material (especially if this process is
   iterated), and by crossposting to, or setting followups to, totally
   unrelated newsgroups.

   Many have argued that "spam", massively multiposted (and to a lesser
   extent massively crossposted) articles, usually for advertising
   purposes, also constitutes a DoS attack in its own regard.  This may
   be so.


C. H. Lindsey                                                  [Page 44]


                News Article Architecture and Protocols    February 2005

   Such articles intended to deny service, or other articles of an
   inflammatory nature, may also have their From or Reply-To addresses
   set to valid but incorrect email addresses, thus causing large
   volumes of email to descend on the true owners of those addresses.

   Similar effects could be caused by any email header which could cause
   every reading agent receiving it to take some externally visible
   action.  For example, the Disposition-Notification-To header defined
   in [RFC 2298] could cause huge numbers of acknowledgements to be
   emailed to an unsuspecting third party (for which reason [RFC 2298]
   declares that that header SHOULD NOT be used in Netnews).

   It is a violation of this standard for a poster to use as his address
   a <mailbox> which he is not entitled to use.  Even addresses with an
   invalid <local-part> but a valid <domain> can cause disruption to the
   administrators of such domains.  Posters who wish to remain anonymous
   or to prevent automated harvesting of their addresses, but who do not
   care to take the additional precautions of using more sophisticated
   anonymity measures, should avoid that violation by the use of
   addresses ending in the ".invalid" top-level-domain (see 7.5).

   A malicious poster may also prevent his article being seen at a
   particular site by preloading that site into the Path header (F-
   3.1.6) and may thus prevent the true owner of a forged From or
   Reply-To address from ever seeing it.

   A malicious complainer may submit a modified copy of an article (e.g.
   with an altered Injection-Info header) to the administrator of an
   injecting agent in an attempt to discredit the author of that article
   and even to have his posting privileges removed. Administrators
   should therefore obtain a genuine copy of the article from their own
   serving agent before taking such precipitate action.

   Administrative agencies with responsibility for establishing policies
   in particular hierarchies can and should set bounds upon the
   behaviour that is considered acceptable within those hierarchies (for
   example by promulgating charters for individual newsgroups, and other
   codes of conduct).

   Whilst this standard places an onus upon injecting agents to bear
   responsibility for the misdemeanours of their posters (which includes
   non-adherence to established policies of the relevant hierarchies as
   provided in section 7.2), and to provide assistance to the rest of
   the network by making proper use of the Injection-Info (F-3.2.13) and
   Complaints-To (a-6.20) headers, it makes no provision for
   enforcement, which may in consequence be patchy. Nevertheless,
   injecting sites which persistently fail to honour their
   responsibilities or to comply with generally accepted standards of
   behaviour are likely to find themselves blacklisted, with their
   articles refused propagation and even subject to cancellation, and
   other relaying sites would be well advised to withdraw peering
   arrangements from them.


C. H. Lindsey                                                  [Page 45]


                News Article Architecture and Protocols    February 2005

8.2.2.  Compromise of System Integrity

   The posting of unauthorized (as determined by the policies of the
   relevant hierarchy) control messages can cause unwanted newsgroups to
   be created, or wanted ones removed, from serving agents.
   Administrators of such agents SHOULD therefore take steps to verify
   the authenticity of such control messages, either by manual
   inspection (particularly of the Approved header) or by checking any
   digital signatures that may be provided (see 6.1).  In addition, they
   SHOULD periodically compare the newsgroups carried against any
   regularly issued checkgroups messages, or against lists maintained by
   trusted servers and accessed by out-of-band protocols such as FTP or
   HTTP.

   Malicious cancel messages (6.3) can cause valid articles to be
   removed from serving agents. Administrators of such agents SHOULD
   therefore take steps to verify that they originated from the
   (apparent) poster, the injector or the moderator of the article, or
   that in other cases they came from a place that is trusted to work
   within established policies and customs. Such steps SHOULD include
   the checking of any digital signatures, or other security devices,
   that may be provided (see 6.1).  Articles containing Supersedes
   headers (F-3.2.5) are effectively cancel messages, and SHOULD be
   subject to the same checks.  Currently, many sites choose to ignore
   all cancel messages on account of the difficulty of conducting such
   checks.

   Improperly configured serving agents can allow articles posted to
   moderated groups onto the net without first being approved by the
   moderator. Injecting agents SHOULD verify that moderated articles
   were received from one of the entities given in their Approved
   headers and/or check any digital signatures that may be provided (see
   6.1).

   The filename parameter of the Archive header (F-3.2.11) can be used
   to attempt to store archived articles in inappropriate locations.
   Archiving sites should be suspicious of absolute filename parameters,
   as opposed to those relative to some location of the archiver's
   choosing.
[This parameter may yet be removed from USEFOR.]

   There may be weaknesses in particular implementations that are
   subject to malicious exploitation. In particular, it has not been
   unknown for complete shell scripts to be included within Control
   headers. Implementors need to be aware of this.

   Reading agents should be chary of acting automatically upon MIME
   objects with an "application" Content-Type that could change the
   state of that agent, except in contexts where such applications are
   specifically expected (as in 5).  Even the Content-Type "text/html"
   could have unexpected side effects on account of embedded objects,
   especially embedded executable code or URIs that invoke non-news
   protocols such as HTTP [RFC 2616].  It is therefore generally
   recommended that reading agents do not enable the execution of such

C. H. Lindsey                                                  [Page 46]


                News Article Architecture and Protocols    February 2005

   code (since it is extremely unlikely to have a valid application
   within Netnews) and that they only honour URIs referring to other
   parts of the same article.

   Non-printable characters embedded in article bodies may have
   surprising effects on printers or terminals, notably by reconfiguring
   them in undesirable ways which may become apparent only after the
   reading agent has terminated.

8.3.  Liability

   There is a presumption that a poster who sends an article to Usenet
   intends it to be stored on a multitude of serving agents, and has
   therefore given permission for it to be copied to that extent.
   Nevertheless, Usenet is not exempt from the Copyright laws, and it
   should not be assumed that permission has been given for the article
   to be copied outside of Usenet, nor for its permanent archiving
   contrary to any Archive header that may be present.

   Posters also need to be aware that they are responsible if they
   breach Copyright, Libel, Harassment or other restrictions relating to
   material that they post, and that they may possibly find themselves
   liable for such breaches in jurisdictions far from their own. Serving
   agents may also be liable in some jurisdictions, especially if the
   breach has been explicitly drawn to their attention.

   Users who are concerned about such matters should seek advice from
   competent legal authorities.

9.  IANA Considerations

   IANA is requested to register the following media types, described
   elsewhere in this standard, for use with the Content-Type header, in
   the IETF tree in accordance with the procedures set out in [RFC
   2048].

      application/news-transmission  (5.1)
      application/news-groupinfo     (5.3)
      application/news-checkgroups   (5.4)

   IANA is also requested to change the status of the following media
   type to "OBSOLETE".

      message/news                   (5.2)

        NOTE: "Application/news-transmission" is an update, with
        clarification and additional optional parameters, to an existing
        registration. "Message/rfc822" should now be used in place of
        the obsoleted "message/news".



C. H. Lindsey                                                  [Page 47]


                News Article Architecture and Protocols    February 2005

10.  References

[To Do: Split this section into Normative and Informative references.
This will probably be delayed until the final draft, for technical
reasons.]

   [ANSI X3.4] "American National Standard for Information Systems -
        Coded Character Sets - 7-Bit American National Standard Code for
        Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986.

   [ARTICLE] Charles H. Lindsey, "News Article Format and Transmission",
        draft-ietf-usefor-article-format-*.txt.

   [NNTP] Clive D.W. Feather, "Network News Transport Protocol", draft-
        ietf-nntpext-base-*.txt.

   [PGPVERIFY] David Lawrence,
        <ftp://ftp.isc.org/pub/pgpcontrol/README.html>.

   [RFC 1036] M. Horton and R. Adams, "Standard for Interchange of
        USENET Messages", RFC 1036, December 1987.

   [RFC 2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail
        Extensions (MIME) Part One: Format of Internet Message Bodies",
        RFC 2045, November 1996.

   [RFC 2048] N. Freed, J. Klensin, and J. Postel, "Multipurpose
        Internet Mail Extensions (MIME) Part Four: Registration
        Procedures", RFC 2048, November 1996.

   [RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate
        Requirement Levels", RFC 2119, March 1997.

   [RFC 2298] R. Fajman, "An Extensible Message Format for Message
        Disposition Notifications", RFC 2298, March 1998.

   [RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names",
        RFC 2606, June 1999.

   [RFC 2616] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter,
        P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol --
        HTTP/1.1", RFC 2616, June 1999.

   [RFC 2822] P. Resnick, "Internet Message Format", RFC 2822, April
        2001.

   [RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration
        procedures for message header fields", RFC 3864.

   [RFC 850] Mark R. Horton, "Standard for interchange of Usenet
        messages", RFC 850, June 1983.

   [RFC 976] Mark R. Horton, "UUCP mail interchange format standard",

C. H. Lindsey                                                  [Page 48]


                News Article Architecture and Protocols    February 2005

        RFC 976, February 1986.

   [Son-of-1036] Henry Spencer, "News article format and transmission",
        <ftp://ftp.zoo.toronto.edu/pub/news.txt.Z>, June 1994.

   [USEAGE] draft-ietf-usefor-useage-*.txt.

   [USEFOR] C. H. Lindsey et al, "News Article Format", draft-ietf-
        usefor-usefor-format-*.txt.

   [USEPRO] This Standard.

11.  Acknowledgements

   TBD

12.  Contact Address

Editor

        Charles. H. Lindsey
        5 Clerewood Avenue
        Heald Green
        Cheadle
        Cheshire SK8 3JU
        United Kingdom
        Phone: +44 161 436 6131
        Email: chl@clw.cs.man.ac.uk

[

Working group chair

        Alexey Melnikov <alexey.melnikov-usefor@isode.com>
]

   Comments on this draft should preferably be sent to the mailing list
   of the Usenet Format Working Group at

        ietf-usefor@imc.org.

Appendix A.1 - A-News Article Format

   The obsolete "A News" article format consisted of exactly five lines
   of header information, followed by the body. For example:





C. H. Lindsey                                                  [Page 49]


                News Article Architecture and Protocols    February 2005

      Aeagle.642
      news.misc
      cbosgd!mhuxj!mhuxt!eagle!jerry
      Fri Nov 19 16:14:55 1982
      Usenet Etiquette - Please Read
      body
      body
      body

   The first line consisted of an "A" followed by an article ID
   (analogous to a message identifier and used for similar purposes).
   The second line was the list of newsgroups. The third line was the
   path. The fourth was the date, in the format above (all fields fixed
   width), resembling an Internet date but not quite the same. The fifth
   was the subject.

   This format is documented for archeological purposes only.  Articles
   MUST NOT be generated in this format.

Appendix A.2 - Early B-News Article Format

   The obsolete pseudo-Internet article format, used briefly during the
   transition between the A News format and the modern format, followed
   the general outline of a MAIL message but with some non-standard
   headers. For example:

      From: cbosgd!mhuxj!mhuxt!eagle!jerry (Jerry Schwarz)
      Newsgroups: news.misc
      Title: Usenet Etiquette -- Please Read
      Article-I.D.: eagle.642
      Posted: Fri Nov 19 16:14:55 1982
      Received: Fri Nov 19 16:59:30 1982
      Expires: Mon Jan 1 00:00:00 1990

      body
      body
      body

   The From header contained the information now found in the Path
   header, plus possibly the full name now typically found in the From
   header. The Title header contained what is now the content of the
   Subject header. The Posted header contained what is now the content
   of the Date header. The Article-I.D. header contained an article ID,
   analogous to a message identifier and used for similar purposes. The
   Newsgroups and Expires headers were approximately as now. The
   Received header contained the date when the latest relaying agent to
   process the article first saw it. All dates were in the above format,
   with all fields fixed width, resembling an Internet date but not
   quite the same.

   This format is documented for archeological purposes only.  Articles
   MUST NOT be generated in this format.


C. H. Lindsey                                                  [Page 50]


                News Article Architecture and Protocols    February 2005

Appendix A.3 - Obsolete Control Messages

   This present standard obsoletes certain control messages defined in
   [RFC 1036] (see 6.5), all of which had the effect of requesting a
   description of a relaying or serving agent's software, or its peering
   arrangements with neighbouring sites, to be emailed to the article's
   reply address. Whilst of some utility when Usenet was much smaller
   than it is now, they had become no more than a tool for the malicious
   sending of mailbombs. Moreover, many organizations now consider
   information about their internal connectivity to be confidential.

      version
      sendsys
      whogets
      senduuname

   "Version" requested details of the transport software in use at a
   site.  "Sendsys" requested the full list of newsgroups taken, and the
   peering arrangements. "Who gets" was similar, but restricted to a
   named newsgroup.  "Senduuname" resembled "sendsys" but restricted to
   the list of peers connected by UUCP.

   Historically, a checkgroups body consisting of one or two lines, the
   first of the form "-n newsgroup", caused check-groups to apply to
   only that single newsgroup.

   Historically, an article posted to a newsgroup whose name had exactly
   three components of which the third was "ctl" signified that article
   was to be taken as a control message.  The Subject header specified
   the actions, in the same way the Control header does now.

   These forms are documented for archeological purposes only; they MUST
   NO LONGER be used.

Appendix B - Notices

Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

C. H. Lindsey                                                  [Page 51]


                News Article Architecture and Protocols    February 2005

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at ietf-
   ipr@ietf.org.

Full Copyright Statement

   Copyright (C) The Internet Society (2005).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Appendix C - Change Log

[This Appendix is to be removed prior to final publication.]

   For version 01

   1    Numerous texts describing protocol features related to
        particular headers in parts of [ARTICLE] which were destined to
        become part of [USEFOR] have been moved to appropriate locations
        within section 7 of this document. Such revised texts will be
        found in sections
        7.2.2 Steps 4, 6, 7, 10, 11, 12;
        7.2.3 Step 1(b);
        7.3 introductory paragraphs, Steps 1, 4, 8, 9, and some final
        paragraphs;
        7.4 introductory and final paragraphs;
        7.9.1 Step 5.

   2    A section on "Duties of a Reading Agent" (7.8) has been added.

   3    Some demotions MUST -> SHOULD -> MAY, as noted in pseudo-
        comments, have been made or proposed in sections
        7.3
        7.3 Step 4.

   4    Part of the procedure for examining Path headers by relaying
        agents has been moved to serving agents, as explained in
        pseudo-comments in section 7.4.

   5    Some renumbering of sections and minor textual clarifications.

   For version 02


C. H. Lindsey                                                  [Page 52]


                News Article Architecture and Protocols    February 2005

   1    2nd para. of a-7 temporarily reinstated in section 6.

   2    Para. in section 6 relating to propagation of control messages
        and local policy removed to [USEAGE].]

   3    Requirement for some relaying agents to examine control messages
        for non-existent groups
        6
        7.3

   4    Text regarding "aliasing out" brought into line with actual
        practice.
        7.3

   5    More realistic wording regarding the expectations of reading
        agents
        7.7
        7.4

   6    "Precursor" is now defined for all cases in which a References
        header may be used (even though "followup" is not always defined
        under Alternative-1).
        2.1

   7    Provision is made for a poster to use a mailbox ending in
        ".invalid" in a From header (formerly in a-5.2).
        7.5

   8    "Inheritable" and "Variant" headers defined (formerly in a-
        4.2.5).
        2.3

   9    Additional wording regarding function of verb/arguments/body in
        control messages (formerly in a-6.13).
        6

   10   NOTE regarding not altering message indentifiers during
        transport or copying added (formerly in a-5.3).
        7.3

   11   All mention of MIME-style parameters and extension-parameters
        removed.
        3.1
        7.6

   For version 03

   1    The term "inheritable header" is no longer defined. Instead, the
        term "inherited' is used in place of "taken" when defining the
        actions of a followup agent.
        7.6


C. H. Lindsey                                                  [Page 53]


                News Article Architecture and Protocols    February 2005

   2    Consequent changes to "variant header", and also mention of
        Injection-Info as sometimes variant.
        2.3

   3    The term "reply address" is no longer defined.

   4    References now made to sections within USEFOR using "F-..."
        notation.

   5    Cross-references to sections within USEFOR added. Consistent use
        of <...> around all mentions of syntactic objects. All
        occurrences of "Foobar-header" changed to "Foobar header". Many
        other minor textual changes.

   6    <control-message> changed to <control-command>, to avoid
        confusion with "control message", which signifies the complete
        article containing the <control-command>.

   7    <ihave-arguments> has been changed to <ihave-argument> (since
        the earlier practice of multiple arguments is now deprecated).
        Likewise <sendme-argument>.

















C. H. Lindsey                                                  [Page 54]


Html markup produced by rfcmarkup 1.124, available from https://tools.ietf.org/tools/rfcmarkup/