INTERNET-DRAFT Charles H. LindseyUsenet Format Working Group R. Allbery, Ed. Internet-Draft Stanford University Obsoletes: 1036 (if approved) C. Lindsey Intended status: Standards Track University of Manchester November 2006 News ArticleExpires: July 7, 2007 January 3, 2007 Netnews Architecture and Protocols <draft-ietf-usefor-usepro-06.txt>draft-ietf-usefor-usepro-07 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. .QPInternet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.html.http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire in Mayon July 7, 2007. Copyright Notice Copyright (C) The Internet Society (2007). Abstract This Draft, together with its companion draft [USEFOR], are intended as standards track documents, together obsoleting RFC 1036, which itself dates from 1987. This Standarddocument defines the architecture of Netnews systems and specifies the requirements to be metcorrect manipulation and interpretation of Netnews articles by software which originates, distributes, storesstores, and displays them. It also specifies the requirements that must be met by any protocol used to transport and serve Netnews articles. Backward compatibility has been a major goal of this endeavour, but where this standardInternet Draft Comments Comments are solicited and earlier documents or practices conflict, this standardshould be followed. In most such cases, current practice is already compatible with these changes. A companion Best Current Practice document [USEAGE], addressing requirements which are present for Social rather than Normative reasons is in preparation. [This is the latest in the line of USEPRO drafts. However,addressed to the USEFORUsenet Format Working Group is currently considering the possibility of a complete rewrite of this document.] News Article Architecture and Protocols November 2006 [The use of the words "this standard" within this document when referring to itself does not imply that this draft yet has pretensions to be a standard, but rather indicates what will become the case if and when it is accepted as an RFC with the status of a proposed or draft standard.] [Remarks enclosed in square brackets and aligned with the left margin, such as this one, are not part of this draft, but are editorial notes to explain matters amongst ourselves, or to point out alternatives, or to assist the RFC Editor.]at firstname.lastname@example.org. Table of Contents 1. Introduction .................................................. 4. . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1. Basic Concepts ............................................ 4. . . . . . . . . . . . . . . . . . . . . . 5 1.2. Objectives ................................................ 4Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3. Historical Outline ........................................Requirements Notation . . . . . . . . . . . . . . . . . . 5 2. Definitions, Notations and Conventions ........................1.4. Syntax Notation . . . . . . . . . . . . . . . . . . . . . 5 18.104.22.168. Definitions ............................................... 5 2.2. Defining the Architecture ................................. 5 2.3. Identification of news servers ............................. . . . . . . . . . . . . . . . . . . . . . . 6 2.4. Variant Header Fields .....................................2. Transport . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Duties of Agents . . . . . . . . . . . . . . . . . . . . . . . 8 2.5. Textual Notations .........................................3.1. General Principles . . . . . . . . . . . . . . . . . . . . 8 3. Transport .....................................................3.2. The Path Header Field . . . . . . . . . . . . . . . . . . 9 4. Definition of new Media Types ................................. 10 4.1. Application/news-transmission .............................3.2.1. Constructing the Path Header Field . . . . . . . . . . 10 4.2. Message/news obsoleted .................................... 11 4.3. Application/news-groupinfo ................................3.2.2. Path Header Field Example . . . . . . . . . . . . . . 11 4.4. Application/news-checkgroups ..............................3.3. Duties of a Posting Agent . . . . . . . . . . . . . . . . 12 5. Control Messages ..............................................3.3.1. Proto-articles . . . . . . . . . . . . . . . . . . . . 12 3.3.2. Reinjection of Articles . . . . . . . . . . . . . . . 13 5.1. Digital Signature3.3.3. Followups . . . . . . . . . . . . . . . . . . . . . . 13 3.3.4. Construction of the References Header Fields ........................ 14 5.2. Group Control Messages ....................................Field . . . . . 14 5.2.1. The 'newgroup' Control Message ........................ 15 22.214.171.124. The Body of the 'newgroup' Control Message ........ 15 126.96.36.199. Initial Articles .................................. 15 188.8.131.52. Example ........................................... 16 5.2.2. The 'rmgroup' Control Message ......................... 17 184.108.40.206. Example ........................................... 17 5.2.3. The 'mvgroup' Control Message ......................... 17 220.127.116.11. Example ........................................... 19 5.2.4. The 'checkgroups' Control Message ..................... 20 5.3. Cancel .................................................... 21 5.4. Ihave, sendme ............................................. 22 5.5. Obsolete control messages. ............................... 23 6. Duties of Various Agents ...................................... 23 6.1. General principles to be followed ......................... 24 18.104.22.168. Duties of an Injecting Agent .............................. 24 6.2.1. Proto-articles ........................................ 25 6.2.2. Procedure to be followed by Injecting Agents .......... 25 6.2.3. Procedure for. . . . . . . . . . . . . . . 15 3.4.1. Forwarding Messages to a Moderator ............... 28 6.3.. . . . . . . . . . 17 3.5. Duties of a Relaying Agent ................................ 28 6.3.1. Path Header Field Example ............................. 32 6.4.. . . . . . . . . . . . . . . . 18 3.6. Duties of a Serving Agent ................................. 33 News Article Architecture and Protocols November 2006 6.5. Duties of a Posting Agent ................................. 35 6.6. Duties of a Followup Agent ................................ 35 6.6.1. Construction of the References header field ........... 36 6.7.. . . . . . . . . . . . . . . . 19 3.7. Duties of a Reading Agent ................................. 37 6.8.. . . . . . . . . . . . . . . . 21 3.8. Duties of a Moderator ..................................... 37 6.9.. . . . . . . . . . . . . . . . . . 21 3.9. Duties of a Gateway ....................................... 39 6.9.1.. . . . . . . . . . . . . . . . . . . 22 3.9.1. Duties of an Outgoing Gateway ......................... 40 6.9.2.. . . . . . . . . . . . 23 3.9.2. Duties of an Incoming Gateway ......................... 40 6.9.3.. . . . . . . . . . . . 24 3.9.3. Gateway Example ............................................... 42 7. Security. . . . . . . . . . . . . . . . . . . 26 4. Media Types . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1. application/news-transmission . . . . . . . . . . . . . . 28 4.2. application/news-groupinfo . . . . . . . . . . . . . . . . 29 4.3. application/news-checkgroups . . . . . . . . . . . . . . . 30 5. Control Messages . . . . . . . . . . . . . . . . . . . . . . . 32 5.1. Authentication and Authorization . . . . . . . . . . . . . 32 5.2. Group Control Messages . . . . . . . . . . . . . . . . . . 33 5.2.1. The newgroup Control Message . . . . . . . . . . . . . 33 5.2.2. The rmgroup Control Message . . . . . . . . . . . . . 35 5.2.3. The checkgroups Control Message . . . . . . . . . . . 35 5.3. The cancel Control Message . . . . . . . . . . . . . . . . 37 5.4. The Supersedes Header Field . . . . . . . . . . . . . . . 37 5.5. The ihave and Relatedsendme Control Messages . . . . . . . . . . 38 5.6. Obsolete Control Messages . . . . . . . . . . . . . . . . 39 6. Security Considerations ........................... 43 7.1. Leakage ................................................... 43 7.2. Attacks ................................................... 44 7.2.1. Denial of Service ..................................... 44 7.2.2.. . . . . . . . . . . . . . . . . . . 40 6.1. Compromise of System Integrity ........................ 45 7.3. Liability ................................................. 46 8.. . . . . . . . . . . . . . 40 6.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 41 6.3. Leakage . . . . . . . . . . . . . . . . . . . . . . . . . 42 7. IANA Considerations ........................................... 47 9.. . . . . . . . . . . . . . . . . . . . . 43 8. References .................................................... 47 9.1.. . . . . . . . . . . . . . . . . . . . . . . . . . 44 8.1. Normative References ...................................... 47 9.2.. . . . . . . . . . . . . . . . . . . 44 8.2. Informative References .................................... 48 10.. . . . . . . . . . . . . . . . . . 44 Appendix A. Changes to the Existing Protocols . . . . . . . . . . 46 Appendix B. Acknowledgements .............................................. . . . . . . . . . . . . . . . . . 47 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 48 11. Contact Address .............................................. 49 Appendix A - Obsolete Control Messages ............................ 49 Appendix B - Differences from the Protocols in RFC 1036 and its derivatives ....................................................... 50 Appendix C - Transitional Arrangements ............................ 50 Appendix D - Notices .............................................. 51 Appendix E - Change Log ........................................... 52 News Article ArchitectureIntellectual Property and Protocols November 2006Copyright Statements . . . . . . . . . . 49 1. Introduction 1.1. Basic Concepts "Netnews" is a set of protocols for generating, storing and retrieving news "articles", as introduced"articles" whose format is defined in [USEFOR], and for exchanging them amongst a readership that is potentially widely distributed. It is organized around "newsgroups", with the expectation that each reader will be able to see all articles posted to each newsgroup in F-1.1.which he participates. These protocols most commonly use a flooding algorithm which propagates copies throughout a network of participating servers. Typically, only one copy is stored per server, and each server makes it available on demand to readers able to access that server. "Usenet" is a particular worldwide publicly accessible network based uponon the Netnews protocols, with the newsgroups being organized into recognized "hierarchies". Anybody can join (itprotocols. It is simply necessary to negotiate an exchange of articles withonly one or more other participating hosts). An important characteristic of Usenet is the lack of any requirement for a central administration or for the establishmentsuch possible network; there are deployments of any controlling host to managethe network. Nevertheless, administrative agencies do exists with varying degrees of authority to establish "policies" applicableNetnews protocols other than Usenet (such as ones internal to particular partsorganizations). This document discusses the more general Netnews architecture and protocols. 1.2. Scope This document defines the architecture of Usenet. A "policy" is a rule intended to facilitateNetnews systems and specifies the smooth operationcorrect manipulation and interpretation of a networkNetnews articles by establishing parameterssoftware which restrict behaviour that, whilst technically unexceptionable, would nevertheless contravene some accepted standard of "Good Netkeeping". Since the ultimate beneficiaries of a networkoriginates, distributes, stores, and displays them. It addresses protocol issues that are its human readers, who will be less tolerant of poorly designed interfaces than mere computers, articles in breachindependent of established policy can cause considerable annoyance to their recipients. [Could omit that last sentence,transport protocols such as NNTP [RFC3977] and perhaps evenspecifies the whole paragraph?] 1.2. Objectives The purpose of this present standard is to definerequirements Netnews places on those underlying transport protocols. It also specifies the overall architecturehandling of control messages. The format and the protocols to be used forsyntax of Netnews articles are specified in general, and for Usenet[USEFOR], which should be read in particular,conjunction with this document. 1.3. Requirements Notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and to set standards"OPTIONAL" in this document are to be followed by software that implements those protocols. The companion standard [USEFOR] sets outinterpreted as described in [RFC2119]. 1.4. Syntax Notation Syntax defined in this document uses the canonical format of news articles exchanged betweenAugmented Backus-Naur Form (ABNF) notation (including the various agents comprising that architecture. InCore Rules) defined in [RFC4234] and constructs defined in [USEFOR] and [RFC2822]. 1.5. Definitions Any term used in this standard, references to individual sectionsdocument that is defined in the companionSection 1.5 of [USEFOR] are prefixedis used with "F-".the definition given there. In addition, the following terms will be used: A "hierarchy" is the set of hosts withinall newsgroups whose names share a network which, by mutual arrangement, operates some variant (whether more or less restrictive)first <component> (as defined in Section 3.1.4 of the Netnews protocols is a "cooperating subnet". It[USEFOR]). A "sub- hierarchy" is NOTthe purpose of this standard to settle matters of policy, nor aspectsset of software behaviour which do not impinge uponall newsgroups whose names share several initial components. A "news server" is further distinguished into the generation, transmission, storage and receptionroles of articles, nor how"injecting agent", "relaying agent", and "serving agent". An "injecting agent" accepts a proto-article with the authoritygoal of various agenciesdistributing it to create such policiesrelaying and serving agents and hence to exercise controlreaders. A "relaying agent" accepts articles from other relaying agents or oversight of the various parts of Usenet is established. For these purposes,injecting agents and distributes them to other relaying agents or serving agents. A "serving agent" receives an article from a separate Best Current Practice document [USEAGE] is being provided. News Article Architecturerelaying agent or injecting agent and Protocols November 2006 Nevertheless,makes it available to readers. A "user agent" is assumed that such agencies with the necessary authority will exist, and tools are provided within the protocols for their use. 1.3. Historical Outline Network news originated asfurther distinguished into the mediumroles of communication for Usenet, circa 1980. Since then, Usenet has grown explosively, and many Internet"posting agent" and non-Internet sites participate in it. In addition, the news technology"reading agent". A "posting agent" is nowsoftware which assists in widespread use for other purposes, onthe Internet and elsewhere. For an accountpreparation of the earlier formats used in Netnews prior to [RFC 1036], see Henry Spencer's 1994 draft, popularly referreda proto-article and then passes it to as "Son of 1036" [Son-of-1036], which has recently been republished asan Informational RFC. [Thatinjecting agent. A "reading agent" is a tentative statement,software which may need revision.] Although never adopted asretrieves articles from a formal standard, [Son-of-1036] hadserving agent for presentation to a considerable effect onreader. "Injecting" an article is the developmentprocessing of Netnews and hence on these present standards, and ita proto-article by an injecting agent. Normally this action is hoped that we have followed its spiritdone once and intentions. .nr H1 1 2. Definitions, Notationsonly once for a given article. "Reinjecting" an article is passing an already- injected article to an injection agent. A "gateway" is software which receives news articles and Conventions 2.1. Definitions All the technical terms defined in F-1.5 areconverts them to be consideredmessages of some other kind (such as defined also, with the same meaning, in this standard. In addition,[RFC2822] mail messages), receives messages of some further terms are defined here,other kind and in the following section. A "hierarchy" is the set of all newsgroups whose names share a first <component> (as defined in F-3.1.4).converts them to news articles, or conveys articles between two separate Netnews networks. 2. Transport The term "sub-hierarchy" is alsoexact means used where several initial components are shared. The "semantic content" (often abbreviatedto just "content" when the contexttransmit articles from one agent to another is clear) of a header fieldnot specified. NNTP [RFC3977] is its semantic interpretation; i.e. what remains after unfolding it and removing its field name with its colon and any leading and trailing whitespace and,the most common transport mechanism for Netnews networks. Other methods in use include the caseUUCP protocol [RFC0976] (extensively used in the early days of structured header fields only, ignoring commentsUsenet) and other semantically invisible itemsphysically delivered magnetic and replacing white space by a single SP. See 6.6.1 for the use ofoptical media. Any mechanism may be used in conjunction with this term. 2.2. Definingprotocol provided that it can meet the Architecture Arequirements specified here. Transports for Netnews system is a distributed database composed of agentsarticles MUST treat news articles as uninterpreted sequences of various types which, acting together according tooctets, excluding the protocols definedvalues 0 (which may not occur in section 6 of this standard, causes articles to be propagated throughout the systemNetnews articles) and to be made available to its readers. The protocols ensure13 and 10 (which MUST only appear in Netnews articles as a pair in that all copies oforder and which together denote a given article, wherever stored,line separator). These octets are identical apart from those header fields defined as variant (2.4). For explainingthe working of the protocols, it is convenient to define particular sub-categories of agent as follows: News Article ArchitectureUS-ASCII [ASCII] characters NUL, CR, and Protocols November 2006 A "posting agent" isLF respectively. NOTE: This corresponds to the software that assists postersrange of octets permitted in MIME 8bit data [RFC2045]. Transports for Netnews are not required to prepare proto-articlessupport transmission of MIME binary data. In particular, transports MUST convey all header fields (including header fields within message/rfc822 objects in compliance with [USEFOR]. The proto-article is then passed onarticle bodies) unmodified even if they contain octets in the range 128 to an "injecting agent"255. Furthermore, transports for final checkingrelaying and injection into the news stream. If theserving agents MUST, and transports for other agents SHOULD, convey lines even if they exceed 998 characters in length, especially in article bodies. (This requirement is not compliant, or is rejected by the injecting agent, thenstricter than MIME 8bit data.) These requirements include the transport paths between posting agent informs the poster with an explanationagents, injecting agents, serving agents, and reading agents. 3. Duties of Agents The following section specifies the error. A "reading agent" is software which presents articles to a reader. A "followup agent" is a combinationduties of reading agent and posting agent that aidsthe agents involved in the preparationcreation, relaying, and postingserving of a followup. An "injecting agent" takes the finished article from the posting agent (often viaNetnews articles. This protocol is described by following the NNTP "POST" command), performs some final checks and passes it on tolife of a "relaying agent" for general distribution. A "relaying agent"typical Usenet article: it is software which receives allegedly compliant articles fromprepared by a posting agent, given to an injecting agents and/or otheragent, transferred through one or more relaying agents, and possibly passes copies on to other relaying agents and "storage agents". A "storage agent" receives an article fromaccepted by a relaying agentserving agent, and files it infinally retrieved by a "news database". It also provides an interface forreading agentsagent. Articles submitted to access the news database. A "news database"moderated groups go through an additional process, which is described separately. Finally, the set of articlesadditional duties and related structural information stored byrequirements of a storagegateway are discussed. At each step, each agent has a set of checks and made available for access by reading agents. A "gateway"transformations of the article that it is software which receives news articles and converts themrequired to messagesperform. These are described as sequences of some other kind (e.g. mailsteps to a mailing list), or vice versa; in essencebe followed, but it is a translating relaying agentshould be understood that straddles boundaries between different methods of message exchange. The most common typeit is the effect of gateway connects newsgroup(s)these sequences that is important, and implementations may use any method that gives rise to mailing list(s), either unidirectionally or bidirectionally, but there are also gateways between news networks usingthe [USEFOR]same effect. Many news format and those using other formats. Posting, reading and followup agents (which are usually just different services provided byservers combine the same piecefunctions of software) together comprise the "user agents" defined in F-1.5. Likewise, injecting,injecting agent, relaying agent, and storage agents (which are often just different services provided byserving agent in a single software package. For the same piecepurposes of software) together comprisethis specification, such combined agents should conceptually be treated as an injecting agent which sends articles to a serving agent and optionally a relaying agent. The requirements of all three agents MUST be still met when the "news servers". 2.3. Identificationnews server is performing the functions of those agents. Control messages may have additional effects than those described below on news servers [Therewhich accept them. Those effects are described in Section 5. 3.1. General Principles There are two alternative texts which have been proposed:] In order to record the passage of articles through the network,important principles that news serversimplementors and administrators need to identify themselves by means of a <path-identity> (F-3.1.5), which can appearkeep in Path, Injection-Info and Xref header fields. Whatever <path-identity>mind. The first is used inthe Path header field News Article Architecturewell-known Internet Robustness Principle: Be liberal in what you accept, and Protocols November 2006conservative in what you send. As applied to Netnews, this primarily means that unwanted or non- compliant articles SHOULD be used alsorejected as early as possible, but once they are in any Injection-Info header field (and it would be normal to use it in any Xref header field also). [Maybe that last sentence moves elsewhere.] NOTE: Such <path-identity>sgeneral circulation, relaying and serving agents may also be suitable for sending emailwish to news server administrators (see [USEAGE]). [1st alternative] <Path-identity>s can take the following forms (in decreasing order of preference): 1. A fully qualified domain name (FQDN) thataccept them where possible rather than lose information. Posting agents and injecting agents SHOULD therefore be resolvable in the DNS (whether via an A, AAAA or MX record or an equivalent CNAME), thus guaranteeing a unique identity. Ideally, it will also provide a means to contact the administrators by email (according to [RFC 2142], the forms "usenet@server" and "news@server" are common addresses for a news server administrator). 2. Some other (arbitrary) namemaximally strict in the formtheir application of a <path-nodot>,both this protocol and believed to be unique[USEFOR], and registered at least with all other news servers sending articles directly to the given one. This optionreading agents SHOULD NOTbe used unless the earlier option is unavailable (e.g. because the serverrobust in question is not connected tothe Internet), or unless it ispresence of violations of longstanding usage and cessation would be unduly disruptive, or unlessthe earlier option is provided as well. [2nd alternative] <Path-identity>s can takeNetnews article format where possible. In the following forms (in decreasing ordercase of preference): 1. A fully qualified domain name (FQDN) that can be resolved to an email server viaNetnews, there is an MX, A or AAAA record according to the procedureseven more important principle, derived from a much older code of [RFC 2821];practice, the Hippocratic Oath (we may thus call this guarantees thatthe nameHippocratic Principle): First, do no harm. It is unique, and makes it easyvital to contact the administrators if needed. 2. A fully qualified domain name (FQDN)realize that is guaranteed todecisions which might be unique by the administrators of the domain; for instance, the uniqueness of "server.example.org" could be guaranteedmerely suboptimal in a smaller context can become devastating mistakes when amplified by the administratoractions of "example.org" even if nothing is stored in the DNS for that name. 3. Some other (arbitrary) name in the formthousands of hosts within a <path-nodot>, and believedfew minutes. No Netnews agent is ever required to be uniqueaccept any article. It is common for injecting, relaying, and registered at least with all other news-servers sendingserving agents to reject well- formed articles directlyfor reasons of local policy (such as not wishing to the given one.carry a particular newsgroup or attempting to filter out unwanted articles). This option SHOULD NOTdocument specifies how articles are to be used unless the earlier optionstreated if they are unavailable, or unless the name is of longstanding usageaccepted and cessation wouldspecifies some cases where they must be unduly disruptive, or unless onerejected, but an agent MAY always reject any article for other reasons than those stated here. A primary goal of the earlier optionsNetnews protocol is provided as well. News Article Architecture and Protocols November 2006 Accordingto [RFC 2142]], the forms "usenet@server" and "news@server" are common addresses forensure that all readers receiving a news server administrator. [endparticular article (as uniquely identified by the content of alternatives] NOTE: Although domain names are case insensitiveits Message-ID header field) see the identical article, apart from allowable divergence in trace headers and it is intended that <path-nodot>s should also be so, it is customary to render them alllocal metadata. Accordingly, agents (other than moderators) MUST NOT modify articles in lowercase, since many implementations compare them case sensitively for reasons of efficiency. NOTE: Aways other than described here. Unacceptable articles MUST be rejected rather than corrected. 3.2. The Path Header Field All news server administrator who chooses a <path-nodot> which turns out not to be unique (disregarding case) will have to bear the consequences. NOTE: An IP address is not permitted as a <path-identity>, although it may still appear in a <diag-identity>. Since the syntax permits a colon (":" which, prior to this standard, wascomponents (injecting agents, relaying agents, and serving agents) MUST identify themselves, when processing an alternative to the "!" delimiter) within any <diag-identity> which takes the formarticle, by prepending their <path-identity> (defined in Section 3.1.5 of an <IPv6address>, it would be unwise[USEFOR]) to choose as a <path-nodot> anything composed solely from four or less hexadecimal digits. 2.4. Variant Header Fields Header fields withthe variant property may differ between (or even be completely absent from) copies ofPath header field. Injecting agents MUST also use the same article as stored or relayed throughout a Netnews system. The manner ofidentity in Injection-Info header fields they add, and serving and relaying agents SHOULD use the difference (or absence) MUST be as specifiedsame identity in this (or some future) standard. Typically, theseany Xref header fields are modified as articles are propagated, orthey reflectadd. The <path-identity> used by an agent may be chosen via one of the statusfollowing methods (in decreasing order of preference): 1. The fully-qualified domain name (FQDN) of the articlesystem on a particular storage agent, or cooperating group of such agents.which the agent is running. 2. A variant header field MAY be placed anywherefully-qualified domain name (FQDN) within a domain affiliated with the header fields (though placing it first is recommended). The following header fields are classified as "variant": o Path (F-3.1.5) - augmented at each relaying agent that an article passes through. o Xref (F-3.2.14) - used to keep trackadministrators of the <article-locator>s of crossposted articles so that reading agents serviced by a particular storageagent can mark such articles as read. o Injection-Info (F-3.2.8) is also considered variant in some special situations involving reinjection (6.2and 6.2.2). 2.5. Textual Notations This standard contains explanatory NOTEs using the following format. These mayguaranteed to be skippedunique by persons interested solely inthe contentadministrators of that domain. For example, the specification. The purpose of the notes is to explain why choices were made, to place them in context, or to suggest possible implementation techniques. NOTE: While such explanatory notes may seem superfluous in principle, they often help the less-than-omniscient reader grasp the purposeuniqueness of server.example.org could be guaranteed by the specification and the constraints involved. Given the limitationsadministrator of natural languageexample.org even if there is no DNS record for descriptive News Article Architecture and Protocols November 2006 purposes, this improves the probability that implementors and users will understand the true intent of the specificationserver.example.org itself. 3. Some other (arbitrary) name in cases wherethe wording is not entirely clear. "US-ASCII" is short for "the ANSI X3.4 character set" [ANSI X3.4]. US-ASCII is a 7 bit character set. Please note that this standard requires that all agentsform <path-nodot> believed to be 8 bit clean; that is, they must acceptunique and transmit data without changingregistered at least with all the other news servers to which that relaying agent or omittinginjecting agent sends articles. This option SHOULD NOT be used unless the 8th bit. Certain words, when capitalized,earlier options are used to defineunavailable or unless the significancename is of individual requirements.longstanding usage. Some existing implementations treat <path-identity> as case- sensitive, some case-insensitive. The key words "MUST", "REQUIRED", "SHOULD", "RECOMMENDED", "MAY" and "OPTIONAL",<path-identity> therefore SHOULD be all lowercase and any of those words associated withimplementations SHOULD compare identities case-insensitively. 3.2.1. Constructing the word "NOT", are to be interpreted as described in [RFC 2119]. NOTE: A requirement imposed onPath Header Field If a relaying or storageserving agent regarding some particularreceives an article should be understood as applying only iffrom an injecting or serving agent that articleis actually accepted for processing (since any agent may always reject any article entirely, for reasons of site policy). Wherever the context permits, usepart of the masculine includessame news server, it MAY leave the feminine and usePath header field of the singular includesarticle unchanged. Otherwise, every injecting, relaying, or serving agent that accepts an article MUST update the plural, and vice versa. Throughout this standard we will give examples of various definitions,Path header fields and other specifications. It needs to be rememberedfield as follows. Note that these samples are for the aid ofthe reader only and do NOT define any specification themselves. In orderPath header field content is constructed from right to left by prepending elements. 1. The agent MUST prepend "!" to prevent possible conflict with "Real World" entities and peoplethe top level domain ".example" is used in all sample domainsPath header field content. 2. An injecting agent SHOULD prepend the <path-diagnostic> "!.POSTED", optionally followed by "." and addresses. The hierarchy "example.*" is also used as a sample hierarchy. Information onthe ".example" top level domain is in [RFC 2606]. 3. Transport As in this standard's predecessors,FQDN or IP address of the exact means usedsource, to transmit articles from one hostthe Path header field content. 3. A relaying or serving agent SHOULD prepend a <path-diagnostic> to another is not specified. NNTP [RFC 3977] isthe most common transmission method onPath header field content, where the Internet, but much transmission takes place entirely independent<path-diagnostic> is chosen as follows: * If the expected <path-identity> of the Internet. Other methods in use includesource of the UUCP protocol [RFC 976] extensively used inarticle matches the early days of Usenet, FTP, tunneling through email using application news/transmission, downloading via satellite, tape archives, and physically delivered magnetic and optical media. Transmission paths for news articles MUST treat news articles as uninterpreted sequencesleftmost <path-identity> of octets, excludingthe values 0 (US-ASCII NUL) and 13 and 10 (US-ASCII CR and LF, which MUST ONLY appear inPath header field's content, use "!" (<diag-match>). * If the combination CRLF which denotes a line separator). NOTE: this corresponds toexpected <path-identity> of the rangesource of octets permitted for MIME "8bit data" [RFC 2045]. Thus raw binary data cannot be transmitted in anthe article body exceptdoes not match, use "!.MISMATCH." followed by the useexpected <path-identity> of a Content- Transfer-Encoding such as base64. News Article Architecture and Protocols November 2006 In particular, transmission paths MUST convey all header fields (including body part header fields and header fields within message/rfc822 objects) intact, even if they contain octets inthe range 128 to 255. Furthermore, relaying agents MUST, and other agents SHOULD, convey lines even if they exceed 998 characters in length, especially in article bodies. These requirements includesource or its IP address. * If the transmissiom paths between posting agents, injecting agents,relaying agents, storage agents and reading agents, but NOTor serving agent is not willing or able to check the paths traversed<path-identity>, use "!.SEEN." followed by Netnews articles that have been gatewayed into Email (6.9.1). [At some point it will be necessary forthe IMAP standards to catch up with these requirements.] 4. DefinitionFQDN, IP address, or expected <path-identity> of new Media Types This standard defines (or redefines) several new Media Types, which requirethe source. 4. The agent MUST then prepend its own <path-identity> to be registered with IANA as provided for in [RFC 4288]. 4.1. Application/news-transmissionthe Path header field content. 5. The Media Type "application/news-transmission" is intendedagent MAY then prepend additional <path-identity>s for itself to the encapsulation of complete news articles where the intentionPath header field content, following each <path-identity> so added with either "!!" or "!". This is permitted for agents that the recipient should then inject them into Netnews. This Application type provideshave multiple <path-identity>s (such as during a transition from one to another). Each of these <path-identity>s MUST meet the methods for mailing articles to moderators (see 6.2.2) and it is alsorequirements set out in Section 3.2. Any agent which modifies the preferred method when sending to an email-to-news gateway (see 6.9.2). NOTE: The benefit of such encapsulation is thatPath header field MAY fold it removes possible conflict between news and email header fields andby inserting FWS immediately after any <path-identity> or <diag-other> it provides a convenient wayadded (see section 3.1.5 of [USEFOR] for allowable locations for FWS). 3.2.2. Path Header Field Example Here is an example of "tunnelling"a newsPath header field created following the rules for injecting and relaying agents. Path: foo.isp.example!.SEEN.isp.example!!foo-news !.MISMATCH.2001:DB:0:0:8:800:200C:417A!bar.isp.example !!old.site.example!barbaz!!baz.isp.example !.POSTED.dialup123.baz.isp.example!not-for-mail This article through a transport mediumwas injected by baz.isp.example as indicated by the <diag-keyword> "POSTED". The injector has recorded that it received the article from dialup123.baz.isp.example. "not-for-mail is a common <tail-entry>. The article was relayed to the relaying agent known, at least to old.site.example, as "barbaz". barbaz relayed it to old.site.example, which does not support 8bit characters. The MIME Media Type definition<diag- keyword> and therefore used the old "!" delimiter. This indicates that the identity of "application/news-transmission" is: MIME type name: application MIME subtype name: news-transmission Required parameters: none Optional parameters: usage=moderate usage=inject usage=relay Encoding considerations: A transfer-encoding (such as Quoted- Printable or Base64) different"barbaz" was not verified and may have been forged. old.site.example relayed it to a news server using the <path- identity> of bar.isp.example and claiming (by using the "!!" <path- delimiter>) to have verified that it came from old.site.example. bar.isp.example relayed it to foo-news which, not being convinced that it truly came from bar.isp.example, inserted the <diag-keyword> "MISMATCH" and then stated that ofit received the article transmitted MAY be supplied (perhaps en route)from the IPv6 address [2001:DB8:0:0:8:800:200C:417A]. (This is not to ensure correct transmission over some 7bit transport medium. Security considerations: A news article may besay that bar.isp.example was not a "control message", which could have effects oncorrect <path-identity> for that source but simply that that identity did not match the recipient host's system beyond just storageexpectations of foo-news. foo-news then passed the article. However, such control messages also occur in normal news flow, so most hosts will already be suitably defended against undesired effects. Published specification: [USEPRO] News Article Architecturearticle to foo.isp.example, which declined to validate its <path-identity> and Protocols November 2006 Body part: A completeinstead appended the <diag- keyword> "SEEN" to indicate it knows the source of the article as isp.example. This may be either an expected <path-identity> or proto-article, ready for injection into Netnews, or a batchthe FQDN of such articles inthe batch format described in section 5.4. NOTE: Itsystem from which it received the article. Presumably foo.isp.example is likelya serving agent that then delivered the recipient of an "application/news- transmission" will be a specialized gateway (e.g. a moderator's submission address) ablearticle to accept articles with only onea reading agent. baz.isp.example, bar.isp.example, and foo-news folded the Path header field. 3.3. Duties of a Posting Agent A posting agent is the three usage parameters "moderate", "inject"component of a user agent that assists a poster in creating a valid proto-article and "relay", hence the reason whyforwarding it to an injecting agent. Posting agents SHOULD ensure that proto-articles they create are optional, being redundant in most situations. Nevertheless, they MAY be usedvalid according to signify[USEFOR] and any other applicable policies. They MUST NOT create any Injection-Date or Injection-Info header fields; these headers will be added by the originator's intention with regardinjecting agent. Contrary to [RFC2822], which implies that the transmission, so removing any possible doubt. When the parameter "relay" is used,mailbox or implied,mailboxes in the body part MAYFrom header field should be a batchthat of articlesthe poster or posters, a poster who does not, for whatever reason, wish to be transmitted together,use his own mailbox MAY use any mailbox ending in which casethe batch format defined in section 5.4 MUST be used. 4.2. Message/news obsoleted The Media Type "message/news", as previously registered with IANA, is hereby declared obsolete. It was never widely implemented, and its default treatment as "application/octet-stream" bytop level domain ".invalid" [RFC2606]. Posting agents that did not recognize it was counter productive. The Media Type "message/rfc822"meant for use by ordinary posters SHOULD be used in its place. 4.3. Application/news-groupinfo The "application/news-groupinfo" is used in conjunction with the "newgroup" (5.2.1) and "mvgroup" (5.2.3) control messages. The <newsgroup-name> inreject any attempt to post an article which cancels or Supersedes another article of which the <newsgroups-line> MUST agree withposter is not the <newsgroup-name>author or sender. 3.3.1. Proto-articles A proto-article is an article in the "newgroup" or "mvgroup" control message. The Media Type "application/news-groupinfo" MUST NOT beformat used except as a part of such control messages. The "application/news-groupinfo" body part contains brief information aboutby a newsgroup, i.e.posting agent for offering to an injecting agent. It may omit certain header fields which can be better-supplied by the group's name, it's <newsgroup- description>injecting agent and will not contain header fields that are added by the <moderation-flag>. NOTE: The presence of the <newsgroups-tag> "For your newsgroups file:"injecting agent. A proto-article is intendedonly for transmission to make the whole newgroup message compatible with current practice as described in [Son-of-1036]. The MIME Media Type definition of "application/news-groupinfo" is: MIME type name: application MIME subtype name: news-groupinfo Required parameters: none Disposition: by default, inline Encoding considerations: "7bit" or "8bit" is sufficientan injecting agent and MUSTSHOULD NOT be usedtransmitted to maintain compatibility. Security considerations: this typeany other agent. A proto-article has the same format as a normal article except that the Injection-Date, Injection-Info, and Xref header fields MUST NOT be used except as part of a control message forpresent; the creation or modification ofPath header field MUST NOT contain a Netnews newsgroup News Article Architecture"POSTED" <diag- keyword>; and Protocols November 2006 Published specification: [USEPRO] The content of the "application/news-groupinfo" body part is defined as: groupinfo-body = [ newsgroups-tag CRLF ] newsgroups-line CRLF newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP %x6E.22.214.171.124.72.6F.75.70.73 SP %x66.69.6C.65.3A ; case sensitive ; "For your newsgroups file:" newsgroups-line = newsgroup-name [ 1*HTAB newsgroup-description ] [ 1*WSP moderation-flag ] newsgroup-description = utext *( *WSP utext ) moderation-flag = %x28.4D.6F.126.96.36.199.188.8.131.52 ; case sensitive "(Moderated)" The <newsgroup-description> MUST NOT containany occurrenceof the string "(Moderated)" within it. Although optional, the <newsgroups- tag> SHOULD be included until such time as this standard has been widely adopted, to ensure compatibility with present practice. Moderated newsgroups MUST be marked by appending the case sensitive text " (Moderated)" at the end. It is NOT recommended that the moderator's email addressfollowing mandatory header fields MAY be included in the <newsgroup-description> as has sometimes been done. NOTE: There is no provision for the use of charsetsomitted: Message-ID, Date, and Path. In all other than US-ASCII withinrespects, a <newsgroup-description>. Suchproto- article MUST be a facilityvalid Netnews article. In particular, the header fields which may be provided inomitted MUST NOT be present with invalid content. If a future extension to this standard. [That may seem harsh, but if we make any such provision now, it will make life more complicated and restrict our freedom when it comesposting agent intends to offer the proposed I18N extension. Therefore I resisted the temptation to include any charset parameter with this Media Type. Note that this also appliessame proto-article to multiple injecting agents, the checkgroups message further on.] 4.4. Application/news-checkgroups The "application/news-checkgroups" Media Type is used in conjunction with the "checkgroups" control message (5.2.4). Itheader fields Message-ID and Date MUST NOTbe used except as a part of such control messages. The "application/news-checkgroups" body part contains a complete list ofpresent and identical in all copies of the newsgroups in a (sub)hierarchy, their <newsgroup- description>s and their moderation status. The MIME Media Type definitionproto-article. 3.3.2. Reinjection of "application/news-checkgroups" is: MIME type name: application MIME subtype name: news-checkgroups Required parameters: none News Article ArchitectureArticles A given article SHOULD be processed by an injecting agent once and Protocols November 2006 Disposition:only once. The Injection-Date or Injection-Info header fields are added by default, inline Encoding considerations: "7bit"an injecting agent and are not permitted in a proto-article. Their presence (or the presence of other unstandardized or obsolete trace headers such as NNTP-Posting-Host, NNTP-Posting-Date, or "8bit"X-Trace) indicates that the proto-article is sufficientinstead an article and MUST be usedhas already been processed by an injecting agent. A posting agent SHOULD normally reject such articles. In the exceptional case that an article needs to maintain compatibility. Security considerations: this type MUST NOTbe used exceptreinjected for some reason (such as part of a checkgroups control message Published specification: [USEPRO] The content oftransferring an article from one Netnews to another where those networks have no relaying agreement), the "application/news-checkgroups" body part is defined as: checkgroups-body = *( valid-group CRLF ) valid-group = newsgroups-line ; see 4.3 5. Control Messages The following sections documentposting agent doing the control messages. "Message" is used herein as a synonym for "article" unless context indicates otherwise. Each <control-command> comprises a <verb>, which indicatesreinjection MUST convert the actionarticle back into a proto-article before passing it to be taken, and <argument>(s), which supply the details (see F- 3.2.3). The following sections contain syntactic definitions foran injecting agent (such as by renaming the <verb>, <argument>s,Injection-Info and possibly the body, for each type of control message. The NewsgroupsInjection-Date header field of each control message SHOULD includefields and removing any Xref header field) and MUST perform the <newsgroup-name>(s) fordate checks on the group(s) affected (i.e. groups to be created, modified or removed,existing Injection-Date or containingDate header fields that would otherwise be done by the injecting agent. Reinjecting articles tomay cause loops, loss of trace information, and other problems and should only be canceled). Thisdone with care and when there is to ensureno available alternative. A posting agent that the message propagatesdoes reinjection is a limited type of gateway and as such is subject to all sites which receive (or would receive) that group(s). It MAY include other <newsgroup-name>s so asof the requirements of an incoming gateway in addition to improve propagation (but this practice may causethe control messagerequirements of a posting agent. 3.3.3. Followups A followup is an article that contains a response to propagate alsothe contents of an earlier article, its precursor. In addition to places where itits normal duties, a posting agent preparing a followup is unwanted, or even cause it notalso subject to propagate where it should, sothe following requirements. Wherever in the following it should not be used without good reason). NOTE: Propagationis controlled by relaying agents, and it may be necessary for relaying agents to take special steps to ensurestated that control messages such as newgroup messages for not-yet- existent newsgroups are propagated correctly (see 6.3). The presence ofby default a Subjectheader field whose content starts with the string "cmsg " followed by a <control-command> was construed under [RFC 1036] as a requestis said to perform that control action (even if no genuine Controlbe inherited from one of those header field was present). Indeed, some implementations went further and addedfields in the implied Control header field before injecting. Likewise,precursor, it means that its initial content is to be a copy of the presencecontent of a <newsgroup-name> ending in ".ctl"that precursor header field (with changes in folding permitted). However, posters MAY then override that default before posting. Despite the Newsgroups header field causedhistoric practice of some posting agents, the SubjectKeywords header field content (not starting with "cmsg" in this case) to be interpreted as a <control-command>. All these practices, which have already largely fallen into disuse, are now declared toSHOULD NOT be Obsolete, and Subjectinherited by default from the precursor article. 1. If the Followup-To header fieldsfield of the precursor article consists of "poster", the followup MUST NOT nowbe interpreted as <control-command>s under any circumstances. News Article Architecture and Protocols November 2006 [Possible addtional text:] In orderposted by default but by default is to prevent continuing interpretation of Subject header fieldsbe emailed to the address given in this waythe precursor's Reply-To or From header field following the rules for an email reply [RFC2822]. This action MAY be overridden by existing agents, posting and injecting agents SHOULD detect and decline to post articlesthe poster, in which case the Subjectposting agent should continue as if the Followup-To header field starts within the word "cmsg"precursor did not exist. 2. The Newsgroups header field SHOULD by default be inherited from the precursor's Followup-To header field if present, and in which there is no Controlotherwise from the precursor's Newsgroups header field. 3. The descriptions below set out REQUIREMENTS to be followedSubject header field SHOULD by sitesdefault be inherited from that receive control messages and choose to honour them. However, nothing in these descriptions shouldof the precursor. The case-sensitive string "Re: " (including the space after the colon) MAY be taken as overridingprepended to the rightcontent of any such site, in accordance withits local policy, to refuse to honour any particular control message, or to referSubject header field unless it to an administrator for approval (either as a class or on a case-by-case basis). 5.1. Digital Signature of Header Fields It is most desirable that group control messages (5.2) in particular be authenticated by incorporating them within some digital signature scheme that encompasses other header fields closely associatedalready begins with them (including at least Approved, Message-ID and Date). At the time of writing, this is usually done by means of athat string. NOTE: Prepending "Re: " serves no protocol known as "PGPverify" ([PGPVERIFY]),function and continued usage of this is encouraged at least as an interim measure. However, PGPverifyhence is not considered suitable for standardization in its present form, for various technical reasons. Itrequired, but it is thereforewidely expected that an early extension to this standard will provide a robustand general purpose digital authentication mechanism with applicability to all situations requiring protection against malicious use of, or interference with, header fields. That extensionnot doing so would also address other Netnews security issues. 5.2. Group Control Messages "Group control messages" arebe surprising. 4. The Distribution header field SHOULD by default be inherited from the sub-classprecursor's Distribution header field, if present. 5. The followup MUST have a References header field referring to its precursor constructed in accordance with Section 3.3.4. 3.3.4. Construction of control messages that requestthe References Header Field The following procedure is to be used whenever some updateprevious article (the "parent") is to be referred to in the configurationReferences header field of a new article, whether because the groups known tonew article is a storage agent, namely "newgroup", "rmgroup", "mvgroup"followup and "checkgroups", plus any others created by extensions to this standard. Group control messages that attempt to create groups with names that are deprecatedthe parent is its precursor or reserved according to F-3.1.4 MUST NOT be issued, except by prior agreement within some cooperating subnet. Moreover, sites receiving such control messages SHOULD check themfor conformance before honouring them. Allsome other reason. The content of the group control messages MUST have an Approvednew article's References header field (F-3.2.1) which, in those hierarchies where appropriate administrative agencies exist (see 1.1), identifiesMUST be formed from the appropriate person or entity as authorized by those agencies. The authorized person or entity SHOULD adhere to any conventionscontent of the parent's References header field if present and restrictions onthe formatcontent of <newsgroup-name>s established for those hierarchies News Article Architecture and Protocols November 2006 [USEAGE]. 5.2.1. The 'newgroup' Control Message control-command =/ Newgroup-command Newgroup-command = "newgroup" Newgroup-arguments Newgroup-arguments = FWS newsgroup-name [ FWS newgroup-flag ] newgroup-flag = "moderated" The "newgroup" control message requests thatthe specified groupMessage-ID header field of the parent. If the parent had a References header, FWS as defined in [USEFOR] MUST be created or haveadded between its moderation status or <newsgroups-line> changed. Whencontent and the request is honoured, ifMessage-ID header field content. If the <newgroup-flag> "moderated" is present thenresulting References header field would, after unfolding, exceed 998 characters in length (including its field name but not the statusfinal CRLF), it MUST be trimmed (and otherwise MAY be trimmed). Trimming means removing any number of message identifiers from its content, except that the group SHOULD be marked as moderated,first message identifier and vice versa. "Moderated" isthe only such flag defined by this standard; other flags MAY be defined for use in cooperating subnets, but newgroup messages containing themlast two MUST NOT be acted on outsideremoved. An essential property of those subnets. NOTE: Specifically, some alternative flags such as "y" and "m", which are sentthe References header field, guaranteed by the above procedure and recognizedREQUIRED to be maintained by some current software, are NOT part of this standard. Moreover, some existing implementations treatany flag other than "moderated" as indicatingextensions to this protocol, is that an unmoderated newsgroup. Botharticle MUST NOT precede one of these usages are contrary to this standardits parents. 3.4. Duties of an Injecting Agent An injecting agent takes a proto-article from a posting agent and control messageseither forwards it to a moderator or passes it to a relaying or serving agent or agents. An injecting agent bears the primary responsibility for ensuring that any article it injects conforms with such non-standard flags should be ignored. 184.108.40.206. The Bodythe rules of the 'newgroup' Control MessageNetnews standards. The bodyadministrator of an injecting agent is also expected to bear some responsibility towards the newgroup message contains the following subparts, preferably in the order shown: 1. An "application/news-groupinfo" part (4.3) containing the name and <newsgroups-line> (4.3)rest of the group. This part MUST be present and SHOULD be usedNetnews network to update any copy of the <newsgroups-line> maintained by the storage agent. 2. Other parts containing useful information about the background ofwhich it is connected for the newgroup message (typically of type "text/plain"). 3. Parts containing initialarticles forthe newsgroup. See section 220.127.116.11injecting agent accepts. Injecting agents, when rejecting articles, are encouraged to communicate the reason for details. Inrejection to the event that thereposting agent using whatever facility is onlyprovided by the single (i.e. application/news- groupinfo) subpart present, it will suffice to includeunderlying transport. The injecting agent is in a "Content- Type: application/news-groupinfo" amongstunique position to communicate the header fieldsreason for rejection; relaying agents and serving agents normally have to reject messages silently. The injecting agent therefore bears much of the control message. Otherwise, a "Content-Type: multipart/mixed" header field will be needed, and each separate part will then need its own Content-Type header field. 18.104.22.168. Initial Articles Some subpartsburden of a "newgroup"diagnosing broken posting agents or "mvgroup" control message MAY contain an initial set of articles to be postedcommunicating policy violations to the affected newsgroup as soon as it has been created or modified. These parts are News Article Architectureposters. An injecting agent MUST have available a list (possibly empty) of moderated groups for which it accepts articles and Protocols November 2006 identified by havingthe Media Type "application/news-transmission", possibly with the parameter "usage=inject". The bodycorresponding submission addresses. It SHOULD have available a list of each such part shouldvalid newsgroups to catch articles not posted to a valid newsgroup and therefore likely to be silently discarded by relaying and serving agents. Usually, an injecting agent is deployed in conjunction with a complete proto-article, ready for posting. This featureserving agent and maintains these lists based on control messages received by the serving agent. An injecting agent processes proto-articles as follows: 1. It SHOULD verify that the article is intendedfrom a trusted source (by, for example, relying on the postingauthorization capability of charters, initial FAQs andthe likeunderlying transport used to talk to the newly formed group. The Newsgroups header field of the proto-article MUST include the <newsgroup-name> of the newly created or modified group.posting agent). 2. It MAY include other <newsgroup-name>s. If the proto-article includes a Message-ID header field, the message identifier in itMUST be different from that ofreject any existing article and fromproto-article that ofdoes not have the control message asproper mandatory header fields for a whole. Alternatively suchproto-article; that has Injection- Date, Injection-Info, or Xref header fields; that has a message identifier MAY be derived by the injecting agent whenPath header field containing the proto-article"POSTED" <diag-keyword>; or that is posted. The proto-articlenot syntactically valid as defined by [USEFOR]. It SHOULD include thereject any proto-article which contains a header field "Distribution: local". Thedeprecated for Netnews. It MAY reject any proto-article SHOULD be injected at the storage agentthat processes the control message AFTER the newsgroup in question has been createdcontains trace header fields indicating that it was already injected by an injecting agent that did not add Injection-Info or modified.Injection-Date. 3. It MUST NOT be injected if the newsgroupSHOULD reject any article whose Date header field is not, in fact, created (for whatever reason).more than 24 hours into the future (and MAY use a margin less than 24 hours). It MUST NOTSHOULD reject any article whose Date header appears to be submittedstale (more than 72 hours into the past, for example, or too old to anystill be recorded in the database of a relaying agent for transmission beyond the storage agent(s) upon whichthe newsgroup creation has just been effected (in other words, it is toinjecting agent will be treated as having a "Distribution: local"using) since not all news servers support Injection-Date. 4. It SHOULD reject any proto-article whose Newsgroups header field, whether such afield is actually presentdoes not contain at least one <newsgroup-name> for a valid group, or not). NOTE: Itcontaining a <newsgroup-name> reserved for specific purposes by Section 3.1.4 of [USEFOR] unless that specific purpose or local agreement applies to the proto-article being processed. Crossposting to unknown newsgroups is not precluded provided that at least one of the proto-articlenewsgroups in the Newsgroups header is itself a control message or other type of special article, tovalid. 5. The Message-ID and Date header fields with appropriate contents MUST be activated only upon creationadded when not present in the proto-article. 6. The injecting agent MUST NOT alter the body of the new newsgroup. However, except as might arise from that possibility,article in any "application/news-transmission" within some nested "multipart/*" structure within the proto-article isway (including any change of Content-Transfer-Encoding). It MAY add other header fields not to be activated. 22.214.171.124. Example A "newgroup" with its charter: From: "example.all Administrator" <email@example.com> Newsgroups: example.admin.info,example.admin.announce Date: 27 Feb 2006 12:50:22 +0200 Subject: cmsg newgroup example.admin.info moderated Approved: firstname.lastname@example.org Control: newgroup example.admin.info moderated Message-ID: <email@example.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nxtprt" Content-Transfer-Encoding: 8bit This is a MIME control message. --nxtprt Content-Type: application/news-groupinfo For your newsgroups file: News Article Architecture and Protocols November 2006 example.admin.info Aboutalready provided by the example.* groups (Moderated) --nxtprt Content-Type: application/news-transmission Newsgroups: example.admin.info From: "example.all Administrator" <firstname.lastname@example.org> Subject: Charterposter, but injecting agents are encouraged to use the Injection-Info header for example.admin.info Message-ID: <email@example.com> Distribution: local Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit The group example.admin.info contains regularly postedsuch information onand to minimize the example.* hierarchy. --nxtprt-- 5.2.2. The 'rmgroup' Control Message control-command =/ Rmgroup-command Rmgroup-command = "rmgroup" Rmgroup-arguments Rmgroup-arguments = FWS newsgroup-name The "rmgroup" control message requests thataddition of other headers. It SHOULD NOT alter, delete, or reorder any existing header field except the specified group be removed fromPath header. 7. If the list of valid groups. The Media Type ofNewsgroups header contains one or more moderated groups and the body is unspecified; it MAYproto-article does not contain anything, usuallyan explanatory text. NOTE: It is entirely proper for a storageApproved header field, the injecting agent MUST either forward it to retain the group until all the articlesa moderator as specified in it have expired, providedSection 3.4.1 or, if that it ceases to accept new articles. 126.96.36.199. Example From: "example.all Administrator" <firstname.lastname@example.org> Newsgroups: example.admin.obsolete, example.admin.announce Date: 4 Apr 2006 22:04 -0900 (PST) Subject: cmsg rmgroup example.admin.obsolete Message-ID: <email@example.com> Approved: firstname.lastname@example.org Control: rmgroup example.admin.obsolete The group example.admin.obsoleteis obsolete. Please remove it from your system. 5.2.3. The 'mvgroup' Control Message control-command =/ Mvgroup-command Mvgroup-command = "mvgroup" Mvgroup-arguments Mvgroup-arguments = FWS newsgroup-name FWS newsgroup-name [ FWS newgroup-flag ] News Article Architecture and Protocols November 2006 The "mvgroup" control message requests that the group specified by the first <(old-)newsgroup-name>not possible, reject it. This forwarding MUST be moved to that specified by the second <(new-)newsgroup-name>. Thus it is broadly equivalent to a "newgroup" control message for the second group followed by a "rmgroup" control message fordone after adding the first group. The message body contains an "application/news-groupinfo" part (4.3) containing machine-Message-ID and human-readable information aboutDate headers if required, and before adding the new group,Injection- Info and possibly other subparts as forInjection-Date headers. 8. Otherwise, a "newgroup" control message.Path header field with a <tail-entry> MUST be added if not already present. 9. The information conveyed ininjecting agent MUST then update the "application/news-groupinfo" body part, notably its <newsgroups-line> (4.3), is applied toPath header field as described in Section 3.2.1. 10. An Injection-Info header field SHOULD be added identifying the new group. When this message is received,source of the new group is created (if it does not exist already) as for a "newgroup" control message,article and SHOULDpossibly other trace information as described in any case be made moderated if a <newgroup-flag> "moderated" is present,Section 3.2.8 of [USEFOR]. 11. The injecting agent MUST then add an Injection-Date header field containing the current date and vice versa. Attime. 12. Finally, the same time, arrangements SHOULD be made to removeinjecting agent forwards the old group (as with a "rmgroup" control message), but only after a suitable overlap periodarticle to allowone or more relaying agents, and the network to adjustinjection process is complete. 3.4.1. Forwarding Messages to the new arrangement. At the same time asa storage agent acts upon this message, allModerator An injecting agents associated with that storageagent SHOULD inhibit the posting of new articles toMUST forward the old group (preferably with some indicationproto-article to the poster thatmoderator of the newleftmost moderated group should have been used). Relaying agents, however, MUST continue to propagate such articles duringlisted in the overlap period. NOTE: ItNewsgroups header field, customarily via email. There are two standard ways in which it may do this: 1. The complete proto-article is toencapsulated, header fields and all, within the email. This SHOULD be expected that different storage agents will act on thisdone by creating an email message at different points of time, userswith a Content-Type of application/news-transmission with the old group will have to become accustomedusage parameter set to "moderate". The body SHOULD NOT contain any content other than the new arrangement, and followups to already established threads will likely continue undermessage. This method has the old group. Therefore, there needs to be an overlap period during which articles may continue to be accepted by relayingadvantage of removing any possible conflict between Netnews and email header fields and storage agents in either group. This standard does not specifyany standard period of overlap (though it would be expectedchanges to be expressed in days rather than in months).those fields during transport through email. 2. The inhibitionproto-article is sent as an email with the addition of injectionany header fields (such as a To header field) required for an email. The existing Message-ID header field SHOULD be retained. Although both of new articles to the old group may seem draconian, but it isthese methods have been used in the surest way to preventpast and the changeover from dragging on indefinitely. Sincefirst has clear technical advantages, the "mvgroup" control messagesecond is newly introducedin this standardmore common use and maymany moderators are not be widely implemented initially, itprepared to deal with messages in the first format. Accordingly, the first method SHOULD NOT be followed shortly afterwards by a corresponding "newgroup" control message; and again, after a reasonable overlap period,used unless the moderator to which it MUSTis being forwarded is known to be followed byable to handle this method. NOTE: Deriving the email address of the moderator of a "rmgroup" control message forgroup is outside the old group. In order to facilitatescope of this document. It is worth mentioning, however, that a smooth changeover, storage agents MAY arrangecommon method is to use a forwarding service requeststhat handles submissions for access tomany moderated groups. For maximum compatibility with existing news servers, such forwarding services generally form the oldsubmission address for a moderated group by providing access toreplacing each "." in the new group, which would<newsgroup-name> with "-" and then contain, or appear to contain, all articles posted to either group (including, ideally, the pre-changeover articles fromusing that value as the old one). Nevertheless, if this feature<local-part> of a <mailbox> formed by appending a set domain. Forwarding proto-articles to moderators via email is implemented,the articles themselves, as supplied to reading agents, MUST NOT be alteredmost general method and most common in large Netnews networks such as Usenet, but any way (and, in particular, News Article Architecture and Protocols November 2006 their Newsgroups header fields MUST contain exactly those newsgroups present when they were injected). On the other hand, the Xref header field (F-3.2.14) MAY contain entries for either group (or even both). NOTE: Some storage agents that use an "active" file permit an entrymeans of forwarding the form "oldgroup xxx yyy =newgroup", which enables any articles arriving for oldgroup toarticle that preserves it without injecting it MAY be divertedused. For example, if the injecting agent has access to newgroup, thus providinga simple implementation of this feature. However, it is known that not all current storage agents will find implementation so easy (especially indatabase used by the short term) which is whymoderator to store proto-articles awaiting processing, it is not mandated by this standard. Nevertheless, its eventual implementation in all storage agents is to be considered highly desirable. Onmay place the other hand, it is recognizedproto-article directly into that this feature would likely notdatabase. Such methods may be implementable if the new group was already in existence with existingmore appropriate for smaller Netnews networks. 3.5. Duties of a Relaying Agent A relaying agent accepts injected articles in it. This situation should not normally arise except when there is already some confusion asfrom injecting and other relaying agents and passes them on to which groups are,relaying or are not, supposedserving agents. To avoid bypass of injecting agent policies and forgery of Path and Injector-Info headers, relaying agents SHOULD accept articles only from trusted agents. An article SHOULD NOT be relayed unless the sending agent has been configured to existsupply and the receiving agent to receive at least one of the <newsgroup-name>s in that hierarchy. Note thatits Newsgroups header field and at least one of the "mvgroup"<dist-name>s in its Distribution header field (if present). Exceptionally, control message is not really intended to be used for merging two existing groups. 188.8.131.52. Example From: "example.all Administrator" <email@example.com> Newsgroups: example.oldgroup,example.newgroup,example.admin.announce Date: 30 Apr 2006 22:04 -0500 (EST) Subject: cmsg mvgroup example.oldgroup example.newgroup moderated Message-ID: <firstname.lastname@example.org> Approved: email@example.com Control: mvgroup example.oldgroup example.newgroup moderated MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=nxt --nxt Content-Type: application/news-groupinfo For yourmessages creating or removing newsgroups file: example.newgroup The new replacement group (Moderated) --nxt The moderated group example.oldgroup is replaced by example.newgroup. Please update your configuration, and please, if possible, arrange to file articles arriving(newgroup or rmgroup control messages, for example.oldgroup asexample) SHOULD be relayed if they werethe affected group appears in example.newgroup. --nxt Content-Type: application/news-transmission Newsgroups: example.admin.info From: "example.all Administrator" <firstname.lastname@example.org> Subject: Charter for example.newgroup Message-ID: <email@example.com> News Article Architectureits Newsgroups header field and and Protocols November 2006 Distribution: local Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit This group (formerly known as example.oldgroup) is forthe discussion of examples. --nxt-- 5.2.4. The 'checkgroups' Control Message The "checkgroups" control message contains a list of all the valid groups in a complete hierarchy. control-command =/ Checkgroup-command Checkgroup-command = "checkgroups" Checkgroup-arguments Checkgroup-arguments= [ chkscope ] [ chksernr ] chkscope = 1*( FWS ["!"] newsgroup-name ) chksernr = FWS "#" 1*DIGIT A "checkgroups" message appliessending agent and receiving relaying agents are both configured to any (sub-)hierarchy withrelay a prefix listed in the <chkscope> argument, providednewsgroup of that the rightmost matching <newsgroup-name> in the list isname (whether or not immediately preceded bysuch a "!". If no <chkscope> argument is given, it appliesnewsgroup exists). In order to all hierarchies for which group statements appear in the body of the message. NOTE: Some existing software does not support the <chkscope> argument. Thus a "checkgroups" messageavoid unnecessary relaying attempts, an article SHOULD also containNOT be relayed if the groups<path-identity> of other subhierarchiesthe sender is not responsible for. "New" software MUST ignore groups which do not fallreceiving agent (or some known alias thereof) appears as a <path-identity> (excluding within the <chkscope> argument of the "checkgroups" message. The <chksernr> argument is a serial number, which can be any positive integer (e.g. just numbered<tail-entry> or the datefollowing a "POSTED" <diag-keyword>) in YYYYMMDD).its Path header field. A relaying agent processes an article as follows: 1. It SHOULD increase byMUST reject any article without a Newsgroups header field or Message-ID header field, or without either an arbitrary value with every change to the group list andInjection-Date or Date header field. 2. It MUST NOT ever decrease. NOTE: This was addedreject any article that has already been successfully sent to circumvent security problems in situations whereit, based on the DateMessage-ID header field cannot be authenticated. Example: Control: checkgroups de !de.alt #248 which includes the wholeof the 'de.*' hierarchy, with the exception of its 'de.alt.*' sub-hierarchy. The bodyarticle. To satisfy this requirement, a relaying agent normally keeps a database of themessage identifiers it has the Media Type "application/news- checkgroups" (4.4).already accepted. 3. It assertsMUST examine the Injection-Date header field or, if absent, the Date header field, and reject the article if that date predates the <valid-group>searliest articles of which it lists arekeeps record or if that date is more than 24 hours into the only newsgroupsfuture. It MAY reject articles with dates in the specified hierarchies. News Article Architecture and Protocols November 2006 NOTE: The "checkgroups"future with a smaller margin than 24 hours. 4. It SHOULD reject any article that does not include all the mandatory header fields. It MAY reject any article that contains header fields that do not have valid contents. 5. It SHOULD reject any article that matches an already-received cancel control message is intended to synchronizeor the listcontents of newsgroups stored by a storage agent, and their <newsgroup-description>s, with the lists stored by other storage agents throughoutthe network. However, it might be inadvisable forSupersedes header field of an accepted article, provided that the storagerelaying agent actuallychose (on the basis of local site policy) to createhonor that cancel control message or deleteSupersedes header field. 6. It MAY reject any newsgroupsarticle without first obtaining the approval of its administrators for such proposed actions. NOTE: The possibility of removing a complete hierarchy by means ofan "invalidation" line beginning withApproved header field posted to a '!' innewsgroup known to be moderated. This practice is strongly encouraged but the checkgroups-bodyinformation necessary to do so is no longer provided by this standard. The intent of the feature was widely misunderstood and it was misused more often than it was used correctly. The same effect, if required, can now be obtainednot required to be maintained by a relaying agent. 7. It MUST update the use of an appropriate <chkscope> argumentPath header field as described in conjunctionSection 3.2.1. 8. It MAY delete any Xref header field present and MAY add a new Xref header field with an empty <checkgroups- body>. 5.3. Cancelany valid content. The "cancel" message requestsXref header field is not used by relaying agents, but relaying agents that a target article be "canceled", i.e. be withdrawn from circulation or access. control-command =/ Cancel-command Cancel-command = "cancel" Cancel-arguments Cancel-arguments = FWS msg-id [FWS] The argument identifiesare also serving agents may generate Xref header fields for their own internal purposes. 9. Finally, it passes the article on to be cancelled by its message identifier. The body SHOULD contain an indication of why the cancellation was requested. The "cancel" message SHOULD be postedother relaying and serving agents to the same newsgroup(s), with the same distribution(s), as the articlewhich it is attemptingconfigured to cancel. A storagesend articles. Relaying agents SHOULD, where possible in the underlying transport, inform the agent that elects to honour a "cancel" message SHOULD makepassed the article unavailable forto the relaying or storage (perhaps by deleting it completely). Ifagent if the targetarticle is unavailable, and the acceptabilitywas rejected. Relaying agents MUST NOT inform any other external entity of the "cancel" message cannot be established without it, activationrejection of the "cancel" message SHOULD be delayed until the targetan article unless that external entity has been seen. See also sections 6.3 and 6.4. NOTE: It is expectedexplicitly requested that the security extension envisaged in section 5.1 will make more detailed provisions for establishing whether honouring a particular "cancel" message is in order. In particular,it is likely that there willbe provision for the digital signatureinformed of 3rd party cancels (i.e. those issued other than by the sender, the moderator,such errors. Relaying agents MUST NOT alter, delete, or the injector). NOTE: A cancel submitted by the poster forrearrange any part of an article in a moderated group will be forwarded toexcept for the moderator of that group,Path and it is up to that moderator to act upon it (6.8). NOTE: The former requirement [RFC 1036] that the From and/or SenderXref header fields offields. They MUST NOT modify the "cancel" message should match thosebody of articles in any way. If an article is not acceptable as-is, the originalarticle has been removedMUST be rejected rather than modified. 3.6. Duties of a Serving Agent A serving agent accepts articles from this standard, News Article Architecturea relaying agent or injecting agent, stores it, and Protocols November 2006 sincemakes it only encouraged cancel issuersavailable to conceal their true identity,reading agents. Articles are normally indexed by newsgroup and it was not<article-locator> (Section 3.2.14 of [USEFOR]), usually checked or enforcedin the form of a decimal number. If the serving agent stores articles by canceling software. Therefore, bothnewsgroup, control messages SHOULD NOT be stored in the From and/or Sender header fields and any Approvednewsgroups listed in the control message's Newsgroups header field should now relate tofield. Instead, they SHOULD be stored in a newsgroup in the entity responsiblehierarchy "control", which is reserved for issuing the "cancel" message. 5.4. Ihave, sendme The "ihave" and "sendme"this purpose. Conventionally, control messages implement a crude batched predecessor of the NNTP [RFC 3977] protocol. Theyare largely obsolete on the Internet, but still see usestored in conjunction with some transport protocols suchnewsgroups named for the type of control message (such as UUCP, especially"control.cancel" for backup feeds that normally are active only whencancel control messages). A serving agent MUST have available a primary feed path has failed. There is no requirementlist (possibly empty) of moderated groups for relaying agentswhich it accepts articles so that do not support such transport protocolsit can reject unapproved articles posted to implement them. NOTE: The ihavemoderated groups. Frequently a serving agent is deployed in combination with an injecting agent and sendme messages defined here have ABSOLUTELY NOTHING TO DO WITH NNTP, despite similarities of terminology. The two messages sharecan use the same syntax: control-command =/ Ihave-command Ihave-command = "ihave" Ihave-argument Ihave-argument = relayer-name control-command =/ Sendme-command Sendme-command = "sendme" Sendme-argument Sendme-argument = Ihave-argument relayer-name = path-identity ; see F-3.1.5 ihave-body = *( msg-id CRLF ) sendme-body = ihave-body The body of the message consists of alist of <msg-id>s, one per line. [RFC 1036] also permittedas the list of <msg-id>s to appear ininjecting agent. A serving agent processes articles as follows: 1. It MUST reject any article that does not include all the <Ihave->mandatory header fields or <Sendme-argument> withany article which contains header fields that do not have valid contents. 2. It MUST examine the syntax Ihave-argument = [FWS] *( msg-id FWS ) [relayer-name] but this form SHOULD NOT now be used, though relaying agents MAY recognizeInjection-Date header field or, if absent, the Date header field, and processreject the article if that date predates the earliest articles of which it for backward compatibility. The "ihave" message stateskeeps record or if that date is more than 24 hours into the named relaying agent has receivedfuture. It MAY reject articles with dates in the specified message identifiers, which may be of interestfuture with a smaller margin than 24 hours. 3. It MUST reject any article that has already been successfully sent to it, based on the relaying agents receiving the ihave message. The "sendme" message requests thatMessage-ID header field of the article. To satisfy this requirement, a relaying agent receiving it send the articles having the specifiednormally keeps a database of message identifiers toit has already accepted. 4. It SHOULD reject any article that matches an already-received and honored cancel message or Supersedes header field following the namedsame rules as a relaying agent. Upon receipt of the sendme message, the receivingagent sends the article(s) requested, often (especially when(Section 3.5). 5. It MUST reject any article without an Approved header field posted to any newsgroup listed as moderated. 6. It MUST update the transport protocol is UUCP)Path header field as described in Section 3.2.1. 7. It MUST (except when specially configured to preserve the form of one or more batches, each containing several articles. The usual form of a <batch> is defined<article-locator>s set by the following syntax (which is also used in the application/news transmission media type (4.1)). News Article Architecture and Protocols November 2006 batch = 1*( batch-header article ) batch-header = "#!" SP rnews SP article-size CRLF rnews = %x72.6E.65.77.73 ; case sensitive "rnews" article-size = 1*DIGIT Thus a <batch> is a sequence of articles,sending site) remove any Xref header field from each prefixed byarticle. It then MAY (and usually will) generate a fresh Xref header line that includes its size. The <article-size> is a decimal count of the octets in the article, counting each CRLF as one octet regardless of howfield. 8. Finally, it is actually represented. NOTE: Despitestores the similarity of this format to an executable UNIX script,article and makes it is EXTREMELY unwise to feed such a batch into a command interpreteravailable for reading agents. Serving agents MUST NOT create new newsgroups simply because an unrecognized <newsgroup-name> occurs in anticipation of it runninga command named "rnews"; the security implications of so doing would be disastrous. These control messages are normally sent essentially as point-to- point messages, by using <newsgroup-name>s in theNewsgroups header field of the form "to." followed by one (or possibly more) <component>s in the form of a <relayer-name> (see section F-3.1.4 which forbids "to" as the first <component> of a <newsgroup-name>). Thefield. Newsgroups are normally created via control message SHOULD then be delivered ONLY to the relaying agent(s) identified by that <relayer-name>, and any relaying agent receiving such a message which includes its own <relayer-name>messages (Section 5.2.1). Serving agents MUST NOT propagate it further. Each pairalter, delete, or rearrange any part of relaying agent(s) sendingan article except for the Path and receiving these messagesXref header fields. They MUST be immediate neighbours, exchanging news directly with each other. Each relaying agent advertises its new arrivals toNOT modify the other using "ihave" messages, and each uses "sendme" messages to requestbody of the articles it lacks. To reduce overhead, ihave and sendme messages SHOULDin any way. If an article is not acceptable as-is, the article MUST be sent relatively infrequently and SHOULD contain reasonable numbersrejected rather than modified. 3.7. Duties of message identifiers. If ihave and sendme are being used to implementa backup feed, it may be desirable to insertReading Agent Since a delay between reception of an ihave and generation ofreading agent is only a sendme, so thatpassive participant in a slightly slow primary feed will not cause large numbersNetnews network, there are no specific protocol requirements placed on it. See [USEAGE] for best-practice recommendations. 3.8. Duties of articlesa Moderator A moderator receives news articles, customarily by email, decides whether to be requested unnecessarily via sendme. 5.5. Obsolete control messages. The following control messages (as described in Appendix A)approve them and, if so, either passes them to an injecting agent or forwards them to further moderators. Articles are declared obsoletenormally received by this standard: sendsys version whogets senduuname 6. Duties of Various Agents The following section sets outthe duties of various agents involvedmoderator in the creation, relaying and storage of Netnews articles. Insofar as these duties are describedemail either encapsulated as sequences of steps to be followed, it should be understood that it is the effectan object of these sequences that is News Article Architecture and Protocols November 2006 important, and implementations may use any method that gives rise to that same effect. In this section, the word "verified",Content-Type application/ news-transmission (or possibly encapsulated but without an explicit Content-Type header field), or else directly as applied to the source of some article, means thatan agent processing that article has established, by some means,email already containing all the identity of that source (whichheader fields appropriate for a Netnews article (see Section 3.4.1). Moderators who may receive articles via email SHOULD be another agentprepared to accept articles in either format. There are no protocol restrictions on what criteria are used for accepting or rejecting messages or on what modifications a poster). NOTE: In many implementations, a single agentmoderator may perform various combinations of the injecting, relayingmake to a message (both header fields and storage functions. Its duties are then the union ofbody) before injecting it. Recommended best practice, however, is to make the various duties concerned. 6.1. General principlesminimal required changes. Moderators need to be followed There are two important principlesaware that news implementors (and administrators) needmodifications made to keep in mind. The first isarticles may invalidate signatures created by the well-known Internet Robustness Principle: Be liberal in what you accept,poster or previous moderators. See [USEAGE] for further best-practice recommendations. Moderators process articles as follows: 1. They decide whether to approve or reject a proto-article, and conservative in what you send. However, inif approved, prepare the case of news there is an even more important principle, derived from a much older code of practice,proto-article for injection. If the Hippocratic Oath (we may thus callproto- article was received as an unencapsulated email message, this the Hippocratic Principle): First, do no harm. It is VITALwill require converting it back to realize that decisions which might be merely suboptimal in a smaller context can become devastating mistakes when amplified by the actions of thousands of hosts withina few minutes. In the case of gateways,valid Netnews proto-article. If the primary corollary to this is: Cause no loops. 6.2. Duties of an Injecting Agent An Injecting Agentarticle is responsible for taking a (proto-)article from a posting (or other) agent and either forwardingrejected, it is normally rejected for all newsgroups to a moderator or injectingwhich it into the relaying system for access by readers. As such, an injecting agentwas posted and nothing further is considered responsible for ensuring that any articledone. If it injects conformsis approved, the moderator proceeds with the rules of [USEFOR]. It is also expected to bearfollowing steps. 2. If the Newsgroups header field contains further moderated newsgroups for which approval has not already been given, they may either reach some responsibility towardsagreement with the restother moderators on the disposition of the network forarticle or, more generally, add an indication (identifying both the behaviour of its posters. Inmoderator and the normal coursename of events, an articlethe newsgroup) that has already been injected into a Netnews network will never pass through another injecting agent. So, if an injecting agent receives an otherwise validthey approve the article that has already been injected (as evidenced byand then forward it to the presence of an Injection-Date header field, an Injection-Info header field, or more than one occurrencemoderator of the <diag-keyword> "POSTED" in a Path header field) it MAY choose to reject it, but otherwiseleftmost unapproved newsgroup. This forwarding SHOULD News Article Architecturebe done following the procedure in Section 3.4.1 and Protocols November 2006 cause it toMAY be relayed, as it stands,done by a relaying agent (6.3). In exceptional circumstances (e.g. as part of some complex gatewaying process, or where a relaying agent considers it essential for fulfilling its responsibility towardsrotating the rest of<newsgroup-name>s in the network) an already injected article MAY be "reinjected" intoNewsgroups header field so that the network. This standard does not prescribe any such circumstance; rather thisleftmost unapproved newsgroup is is a matter of policy to be determined bythe administrators of eachleftmost moderated newsgroup in that field and then posting it, letting the injecting agent, who haveagent do the responsibility toforwarding. However, if using this mechanism, they MUST first ensure that the article contains no harm arises. In all other circumstances, unintented reinjection is to be avoided (see 6.9). Nevertheless, in order to preserveApproved header field. 3. If the integrityNewsgroups header field contains no further unapproved moderated groups, they add an Approved header field (see Section 3.2.1 of [USEFOR]) identifying the network in these special cases, this standard does set out the correct way to reinject (see special provisions in 6.2.2 Steps 3, 7 and 9). Itmoderator and, insofar as is usualpossible, all the other moderators who have approved the article. The moderator who takes this step assumes responsibility for an injecting agent to be closely associated with a storage agent, thus giving it access toensuring that the list (6.4) showingarticle was approved by the moderation statusmoderators of theall moderated newsgroups it is likelyto handle. In the event thatwhich it does not have such an associated storage agent, it MUST maintain that list itself. 6.2.1. Proto-articles A proto-article SHOULD NOT be propagated in that formwas posted. 4. Moderators are encouraged to other than injecting agents. A proto-article has the same format as a normal article except that some ofretain the following mandatoryMessage-ID header fields MAY be omitted: Message-Id, Date, Path (and even From if the particular injecting agent can derive that information from other sources). However,field if it is intended to offervalid, and also retain the proto-articleDate header field unless it appears to twobe stale (72 hours or more injecting agentsin parallel, then it is onlythe Path header field thatpast) for reasons understood by the moderator (such as delays in the moderation process) in which case they MAY be omitted. Thesubstitute the current date. Any Injection-Date, Injection-Info, or Xref header fields that canalready present (though there should be omittednone) MUST NOT contain invalid values; theybe removed. 5. Any Path header field MUST either be correctremoved or not present at all. [Maybe omit that last sentence.] NOTE: An article that is offered for reinjection has, by definition, already been injected once, and is not thereforetruncated to be considered as a proto-article. Hence a genuine proto-article will not contain any Injection-Date header field nor the <diag- keyword> "POSTED" anywhere inonly those entries following its Path header field, though that header field MAY contain <path-identity>s corresponding to machines traversed between the posting agent and"POSTED" <diag-keyword>, if any. 6. The moderator then passes the injecting agent proper. 6.2.2. Procedurearticle to be followed by Injecting Agents Anan injecting agent receives (proto-)articles from posting and followup agents. It verifies them, adds header fields where required, and then either forwards them toagent, having first observed all the duties of a moderator or injects them by passing them to storage or relaying agents. It MUST NOT forwardposting agent. 3.9. Duties of a Gateway A gateway transforms an already injectedarticle to a moderator. News Article Architecture and Protocols November 2006 An injecting agent processes articles as follows: 1. It MUST remove any Injection-Info header field already present (though it might be useful to copy it to a suitable "X-" header field). It SHOULD likewise remove any NNTP-Posting-Host, X-Trace,into the native message format of another medium, or other non-standard tracing header field. 2. It SHOULD ensure thattranslates the article is from a verified source, and MAY rejectmessages of another medium into news articles, or transforms articles in which header fields contain unverified email addresses, that is, addresses which are not known to be validinto proto-articles for the verified source, though it would be perverse to reject intentionally unverifiable addresses such as those ending in ".invalid" (6.5). 3. It SHOULD reject anyinjection into a separate Netnews network. Encapsulation of a news article whose Date header field (F-3.1.1) is more than 24 hoursinto the future (and MAY usea margin less thanmessage of MIME type application/news-transmission, or the subsequent undoing of that 24 hours). It MUST (except when reinjecting) reject any article with an Injection-Date header field already present (and SHOULD do likewise with any NNTP-Posting-Date header field). When reinjectingencapsulation, is not gatewaying, since it MAY, ininvolves no transformation of the absencearticle. There are two basic types of any Injection-Date header field, reject anygateway, the outgoing gateway that transforms a news article whose Date header field appears to be stale (e.g. more than 72 hoursinto a different type of message, and the past). 4. It MUST reject any articleincoming gateway that does not have the proper mandatory header fields fortransforms a message from another network into a news proto-article or which contains any header field that does not have legal contents. It SHOULD reject any article which contains any header field deprecated for Netnews (e.g. as in [RFC 2298]). It SHOULD reject any article whose Newsgroups header field does not contain at least one <newsgroup- name> for an existing group (as listed by its associated storage agent)and injects it MAY reject any <newsgroup-name> which violates one of the restrictions in F-3.1.4 or which, although otherwise correct, violatesinto a policy restriction established, for some (sub-)hierarchy, byNetnews network. These are handled separately below. Transformation of an agencyarticle into another medium stands a very high chance of discarding or interfering with the appropriate authority (1.2). Observe that crossposting to unknown newsgroups is not precluded provided at least one of thoseprotection inherent in the Newsgroups header fieldnews system against duplicate articles. The most common problem caused by gateways is listed. NOTE: This ability to reject <newsgroup-name>s in breach of established policy does not extend to relaying agents, though it might be reasonable for posting agentsloops that repeatedly reinject previously posted articles. To prevent this, a gateway MUST take precautions against loops, as detailed below. The transformations applied to do it. 5. If the article is rejected (for reasons given above, or for other formatting errors or matters of site policy)the posting agentmessage SHOULD be informed (suchas via an NNTP 44x response code) that posting has failed andminimal as possible while still accomplishing the article MUST NOT then be processed further. 6. The Message-ID, Dategatewaying. Every change made by a gateway potentially breaks a property of one of the media or loses information, and From header fields (with appropriate contents) MUST be added when not already present. A User-Agent header field MAYtherefore only those transformations made necessary by the differences between the media should be added (orapplied. If bidirectional gatewaying (both an already present User-Agent header field MAY be augmented) so as to identify the software (e.g. "INN/1.7.2") used byincoming and an outgoing gateway) is being set up between Netnews and some other medium, the injecting agent. News Article Architectureincoming and Protocols November 2006 [I think that last sentence needsoutgoing gateways SHOULD be coordinated to go (in which case see consequential change in 7.3). Did we discuss this when we looked at User-Agent in USEFOR?] NOTE: The Message-ID, Dateavoid unintended reinjection of gated articles. Circular gatewaying (gatewaying a message into another medium and From fields will already be present during reinjection. 7. The injecting agent MUSTthen back into Netnews) SHOULD NOT alter the bodybe done; encapsulation of the article in any way (including any change of Content-Transfer-Encoding). It MAY (except when reinjecting) add other header fields not already provided by the poster, butSHOULD NOT alter, delete, or reorder any existing header field, with the specific exception ofbe used instead where this is necessary. Safe bidirectional gatewaying between a mailing list and a newsgroup is far easier if the "tracing" header field Injection-Info, whichnewsgroup is moderated. Posts to be removed as already mentioned. 8. Ifthe Newsgroups header field contains one or moremoderated groupsgroup and submissions to the articlemailing list can then go through a single point that does NOT contain an Approved header field,the injecting agent MUST forward itnecessary gatewaying and then sends the message out to a moderator as specified in section 6.2.3 below. 9. Otherwise, a Path header field with a <tail-entry> (F-3.1.5) MUST be correctly added if not already present. During reinjection,both the existing Path header field SHOULD be retained. 10.It MUST then prependnewsgroup and the <path-identity>mailing list at the same time, eliminating most of the injecting agent, followed by '!.'possibility of loops. Bidirectional gatewaying between a mailing list and the <diag-keyword> "POSTED",an unmoderated newsgroup, in contrast, is difficult to do correctly and then a further "!",is far more fragile. Newsgroups intended to the content of the Path header field; this header fieldbe bidirectionally gated to a mailing list SHOULD thentherefore be foldedmoderated where possible, even if it would otherwise result inthe moderator is a header line of excessive length. NOTE: This could result in moresimple gateway and injecting agent that one "POSTED" <path-keyword> in the casecorrectly handles crossposting to other moderated groups and otherwise passes all traffic. 3.9.1. Duties of reinjection. 11.An Injection-Info header field (F-3.2.8) SHOULD be added, identifyingan Outgoing Gateway From the verified sourceperspective of the article and possiblyNetnews, an address for mailing complaints to. Each injecting agent SHOULD useoutgoing gateway is just a consistent formspecial type of reading agent. The exact nature of what the Injection-Info header field for alloutgoing gateway will need to do to articles emanating fromdepends on the same or similar origins. NOTE:medium to which the articles are being gated. The step above isoperation of the only place in which an Injection- Info header fieldoutgoing gateway is subject to be created. It follows that this header field MUST NOT be created, replaced, changed or deleted by any other agent (except during reinjection, in which case it will always relateadditional constraints due to the latest injection and is, to that extent, regarded as a variant header field). 12.The injecting agent MUST then add an Injection-Date header field (F-3.2.7) if one is not already present, but it MUST NOT alter, or delete, an already present Injection-Date header field (and likewise SHOULD NOT alter, or delete, an already present NNTP- Posting-Date header field). Finally, it forwards the article topossibility of one or more relaying or storage agents, andcorresponding incoming gateways back from that medium to Netnews, since this raises the injection processdanger of loops. The following practices are encouraged for all outgoing gateways, regardless of whether there is known to be considered complete. News Article Architecturea related incoming gateway, both as precautionary measures and Protocols November 2006 NOTE:as a guideline to quality of implementation: 1. The step above ismessage identifier of the only place where an Injection-Date header field is to be created It follows that it MUST NOT subsequently be replaced, changed or deleted by any other agent, even during reinjection. 6.2.3. Procedure for Forwarding to a Moderator An injecting agent forwards annews article to a moderatorshould be preserved if at all possible, preferably as follows: 1. It MUST forward it toor within the moderatorcorresponding unique identifier of the first (leftmost) moderated group listedother medium, but if not at least as a comment in the Newsgroups header field, customarily via email, (see 6.8 for how that moderator may forward it to further moderators). There are two possibilities for doing this: (a)message. This helps greatly with preventing loops. 2. The complete article is encapsulated (header fieldsDate and all) within the email, preferably usingInjection-Date of the Content-Type "application/news-transmission" (4.1) with any usage parameter set to "moderate". Moreover, there SHOULD NOT be more than one encapsulatednews article within the one email. This method has the advantage of removing any possible conflict between Netnews and Email header fields, or of changes to those fields during transport through email. (b) The article is sent as an email as it stands, with the addition of such extra header fields (e.g. a To header field) as are necessaryshould also be preserved if possible, for an email.similar reasons. 3. The existing Message-ID header field SHOULDmessage should be retained. Although both of these methods have seen usetagged in the past, the preponderance of current usage on Usenet has been for method (b) and many moderators are ill-preparedsome way so as to deal with method (a). Therefore, method (a) SHOULD NOTprevent its reinjection into Netnews. This may be used until such time as the majorityimpossible to do without knowledge of moderators are ablepotential incoming gateways, but it is better to accept it. 2. This standard doestry to provide some indication even if not prescribe howsuccessful; at the email address ofleast, a human-readable indication that the moderator is toarticle should not be determined, that beinggated back to Netnews can help locate a matter of policyhuman problem. 4. Netnews control messages should not be gated to another medium unless they would somehow be arranged by the agency responsible formeaningful in that medium. 3.9.2. Duties of an Incoming Gateway The incoming gateway has the oversightresponsibility of ensuring that all of each hierarchy. Nevertheless, there do exist various agents worldwide which providethe servicerequirements of forwarding to moderators, andthis protocol are met by the addressarticles that it forms. In addition to use with them is obtainedits special duties as follows: (a) Each '.' in the <newsgroup-name> is replaced witha '-'. (b) The result of these operations is used as the <local-part>gateway, it bears all of the <mailbox>duties and responsibilities of a posting agent as well, and additionally has the agent. For example, articles intended for "news.announce.important" would be emailed to "news- firstname.lastname@example.org". 6.3. Dutiessame responsibility of a Relaying Agent A Relaying Agent accepts injected articles from injecting and other relaying agents and passes them on torelaying or storage agents accordingagent to mutually agreed policy. Relaying agents SHOULD acceptreject articles ONLY from verified sources. News Article Architecture and Protocols November 2006that it has already gatewayed. An article SHOULDincoming gateway MUST NOT be relayed unlessgate the sending agent has been configuredsame message twice. It may not be possible to supply andensure this in the receiving agent to receive at least oneface of mangling or modification of the <newsgroup-name>s in its Newsgroups header field andmessage, but at the very least onea gateway, when given a copy of the <dist-name>s in its Distributiona message it has already gated identical except for trace header field, if any. Exceptionally, ALL relaying agents are deemed willing to supply or accept the <dist-name> "world", and NO relaying agent should supplyfields (like Received in Email or accept the <dist-name> "local". However, if the particular implementation does not relay non-existent newsgroups, even when includedPath in Netnews) MUST NOT gate the Newsgroups header field and implied (e.g.message again. An incoming gateway SHOULD take precautions against having this rule bypassed by some "wild card" notation) inmodifications of the message that can be anticipated. News articles prepared by gateways MUST be valid news proto-articles (see Section 3.3.1). This often requires the configuration tables,gateway to synthesize a conforming article from non-conforming input. The gateway MUST then pass the agentarticle to an injecting agent, not directly to a relaying agent. Incoming gateways MUST examine all groupNOT pass control messages (5.2) in order to ensure(articles containing a Control or Supersedes header field) without removing or renaming that relaying of thoseheader field. Gateways MAY, however, generate cancel control messages proceeds normally. NOTE: Althoughfor messages they have gatewayed. If a gateway receives a message that it would seem redundant to filter out unwanted distributions at both endscan determine is a valid equivalent of a relaying link (andcancel control message in the medium it is clearly more efficient to do so at the sending end), many sending sites have been reluctant, historically speaking, to apply such filters (except to ensure that distributions local to their own site or cooperating subnet did not escape); moreover they tended to configure their filters on an "all but those listed" basis, sogatewaying, it SHOULD discard that newmessage without gatewaying it, generate a corresponding cancel control message of its own, and hithertoinject that cancel control message. NOTE: It is not unheard of distributions would not be caught. Indeed many "hub" sites actually wantedfor mail-to-news gateways to receive all possible distributions so that they could feed onbe used to post control messages, but encapsulation should be used for these cases instead. Gateways by their clientsvery nature are particularly prone to loops. Spews of normal articles are bad enough; spews of control messages with special significance to the news system, possibly resulting in all possible geographical (or organizational) regions. Therefore, ithigh processing load or even email sent for every message received, are catastrophic. It is desirablefar preferable to provide facilitiesconstruct a system specifically for rejecting unwanted distributions at the receiving end. Indeed, it may be simpler toposting control messages that can do so locally than to inform each sending siteappropriate consistency checks and authentication of what is required, especially inthe caseoriginator of specialized distributions (for example for control messages, such as cancels from certain issuers) which might need to be added at short notice. A similar possibility for reading agents to filter distributions is also suggested in [USEAGE]) for the same reason. In order to avoid unnecessary relaying, an article SHOULD NOT be relayed ifthe <path-identity> of the receiving agent (or some known alias thereof) appears asmessage. If there is a <path-identity> in its Path header field. But notemessage identifier that the <tail-entry> (which follows the last "!") is notfills a <path-identity>, although not all current implementations observe this distinction. For this to work, each relaying agent needs to insert it own <path- identity> (chosen accordingrole similar to 2.3) intothat of the PathMessage-ID header field. It MAY insert more than one <path-identity> for itself (in which case the leftmost should be the one by whichfield in news, it is known to its immediate neighbours), but where an article passes through several relaying agents atSHOULD be used in the same site it MAY omitformation of the <path-identity>s for somemessage identifier of them (but NOTthe one which finally relays itnews article, perhaps with transformations required to an outside site). News Article Architecture and Protocols November 2006 It MAY (and usually SHOULD) also add a <path-diagnostic> giving additional information concerningmeet the route taken byuniqueness requirement of Netnews and with the article throughremoval of any comments so as to comply with the network. A <path-diagnostic> consistssyntax in Section 3.1.3 of either[USEFOR]. Such transformations SHOULD be designed so that two messages with the special <diag-match> "!" (which effectively replacessame identifier generate the standard delimiter "!" by "!!"), or it is composed from a <diag-keyword> usually followed bysame Message-ID header field. NOTE: Message identifiers play a <diag-identity>. The following are the only <diag-keyword>s defined by this standard: o "POSTED" (already introducedcentral role in Step 10the prevention of 6.2.2), which is never followedduplicates, and their correct use by a <diag-identity>; o "SEEN", whose following <diag-identity> indicates the verified identity (see 6) of the agent from which the article was received (but makes no claim asgateways will do much to whether it matched the <path-identity) inserted byprevent loops. Netnews does, however, require that agent); o "MISMATCH", whose following <diag-identity> indicates the verified identity of the agent from which the article was receivedmessage identifiers be unique, and asserts that it couldtherefore message identifiers from other media may not be reconciled with the <path-identity> (supposedly) inserted by that agent. Other <diag-keyword>s beginning with "X" MAYsuitable for use without modification. A balance must be usedstruck by a relaying agent to make some assertion not envisaged here, but other (non-"X") <diag-keyword>s MUST NOT bethe gateway between preserving information used unless defined by some extensionto this standard. NOTE: The <diag-keyword> "MATCH", which might have indicated the verified identityprevent loops and generating unique message identifiers. Exceptionally, if there are multiple incoming gateways for a particular set of the source agent with an assertionmessages, each to a different newsgroup(s), each one SHOULD generate a message identifier unique to that it agreed with the <path-identity> inserted bygateway. Each incoming gateway nonetheless MUST ensure that agent, has NOT been provided, sinceit does not gate the special <diag-match> conveys exactly that meaning for this commonly occurring case.same message twice. NOTE: Whilst <diag-keywords>s are case insensitive, it is intended that they should normally be rendered in uppercase. A relaying agent processes articles as follows: 1. It MUST/SHOULD establishConsider the verified identityexample of the sourcetwo gateways of the article and compare it with the leftmost <path-identity>a given mailing list into two separate Usenet newsgroups, both of which preserve the existing Path header field's content. Except possibly when relaying to other hosts on the same site, Itemail message identifier. Each newsgroup may then MUST or SHOULD, as indicated, prepend to that content (from left to right) the following: o (MUST) its own <path-identity>; o (MUST) a "!" delimiter; o (MUST/SHOULD) if the verified and existing identities match,receive a <diag-match> (effectively convertingportion of the "!" delimiter into "!!"); o (MUST/SHOULD) alternatively,messages (different sites seeing different portions). In these cases, where the identities do not match (or havethere is no one "official" gateway, some other method of generating message identifiers has to be used to avoid collisions. It would obviously be preferable for there to be only one gateway which crossposts, but this may not been determiedbe possible to match), a ".", the <diag-keyword> "MISMATCH" (or "SEEN"), another ".", a <diag- identity> indicatingcoordinate. If no date information is available, the verified identity, and finallygateway MAY supply a further "!"; o possibly further <path-identity>s etc. as above, identifying itself. ThisDate header field with the gateway's current date. If only partial information is available (such as date but not time), this SHOULD thenbe folded iffleshed out to a full Date by adding default values rather than discarding this information. Only in very exceptional circumstances should Date information be discarded, as it would otherwise resultplays an important role in preventing reinjection of old messages. An incoming gateway MUST add a Sender header line of excessive length. News Article Architecture and Protocols November 2006 [The "MUST/SHOULD"s above were all "MUST" in the previous drafts. Discussion is needed to resolve this.] NOTE: Since each agent at one site can be assumedfield to be aware ofthe identitynews article it forms containing the <mailbox> of the others (andadministrator of itself), it wouldthe gateway. Problems with the gateway may be most unusual for their <path-identity>sreported to be separated other than by "!!". Thusthis <mailbox>. The <display-name> portion of this <mailbox> SHOULD indicate that the presenceentity responsible for injection of the message is a single "!", unless followed bygateway. If the original message already had a "." and some <diag-keyword>,Sender header field, it SHOULD be renamed so that its contents can be preserved. 3.9.3. Gateway Example To illustrate the type of precautions that should be taken as signifyingagainst loops, here is an agent that has not yet been upgraded to conformexample of the measures taken by one particular combination of mail-to-news and news-to-mail gateways designed to this standard. NOTE: Whilsthandle bidirectional gatewaying between mailing lists and unmoderated groups: 1. The news-to-mail gateway preserves the presencemessage identifier of a "MISMATCH" would normally indicate thatthe existing Path was bogusnews article in some sense, it could also indicate thatthe ralaying agent was improperly configured to recognisegenerated email message. The mail-to-news gateway likewise preserves the identities or aliases used by its neighbours. Administators of relaying agents should therefore periodically monitoremail message identifier provided that it is syntactically valid for Netnews. This allows the <path-diagnostic> being inserted so as to avoid this. NOTE: In ordernews system's built-in suppression of duplicates to prevent overloading, relaying agents should not routinely query an external entity (suchserve as a DNS-server) in order to determine a verified identity (though a local cache ofthe required information might usefully be consulted).first line of defense against loops. 2. It MUST examine the Injection-DateThe news-to-mail gateway adds an X-Gateway header field (or, if thatto all messages it generates. The mail-to-news gateway discards any incoming messages containing this header field. This is absent,robust against mailing list managers that replace the Date header field)message identifier, and rejectagainst any number of email hops, provided that the article as stale (F-3.2.7) if that predatesother message header fields are preserved. 3. The mail-to-news gateway prepends the earliest articles ofhost name from which it normally keeps record, or if it is more than 24 hours intoreceived the future (the margin MAY be less than that 24 hours). 3. It SHOULD rejectemail message to the content of the Path header field. The news-to-mail gateway refuses to gateway any articlemessage that does not include allcontains the mandatorylist server name in its Path header fields (section F-3.1). 4. It MAY rejectfield (including in the tail section). This is robust against any article whoseamount of munging of the message header fields do not have legal contents. 5. It SHOULD reject any articleby the mailing list, provided that has already been sent to it (a database of message identifiers of recent messagesthe email only goes through one hop. 4. The mail-to-news gateway is usually kept and matched against). NOTE: Even though commonly derived fromdesigned never to generate bounces to the domain nameenvelope sender. Instead, articles that are rejected by the news server (for reasons not warranting silent discarding of the originating site (and domain names are case-insensitive), a <msg-id-core> MUST NOT be alteredmessage) result in any way during transport, or when copied (as when forming a References header field), and thusa simple (case-sensitive) comparison of octets will always suffice to recognize that samebounce message identifier wherever it subsequently reappears. NOTE: These requirements aresent to an errors address known not to forward to any mailing lists, so that they can be contrasted with those of the un-normalized msg-ids definedhandled by [RFC 2822], which may perfectly legitimately become normalized (or vice versa) during transport or copyingthe news administrators. These precautions have proven effective in email systems. News Article Architecture and Protocols November 2006 6. It SHOULD reject any article that matches an already received cancel message (or an equivalent Supersedes header field) issued by its poster or by some other trusted entity. 7. It MAY reject any article without an Approved header field posted topractice at preventing loops for this particular application (bidirectional gatewaying between mailing lists and locally distributed newsgroups knownwhere both gateways can be designed together). General gatewaying to world-wide newsgroups poses additional difficulties; one must be moderated (this practicevery wary of strange configurations, such as a newsgroup gated to a mailing list which is strongly recommended, but the information necessaryin turn gated to do so may nota different newsgroup. 4. Media Types This document defines several media types, which shall be available to all agents). 8.registered with IANA as provided for in [RFC4288]. The media type message/news, as previously registered with IANA, is hereby declared obsolete. It MAY delete any Xref header fieldwas never widely implemented, and its default treatment as application/octet-stream by agents that is present. 9. Finally,did not recognize it passeswas counter-productive. The media type message/rfc822 SHOULD be used in its place. 4.1. application/news-transmission The media type application/news-transmission is intended for the encapsulation of complete news articles on to neighbouring relaying and storage agents. Ifwhere the articleintention is rejected as being invalid, unwanted or unacceptable due to site policy, the agentthat passedthe article torecipient should then inject them into Netnews. This application type provides one of the relaying agent SHOULDmethods for mailing articles to moderators (see Section 3.4.1) and may be informed (such as via an NNTP 43x response code) that relaying failed. In orderused to prevent a large number of errorconvey messages being sentto one location, relaying agents MUST NOT inform any other external entity thatan injecting agent. This encapsulation removes the need to transform an email message into a Netnews proto-article and provides a way to send a Netnews article was not relayed UNLESS that external entity has explicitly requestedusing MIME through a transport medium that it be informeddoes not support 8bit data. The MIME media type definition of such errors. Relaying agents MUST NOT alter, deleteapplication/news-transmission is: MIME type name: application MIME subtype name: news-transmission Required parameters: none Optional parameters: One and only one of "usage=moderate", "usage=inject", or rearrange any part"usage=relay". Encoding considerations: A transfer-encoding different from that of anthe article except for header fields designated as variant (2.4). In particular o they MUST NOT create or augment a User-Agent header field in ordertransmitted MAY be supplied to identify themselves; o they MUST NOT rewrite the Newsgroups header field in any way, even ifensure correct transmission over some supposedly non-existent newsgroup is included; o they MUST NOT refold any header field (i.e. they must pass7bit transport medium. Security considerations: A news article may be a control message, which if processed could have effects on the folding as received); o they MUST NOT alterrecipient host's system beyond just storage of the Date header fieldarticle. Published specification: This specification. Body part: A complete proto-article ready for injection into Netnews or an article being relayed to another agent usage=moderate indicates the Injection-Date header field; o they MUST NOT delete any unrecognized header field whose field name is syntactically correct (whether or not itarticle is registered with IANA [RFC 3864]); o they MUST NOT change the Content-Transfer-Encoding of the body or any body part; o they MUST transmit lines of arbitrary lengthintended for a moderator, usage=inject for an injecting agent, and articles of arbitrary size. 6.3.1. Path Header Field Example Path: bar.isp.example!.SEEN.news3.foo.isp.example!foo.isp.example !!foo-server!.MISMATCH.2001:DB8:0:0:8:800:200C:417A !dubious.site.example!!old.site.example!barbaz!!baz.isp.example !.POSTED!dialup123.baz.isp.example!not-for-mail NOTE: Thatusage=relay for a relaying agent. The entity receiving the article was injected intomay only implement one type of agent, in which case the news stream by baz.isp.example, as indicated byparameter MAY be omitted. Contrary to the <diag-keyword> "POSTED" (complaintsprior registration of this media type, article batches are not permitted as a body part. Multiple messages or a message with multiple application/news-transmission parts may be addressed to email@example.com).used instead. 4.2. application/news-groupinfo The injector has chosen to record that it got it from dialup123.baz.isp.example. "not-for-mail"application/news-groupinfo media type is used in conjunction with the newgroup control message (see Section 5.2.1). Its body part contains brief information about a dummy <tail- News Article Architecturenewsgroup: the newsgroup name, its description, and Protocols November 2006 entry>, though sometimes a real userid is put there.its moderation status. The article was relayed, perhaps by UUCP, to the machine known, at least to old.site.example, as "barbaz". Barbaz relayed it to old.site.example,MIME media type definition of application/news-groupinfo is: MIME type name: application MIME subtype name: news-groupinfo Required parameters: none Optional parameters: charset, which does not yet conform to this standard (hence the single '!' delimiter). So one cannotMUST be sure that it really came from barbaz. Old.site.example relayed it toa site claiming to be dubious.site.example,charset registered for use with MIME text types and claiming (by usinghas the '!!' delimiter) to have verified that it came from old.site.example. Dubious.site.example relayed it to "foo-server" which, not being convinced that it truly came from dubious.site.example, and knowing that it in fact arrived from a host withsame syntax as the IPv6address [2001:DB8:0:0:8:800:200C:417A], insertedparameter defined for text/plain [RFC2046]. Specifies the <path-diagnostic> "!.MISMATCH.2001:DB8:0:0:8:800:200C:417A" (that ischarset of the body part. If not given, the charset defaults to say that [2001:DB8:0:0:8:800:200C:417A] was not a correct IPv6 address for dubious-site.example, but simply that that connection could not be substantiatedUS-ASCII [ASCII]. Disposition: by foo-server). "foo-server"default, inline Encoding considerations: 7bit or 8bit MUST be used to maintain compatibility. Security considerations: None. Published specification: This specification. The content of the application/news-groupinfo body part is a locally significant name withindefined as: groupinfo-body = [ newsgroups-tag CRLF ] newsgroups-line CRLF newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP %x6E.184.108.40.206.72.6F.75.70.73 SP %x66.69.6C.65.3A ; case sensitive ; "For your newsgroups file:" newsgroups-line = newsgroup-name [ 1*HTAB newsgroup-description ] [ 1*WSP moderation-flag ] newsgroup-description = eightbit-utext *( *WSP eightbit-utext ) moderation-flag = %x28.4D.6F.220.127.116.11.18.104.22.168 ; case sensitive "(Moderated)" eightbit-utext = utext / %d127-255 This unusual format is backward-compatible with the complex sitescanning of many machines run by foo.isp.example, sothe latter should have no problem recognizing foo-server and using a '!!' delimiter. It was not strictly necessary to insert the <path- identity> "foo-server" as well as "foo.isp.example" (but maybe some site elsewhere had some reason to testbody of newgroup control messages for it). [Please could Richard Clayton provide a more plausible reason why "foo- server" might be a <path-nodot> here?] It then went to bar.isp.example which determined (by reverse DNS) that it had come from news3.foo.isp.example, but had taken no steps to check whetherdescriptions done by Netnews implementations that was a known alias for foo.isp.example (which it probably was). Strictly, it SHOULD have made such a check, and then inserted either a "!!" or a "!MISMATCH..." according topredate this specification. Although optional, the result. Presumably bar.isp.example then delivered<newsgroups-tag> SHOULD be included for backward compatibility. The <newsgroup-description> MUST NOT contain any occurrence of the article to its direct clients. It appears that foo.isp.example, foo-server and baz.isp.example decided to foldstring "(Moderated)" within it. Moderated newsgroups MUST be marked by appending the line, oncase sensitive text " (Moderated)" at the grounds that it seemed to be gettingend. While a little too long. Note that folding and whitespacecharset parameter is permitted before (but not after) any "!" (butdefined for this media type, most existing software does not withinunderstand MIME header fields or correctly handle descriptions in a "!!"); hence continuation lines will always startvariety of charsets. Using a charset of US- ASCII where possible is therefore RECOMMENDED; if not possible, UTF-8 [RFC3629] SHOULD be used. Regardless of the charset used, the constraints of the above grammar MUST be met and the <newsgroup-name> MUST be represented using the same octets as would be used with either "!" or "!!". 6.4. Dutiesa charset of US-ASCII. 4.3. application/news-checkgroups The application/news-checkgroups media type contains a Serving Agent A Serving Agent takes an article fromlist of newsgroups within a relayinghierarchy or injecting agenthierarchies, including their descriptions and files it in a "news database".moderation status. It also provides an interfaceis primarily for reading agents to accessuse with the news database. This database is normally indexed by newsgroupcheckgroups control message (see Section 5.2.3). The MIME media type definition of application/news-checkgroups is: MIME type name: application MIME subtype name: news-checkgroups Required parameters: none Optional parameters: charset, which MUST be a charset registered for use with articles in each newsgroup identified by an News Article ArchitectureMIME text types and Protocols November 2006 <article-locator> (usually inhas the formsame syntax as the parameter defined for text/plain [RFC2046]. Specifies the charset of a decimal number - see F- 3.2.14). A storage agentthe body part. If not given, the charset defaults to US-ASCII [ASCII]. Disposition: by default, inline Encoding considerations: 7bit or 8bit MUST be used to maintain compatibility. Security considerations: This media type provides only a means for conveying a list of thenewsgroups it stores in its news database showing the moderation status of each one (see 5.2.1),and SHOULD include in that list all groups likely to be crossposteddoes not provide any information indicating whether the sender is authorized to from those groups (e.g. allstate which newsgroups should exist within a hierarchy. Such authorization must be accomplished by other groups inmeans. Published specification: This specification. The content of the application/news-groupinfo body part is defined as: checkgroups-body = *( valid-group CRLF ) valid-group = newsgroups-line ; see 4.2 The same hierarchy(ies)). NOTE: Since control messages are often of interest, but should not be displayedrestrictions on charset, <newsgroup-name>, and <newsgroup- description> apply for this media type as normal articles in regular newsgroups, itfor application/ news-groupinfo. One application/news-checkgroups message may contain information for one or more hierarchies and is commonconsidered complete for storage agents to make them available inany hierarchy for which it contains a pseudo-<valid-group>. In other words, an application/news-checkgroups body part consisting of: example.moderated A moderated newsgroup named "control" or in(Moderated) example.test An unmoderated test group is a pseudo-newsgroup instatement that the example.* hierarchy contains two newsgroups, example.moderated and example.test, and no others. This media type therefore MUST NOT be used for conveying partial information about a sub-hierarchy; if a group from a given hierarchy under "control." (e.g. "control.cancel").is present, all groups that exist in that hierarchy MUST be listed. 5. Control Messages A storage agent MAY decline to acceptcontrol message is an article if the Pathwhich contains a Control header field containsand thereby indicates that some <path-identity> whose articles the storageaction should be taken by an agent does not want, asother than distribution and display. Any article containing a matterControl header field (defined in Section 3.2.3 of local policy. NOTE: This last facility[USEFOR]) is sometimes used to detect and declinea control messages (notably cancel messages) which have been deliberately seeded withmessage. Additionally, the action of an article containing a <path-identity> to be "aliased out" by sites not wishing to act upon them. [INN at least does this. It might be argued that itSupersedes header field is described here; while such an article is not necessary to mentiona control message, it here.] A storage agent processes articles as follows: 1. It MUST/SHOULD establish the verified identity ofspecifies an action similar to the sourcecancel control message. The <control-command> of the article and modify the Path header field as fora relaying agent (6.3). 2. It MUST examine the Injection-DateControl header field (or, if that is absent,comprises a <verb>, which indicates the Date header field)action to be taken, and rejectone or more <argument> values, which supply the article as stale (F-3.2.7) if that predatesdetails. For some control messages, the earliest articlesbody of which it normally keeps record, or if it is more than 24 hours intothe futurearticle is also significant. Each recognized <verb> (the margincontrol message type) is described in a separate section below. Agents MAY be lessaccept other control message types than that 24 hours). 3. Itthose specified below, and MUST reject any article that does not include all the mandatory header fields (section F-3.1),either ignore or which contains any header field that does not have legal contents. 4. It SHOULDreject any article that has already been sent to it (a database of message identifierscontrol messages with unrecognized types. Syntactic definitions of recent articles is usually keptvalid <argument> values and matched against). 5. It SHOULD reject any article that matches an already received cancelrestrictions on control message (or an equivalent Supersedes header field) issued by its poster or by some other trusted entity. Likewise, a newly received cancelbodies are given in the section for each control message (or equivalent Supersedes) should cause immediate deletion (or deactivation) oftype. Contrary to [RFC1036], the canceled article. News Article Architecture and Protocols November 2006 NOTE: An article withpresence of a SupersedesSubject header field is itself stored normally. 6. Itstarting with the string "cmsg " MUST reject any article withoutNOT cause an Approved header field postedarticle to any newsgroup listedinterpreted as moderated. 7. It MUST (exept when specially configured to preserve the <article-locator>s set by the sending site) remove any Xrefa control message. Agents MAY reject an article with no Control header field (F-3.2.14) from each article. It then MAY (and usually will) generateand such a fresh XrefSubject header field. 8. Finally, it storesfield as ambiguous. Likewise, the article in its news database. Serving agents MUST NOT create new newsgroups simply because an unrecognizedpresence of a <newsgroup-name> occursending in a".ctl" in the Newsgroups header field (see 5.2.1 foror the correct methodpresence of newsgroup creation). Serving agentsan Also-Control header field MUST NOT alter, delete or rearrange any part of ancause the article in any other way. The list of particular cases given for relaying agents (6.3) applies here also. 6.5. Duties of a Posting Agent A Posting Agent is usedto assist the poster in creatingbe interpreted as a valid proto-articlecontrol message. 5.1. Authentication and forwarding it toAuthorization Control messages specify actions above and beyond the normal processing of an injecting agent. Postings agents SHOULD ensure that proto-articles they createarticle and are valid according to [USEFOR]therefore potential vectors of abuse and other applicable policies. In particular, they MUST NOT create any Injection-Dateunauthorized action. There is, at present, no standardized means of authenticating the sender of a control message or Injection-Info header field. Contrary to [RFC 2822], which impliesverifying that the mailbox(es) in the From header field should be thatcontents of the poster(s),a poster who does not, for whatever reason, wish to use his own mailbox MAY use any mailbox ending in the top level domain ".invalid" [RFC 2606]. Posting agents meant for usecontrol message were sent by ordinary postersthe claimed sender. There are, however, some unstandardized authentication mechanisms in common use. Agents acting on control messages SHOULD reject any attempttake steps to postauthenticate control messages before acting on them, as determined by local authorization policy. Whether this is done via the use of an article which cancels or Supersedesunstandardized authentication protocol, by comparison with information obtained through another articleprotocol, by human review, or by some other means is left unspecified by this document. Further extensions or revisions of this protocol are expected to standardize a digital signature mechanism. Agents are expected to have their own local authorization policies for which the postercontrol messages will be honored. No Netnews agent is notever required to act on any control message. The following descriptions specify the author or sender. 6.6. Duties ofactions that a Followup Agentcontrol message requests, but an agent MAY always decline to act on any given control message. 5.2. Group Control Messages A Followup Agentgroup control message is a special caseany control message type that requests some update to the list of newsgroups known to a posting agent,news server. The standard group control message types are "newgroup", "rmgroup", and as such is bound by all the posting agent's requirements. Followup agents"checkgroups". Before honoring any group control message, an agent MUST create valid followupscheck the newsgroup or newsgroups affected by that control message and are subjectdecline to special requirements involvingcreate any newsgroups not in conformance with the Newsgroups, Subject, Distribution and References header fields. Whereverrestrictions in Section 3.1.4 of [USEFOR]. All of the following it is stated that, "by default", agroup control messages MUST have an Approved header field is to be "inherited" from one(Section 3.2.1 of those header fields in the precursor, it means that its initial (semantic) content is to be a copy of the content of that precursor header field. However, posters MAY then override that default before posting if they so wish. News Article Architecture and Protocols November 2006 NOTE: The Keywords header field is not inheritable, though some older user agents treated it as such. 1. The Newsgroups[USEFOR]). Group control messages without an Approved header field (F-3.1.4)SHOULD by defaultNOT be inherited from the precursor's Followup-To header field if present,honored. Group control messages affecting specific groups (newgroup and otherwise fromrmgroup control messages, for example) SHOULD include the precursor's<newsgroup- name> for the group or groups affected in their Newsgroups header field. However, ifOther newsgroups MAY be included in the content of that Followup-ToNewsgroups header field consists of "poster" (andso that the user does not override it), thencontrol message will reach more news servers, but due to the followup MUST NOT be posted but, rather,special relaying rules for group control messages (see Section 3.5) this is tonormally unnecessary and may be emailed toexcessive. 5.2.1. The newgroup Control Message The newgroup control message requests the precursor's poster. 2.specified group be created or, if already existing, have its moderation status or description changed. The Subjectsyntax of its Control header field is: control-command =/ Newgroup-command Newgroup-command = "newgroup" Newgroup-arguments Newgroup-arguments = 1*WSP newsgroup-name [ 1*WSP newgroup-flag ] newgroup-flag = "moderated" If the request is honored, the moderation status of the group SHOULD by defaultbe inherited from thatset in accordance with the presence or absence of the precursor. The case sensitive string "Re: "<newgroup- flag> "moderated". "moderated" is the only flag defined by this protocol. Other flags MAY be prependeddefined by extensions to this protocol and accepted by agents. If an agent does not recognize the content<newgroup-flag> of its Subject header field, unlessa newgroup control message, it already begins withSHOULD ignore that string. 3.control message. The Distribution header field (F-3.2.4)body of a newgroup message SHOULD by default be inherited from the precursor's Distribution header field, if any. 4. The followup MUST (in accordance withcontain an entity of type application/news-groupinfo specifying the definitiondescription of that term - F-1.5) havethe newsgroup, either as the entire body or as an entity within a References header field referring to its precursor, constructed in accordance with section 6.6.1 below. NOTE: That "MUST"multipart/mixed object [RFC2046]. If such an entity is to be contrasted withpresent, the weaker recomendation using "SHOULD" applied, in [RFC 2822], tomoderation status specified therein MUST match the generation of "replies" in email. Moreover, in Netnews, there is no expectation of any In-Reply-To header field in a followup. 6.6.1. Construction ofmoderation status specified by the References header field<newgroup-flag>. The following procedure is to be used whenever some previous article (the "parent") is to be referred to in the References header field (F-3.2.10)body of a new article, whethernewgroup message MAY contain other entities (encapsulated in multipart/mixed) that provide additional information about the course of generating a followupnewsgroup or for some other reason (e.g.the later partscircumstances of a multipart posting such as a FAQ, orthe later parts of a message/partial as suggested in [RFC 2046]). The (semantic) content (2.1) ofcontrol message. In the new article's References header field consistsabsence of an application/news-groupinfo entity, a news server MAY search the contentbody of the Message-ID header field ofmessage for the parent preceded, ifline "For your newsgroups file:" and take the parent hadfollowing line as a References header field, by<newsgroups-line>. Prior to the contentstandardization of that References header field andapplication/news-groupinfo, this was the convention for providing a SP (subject to trimming as described below).newsgroup description. If the resulting References header field would, after unfolding, exceed 998 characters in length (including its field name but notrequest is honored and contains a newsgroup description, and if the final CRLF), it MUST be trimmed (and otherwisenews server honoring it MAYstores newsgroup descriptions, the stored newsgroup description SHOULD be trimmed). Trimming involves removing any number of message identifiers from its content, except thatupdated to the first message identifier anddescription specified in the last two MUST NOT be removed. NOTE: There iscontrol message, even if no provision in this standard for an article to have more than one parent. The essentialother property of the References header field, guaranteed bygroup has changed. 22.214.171.124. newgroup Control Message Example A newgroup control message requesting creation of the procedure above and to be preserved in any future extension,moderated newsgroup example.admin.info. From: "example.* Administrator" <firstname.lastname@example.org> Newsgroups: example.admin.info Date: 27 Feb 2002 12:50:22 +0200 Subject: cmsg newgroup example.admin.info moderated Approved: email@example.com Control: newgroup example.admin.info moderated Message-ID: <firstname.lastname@example.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nxtprt" Content-Transfer-Encoding: 8bit This is that no article can News Article Architecture and Protocols November 2006 ever precede one of its own parents. 6.7. Duties of a Reading Agent A reading agent downloads articles froma storage agent, as directed by the reader, and displays them toMIME control message. --nxtprt Content-Type: application/news-groupinfo For your newsgroups file: example.admin.info About the reader (or processes themexample.* groups (Moderated) --nxtprt Content-Type: text/plain A moderated newsgroup for announcements about new newsgroups in some other manner). It SHOULD also havethe capability to showexample.* hierarchy. --nxtprt-- 5.2.2. The rmgroup Control Message The rmgroup control message requests the raw article exactly as received. It MAY present listsspecified group be removed from a news server's list of articles available for display, and MAY structure those lists so as to show the relationships between the articles, as determined by the References, Subject, Date and othervalid groups. The syntax of its Control header fields (see [USEAGE] for some usual methodsfield is: control-command =/ Rmgroup-command Rmgroup-command = "rmgroup" Rmgroup-arguments Rmgroup-arguments = 1*WSP newsgroup-name The body of doing this). [This whole section may yet get omitted] 6.8. Dutiesthe control message MAY contain anything, usually an explanatory text. 5.2.3. The checkgroups Control Message The checkgroups control message contains a list of all the valid groups in a hierarchy with descriptions and moderation status. It requests a Moderator A Moderator receivesnews articles, customarily by email, decides whetherserver update its valid newsgroup list for that hierarchy to approve them and, if so, either injects them intoinclude the news stream or forwards themgroups specified, remove any groups not specified, and update group descriptions to further moderators. Articles will be received bymatch those given in the moderator either encapsulated as an objectcheckgroups control message. The syntax of Content-Type application/news-transmission (or possibly encapsulated but without an explicit Content-Typeits Control header field), or else directlyfield is: control-command =/ Checkgroup-command Checkgroup-command = "checkgroups" Checkgroup-arguments Checkgroup-arguments= [ chkscope ] [ chksernr ] chkscope = 1*( 1*WSP ["!"] newsgroup-name ) chksernr = 1*WSP "#" 1*DIGIT A checkgroups message is interpreted as an email already containing allexhaustive list of the header fields appropriate forvalid groups in all hierarchies or sub-hierarchies with a Netnews article (see 6.2.2). Moderators SHOULD be prepared to accept articlesprefix listed in either format. A moderator processes an article, as submitted tothe <chkscope> argument, excluding any newsgroup that he moderates, as follows: 1. He decides, onsub-hierarchy where the basis of whatever moderation policy applies to his group, whether to approve or reject the article. He MAY do this manually, or else partially or wholly with the aid of appropriate software for whose operation he is then responsible. If the article<chkscope> argument is a cancel nessage (5.3) issuedprefixed by the poster of an earlier article, then he"!". If no <chkscope> argument is expectedgiven, it applies to cancel that earlier article (inall hierarchies for which case there is no more to be done). He MAY modify the article if that isgroup statements appear in accordance withthe applicable moderation policy (and in particular he MAY remove redundant header fields and add Comments and other informational header fields). He also needs to be aware if any change he makes tobody of the article will invalidate some authentication check provided bymessage. Since much existing software does not honor the poster or by an earlier moderator. If<chkscope> argument, the article is rejected, then it normally fails for allbody of the checkgroups control message MUST NOT contain group statements for newsgroups outside the intended scope and SHOULD contain a correct newsgroup list even for which it was intended.sub-hierarchies excluded with "!" <chkscope> terms. News servers, however, MUST honor <chkscope> as specified here. The <chksernr> argument may be any positive integer. If present, it is approved, the moderator proceedsMUST increase with every change to the following steps. 2. If the Newsgroups header field contains further moderated newsgroups for which approval has not already been given, he adds an indication (identifying both himselfnewsgroup list and MUST NOT ever decrease. If provided, news servers SHOULD remember the name<chksernr> value of the group) that he approves the article,previous checkgroups control message honored for a particular hierarchy or sub-hierarchy and then forwards itdecline to honor any subsequent checkgroups control message for the moderator of the leftmost unapproved group (which, if this News Article Architecture and Protocols November 2006 standard has been followed correctly, will generally be the next moderated group to the right of his own). There are two ways to do this: (a) He emails it to the submission address of the next moderator (see section 6.2.2 for the proper method of doing this),same hierarchy or (b) he rotates the <newsgroup-name>s insub- hierarchy with a smaller <chksernr> value. For example, the Newsgroupsfollowing Control header field to the left so thatControl: checkgroups de !de.alt #248 indicates the targeted group isbody of the leftmost moderated groupmessage will list every newsgroup in that field, and injects it again (thus causing the injecting agent to forward it tothe correct moderator). However, he MUST first ensure thatde.* hierarchy, excepting the article contains no Approved header field. NOTE: This standard doesde.alt.* sub-hierarchy, and should not prescribe how a moderator's approval is tobe indicated (thoughhonored if a future standard may do so). Possible methods include adding an Approved header field (orcheckgroups control message with a similarserial number greater than 248 was previously honored. The body of the message is an entity of type application/ news-checkgroups. It SHOULD be declared as such with appropriate MIME headers, but differently named header fieldnews servers SHOULD interpret checkgroups messages that lack the appropriate MIME headers as if method (b) is being used) listing allthe approvals made so far, or adding a separate header field for each individual approval (the header field X-Auth is sometimes usedbody were of type application/news-checkgroups for this purpose).backward compatibility. 5.3. The approval may alsocancel Control Message The cancel control message requests that a target article be confirmed with some formwithdrawn from circulation and access. The syntax of digital signature (5.1). 3. If the Newsgroupsits Control header field contains no further unapproved moderated groups, he adds an Approved header field (F-3.2.1) identifying himself and, insofar as is possible, all the other moderators who have approved the article. He thus assumes responsibility for having ensured that the article was approved by the moderators of all the moderated groups involved. 4.is: control-command =/ Cancel-command Cancel-command = "cancel" Cancel-arguments Cancel-arguments = 1*WSP msg-id The Date header field SHOULD be retained. Any Injection-Date header field already present (though there should be none) MUST be removed. Exceptionally, if it is known that the injecting agent does not yet support the Injection-Date header field andargument identifies the Date header field appearsarticle to be stale (F-3.2.7) for reasons understoodcancelled by its message identifier. The body of the moderator (e.g. delays in the moderation process) hecontrol message MAY substitute the current date. The Message-ID header field SHOULD also be retained unless it is obviously non-compliant with [USEFOR]. NOTE:contain anything, usually an explanatory text. A <msg-id-core> created by a conforming posting or injecting agent, or even by a mail user agent conforming to [RFC 2822], may reasonably be supposed to be conformant (and will, in any case, be caught by the injectingserving agent if it is not). 5. Any variant header fields (2.4) MUST be removed, exceptthat a Path header field MAY be truncatedelects to only those entries following its "POSTED" <diag-keyword>. Any Injection-Info header field (F- 3.2.8)honor a cancel message SHOULD be removed (and if not, the injecting agent will do so, as required in 6.2.2). 6. He then causesmake the article unavailable to be injected, having first observed all the duties of a posting agent. News Article Architecture and Protocols November 2006 NOTE: This standard does not prescribe how the moderator or moderation policy for each newsgroup is established; ratherreading agents (perhaps by deleting it assumes that whatever agencies are responsible for the relevant network or hierarchy (1.1) will have made appropriate arrangements in that regard. 6.9. Duties of a Gateway A Gateway transforms an article into the native message format of another medium, or translates the messages of another medium into news articles. Encapsulation of a news article into a message of MIME type application/news-transmission, or the subsequent undoing of that encapsulation, is not gatewaying, since it involves no transformation of the article. There are two basic types of gateway, the Outgoing Gateway that transforms a news article into a different type of message, and the Incoming Gateway that transforms a message from another medium into a news article and injects it into a news system. These are handled separately below. The primary diktat for a gateway is: Above all, prevent loops. Transformation of an article into another medium stands a very high chance of discarding or interfering with the protection inherent in the news system against duplicate articles. The most common problem caused by gateways is "spews", gateway loops that cause previously posted articles to be reinjected repeatedly into Usenet. To prevent this, a gateway MUST take precautions against loops, as detailed below. If bidirectional gatewaying (both an incoming and an outgoing gateway) is being set up between Netnews and some other medium, the incoming and outgoing gateways SHOULD be coordinated to avoid unintended reinjection of gated articles. Circular gatewaying (gatewaying a message into another medium and then back into Netnews) SHOULD NOT be done; encapsulation of the article SHOULD be used instead where this is necessary. A second general principal of gatewaying is that the transformations applied to the message SHOULD be as minimal as possible while still accomplishing the gatewaying. Every change made by a gateway potentially breaks a property of one of the media or loses information, and therefore only those transformations made necessary by the differences between the media should be applied. It is worth noting that safe bidirectional gatewaying between a mailing list and a newsgroup is far easier if the newsgroup is moderated. Posts to the moderated group and submissions to the mailing list can then go through a single point that does the necessary gatewaying and then sends the message out to both the newsgroup and the mailing list at the same time, eliminating most of News Article Architecture and Protocols November 2006 the possibility of loops. Bidirectional gatewaying between a mailing list and an unmoderated newsgroup, in contrast, is difficult to do correctly and is far more fragile. Newsgroups intended to be bidirectionally gated to a mailing list SHOULD therefore be moderated where possible, even if the moderator is a simple gateway and injecting agent that correctly handles crossposting to other moderated groups and otherwise passes all traffic. 6.9.1. Duties of an Outgoing Gateway From the perspective of Netnews, an outgoing gateway is just a special type of reading agent. The exact nature of what the outgoing gateway will need to do to articles depends on the medium to which the articles are being gated. The operation of the outgoing gateway is subject to additional constraints due to the possibility of one or more corresponding incoming gateways back from that medium to Netnews, since this opens the possibility of loops. In general, the following practices are recommended for all outgoing gateways, regardless of whether there is known to be a related incoming gateway, both as a precautionary measure and as a guideline to quality of implementation. 1. The message identifier of the news article should be preserved if at all possible, preferably as or within the corresponding unique identifier of the other medium, but if not at least as a comment in the message. This helps greatly with preventing loops. 2. The Date and Injection-Date of the news article should also be preserved if possible, for similar reasons. 3. The message should be tagged in some way so as to prevent its reinjection into Netnews. This may be impossible to do without knowledge of potential incoming gateways, but it is better to try to provide some indication even if not successful; at the least, a human-readable indication that the article should not be gated back to Netnews can help locate a human problem. 4. Netnews control messages should not be gated to another medium unless they would somehow be meaningful in that medium. 5. Changes MAY be made to the Content-Transfer-Encoding of some or all parts of the body, and even to the charsets specified in <encoded-word>s or in Content-Type header fields, but such changes SHOULD NOT be made unless absolutely necessary. 6.9.2. Duties of an Incoming Gateway The incoming gateway has the serious responsibility of ensuring that all of the requirements of this standard are met by the articles that it forms. In addition to its special duties as a gateway, it bears all of the duties and responsibilities of an injecting agent as well, News Article Architecture and Protocols November 2006 and additionally has the same responsibility of a relaying agent to reject articles that it has already gatewayed. An incoming gateway MUST NOT gate the same message twice. It may not be possible to ensure this in the face of mangling or modification of the message, but at the very least a gateway, when given a copy of a message it has already gated identical except for trace header fields (like Received in Email or Path in Netnews) MUST NOT gate the message again. An incoming gateway SHOULD take precautions against having this rule bypassed by modifications of the message that can be anticipated. News articles prepared by gateways MUST be legal news articles. In particular, they MUST include all of the mandatory header fields, MUST fully conform to the restrictions on those fields, and SHOULD exclude any deprecated header fields (e.g. as in [RFC 2298]). This often requires that a gateway function not only as a relaying agent, but also partly as a posting agent, aiding in the synthesis of a conforming article from non-conforming input. Incoming gateways MUST NOT pass control messages (articles containing a Control or Supersedes header field) without removing or renaming that header field. Gateways MAY, however, generate their own cancel messages, under the general allowance for injecting agents to cancel their own messages ([USEAGE]). If a gateway receives a message that it can determine is a valid equivalent of a cancel message in the medium it is gatewaying, it SHOULD discard that message without gatewaying it, generate a corresponding cancel message of its own, and inject that cancel message. Incoming gateways MUST NOT inject control messages other than cancels. Encapsulation SHOULD be used instead of gatewaying, when direct posting is not possible or desirable. NOTE: It is not unheard of for mail-to-news gateways to be used to post control messages, but encapsulation should be used for these cases instead. Gateways by their very nature are particularly prone to loops. Spews of normal articles are bad enough; spews of control messages with special significance to the news system, possibly resulting in high processing load or even email sent for every message received, are catastrophic. It is far preferable to construct a system specifically for posting control messages that can do appropriate consistency checks and authentication of the originator of the message. If there is a message identifier that fills a role similar to that of the Message-ID header field in news, it SHOULD be used in the formation of the message identifier of the news article, perhaps with transformations required to meet the uniqueness requirement of Netnews and with the removal of any comments so as to comply with the syntax in section F-3.1.3. Such transformations SHOULD be designed so that two messages with the same identifier generate the same Message-ID header field. News Article Architecture and Protocols November 2006 NOTE: Message identifiers play a central role in the prevention of duplicates, and their correct use by gateways will do much to prevent loops. Netnews does, however, require that message identifiers be unique, and therefore message identifiers from other media may not be suitable for use without modification. A balance must be struck by the gateway between preserving information used to prevent loops and generating unique message identifiers. Exceptionally, if there are multiple incoming gateways for a particular set of messages, each to a different newsgroup(s), each one SHOULD generate a message identifier unique to that gateway. Each incoming gateway nonetheless MUST ensure that it does not gate the same message twice. NOTE: Consider the example of two gateways of a given mailing list into the world-wide Usenet newsgroups, both of which preserve the email message identifier. Each newsgroup may then receive a portion of the messages (different sites seeing different portions). In these cases, where there is no one "official" gateway, some other method of generating message identifiers has to be used to avoid collisions. It would obviously be preferable for there to be only one gateway which crossposts, but this may not be possible to coordinate. If no date information is available, the gateway MAY supply a Date header field with the gateway's current date. If no injection-date information is available, the gateway MUST supply an Injection-Date header field with whatever date information is available, and otherwise with the gateway's current date. If only partial information is available (e.g. date but not time), this SHOULD be fleshed out to a full Date and/or Injection-Date header field by adding default values rather than discarding this information. Only in very exceptional circumstances should Date information be discarded, as it plays an important role in preventing reinjection of old messages. An incoming gateway MUST add a Sender header field to the news article it forms containing the <mailbox> of the administrator of the gateway. Problems with the gateway may be reported to this <mailbox>. The <display-name> portion of this <mailbox> SHOULD indicate that the entity responsible for injection of the message is a gateway. If the original message already had a Sender header field, it SHOULD be renamed so that its contents can be preserved. 6.9.3. Example To illustrate the type of precautions that should be taken against loops, here is an example of the measures taken by one particular combination of mail-to-news and news-to-mail gateways at Stanford University designed to handle bidirectional gatewaying between mailing lists and unmoderated groups. News Article Architecture and Protocols November 2006 1. The news-to-mail gateway preserves the message identifier of the news article in the generated email message. The mail-to-news gateway likewise preserves the email message identifier provided that it is syntactically valid for Netnews. This allows the news system's built-in suppression of duplicates to serve as the first line of defense against loops. 2. The news-to-mail gateway adds an X-Gateway header field to all messages it generates. The mail-to-news gateway discards any incoming messages containing this header field. This is robust against mailing list managers that replace the message identifier, and against any number of email hops, provided that the other message header fields are preserved. 3. The mail-to-news gateway prepends the host name from which it received the email message to the content of the Path header field. The news-to-mail gateway refuses to gateway any message that contains the list server name in its Path header field. This is robust against any amount of munging of the message header fields by the mailing list, provided that the email only goes through one hop. 4. The mail-to-news gateway is designed never to generate bounces to the envelope sender. Instead, articles that are rejected by the news server (for reasons not warranting silent discarding of the message) result in a bounce message sent to an errors address known not to forward to any mailing lists, so that they can be handled by the news administrators. These precautions have proven effective in practice at preventing loops for this particular application (bidirectional gatewaying between mailing lists and locally distributed newsgroups where both gateways can be designed together). General gatewaying to world-wide newsgroups poses additional difficulties; one must be very wary of strange configurations, such as a newsgroup gated to a mailing list which is in turn gated to a different newsgroup. 7. Security and Related Considerations There is no security. Don't fool yourself. Usenet is a prime example of an Internet Adhocratic-Anarchy; that is, an environment in which trust forms the basis of all agreements. It works. See also F-5 for further security considerations related to the format of articles. 7.1. Leakage Articles which are intended to have restricted distribution are dependent on the goodwill of every site receiving them. The "Archive: no" header field (F-3.2.2) is available as a signal to automated archivers not to file an article, but that cannot be guaranteed. News Article Architecture and Protocols November 2006 The Distribution header field makes provision for articles which should not be propagated beyond a cooperating subnet. The key security word here is "cooperating". When a machine is not configured properly, it may become uncooperative and tend to distribute all articles. The flooding algorithm is extremely good at finding any path by which articles can leave a subnet with supposedly restrictive boundaries, and substantial administrative effort is required to avoid this. Organizations wishing to control such leakage are strongly advised to designate a small number of official gateways to handle all news exchange with the outside world (however, making such gateways too restrictive can also encourage the setting up of unofficial paths which can be exceedingly hard to track down). The sendme control message (5.4), insofar as it is still used, can be used to request articles with a given message identifier, even one that is not supposed to be supplied to the requestor. 7.2. Attacks 7.2.1. Denial of Service The proper functioning of individual newsgroups can be disrupted by the massive posting of "noise" articles, by the repeated posting of identical or near identical articles, by posting followups unrelated to their precursors, or which quote their precursors in full with the addition of minimal extra material (especially if this process is iterated), and by crossposting to, or setting followups to, totally unrelated newsgroups. Many have argued that "spam", massively multiposted (and to a lesser extent massively crossposted) articles, usually for advertising purposes, also constitutes a DoS attack in its own regard. This may be so. Such articles intended to deny service, or other articles of an inflammatory nature, may also have their From or Reply-To addresses set to valid but incorrect email addresses, thus causing large volumes of email to descend on the true owners of those addresses. Similar effects could be caused by any email header field which could cause every reading agent receiving it to take some externally visible action. For example, the Disposition-Notification-To header field defined in [RFC 2298] could cause huge numbers of acknowledgements to be emailed to an unsuspecting third party (for which reason [RFC 2298] declares that that header field SHOULD NOT be used in Netnews). It is a violation of this standard for a poster to use as his address a <mailbox> which he is not entitled to use. Even addresses with an invalid <local-part> but a valid <domain> can cause disruption to the administrators of such domains. Posters who wish to remain anonymous or to prevent automated harvesting of their addresses, but who do not News Article Architecture and Protocols November 2006 care to take the additional precautions of using more sophisticated anonymity measures, should avoid that violation by the use of addresses ending in the ".invalid" top-level-domain (see 6.5). A malicious poster may also prevent his article being seen at a particular site by preloading that site into the Path header field (F-3.1.5) and may thus prevent the true owner of a forged From or Reply-To address from ever seeing it. A malicious complainer may submit a modified copy of an article (e.g. with an altered Injection-Info header field) to the administrator of an injecting agent in an attempt to discredit the author of that article and even to have his posting privileges removed. Administrators should therefore obtain a genuine copy of the article from their own storage agent before taking such precipitate action. Administrative agencies with responsibility for establishing policies in particular hierarchies can and should set bounds upon the behaviour that is considered acceptable within those hierarchies (for example by promulgating charters for individual newsgroups, and other codes of conduct). Whilst this standard places an onus upon injecting agents to bear responsibility for the misdemeanours of their posters (which includes non-adherence to established policies of the relevant hierarchies as provided in section 6.2), and to provide assistance to the rest of the network by making proper use of the Injection-Info (F-3.2.8) header field, it makes no provision for enforcement, which may in consequence be patchy. Nevertheless, injecting sites which persistently fail to honour their responsibilities or to comply with generally accepted standards of behaviour are likely to find themselves blacklisted, with their articles refused propagation and even subject to cancellation, and other relaying sites would be well advised to withdraw peering arrangements from them. 7.2.2. Compromise of System Integrity The posting of unauthorized (as determined by the policies of the relevant hierarchy) control messages can cause unwanted newsgroups to be created, or wanted ones removed, from storage agents. Administrators of such agents SHOULD therefore take steps to verify the authenticity of such control messages, either by manual inspection (particularly of the Approved header field) or by checking any digital signatures that may be provided (see 5.1). In addition, they SHOULD periodically compare the newsgroups carried against any regularly issued checkgroups messages, or against lists maintained by trusted servers and accessed by out-of-band protocols such as FTP or HTTP. Malicious cancel messages (5.3) can cause valid articles to be removed from storage agents. Administrators of such agents SHOULD therefore take steps to verify that they originated from the (apparent) poster, the injector or the moderator of the article, or that in other cases they came from a place that is trusted to work News Article Architecture and Protocols November 2006 within established policies and customs. Such steps SHOULD include the checking of any digital signatures, or other security devices, that may be provided (see 5.1). Articles containing Supersedes header fields (F-3.2.12) are effectively cancel messages, and SHOULD be subject to the same checks. Currently, many sites choose to ignore all cancel messages on account of the difficulty of conducting such checks. Improperly configured injecting and relaying agents can allow articles posted to moderated groups onto the net without first being approved by the moderator. Injecting agents SHOULD verify that moderated articles were received from one of the entities given in their Approved header fields and/or check any digital signatures that may be provided (see 5.1). There may be weaknesses in particular implementations that are subject to malicious exploitation. In particular, it has not been unknown for complete shell scripts to be included within Control header fields. Implementors need to be aware of this. Reading agents should be chary of acting automatically upon MIME objects with an "application" Content-Type that could change the state of that agent, except in contexts where such applications are specifically expected (as in 4). Even the Content-Type "text/html" could have unexpected side effects on account of embedded objects, especially embedded executable code or URIs that invoke non-news protocols such as HTTP [RFC 2616]. It is therefore generally recommended that reading agents do not enable the execution of such code (since it is extremely unlikely to have a valid application within Netnews) and that they only honour URIs referring to other parts of the same article. Non-printable characters embedded in article bodies may have surprising effects on printers or terminals, notably by reconfiguring them in undesirable ways which may become apparent only after the reading agent has terminated. 7.3. Liability [This whole section might be better removed to [USEAGE].] There is a presumption that a poster who sends an article to Usenet intends it to be stored on a multitude of storage agents, and has therefore given permission for it to be copied to that extent. Nevertheless, Usenet is not exempt from the Copyright laws, and it should not be assumed that permission has been given for the article to be copied outside of Usenet, nor for its permanent archiving contrary to any Archive header field that may be present. Posters also need to be aware that they are responsible if they breach Copyright, Libel, Harassment or other restrictions relating to material that they post, and that they may possibly find themselves liable for such breaches in jurisdictions far from their own. Serving agents may also be liable in some jurisdictions, especially if the News Article Architecture and Protocols November 2006 breach has been explicitly drawn to their attention. Users who are concerned about such matters should seek advice from competent legal authorities. 8. IANA Considerations IANA is requested to register the following media types, described elsewhere in this standard, for use with the Content-Type header field, in the IETF tree in accordance with the procedures set out in [RFC 4288]. application/news-transmission (4.1) application/news-groupinfo (4.3) application/news-checkgroups (4.4) IANA is also requested to change the status of the following media type to "OBSOLETE". message/news (4.2) NOTE: "Application/news-transmission" is an update, with clarification and additional optional parameters, to an existing registration. "Message/rfc822" should now be used in place of the obsoleted "message/news". nr H1 7 9. References 9.1. Normative References [ANSI X3.4] "American National Standard for Information Systems - Coded Character Sets - 7-Bit American National Standard Code for Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986. [RFC 2119] S. Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. [RFC 2606] D. Eastlake and A. Panitz, "Reserved Top Level DNS Names", RFC 2606, June 1999. [RFC 2822] P. Resnick, "Internet Message Format", RFC 2822, April 2001. [RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration procedures for message header fields", RFC 3864. [USEFOR] K. Murchison et al, "News Article Format", draft-ietf- usefor-usefor-11.txt. [USEPRO] This Standard. News Article Architecture and Protocols November 2006 9.2. Informative References [PGPVERIFY] David Lawrence, <ftp://ftp.isc.org/pub/pgpcontrol/README.html>. [RFC 1036] M. Horton and R. Adams, "Standard for Interchange of USENET Messages", RFC 1036, December 1987. [RFC 2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. [RFC 2046] N. Freed and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. [RFC 2142] D. Crocker, "Mailbox Names for Common Services, Roles and Functions", RFC 2142, May 1997. [RFC 2298] R. Fajman, "An Extensible Message Format for Message Disposition Notifications", RFC 2298, March 1998. [RFC 2616] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [RFC 2821] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer Protocol", RFC 2821, April 2001. [RFC 3977] C. Feather, "Network News Transport Protocol (NNTP)", RFC 3977. [RFC 4288] N. Freed and J. Klensin, "Media Type Specifications and Registration Procedures", RFC 4288, December 2005. [RFC 976] Mark R. Horton, "UUCP mail interchange format standard", RFC 976, February 1986. [Son-of-1036] Henry Spencer, "News article format and transmission", <ftp://ftp.zoo.toronto.edu/pub/news.txt.Z>, June 1994. [USEAGE] draft-ietf-usefor-useage-*.txt. 10. Acknowledgements As this document iscompletely). If the result of a ten year effort,cancel control message arrives before the number of people that have contributed to its content are too numerous to mention individually. Many thanks go outarticle it targets, news servers choosing to all pasthonor it SHOULD remember the message identifier that was cancelled and present members ofreject the USEFOR Working Group ofcancelled article when it arrives. To best ensure that it will be relayed to the Internet Engineering Task Force (IETF) andsame news servers as the accompanying mailing list. News Article Architecture and Protocols November 2006 11. Contact Address Editor Charles. H. Lindsey 5 Clerewood Avenue Heald Green Cheadle Cheshire SK8 3JU United Kingdom Phone: +44 161 436 6131 Email: email@example.com [ Working group chairs Alexey Melnikov <firstname.lastname@example.org> Harald Tveit Alvestrand <email@example.com> ] Comments on this draft should preferablyoriginal message, a cancel control message SHOULD have the same Newsgroups header field as the message it is cancelling. Cancel control messages listing moderated newsgroups in their Newsgroups header field MUST contain an Approved header field like any other article in a moderated newsgroup. This means that cancels posted to a moderated newsgroup will normally be sent to the mailing listmoderator first for approval. Outside of the Usenet Format Working Group at firstname.lastname@example.org. Appendix A - Obsolete Control Messages This present standard obsoletes certainmoderated newsgroups, cancel messages are not required to contain an Approved header field. Contrary to [RFC1036], cancel control messages defined in [RFC 1036] (see 5.5), allare not required to contain From and Sender header fields matching the target message. This requirement only encouraged cancel issuers to conceal their identity and provided no security. 5.4. The Supersedes Header Field The presence of which hada Supersedes header field in an article requests that the effect of requesting a description of a relaying or storage agent's software, or its peering arrangements with neighbouring sites, tomessage identifier given in that header field be emailed towithdrawn in exactly the article's reply address. Whilst of some utility when Usenet was much smaller thansame manner as if it is now, they had become no more than a tool forwere the malicious sending of mailbombs. Moreover, many organizations now consider information about their internal connectivity to be confidential. version sendsys whogets senduuname "Version" requested detailstarget of the transport software in use ata site. "Sendsys" requestedcancel control message. Accordingly, news servers SHOULD use the full list of newsgroups taken,same authentication and the peering arrangements. "Whogets" was similar, but restrictedauthorization checks for deciding whether to honor a named newsgroup. "Senduuname" resembled "sendsys" but restricted toSupersedes header field as they use for cancel control messages. If the list of peers connected by UUCP. Historically,Supersedes header field is honored, the news server SHOULD take the same actions as it would take when honoring a checkgroups body consisting of one or two lines,cancel control message for the firstgiven target article. 5.5. The ihave and sendme Control Messages The ihave and sendme control messages implement a predecessor of the form "-n newsgroup", caused check-groups to apply to only that single newsgroup.NNTP [RFC3977] protocol. They are largely obsolete on the Internet but still see use in conjunction with some transport protocols such as UUCP. News Article Architecture and Protocols November 2006 Historically, an article postedservers are not required to a newsgroup whose name had exactly three componentssupport them. ihave and sendme control messages share similar syntax for their Control header fields and bodies: control-command =/ Ihave-command Ihave-command = "ihave" Ihave-arguments Ihave-arguments = 1*WSP relayer-name / 1*( 1*WSP msg-id ) [ 1*WSP relayer-name ] control-command =/ Sendme-command Sendme-command = "sendme" Sendme-arguments Sendme-arguments = Ihave-arguments relayer-name = path-identity ; see 3.1.5 of [USEFOR] ihave-body = *( msg-id CRLF ) sendme-body = ihave-body The body of whichthe third was "ctl" signified thatarticle was to be taken asconsists of a control message.list of <msg-id>s, one per line. The Subject header field specifiedmessage identifiers SHOULD be put in the body of the actions,article, not in the same wayControl header field, but news servers MAY recognize and process message identifiers in the Control header field does now. These forms are documentedfor archaeological purposes only; theybackward compatibility. Message identifiers MUST NO LONGERNOT be used. Appendix B - Differences fromput in the ProtocolsControl header field if they are present in RFC 1036 and its derivatives This apendix contains a listthe body of changesthe control message. The ihave message states that have been madethe named relaying agent has received articles with the specified message identifiers, which may be of interest to the protocolsrelaying agents receiving the ihave message. The sendme message requests that the agent receiving it send the articles having the specified message identifiers to the named relaying agent. If <relayer-name> is not given, it is determined from the origin of the earlier standards, specifically [RFC 1036]. See F- Appendix B. forcontrol message. Upon receipt of the related syntax changes. o Theresendme message (and a decision to honor it), the receiving agent sends the article or articles requested. The mechanism for this transmission is unspecified by this document and is arranged between the sites involved. These control messages are normally sent as point-to-point articles between two sites and not then sent on to other sites. Newsgroups beginning with "to." are reserved for such point-to-point communications and are formed by prepending "to." to a new Control message 'mvgroup'<relayer-name> to facilitate movingform a <newsgroup-name>. Articles with such a group to a different place (name)in a hierarchy. o Certain Control messages (Appendix A) have been made obsolete, and the special significance of "cmsg" when at the start of a Subjecttheir Newsgroups header field has been removed (section 5). o Additional mediafields SHOULD NOT be sent to any news server other than the one identified by <relayer-name>. 5.6. Obsolete Control Messages The following control message types are defineddeclared obsolete by this document and SHOULD NOT be sent or honored: sendsys version whogets senduuname 6. Security Considerations Netnews is designed for better structuringbroad dissemination of controlpublic messages (4.3and 4.4). o Distributions are expected to be checked at the receiving end, as well asoffers little in the sending end,way of a relaying link. o There are numerous other small changes, clarificationssecurity. What protection Netnews has against abuse and enhancements. Appendix C - Transitional Arrangements Itimpersonation is provided primarily by the intention thatunderlying transport layer. In large Netnews networks where news servers cannot be relied upon to enforce authentication and authorization requirements at the new featurestransport layer, articles may be trivially forged and widely read, and the identities of [USEFOR]article senders and privacy of this standard canarticles cannot be assimilated into Usenet as it presently operates without major interruptionassured. See Section 5 of [USEFOR] for further security considerations related to the service, though someformat of the new featuresarticles. 6.1. Compromise of System Integrity Control messages pose a particular security concern since acting on unauthorized control messages may not begincause newsgroups to show benefit until they become widely implemented. An important distinction mustbe made between news servers, which are responsible for the distributionremoved, articles to be deleted, and storageunwanted newsgroups to be created. Administrators of news articles, and user agents, which are responsible for interactions with users. It is important thatservers SHOULD therefore take steps to verify the former shouldauthenticity of control messages as discussed in Section 5.1. Articles containing Supersedes header fields are effectively cancel control messages and SHOULD be upgraded to conformsubject to this standard as soonthe same checks as possiblediscussed in Section 5.4. Currently, many sites are ignoring all cancel control messages and Supersedes header fields due to providethe benefitdifficulty of authenticating them and their widespread abuse. Cancel control messages are not required to have the enhanced facilities. Fortunately, the number of distinct implementations of such servers is rather small, at least so farsame Newsgroups header field as the main "backbone" of Usenet is concerned, and many of the new featuresmessages they are already supported. Contrariwise, therecancelling and, since they are sometimes processed before the original message is received, it may not be possible to check that they do. This allows a great number of implementations of user agents, installed onmalicious poster to inject a vastly greater number of small sites. Therefore,cancel control message for an article in a moderated newsgroup without adding an Approved header field to the new functionality has been designedcontrol message, and to hide malicious cancel control messages from some reading agents by using a different Newsgroups header field so that existing user agents may continue to be used, althoughthe full benefits maycancel control message is not be realised until a substantial proportion of them have been upgraded. Inaccepted by all news servers that accepted the list which follows, care has been taken to distinguishoriginal message. All agents should be aware that all article content, most notably including the implications for both kinds of agent. News Article Architecture and Protocols November 2006 o The introductioncontent of whitespace and folding intothe Newsgroups and relatedControl header fields (F-3.1.4, F-3.2.4, F-3.2.6)field, is potentially untrusted and malicious. Articles may be constructed in syntactically invalid ways to attempt to overflow internal buffers, violate hidden assumptions, or exploit implementation weaknesses. For example, some news server implementations have been successfully attacked via inclusion of <comment>s intoUnix shell code in the ReferencesControl header field (F-3.2.10) will cause problems for existing agents,field. All article contents, and [USEFOR] provides that theyparticularly control message contents, SHOULD NOTbe generated athandled with care and rigorously verified before any action is taken on the present time. o The requirement in [USEFOR]basis of the contents of the article. A malicious poster may add an Approved header field to support MIME reflectsbypass the moderation process of a practicemoderated newsgroup. Injecting agents SHOULD verify that is already widespread. Articles in strict compliance withmessages approved for a moderated newsgroup are being injected by the previous standards (using strict US-ASCII) will be unaffected. o Allmoderator using authentication information from the header fields newly introduced by [USEFOR] can safely be ignored by existing software, albeitunderlying transport or some other authentication mechanism arranged with loss ofthe new functionality. o The new style ofmoderator. A malicious news server participating in a Netnews network may bypass checks performed by injecting agents, forge Path header field, using a <diag-match> to allow "!!"fields and other trace information (such as a <delimiter>Injection-Info header fields), and introducing <path-diagnostic>s (which can be distinguished from <path-identity>s by their leading ".") is already consistent withotherwise compromise the previous standards. However,authorization requirements of a Netnews network. News servers SHOULD use the intention is that relaying agents should eventuallyfacilities of the underlying transport to authenticate their peers and reject articles in the old style,from injecting and so this possibility should be offered as a configurable option inrelaying agents. Useragents are unaffected. othat do not follow the requirements of this protocol or the Netnews network. 6.2. Denial of Service The new Control: mvgroup command will need toproper functioning of individual newsgroups can be implementeddisrupted by the excessive posting of unwanted articles; by the repeated posting of identical or near identical articles; by posting followups unrelated to their precursors or which quote their precursors in storage agents. Forfull with the benefitaddition of older storage agents itminimal extra material (especially if this process is therefore RECOMMENDED that it be followed shortlyiterated); by a corresponding newgroup commandcrossposting to, or requesting followups to, totally unrelated newsgroups; and it MUST always be followedby a rmgroup command for the old group after a reasonable overlap period. An implementation of the mvgroup command as an alias for the newgroup command would thus be minimally conforming. User agents are unaffected. o Provision is made for relayingabusing control messages and storage agents to usethe DateSupersedes header field in the case ofto delete articles injected through existing agents which do not yet provide an Injection-Date header field. Appendix D - Notices Intellectual Property The IETF takes no position regarding the validityor scope of any Intellectual Property Rightsnewsgroups. Such articles intended to deny service, or other rights that might be claimedarticles of an inflammatory nature, may also have their From or Reply-To addresses set to pertainvalid but incorrect email addresses, thus causing large volumes of email to descend on the implementation or usetrue owners of the technology describedthose addresses. Users and agents should always be aware that identifying information in this document or the extent to which any license under such rights might or might notarticles may be available; nor does it representforged. A malicious poster may prevent an article from being seen at a particular site by including in the Path header field of the proto- article the <path-identity> of that it has made any independent effort to identify any such rights. Information onsite. Use of the procedures with respect<diag-keyword> "POSTED" by injecting agents to rights in RFC documentsmark the point of injection can prevent this attack. Primary responsibility for preventing such attacks lies with injecting agents, which can be found in BCP 78apply authentication and BCP 79. Copies of IPR disclosures made toauthorization checks via the IETF Secretariatunderlying transport and any assurancesprevent those attempting denial of licensesservice attacks from posting messages. If specific injecting agents fail to live up to their responsibilities, they may be made available, orexcluded from the resultNetnews network by configuring relaying agents to reject articles originating from them. A malicious complainer may submit a modified copy of an article (with an altered Injection-Info header field, for instance) to the administrator of an injecting agent in an attempt madeto discredit the author of that article and even to have his posting privileges removed. Administrators SHOULD therefore obtain a general license or permission for the usegenuine copy of the article from their own serving agent before taking action in response to such proprietary rights by implementers or usersa complaint. 6.3. Leakage Articles which are intended to have restricted distribution are dependent on the goodwill of this specification canevery site receiving them. Restrictions on dissemination and retention of articles may be obtained fromrequested via the IETF on-line IPR repository at http://www.ietf.org/ipr. News Article ArchitectureArchive and Protocols November 2006Distribution header fields, but such requests cannot be enforced by the protocol. The IETF invitesflooding algorithm used by Netnews transports such as NNTP [RFC3977] is extremely good at finding any interested partypath by which articles can leave a subnet with supposedly restrictive boundaries, and substantial administrative effort is required to bringavoid this. Organizations wishing to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be requiredcontrol such leakage are advised to designate a small number of gateways to implement this standard. Please addresshandle all news exchange with the informationoutside world. The sendme control message Section 5.5, insofar as it is still used, can be used to request articles the IETF at ietf- email@example.com. Full Copyright Statement Copyright (C) The IETF Trust (2006). This documentrequester should not have access to. 7. IANA Considerations IANA is subjectrequested to register the rights, licenses and restrictions containedfollowing media types, described elsewhere in BCP 78, and except as set forth therein,this document, for use with the authors retain all their rights. This document andContent-Type header field, in the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THEIETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." Appendix E - Change Log [This Appendixtree in accordance with the procedures set out in [RFC4288]. application/news-transmission (4.1) application/news-groupinfo (4.2) application/news-checkgroups (4.3) application/news-transmission is a change to be removed priora previous registration. IANA is also requested to final publication.] For version 01 1 Numerous texts describing protocol features relatedchange the status of the message/news media type to particular header fields"OBSOLETE". message/rfc822 should be used instead. 8. References 8.1. Normative References [ASCII] "American National Standard for Information Systems - Coded Character Sets - 7-Bit American National Standard Code for Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986. [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. [RFC2119] Bradner, S., "Key words for use in parts of [ARTICLE] which were destinedRFCs to become partIndicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2822] Resnick, P., "Internet Message Format", RFC 2822, April 2001. [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003. [RFC4234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 4234, October 2005. [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and Registration Procedures", BCP 13, RFC 4288, December 2005. [USEFOR] have been moved to appropriate locations within section 7 of this document. Such revised texts will be found in sections 7.2.2 Steps 4, 6, 7, 10, 11, 12; 7.2.3 Step 1(b); 7.3 introductory paragraphs, Steps 1, 4, 8, 9,Murchison, K., Ed., Lindsey, C., and some final paragraphs; 7.4 introductoryD. Kohn, "Netnews Article Format". 8.2. Informative References [RFC0976] Horton, M., "UUCP mail interchange format standard", RFC 976, February 1986. [RFC1036] Horton, M. and final paragraphs; 7.9.1 Step 5. 2 A section on "DutiesR. Adams, "Standard for interchange of a Reading Agent" (7.8) has been added. 3 Some demotions MUST -> SHOULD -> MAY, as noted in pseudo- comments, have been made or proposed in sections 7.3 7.3 Step 4. 4USENET messages", RFC 1036, December 1987. [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. [RFC2606] Eastlake, D. and A. Panitz, "Reserved Top Level DNS Names", BCP 32, RFC 2606, June 1999. [RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)", RFC 3977, October 2006. [USEAGE] Lindsey, C., "Usenet Best Practice". Appendix A. Changes to the procedure for examiningExisting Protocols This document prescribes many changes, clarifications, and new features since the protocol described in [RFC1036]. Most notably: o A new, backward-compatible Path header fields by relaying agents has been moved to storage agents, as explained in pseudo-comments in section 7.4. 5 Some renumberingfield format that permits standardized embedding of sectionsadditional trace and minor textual clarifications. For version 02 News Article Architectureauthentication information is now RECOMMENDED. See Section 3.4 and Protocols November 2006 1 2nd para. of a-7 temporarily reinstated in section 6. 2 Para. in section 6 relating to propagationSection 3.5. Folding of control messages and local policy removed to [USEAGE].] 3 Requirement for some relaying agents to examine control messages for non-existent groups 6 7.3 4 Text regarding "aliasing out" brought into line with actual practice. 7.3 5 More realistic wording regardingthe expectationsPath header is permitted. o Trimming of reading agents 7.7 7.4 6 "Precursor"the References header field is now defined for all cases in whichREQUIRED and a Referencesmechanism for doing so is defined. o Addition of the new Injection-Date header field mayis required for injecting agents (Section 3.4) and MUST be used (even though "followup"by news servers for date checks (Section 3.5). Injecting agents are strongly encouraged to put all local trace information in the new Injection-Info header field. o A new media type is not alwaysdefined under Alternative-1). 2.1 7 Provision is madefor a postertransmitting Netnews articles through other media (Section 4.1), and moderators SHOULD prepare to use a mailbox ending in ".invalid"receive submissions in that format (Section 3.4.1). o Certain control messages (Section 5.6) are declared obsolete, and the special significance of "cmsg" at the start of a FromSubject header field (formerly in a-5.2). 7.5 8 "Inheritable" and "Variant" header fields defined (formerly in a-4.2.5). 2.3 9is removed. o Additional wording regarding functionmedia types are defined for improved structuring, specification, and automated processing of verb/arguments/body incontrol messages (formerly in a-6.13). 6 10 NOTE regarding not altering message indentifiers during transport or copying added (formerly in a-5.3). 7.3 11 All mention of MIME-style parameters(Section 4.2 and extension-parameters removed. 3.1 7.6 For version 03 1Section 4.3). o Two new optional parameters are added to the checkgroups control message. o The term "inheritable header"message/news media type is declared obsolete. o Cancel control messages are no longer defined. Instead,required to have From and Sender header fields matching those of the term "inherited' is usedtarget message, as this requirement added no real security. In addition, many protocol steps and article verification requirements unmentioned or left ambiguous by [RFC1036] but widely implemented by Netnews servers have been standardized and specified in placedetail. Appendix B. Acknowledgements This document is the result of a ten year effort and the number of people that have contributed to its content are too numerous to mention individually. Many thanks go out to all past and present members of "taken" when definingthe actionsUSEFOR Working Group of a followup agent. 7.6 News Article Architecturethe Internet Engineering Task Force (IETF) and Protocols November 2006 2 Consequent changesthe accompanying mailing list. Special thanks are due to "variant header field", and also mention of Injection-InfoHenry Spencer, whose Son-of-1036 draft served as sometimes variant. 2.3 3the initial basis for this document. Authors' Addresses Russ Allbery (editor) Stanford University P.O. Box 20066 Stanford, CA 94309 US Email: firstname.lastname@example.org URI: http://www.eyrie.org/~eagle/ Charles H. Lindsey University of Manchester 5 Clerewood Avenue Heald Green Cheadle Cheshire SK8 3JU United Kingdom Phone: +44 161 436 6131 Email: email@example.com Full Copyright Statement Copyright (C) The term "reply address"Internet Society (2007). This document is no longer defined. 4 References now made to sections within USEFOR using "F-..." notation. 5 Cross-referencessubject to sections within USEFOR added. Consistent use of <...> aroundthe rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all mentions of syntactic objects. All occurrencestheir rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of "Foobar-header" changed to "Foobar header". Manyany Intellectual Property Rights or other minor textual changes. 6 <control-message> changedrights that might be claimed to <control-command>,pertain to avoid confusion with "control message", which signifiesthe complete article containingimplementation or use of the <control-command>. 7 <ihave-arguments>technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has been changedmade any independent effort to <ihave-argument> (since the earlier practice of multiple arguments is now deprecated). Likewise <sendme-argument>. For version 04 1 References divided into Normative and Informational. 2 All mention ofidentify any such rights. Information on the Mail-Copies-To, Posted-And-Mailed and Complaints-To header fields removed. 3 NOTE added to contrast MUST for References header fieldprocedures with SHOULDrespect to rights in RFC 2822. 7.6 4 Changes arising from the new syntax of <path-delimiter>sdocuments can be found in BCP 78 and <path-keyword>s. 7.3 5 ChangesBCP 79. Copies of IPR disclosures made to clarifythe constructionIETF Secretariat and any assurances of the References header field. 7.6.1 6 Changes duelicenses to removal of <comment>s from further header fields. 7 New section on Identification of news-servers describing acceptable forms for <path-identity>s. 2.3 8 Definition of "semantic content"be made available, or the result of an attempt made to obtain a header field. 2.1 9 Systematic replacement of "header" by "header field". News Article Architecture and Protocols November 2006 10 More stringent rules for checking <newsgroup-name>s in control messagesgeneral license or permission for compliance with USEFOR. 6.2 For version 05 1 Historical Appendices A.1 and A.2 removed, in anticipation of republication of Son-of-1036 as an Informational RFC. 1.3 2 Definitions of technical terms adopted from USEFOR rather than defining them separately. 3 Discussionthe use of <path-identity> rewritten to reflect recent developments (but still awaiting further work in USEFOR). 2.3 4 Items now included in Appendix Asuch proprietary rights by implementers or users of USEFOR have been removedthis specification can be obtained from section 3, andthe "Transitional Arrangements" (which stillIETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the USEFOR changes) have been modifiedinformation to reflect this. For version 06 1 Proceduresthe IETF at firstname.lastname@example.org. Acknowledgment Funding for constructingthe Path header field updated to conform with USEFOR. 2 s/serving agent/storage agent/ throughout. 3 s/trusted source/verified source/ 4 Chapter 3 (Changes toRFC Editor function is provided by the existing protocols) moved to Appendices B & C, and subsequent chapters renumbered.IETF Administrative Support Activity (IASA).