[Docs] [txt|pdf] [Tracker] [Email] [Nits]



Network Working Group                                             E. Sam
Internet-Draft                                            March 18, 2020
Intended status: Informational
Expires: September 19, 2020


                      Updates to the NNTP Protocol
                        draft-sam-nntpupdates-00

Abstract

   This Internet Draft proposes and suggests updates to the Network News
   Transfer protocol, or NNTP.  The NNTP powers Usenet, one of the
   largest social media platforms in the world.  It is primarily used
   for transferring text and binary posts.  The aim of these updates is
   to improve efficiency between NNTP peers and NNTP clients.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 19, 2020.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.



Sam                    Expires September 19, 2020               [Page 1]


Internet-Draft        Updates to the NNTP Protocol            March 2020


Table of Contents

   1.  Background  . . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Conventions and Definitions . . . . . . . . . . . . . . . . .   3
   3.  Updates to Article Posting and Retrieval  . . . . . . . . . .   3
   4.  Command Updates/Additions . . . . . . . . . . . . . . . . . .   3
     4.1.  XPAT  . . . . . . . . . . . . . . . . . . . . . . . . . .   3
       4.1.1.  Usage . . . . . . . . . . . . . . . . . . . . . . . .   3
       4.1.2.  Description . . . . . . . . . . . . . . . . . . . . .   4
       4.1.3.  Examples  . . . . . . . . . . . . . . . . . . . . . .   4
     4.2.  WHOAMI  . . . . . . . . . . . . . . . . . . . . . . . . .   5
       4.2.1.  Usage . . . . . . . . . . . . . . . . . . . . . . . .   5
       4.2.2.  Description . . . . . . . . . . . . . . . . . . . . .   5
       4.2.3.  Examples  . . . . . . . . . . . . . . . . . . . . . .   6
   5.  Dynamic Feeds . . . . . . . . . . . . . . . . . . . . . . . .   6
     5.1.  LIST CRITERIA command . . . . . . . . . . . . . . . . . .   7
     5.2.  MAXARTSIZE attribute  . . . . . . . . . . . . . . . . . .   7
     5.3.  GROUPWILDMAT attribute  . . . . . . . . . . . . . . . . .   7
     5.4.  MAXGROUPS attribute . . . . . . . . . . . . . . . . . . .   8
     5.5.  DIST attribute  . . . . . . . . . . . . . . . . . . . . .   8
     5.6.  Example . . . . . . . . . . . . . . . . . . . . . . . . .   8
   6.  Header Related Commands . . . . . . . . . . . . . . . . . . .   8
     6.1.  IHAVEHDR  . . . . . . . . . . . . . . . . . . . . . . . .   9
       6.1.1.  Usage . . . . . . . . . . . . . . . . . . . . . . . .   9
       6.1.2.  Description . . . . . . . . . . . . . . . . . . . . .   9
       6.1.3.  Examples  . . . . . . . . . . . . . . . . . . . . . .  10
     6.2.  Changes to Certain Commands with HEADER capability  . . .  11
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  11
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .  11
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  11
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  11
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  12
   Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  12
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  12

1.  Background

   Usenet is one of the largest social media networks in the world, with
   many endpoints and users posting both text and binary content into
   the network.  This draft proposes various changes and improvements to
   keep up with the demand for Usenet.

   Today, many posts are transmitted through the Usenet network and this
   can cause stress on a server.  Header-only feeds, which are
   standardized in this document, help test servers capability to handle
   a Usenet feed with billions of articles.





Sam                    Expires September 19, 2020               [Page 2]


Internet-Draft        Updates to the NNTP Protocol            March 2020


2.  Conventions and Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

3.  Updates to Article Posting and Retrieval

   News servers store and index articles with three primary keys, a
   Message ID (which is globally unique and non reusable), a article
   number (a unique key for an article), and an arrival timestamp.  More
   information can be found in [RFC3977].  This draft aims to make
   changes to some of the rules set in [RFC3977].

   Article numbers MUST lie between 1 and 9,223,372,036,854,775,807,
   inclusive.  The client and server MAY use leading zeroes in
   specifying article numbers but MUST NOT use more than 19 digits.

4.  Command Updates/Additions

   This draft proposes additional commands and changes to existing
   commands put forth by previous NNTP RFCs.

4.1.  XPAT

4.1.1.  Usage

   Syntax

   XPAT [extended-wildmat] header range|<message-id> (pat
   [pat...])|wildmat

   Responses

   221 Header follows
   430 no such article
   502 no permission

   Parameters

   pat wildmat-pattern
   message-id Article message ID
   extended-wildmat Use RFC 3977 extended wildmat






Sam                    Expires September 19, 2020               [Page 3]


Internet-Draft        Updates to the NNTP Protocol            March 2020


4.1.2.  Description

   The XPAT command is used to retrieve specific headers from specific
   articles, based on pattern matching on the contents of the header.
   This command was first available in INN.  This command is backward
   compatible with the XPAT specification in [RFC2980].

   In this command, a pat (pattern) is equivalent to a wildmat-pattern
   in [RFC3977].

   The required header parameter is the name of a header line (e.g.
   "subject") in a news group article.  See [RFC5536] for a list of
   valid header lines.  The required range argument may be any of the
   following:

   o  an article number

   o  an article number followed by a dash to indicate all following

   o  an article number followed by a dash followed another article
      number

   The required message-id argument indicates a specific article.  The
   range and message-id arguments are mutually exclusive.

   If the optional extended-wildmat parameter was specified in the
   command, then the last argument MUST be ONE [RFC3977] wildmat.

   At least one single wildmat-pattern (as defined by [RFC3977]) must be
   specified as well if the extended-wildmat parameter is not specified.
   If there are additional wildmat-patterns they are joined together
   separated by a single space to form a collection of wildmat-
   pattern(s).  Successful responses start with a 221 response followed
   by a the headers from all messages in which the pattern/wildmat
   matched the contents of the specified header line.  This includes an
   empty list.  Once the output is complete, a period is sent on a line
   by itself.  If the optional argument is a message-id and no such
   article exists, the 430 error response is returned.  A 502 response
   will be returned if the client only has permission to transfer
   articles.

4.1.3.  Examples

   Examples of a client using the PAT command to get the Path header for
   2 posts:

   [C] GROUP alt.example
   [S] 211 80 1 80 alt.example



Sam                    Expires September 19, 2020               [Page 4]


Internet-Draft        Updates to the NNTP Protocol            March 2020


   [C] PAT path 79-80 *example*
   [S] 221 Header information for path follows (from articles)
   [S] 79 patton.com!example.com!not-for-mail
   [S] 80 juju.org!kubellis.com!bowdoin.com!example.com!not-for-mail
   [S] .

   Examples of a client using the PAT command with extended-wildmat to
   filter out all path headers with "patton.com":

   [C] GROUP alt.example
   [S] 211 80 1 80 alt.example
   [C] PAT path 79-80 *,!*patton.com*
   [S] 221 Header information for path follows (from articles)
   [S] 80 juju.org!kubellis.com!bowdoin.com!example.com!not-for-mail
   [S] .

4.2.  WHOAMI

4.2.1.  Usage

   Syntax

   WHOAMI

   Responses

   102 username server-name Logged in
   103 server-name Not logged in

   Parameters

   username The username of the current user (specified in AUTHINFO
   USER)
   server-name The name of the NNTP server

4.2.2.  Description

   Upon receiving the WHOAMI command, the server will send the response
   code 102 along with the username and mnemonic name of the server if
   the user is authenticated.  If the user is not logged in, then the
   response code 103 must be returned with the server-name.

   Severs MUST only send the 102 response code when successful
   authentication has been achieved through the AUTHINFO command in
   [RFC4643].

   This command MUST always return the response code 102 or 103,
   regardless of the clients authentication status.



Sam                    Expires September 19, 2020               [Page 5]


Internet-Draft        Updates to the NNTP Protocol            March 2020


   The server-name can be the hostname, FQDN, or mnemonic name of the
   server.  It can also be the name that the news server injects into
   its path header.  Multiple news servers can have the same server-name
   (server operators might due this for load balancing), but news
   servers MUST make sure servers with the same server-name have the
   same article numbering for newsgroups.  News clients SHOULD assume a
   different server-name means the article numbering is different.  News
   clients can be assured that receiving the same server-name on the
   invoking of the WHOAMI command means the article numbering is the
   same.

   This command SHOULD NOT be used to confirm if posting/reading is
   allowed on the server.  To see if those abilities are allowed,
   clients should use the specific commands (POST, GROUP, ARTICLE) to
   see if they can post and read articles.

4.2.3.  Examples

   Examples of a client using the WHOAMI command while unauthenticated:

   [C] WHOAMI
   [S] 103 example.com Not logged in

   Examples of a client using the WHOAMI command while authenticated:

   [C] AUTHINFO USER ama
   [S] 381 PASS required
   [C] AUTHINFO PASS dublin
   [S] 281 Authentication accepted
   [C] WHOAMI
   [S] 102 ama example.com Logged in

5.  Dynamic Feeds

   A news server may like to filter incoming feeds to conserve space and
   reduce spam on their server.  The LIST CRITERIA capability will aim
   to accomplish this by allowing clients to request from the server
   what types of articles it is looking for.  This is not intended to be
   a complete replacement for out of band feed control, but should be
   good enough for temporary feed adjustment.  On the absence of certain
   attributes in the LIST CRETERIA command or the absence of the LIST
   CRETERIA capability altogether, news servers and clients SHOULD
   revert to the out of band peering info they have stored or their
   default behavior.







Sam                    Expires September 19, 2020               [Page 6]


Internet-Draft        Updates to the NNTP Protocol            March 2020


5.1.  LIST CRITERIA command

   When the LIST CRITERIA command is sent to a server, the server will
   respond with one of the following attributes.  Clients must send the
   LIST CRITERIA command ONLY if the server advertises the LIST CRITERIA
   capability.

   MAXARTSIZE nnn
   GROUPWILDMAT wildmat
   MAXGROUPS nnn
   DIST string[,string]

   When it is done with all of its responses, the server should
   terminate the sequence with a single dot (".") on a line.

5.2.  MAXARTSIZE attribute

   MAXARTSIZE nnn

   Argument: byte amount of the maximum size of desired articles.

   The MAXARTSIZE attribute specifies the maximum size of the articles
   the server wishes to recieve (in bytes).  Clients SHOULD NOT send
   articles bigger than then the size specified in MAXARTSIZE.

5.3.  GROUPWILDMAT attribute

   GROUPWILDMAT wildmat

   Argument:[RFC3977] wildmat

   The GROUPWILDMAT attribute specifies the newsgroups the server wishes
   to receive.
   Using the extended wildmat format, servers can specify the articles
   they want and don't want in one attribute.  Clients SHOULD NOT send
   articles if all of the newsgroups in the article are prohibited by
   the wildmat.  Clients SHOULD send articles if at least one of the
   newsgroups in the article are allowed by the wildmat.

   Some examples of some wildmats that might be used in GROUPWILDMAT:

   * Send all newsgroups
   *,!alt.usenet.kooks Send all newsgroups except "alt.usenet.kooks"
   alt.example Only send articles that were posted to "alt.example"
   *bin* Only send articles that were posted to newsgroups with the
   phrase "bin"





Sam                    Expires September 19, 2020               [Page 7]


Internet-Draft        Updates to the NNTP Protocol            March 2020


5.4.  MAXGROUPS attribute

   MAXGROUPS nnn

   Argument: A positive integer specifying the maximum number of
   newsgroups to which an article may be posted for the site to be
   willing to receive it.

   The MAXGROUPS attribute allows for a sever to inform the client that
   it should not send articles that have been cross posted to more
   newsgroups than the argument in MAXGROUPS.

5.5.  DIST attribute

   DIST string[,string]

   Argument: A group of strings containing article distributions the
   server does not want to receive.

   The DIST attribute allows for the server to inform the client that it
   should not send any articles that have the following distributions
   (determined by the Distribution: header).

5.6.  Example

   This is an example of a server who specifies and returns all
   attributes listed above after the client sends the LIST CRETERIA
   command.

   [C] LIST CRETERIA
   [S] MAXARTSIZE 256000
   [S] GROUPWILDMAT *,!alt.example,!alt.test
   [S] MAXGROUPS 7
   [S] DIST local,internal
   [S] .

6.  Header Related Commands

   The massive growth of Usenet has caused some peers to engage in
   header-only feeds.  Header-only feeds exist to help test system
   reliability and test systems between two peers before a full binary
   feed is established.

   In addition, caching software and proxy servers may wish to have a
   copy of headers and then request the actual article from a upstream
   server when requested from a user.  This may be accomplished from a
   two servers: a faster one that only has headers, and a slower one
   that actually has the articles request.



Sam                    Expires September 19, 2020               [Page 8]


Internet-Draft        Updates to the NNTP Protocol            March 2020


   These commands aim to standardize commands that can be used in the
   transmission of header only feeds.  In order to use any of the
   commands listed below, a server MUST advertise the HEADERS capability
   and the client MUST be in MODE TRANSIT.  More information about
   adding capabilities and modes can be found in [RFC3977].

6.1.  IHAVEHDR

6.1.1.  Usage

   In order to use this command, the server MUST advertise the HEADER
   capability.  This command MUST NOT be pipelined.

   Syntax

   IHAVEHDR message-id

   Responses

   Initial Responses

   331 Send article headers to be transferred
   431 Headers not wanted
   432 Transfer not possible; try again later

   Subsequent Responses

   232 Header transfer successful
   432 Transfer failed; try again later
   433 Transfer failed; do not retry

   Parameters

   message-id Article message ID

6.1.2.  Description

   The IHAVEHDR command informs the server that the client has the
   headers of an article with the specified message-id.  If the server
   wants a copy of the article headers, it MUST return a 331 response
   code.  If the server does not want a copy from
   the client (example - the server already has a copy of the article
   headers or has the full article), the server MUST return a 431
   response code.
   If the server does not wish to receive the headers at the moment
   (example - if it is in the process of receiving a full article/
   headers from another client), then the serer MUST send a 432 response
   code.



Sam                    Expires September 19, 2020               [Page 9]


Internet-Draft        Updates to the NNTP Protocol            March 2020


   If the server indicates it wants to receive the headers by a 331
   response code, then the client will then send back a copy of the
   headers.  The end of header transmission is marked by a single dot
   (".") on a line by itself.  The actual content of the article MUST
   NOT be transmitted using IHAVEHDR, instead the client can use IHAVE
   to transmit both the headers and content of an article.

   Once the transmission of the articles have been completed, a server
   MUST send a 232 response code if the header transfer is successful.
   Unless a 232 response code is sent, clients MUST NOT assume that the
   transfer of the headers was successful.  A lack of response SHOULD BE
   treated the same as a 432 response.  Once received, a server MUST add
   the article to the newsgroups and assign the article a article
   number, like it would with a complete article.  However, certain
   commands must return a different response code if only headers are
   available.

6.1.3.  Examples

   Example of a successful header transfer to another site:

   [C] IHAVEHDR <article.with.headers@example.com>
   [S] 331 Send headers; end with <CR-LF>.<CR-LF>
   [C] From: "A Usenet User" <usenet@example.com>
   [C] Newsgroups: alt.test
   [C] Path: example.com!not-for-mail
   [C] Subject: Test Article
   [C] Date: 20 Jan 2020 09:30:40 -0500
   [C] Organization: Example Corp
   [C] Message-ID: <article.with.headers@example.com>
   [C] .
   [S] 232 Header transfer successful

   Example of a unsuccessful header transfer to another site:

   [C] IHAVEHDR <article.with.headers@example.com>
   [S] 331 Send headers; end with <CR-LF>.<CR-LF>
   [C] From: "A Usenet User" <usenet@example.com>
   [C] Newsgroups: alt.test
   [C] Path: example.com!not-for-mail
   [C] Subject: Test Article
   [C] Date: 20 Jan 2020 09:30:40 -0500
   [C] Organization: Example Corp
   [C] Message-ID: <article.with.headers@example.com>
   [C] .
   [S] 433 Header transfer unsuccessful; don't try again





Sam                    Expires September 19, 2020              [Page 10]


Internet-Draft        Updates to the NNTP Protocol            March 2020


6.2.  Changes to Certain Commands with HEADER capability

   Advertising the HEADER capability means that certain commands will
   return an extra response code when dealing with a header-only message
   ID.  There are also certain commands that must return a successful
   response code only if a full article is available:

   o  STAT SHOULD respond with code 226 (Header only article) along with
      the article number and message ID in that order.

   o  ARTICLE SHOULD respond with code 227 for header only articles.  It
      should then follow the conventional format (article number,
      message-id, content of article).  To maintain compatibility with
      newsreaders, a empty line in between the headers and the dot
      terminator (".") MUST be present.  A news server can also opt, but
      SHOULD NOT, to place a message in the body explaining it only has
      the headers for that article.  If the server manages to get the
      full article, then it can respond normally with a 220 code and the
      article information.

   o  When a server receives a IHAVE, CHECK or TAKETHIS command, it
      SHOULD act like the article does not exist if the server only has
      the headers.  This will allow the server to get the full article.

   o  IHAVE MUST only be used to send full articles.  For sending
      headers only, use IHAVEHDR.

   o  The header only article MUST be assigned an article number and the
      headers MUST be available in header commands like PAT, OVER, HDR,
      etc.

7.  IANA Considerations

   This document has no IANA actions.

8.  Security Considerations

   This document has no security considerations.

9.  References

9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.




Sam                    Expires September 19, 2020              [Page 11]


Internet-Draft        Updates to the NNTP Protocol            March 2020


   [RFC2980]  Barber, S., "Common NNTP Extensions", RFC 2980,
              DOI 10.17487/RFC2980, October 2000,
              <https://www.rfc-editor.org/info/rfc2980>.

   [RFC3977]  Feather, C., "Network News Transfer Protocol (NNTP)",
              RFC 3977, DOI 10.17487/RFC3977, October 2006,
              <https://www.rfc-editor.org/info/rfc3977>.

   [RFC4643]  Vinocur, J. and K. Murchison, "Network News Transfer
              Protocol (NNTP) Extension for Authentication", RFC 4643,
              DOI 10.17487/RFC4643, October 2006,
              <https://www.rfc-editor.org/info/rfc4643>.

   [RFC4644]  Vinocur, J. and K. Murchison, "Network News Transfer
              Protocol (NNTP) Extension for Streaming Feeds", RFC 4644,
              DOI 10.17487/RFC4644, October 2006,
              <https://www.rfc-editor.org/info/rfc4644>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

9.2.  Informative References

   [RFC5536]  Murchison, K., Ed., Lindsey, C., and D. Kohn, "Netnews
              Article Format", RFC 5536, DOI 10.17487/RFC5536, November
              2009, <https://www.rfc-editor.org/info/rfc5536>.

Acknowledgments

   Thanks to everyone who helped create the tools that let us use
   Markdown to create Internet Drafts.

   Thanks to everyone who contributed to ideas for this draft.

   Thanks to everyone in the DISPATCH WG.

Author's Address

   Ekow Sam

   Email: winshell64@gmail.com









Sam                    Expires September 19, 2020              [Page 12]


Html markup produced by rfcmarkup 1.129d, available from https://tools.ietf.org/tools/rfcmarkup/