[Docs] [txt|pdf] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01

IPN Research Group                                               V. Cerf
INTERNET-DRAFT                        Worldcom/Jet Propulsion Laboratory
<draft-irtf-ipnrg-arch-01.txt>                               S. Burleigh
August 2002                                                     A. Hooke
Expires February 2003                                       L. Torgerson
                                          NASA/Jet Propulsion Laboratory
                                                                R. Durst
                                                                K. Scott
                                                   The MITRE Corporation
                                                                 K. Fall
                                                       Intel Corporation
                                                                H. Weiss
                                                            SPARTA, Inc.
                  Delay-Tolerant Network Architecture:
                  The Evolving Interplanetary Internet

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that other
   groups may also distribute working documents as Internet-Drafts.
   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at

   The list of Internet-Draft Shadow Directories can be accessed at


   This document describes an architecture for delay-tolerant networks,
   and is a generalization of the architecture designed for the
   Interplanetary Internet: a communication system to provide Internet-
   like services across interplanetary distances in support of deep
   space exploration.  This generalization addresses networks whose
   operational characteristics make conventional networking approaches
   become either unworkable or impractical.  We define a message-based
   overlay that exists above the transport layer of the networks on
   which it is hosted.  The draft presents an architectural overview
   followed by discussions of services, topology, routing, security,
   reliability and state management.  It concludes with a discussion of
   end-to-end information exchange, including an example.

Cerf, et al.            Expires February 2003                [Page 1]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

Table of Contents
   Status of this Memo................................................1
   Table of Contents..................................................2
   Copyright Notice...................................................3
   1     Introduction................................................6
   2     Why an Architecture for Delay-Tolerant Networking?..........6
         2.1  Extreme Environments...................................7
         2.2  Problems with Internet Protocols and Applications.....11
         2.3  DTN Principles of Design..............................14
   3     DTN Architectural Overview.................................16
         3.1  The DTN Architecture is Based on Message Switching....16
         3.2  DTN Classes of Service Mimic Postal Operation.........17
         3.3  Regions and Region Identifiers........................17
         3.4  Tuples................................................18
         3.5  Late Binding..........................................18
         3.6  The "Bundle Layer" Terminates Local Transport Protocols
              and Operates End-to-End...............................19
         3.7  Routing...............................................19
         3.8  Bundle Layer Reliability and Custodianship............19
         3.9  Security..............................................20
         3.10 Time Synchronization..................................21
   4     Service Considerations:  Application Instances and Bundles.21
         4.1  Networking Style Issues...............................21
         4.2  Bundle Lifetime.......................................22
         4.3  Classes of Service....................................23
         4.4  Delivery Options......................................23
         4.5  Naming and Addressing.................................24
   5     Topological Considerations:  Nodes, Regions, and Gateways..24
         5.1  Nodes.................................................25
         5.2  Regions...............................................25
         5.3  Gateways..............................................26
         5.4  Discussion............................................26
   6     Routing considerations.....................................26
         6.1  Types of Routes.......................................27
         6.2  Store-and-forward operation...........................28
         6.3  Contact-oriented routing..............................29
         6.4  Routing protocols.....................................29
   7     Security considerations....................................30
         7.1  User-oriented security services.......................30
         7.2  Infrastructure security services......................32
   8     Reliability Considerations.................................35
         8.1  Custodial Operation...................................35
         8.2  End-to-end Retransmission.............................36
         8.3  Congestion and Flow Control at the Bundle Layer.......37
   9     State Management Considerations............................38
         9.1  Application Interface State...........................39
         9.2  Bundle retransmission state...........................39

Cerf, et al.            Expires February 2003                [Page 2]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

         9.3  Bundle routing state..................................40
         9.4  Transmission queue state..............................41
         9.5  Receive queue state...................................41
         9.6  Network management state..............................41
   10    Convergence Layer Considerations for Use of Underlying
   11    Bundle Header Information..................................42
   12    An Example Bundle Transfer.................................44
         12.1 Rules for forming tuples in the example network.......44
         12.2 Example Network Topology at the Region Level..........45
         12.3 DTN Gateway routing...................................47
         12.4 Systems participating in example bundle data transfer.48
         12.5 End-to-end Transfer...................................50
         12.6 Error Conditions at the Bundle Layer..................53
   13    Summary....................................................55
   14    References.................................................56
   15    Security Considerations....................................57
   16    Authors' Addresses.........................................58

Copyright Notice
   Copyright (C) The Internet Society (2002).  All Rights Reserved.

Cerf, et al.            Expires February 2003                [Page 3]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002


   John Wroclawski, David Mills, Greg Miller, James P. G. Sterbenz, Joe
   Touch, Steven Low, Lloyd Wood, Robert Braden, Deborah Estrin, Stephen
   Farrell and Craig Partridge all contributed useful thoughts and
   criticisms to this document.  We are grateful for their time and

   This work was performed in part under DOD Contract DAA-B07-00-CC201,
   DARPA AO H912; JPL Task Plan No. 80-5045, DARPA AO H870; and NASA
   Contract NAS7-1407.

Cerf, et al.            Expires February 2003                [Page 4]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002


   "The term 'sonata' can be applied to a large work in three movements
   or to the three sections of a single movement: exposition,
   development, recapitulation.  They resonate with the three acts of a
   movie or play, the three elements of a primal myth or legend (life,
   death, rebirth), the cycle of days (daylight, night, then daylight
   again) and of the seasons in temperate climates (nice weather, then
   winter, then it gets nice again), the Star Wars trilogy, etc.
   Basically, in any human experience we start off brightly, full of
   energy and hope; then we experience reality and become thoughtful and
   reflective, darker and slower; then Something Happens and hope is
   reborn, our energy returns, and we reach that happy synthesis of
   Innocence and Experience that is maturity.  Allegro, andante,

      -- Scott Burleigh

   Release Notes

   draft-irtf-ipnrg-arch-00.txt, May 2001:  Allegro.

   Original Issue.

   draft-irtf-ipnrg-arch-01.txt, August 2002:  Andante.

   Restructured document to distinguish between architecture for delay-
   tolerant networking and the *application* of that architecture to a
   number of different environments, potentially including
   interplanetary internetworking, military tactical networking, and
   sensor internetworking.

   Refined DTN classes of service and delivery options.  Added a "reply-
   to" address to have response information such as error messages or
   end-to-end acks directed to a source-specified third party.

   Further defined the topological elements of delay tolerant networks.

   Elaborated routing and reliability considerations.

   Presented a model for securing the delay tolerant network

   Expanded discussion of flow and congestion control.

   Added section discussing state information at the bundle layer.

   Updated bundle header information and end-to-end data transfer.

Cerf, et al.            Expires February 2003                [Page 5]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

1  Introduction

   This document describes an architecture for delay-tolerant networks.
   We present this as a generalization and extension of our ongoing work
   on the Interplanetary Internet (http://www.ipnsig.org).  We have come
   to the realization that there are a number of different environments
   that share essential characteristics for which the architecture
   presented herein is appropriate.  For example, sensor-based networks
   may have scheduled intermittent connectivity with a "conventional"
   internet with long delays between contacts.  These delays may result
   from limited availability of communication assets (e.g. Low Earth
   Orbiting satellite contacts) or from limited availability of other
   communication assets (e.g., power to run a transmitter).
   Applications using the network may perceive long, highly variable
   end-to-end delays, particularly if multiple instances of intermittent
   connectivity affect a path from source to destination.  This fully
   terrestrial example may appear no different to an application than an
   interplanetary communication, even though each individual link may be

   The particular approach we employ is that of a message-based overlay
   that exists above the transport layers of the networks on which it is

2  Why an Architecture for Delay-Tolerant Networking?

   The existing TCP/IP-based internet, while fabulously successful in
   many environments, does not suit all environments.  The ability of
   the "TCP/IP suite" to provide service depends on a number of
   important assumptions:
   . that an end-to-end path between source and destination exists for
     the duration of the communication session;
   . (for reliable communication) that the maximum round-trip time over
     that path is not excessive and not highly variable from packet to
     packet; and
   . that the end-to-end loss is relatively small.

   A class of "challenged networks," which may violate one or more of
   these assumptions, is becoming important and may not be well served
   by the current end-to-end TCP/IP model.  These networks typically
   serve environments in which it is either impractical or impossible to
   configure a communication environment that supports the assumptions
   on which the TCP/IP suite depends.

   This section examines some of the constraints posed by extreme
   environments that may result in "challenged networks," and considers
   the problems that these environments might cause for common Internet
   protocols and applications.  Finally, we derive some "principles of
   design" for the DTN.

Cerf, et al.            Expires February 2003                [Page 6]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

2.1  Extreme Environments

2.1.1  Constraints Posed by Extreme Environments

   The kinds of extreme environments that this DTN architecture
   addresses constrain both the communication system and the
   applications using that communication system.  This section describes
   some of the characteristics that make environments "extreme."
   Clearly, not all environments that would be considered "extreme"
   exhibit all of these characteristics.

   Long and/or Variable Delays

   Delay affects networks in at least two different ways.  First,
   interactivity is directly affected by delay. For example, telephone
   conversations over a terrestrial telephone line are markedly
   different than telephone conversations that are routed over a
   geostationary satellite hop.  Second, variability in delay can affect
   applications and protocols.  Consider a stream of packets carrying
   voice data over a network.  If the delay in the network varies
   greatly from one packet to the next, a buffer is required to avoid
   the perception of pauses in the reconstructed voice track.  If the
   application is hosted on a network with greater delay variability
   than the application was designed for, the stream will again have
   pauses, possibly of sufficient magnitude to make the data

   Delays are comprised primarily of three components: propagation delay
   through the medium; queuing delay within relay points, source, and
   destination; and clocking delays associated with transmitting an
   atomic unit of data onto the medium.

   Propagation delays can be long due to speed-of-light delays to cross
   long distances (e.g., deep space).  Alternatively, propagation delays
   could be long due to the propagation medium (e.g.,
   acoustic/underwater).  How long is long?  Light time (round trip)
   between Earth and Mars ranges from ~8 minutes to ~40 minutes.
   (Mars's average distance from the sun is 1.5AU, and light in a vacuum
   travels at roughly 8 minutes per AU.)

   Queuing delays are affected by traffic and service rate.  In extreme
   environments, the service rate of a node may be low due to power
   limitations requiring a slow processor or a long wait while the unit
   is quiescent.  Arrival distributions may cause significant
   variability in delays, particularly for heavy-tailed distributions
   [KP97].  (Heavy-tailed distributions are those that asymptotically
   follow a power law.  That is, P[X > x] ~ x^-alpha as x approaches
   infinity, for 0 < alpha < 2.  For alpha < 2, the variance of a heavy-

Cerf, et al.            Expires February 2003                [Page 7]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   tailed distribution is infinite.  For alpha < 1, the mean is also
   infinite.  "Thus, as alpha decreases, a large portion of the
   probability mass resides in the tail of the distribution." [KP97])

   Clocking delays accrue each time a packet is transmitted if the
   packet was fully received prior to transmission.  "Cut-through"
   routing, in which the first bits/bytes of a data unit are operated
   upon before the last bits/bytes have been received, is a fairly rare
   operation.  For many types of data, the latency advantages of cut-
   through routing are more than offset by the reluctance of network
   designers to forward corrupt data.  (Although some types of data are
   useful even if partially corrupted.)  For data that are not forwarded
   before being fully received (and error checked), clocking delays
   accumulate at each hop in the network.  For relatively slow, multi-
   hop networks, this can result in high per-packet delays, particularly
   if packet sizes are large.  (Which can, of course, increase queuing
   delays for other packets, increasing both the round trip times and
   the variability of delay.)

   We assume that processing delay, another contributor to overall
   delay, is comparatively low.  However for some simple processors,
   cryptographic processing, and in particular key generation, can
   contribute sufficiently to processing delay and in some circumstances
   make it become a noticeable contributor to overall delay.

   Variability of delay can have a significant effect on communication
   in addition to absolute delay.  Many protocols attempt to provide
   reliability through retransmission (ARQ) mechanisms.  Timers are a
   critical element of most retransmission mechanisms, and establishing
   a retransmission time out value is an important aspect of the
   mechanism.  Some protocols, particularly those at the link layer,
   have the ability to eliminate all components of round trip time other
   than propagation and clocking delays from their timings, and are able
   to produce retransmission time out estimates that are quite close to
   the actual round trip time.  When queuing delay must be considered
   and the timing does not have direct knowledge of the delays at all of
   the queues, estimating a reasonable retransmission time out value
   becomes more complex.  A timeout value that is too low will cause
   unnecessary retransmission, while one that is too high will cause
   excessive delay in data delivery.

   Frequent Partitioning (Intermittent Reachability)

   Network partitioning typically adds to data loss, because data that
   cannot be forwarded essentially immediately is generally discarded.
   If losses become too severe, applications typically fail. In addition
   to contributing to loss, network partitioning adds significantly to
   overall delay if nodes are configured to store data until the
   outbound link is restored.  This behavior can also contribute to

Cerf, et al.            Expires February 2003                [Page 8]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   misordered data arriving at the destination, which can cause poor
   performance in some protocols.

   Data Rate Asymmetry

   Data rate asymmetry measures the ratio of forward-path minimum
   capacity versus reverse-path minimum capacity.  Data rate asymmetry
   adversely affects protocols using ARQ for reliable delivery by
   altering the ACK or NACK return path.  This effect has been studied
   extensively with TCP, and is discussed below.  At a higher layer,
   request/response applications that have not been appropriately tuned
   to balance the amount of request versus response traffic can time out
   waiting for data to travel across the lower-capacity link.  In the
   most extreme case, the asymmetry is infinite (as is the case with
   communications to radio-silent submarines, for example) ARQ is simply
   not possible.  In these cases, forward error correction and periodic
   retransmission are typically used to improve communication
   reliability.  See [BLMR98] as an example of how such techniques are
   used in the Internet.

   Data rate asymmetry can arise due to routing path asymmetry,
   different forward/reverse path link technologies, or intentional
   engineering tradeoffs.  Moderate asymmetries in the wired Internet
   are found most commonly in asymmetric access technologies such as in
   CATV or ADSL-based subscriber lines.  The largest asymmetries for
   wireless Internet access appear to be prevalent in in-flight Internet
   access systems.  The AirTV system, for example, being proposed for
   2004 and beyond, could provide a ground-to-air speed of up to 45Mb/s
   using DVB satellite technology with a comparatively anemic air-to-
   ground bandwidth of a 128kb/s or less using INMARSAT [ARINC].  In the
   case of deep space communications, downlink data rate is
   intentionally engineered to be much greater than uplink data rate.
   This tradeoff is not surprising given the goal of most science
   missions: the capacity demands of downlinking high-resolution images
   and telemetry from a spacecraft dwarf the capacity required to uplink
   short commands to that spacecraft.

   Packet Loss and Errors

   Consider the following probability of success metric:

   Pr = (1-pe)^(i*(8*m))

   This metric is the probability of successful end-to-end delivery of a
   particular message of size m bytes across i links assuming an
   identical, independently distributed (IID) stationary loss process
   with constant bit error rate pe across all intervening forwarders.
   (In this equation, the carat -- ^ -- indicates exponentiation.)  This
   metric, as expressed, does not account for congestive loss but does

Cerf, et al.            Expires February 2003                [Page 9]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   account for message size and number of cascaded links given a fixed
   BER.  For packet networks, most links employ some form of error
   detection, implying that any bit loss creates an end-to-end packet
   loss.  As can be clearly seen from the formula, the end-to-end
   probability of successful delivery decays exponentially with path hop
   count.  Any congestive loss would only worsen the performance.

   For reliable transfer, excessive errors require repair using either
   error correction or retransmission.  If the path is sufficiently
   lossy, end-to-end retransmission can become essentially useless.  To
   understand why this is so, consider a probability of packet loss pp =
   1 ¡ Pr for some fixed m.  Given the assumptions listed above, the
   expected number of retransmissions required before successful
   delivery is given by: (1-(1-pp)^i)/(1-pp)^i.  Considering a packet
   error probability of 0.3.  In this case, 4 hops requires 3
   retransmissions, 10 hops requires 34, and 20 hops requires about 1200
   retransmissions.  If a hop-by-hop retransmission scheme were used
   instead, the total (network wide) number of retransmissions is given
   by i*pp/(1-pp).  For the same error probability of 0.3, the number of
   retransmissions for 4, 10, and 20 hops would be 2, 5, and 9,
   respectively.  Thus, for very lossy environments, hop-by-hop
   retransmission is preferable to end to end retransmission.

2.1.2  Interoperating Among Differently-Challenged Networks

   Two complicating factors in building internetworks comprised of
   networks operating in multiple different extreme environments are

   1. Network adaptations necessary to achieve required operational
     characteristics in one extreme environment may be mutually
     exclusive with those required in another environment, and
   2. Evolution to achieve interoperability with networks not
     anticipated at design time may be technically or politically

   There is no guarantee of support for any common communication
   technology across networks designed for operation in extreme
   environments.  In many instances, at the time of deployment only the
   barest minimum of capabilities can be fielded to support the
   network's mission.

   If these networks are successful, they evolve.  In many cases, this
   evolution leads to an expansion of scope and span for the network.
   Often this can result in a requirement to interoperate with another
   network in an environment that presents different challenges that
   have been addressed by dramatically different solutions.

   The ability to operate over many different types of networks that
   have been adapted to support a range of benign and extreme

Cerf, et al.            Expires February 2003               [Page 10]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   environments is both a challenge and a requirement for this Delay
   Tolerant Networking Architecture.

2.1.3  Examples of some extreme environments

   Several types of extreme environments currently exist, with more
   appearing almost daily.  If one considers as the norm the wired
   Internet, with multi-gigabit backbones, almost no corruption-related
   data losses and relatively short round trip times, then many
   different types of environments seem extreme.

   This architecture was originally conceived to support the
   Interplanetary Internet, which exhibits round trip delays on the
   order of tens of minutes and intermittent connectivity that can
   result in disconnection for weeks.

   Military tactical networks, in which data rates are low, error rates
   are high, and link interruptions are frequent, can likely derive
   great benefit from application of this architecture.

   Similarly, sensor networks deployed in oceanic environments exhibit
   long delays, significant data loss, and the potential for long link

   An individual seeking a networking solution for a particularly
   intriguing and challenging environment has recently approached us to
   determine if the DTN architecture was right for the following
   environment.  The Sami people in Lapland live in widely dispersed
   communities in remote areas and are not well served by either wired,
   fixed wireless, or satellite internet service.  They do, however,
   frequently travel on snowmobiles from community to community, and
   congregate at work sites and larger towns.  To effectively serve this
   environment, an architecture is required that can embrace
   intermittent, probabilistic connectivity, store and forward
   operation, and high and variable delays.  (The architecture described
   in this document meets these needs.)

2.2  Problems with Internet Protocols and Applications

   This section discusses some of the issues associated with attempting
   to serve challenged networks with the Internet suite of protocols.

2.2.1  Internet "Core" Protocols

   The performance characteristics of challenged networks confound the
   efficient operation of the core Internet protocols.  By the 'core'
   Internet protocols we mean IP, TCP, UDP, BGP, common IGPs (RIP, OSPF,
   or EIGRP) and DNS.  These protocols span the services of end-to-end
   datagram delivery, reliable two-party stream delivery, regional

Cerf, et al.            Expires February 2003               [Page 11]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   (aggregated) routing path discovery with policy, intra-domain path
   selection and distributed support for name resolution.  Although some
   of these protocols are technically "application" protocols from a
   layering point of view, we treat them here together as core protocols
   for the purposes of discussion.

   Excessive latency affects TCP directly by severely limiting its
   throughput performance or interfering with connection establishment
   [DMT96].  Any application-layer protocol using TCP as its underlying
   transport is therefore affected (for example, BGP and DNS zone
   transfers).  Excessive latency also adversely affects the proper
   operation of conventional routing protocols as links will be
   incorrectly discovered to be non-operating when soft-state refreshes
   are delayed too long (RIP), or a lack of response to link state
   "hello" messages (OSPF, EIGRP) is observed.   UDP is not sensitive to
   excessive latency because it does not contain timers that affect its
   operation.  However, core application protocols that require it (for
   example, DNS queries/responses; also SNMP, if used) will take too
   long to complete when faced with excessive delay, triggering early
   application failure (or the lack of ability to use DNS name/address
   mappings in the case of address-to-name queries).

   Data rate asymmetry adversely affects TCP by altering the smooth flow
   of acknowledgments.  Extensive studies have been conducted on TCP
   using the normalized asymmetry ratio, which takes into account the
   asymmetric message sizes between data packets and returning ACKS
   [MLS00].  Results indicate that bandwidth asymmetry can lead to poor
   performance of unidirectional transfers due to alterations in the
   time series of the ACK channel.  In particular, if ACKs are queued,
   their spacing in time causes the TCP sender to clock out subsequent
   packets less frequently.  If ACKs are lost, burstiness, slow
   congestion window growth, or defeat of the fast retransmit/recovery
   algorithms can occur.  (See [PILC-ASYM] characterization of this
   problem and some possible approaches to ameliorate it).  Any
   application layer protocols attempting to use TCP over asymmetric
   links are therefore affected as well.

   Significant path loss affects the TCP transport most strongly,
   causing it a number of problems.  After multiple loss events it will
   continue retrying with a backed-off retransmission timer until it
   gives up on retransmitting altogether and closes the connection.
   Somewhat more moderate losses will contribute to problems invoking
   the fast retransmit and recovery algorithms, even in the presence of
   SACK.  IP performance can also be affected by path loss if fragmenta-
   tion is required.  IP provides no mechanism for fragment
   retransmission, thereby causing the overall probability of successful
   datagram delivery to be further reduced if datagrams are fragmented

Cerf, et al.            Expires February 2003               [Page 12]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   Node longevity affects the probability of end-to-end delivery, as
   round-trip or even one-way delivery time of a particular message may
   exceed the sender's lifetime.  Clearly, in such cases it is useless
   to arrange for immediate acknowledgements to be returned.  In such
   cases, any notification of successful or unsuccessful delivery needs
   to be directed to some alternative node that remains functional.

2.2.2  Common Internet Applications and Supporting Protocols

   The most common user services within the Internet today are the web,
   e-mail, and perhaps FTP. These services employ a number of different
   application protocols, including HTTP, SMTP, POP, IMAP, Microsoft's
   Exchange Protocol and FTP. These protocols, or in some cases the
   applications that implement them, are either severely degraded or
   fail to operate entirely under challenging conditions.  Two primary
   reasons are application-level timeouts mismatched to the actual
   network latency and excessive numbers of round-trip request/response
   message exchanges required in order to accomplish a protocol
   transaction.  For example, web browsers and mail clients will often
   time out during connection establishment after a minute or two.  The
   SMTP protocol requires peers to mutually identify themselves prior to
   actually exchanging the mail payload, even though this early name
   exchange is not fundamental to the process of delivering e-mail.
   Such protocols have been called "chatty" due to their excessive
   number of round-trip times required to accomplish simple tasks.  FTP
   has a similarly chatty interaction during its initial association as
   user login and authentication information is exchanged between client
   and server.

2.2.3  Discussion

   When considering the use of existing Internet protocols as a basis
   for interconnecting challenged networks, certain properties of the
   operational environment seem insurmountable.  The possibility of
   intermittently infinite data rate asymmetry appears to preclude the
   use of conventional TCP-like protocols based on ARQ for reliability
   because an ACK channel may not be available.  With exponentially
   decaying probability of successful end-to-end delivery as path length
   grows, end-to-end reliability should be avoided in favor of hop-by-
   hop retransmission in lossy environments.  Given the possibility of
   very high round-trip times, any expectation of timely performance in
   request/response protocols must also be avoided.  Finally, the
   standard Internet routing protocols assume the immediate existence of
   an end-to-end path and do not accommodate the notion of future
   (scheduled) routing opportunities that are needed for operation in
   frequently partitioned networks.

Cerf, et al.            Expires February 2003               [Page 13]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

2.3  DTN Principles of Design

   After examining the environmental conditions cited above and seeing
   the problems that the Internet protocols have with these conditions,
   we derived the following principles of design to guide the definition
   of a Delay Tolerant Networking Architecture.

2.3.1  Use transport and network layer technologies appropriate to the
       local environment

   Where Internet protocols work well, there is a distinct advantage to
   using them.  However, in other environments, such as sensor networks
   or certain radio networks, the internet suite may not be the best fit
   for the task at hand.  Some sensor networks, for example, use very
   different routing and node naming concepts (e.g., directed diffusion
   [IGE00]).  The DTN architecture should not inhibit designers from
   building networks using transport, network, link, and physical layer
   technologies that are best suited for their environments.  Rather,
   the DTN architecture should provide the means for dissimilar networks
   to interoperate.  The dissimilarity of the extreme environments
   discussed precludes the use of a single end-to-end transport
   protocol.  Instead, end-to-end operations require us to provide a
   common substrate above the *transport* layers of the constituent
   networks.  Thus, whereas IP provides the common substrate for the
   Internet above dissimilar link layers, the DTN architecture creates a
   "network of Internets" by providing an end-to-end layer above the
   transport layer.  We call this the "bundle layer."

2.3.2  Employ a "non-chatty" communication model

   Because delays may be long and network partitioning may be frequent,
   we suggest that highly interactive styles of communication be avoided
   in the kinds of extreme environments described above.  Rather, entire
   application-layer messages, which we call "bundles," move through the
   network as units.  These bundles are intended to have all of the
   primary data and meta data necessary for their use at the

   Although this *may* mean that bundles are large (relative to packets
   in a TCP/IP network), it does not require it.  The intent simply is
   to not have multiple exchanges between source and destination (e.g.
   "user:", response, "password:", response, etc.) when such exchanges
   can be "bundled" together into a single communication - a single
   bundle or a unidirectional sequence of bundles.

2.3.3  Base transfers between nodes on store-and-forward techniques

   The Internet Protocol (IP) is based on store and forward
   transmission.  However, in IP routers, the "store" portion of the

Cerf, et al.            Expires February 2003               [Page 14]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   operation typically lasts only long enough to determine the next hop
   destination and to wait for any packets to be transmitted that are
   ahead of the current one.  If no route to the destination exists at
   the time that the routing engine examines the packet, that packet is
   typically discarded.

   In a Delay-Tolerant Network, it is considered *normal* for there to
   be no path to the destination at the time a bundle is received.
   Bundles are routed based not on current connectivity but on the
   expectation of connectivity, drawn either from a firm schedule or
   from previous experience.  If an episode of connectivity doesn't
   occur as expected, bundles are rerouted accordingly.

   This approach assumes that there is a considerable amount of storage
   available within the network, which is a fundamental assumption for
   these types of delay-tolerant networks.

   The result of this interpretation of store-and-forward operation is
   that bundles of data can still make forward progress toward their
   destination even if an end to end path never exists at any single
   moment in time.

2.3.4  Advance the point of retransmission toward the destination

   The retransmission of lost data in the Internet is (nominally) always
   end-to-end: the source must commit resources to the retention of data
   for possible retransmission by TCP until reception by the destination
   is acknowledged, and retransmitted data must always traverse the
   entire route from source to destination.  In a Delay-Tolerant Network
   such behavior might impose high performance costs due to the length
   of time needed to traverse that route, so the point of data
   retransmission is advanced toward the destination whenever possible.
   This allows earlier release of retransmission resources at any given
   node and enables retransmitted data to arrive sooner.

   There is an obvious benefit to this approach if the point of
   retransmission progresses across a *very* long delay hop (such as one
   that spans interplanetary distance).  However, in a network composed
   of a series of lossy links (such as a congested multi-hop wireless
   network), there is similar benefit, as discussed in section 2.1.1,
   "Packet Loss and Errors."

   Retransmission point advance occurs on two of the three levels at
   which the DTN Architecture supports retransmission (the third being
   the optional end-to-end retransmission discussed in section 8.2).

   First, wherever transmission between pairs of DTN nodes is
   accomplished by underlying reliable transport-layer protocols such as
   TCP, the probability of correct end-to-end delivery at the bundle

Cerf, et al.            Expires February 2003               [Page 15]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   layer is enhanced simply by the concatenation of episodes of point-
   to-point reliability provided by those protocols.  (The "points" of
   transmission at the bundle layer look like "ends" at the transport
   layer.)  Data arrival at a transport-layer destination informally
   advances the point of retransmission within the DTN:  the destination
   of one transmission that is reliable at the transport layer becomes
   the source of the next one.

   Second, upon receipt of a bundle, a node may *take custody* of it.
   This is a commitment made by the node that it will shoulder the
   burden of retransmission *at the bundle layer* to insure that the
   bundle reaches its destination, and is accomplished by issuing a
   bundle-layer acknowledgment to the previous custodian (which might be
   the source).  This allows the previous custodian to release the
   resources necessary to ensure delivery and formally moves the point
   of bundle-layer retransmission closer to the destination.

2.3.5  Provide security services to secure both the infrastructure and
       the information

   Networks in extreme environments may range from being completely
   disposable to irreplaceably precious.  Those that are disposable are
   typically so resource starved that difficult tradeoffs must be made
   regarding provision of service versus protection of infrastructure.
   These tradeoffs must be made by those who instantiate the
   architecture, and the architecture must provide for a range of

   Minimally, we have concluded that the architecture must provide for
   infrastructure protection, up to and including mutual suspicion among
   DTN routers.  However, this must be optional at the discretion of
   those who actually instantiate a particular delay tolerant network.
   Security services offered to users should include key management,
   authentication, integrity, and confidentiality, but availability and
   use of those services may vary from DTN to DTN.

3  DTN Architectural Overview

   The previous section presented the design principles that guide the
   definition of the DTN architecture.  This section presents an
   overview of the decisions that result from those design principles.

3.1  The DTN Architecture is Based on Message Switching

   In order to avoid unnecessary interaction between the source and
   destination, a DTN transmits "bundles" that contain both user data
   and relevant metadata.  These bundles are the "messages" on which the
   DTN architecture is based.

Cerf, et al.            Expires February 2003               [Page 16]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   A message-switched architecture provides the network with a priori
   knowledge of the size and performance requirements of requested data
   transfers.  When there is a significant amount of queuing that can
   occur prior to transmission over an outbound route (as is the case in
   the DTN version of store-and-forward) the advantage provided by this
   information provided is significant.  In an environment that
   frequently experiences network partitioning, messages are efficient
   units for queuing while awaiting network reconnection.

3.2  DTN Classes of Service Mimic Postal Operation

   The (U.S. or similar) Postal Service provides a strong metaphor for
   the services that a Delay-Tolerant Network offers.  Traffic is
   generally not interactive.  It may be one-way in nature.  There are
   generally no strong guarantees of timely delivery.

   The DTN Architecture, like the Postal Service, offers *relative*
   measures of priority (low, medium, high).  It may offer basic
   notifications, for example: "the intended recipient has accepted
   delivery of the message," "the route taken by this message is as
   follows..." and "the message has been transmitted."

   An essential element of Postal Service operation is that mail has a
   place to wait in queue until an outbound hop is available.  This
   highlights the previously stated assumptions:

    1. That there is storage available in the network,
    2. that storage is well-distributed throughout the network,
    3. that it sufficiently persistent to store bundles until custody
      transfer can occur, and
    4. (implicitly) that this 'store-and-forward' model is a better
      trade-off than using resources to attempt to effect continuous
      connectivity or other alternatives.

   For a network to effectively support the DTN architecture, these
   assumptions must be considered and must be found to hold.

3.3  Regions and Region Identifiers

   The DTN architecture defines a "network of Internets."  Each of these
   Internets is called a "DTN region."  Regions may be delimited by
   differences in network protocol, transport protocol, or addressing
   mechanism.  Regions may also be delimited by other criteria, such as
   trust boundaries [NEWARCH].

   Each DTN region has a unique identifier that is known (or knowable)
   among all regions of the DTN.

Cerf, et al.            Expires February 2003               [Page 17]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   All inter-region communication takes place via DTN *gateways*, which
   provide the interconnection points between regions.  These correspond
   to "waypoints" in [META], and to the gateways described in the
   original ARPANET design [CK74].  DTN gateways differ from ARPANET
   gateways because they operate above the transport layer and are
   focused on message switching rather than packet switching.  However,
   both DTN gateways and ARPANET gateways provide conversions between
   the protocols specific to one region and the protocols specific to

   Region definition and region identification are key concepts in the
   DTN architecture.  DTN bundles that originate outside the destination
   region are first transmitted to one of the DTN gateways that connect
   the source region with one or more other regions.  Routing outside
   the destination region is based on the destination region's
   identification, meaning that there is a single region identifier
   space that is common across the end-to-end DTN.

3.4  Tuples

   The region identifier described above is sufficient to route a bundle
   of data to its destination region, but not to deliver it to the
   specific entity for which it is intended.  The DTN architecture uses
   a tuple of the region identifier and a region-specific entity
   identifier to identify uniquely a particular entity in the DTN.

   The tuple notion allows us to distinguish data that is necessary for
   routing *across* regions from that which is necessary for delivery
   within a region.  As a result of this, the entity ID is *opaque*
   outside the region of definition. The referenced entity might be a
   host, a protocol, an application, or something else.  Further, the
   entity identifier might be a name, an address, or a descriptor such
   as a URL.  This is a local issue within the entity's region.  (If the
   entity ID is a name, there is an assumption that a name-to-address
   binding function exists within the region to support eventual routing
   of the bundle to the destination entity.)

3.5  Late Binding

   The opacity of entity IDs outside their local region enforces another
   key concept in the DTN Architecture: that of late binding.  Late
   binding refers to the fact that the entity ID of a tuple is not
   interpreted (e.g., bound to an address) outside its local region.
   This avoids having a universal name-to-address binding space (and its
   associated database and synchronization issues).

   This approach preserves a significant amount of autonomy within each
   region.  The entity IDs used in bundles might be built on DNS names,

Cerf, et al.            Expires February 2003               [Page 18]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   or URLs, but they might also be "expressions of interest" as in a
   directed diffusion-routed network [IGE00].

   Additionally, late binding avoids the delay-related issue that the
   binding information might be highly ephemeral relative to the transit
   time of a bundle.  We assume that the internal details of a
   destination network will be sufficiently stable for the duration of a
   transmission of a message within that region, or that delay-tolerant
   mechanisms will be employed *within* the region to compensate.  (This
   is, by definition, a local issue within the region and may be
   accommodated in whatever manner is most practical for that region.)

3.6  The "Bundle Layer" Terminates Local Transport Protocols and
     Operates End-to-End

   The bundle layer provides an end-to-end protocol that employs custody
   in-network retransmission, both point-to-point retransmission between
   transport-layer endpoints and also custody transfer between (possibly
   non-adjacent) DTN nodes.  The bundle layer makes use of the
   reliability mechanisms available from the underlying transport layers
   of each region, and a single bundle-layer hop *may* span an entire

3.7  Routing

   The bundle layer provides routing among DTN-capable nodes, including
   DTN gateways.  There may be many DTN gateways that interconnect
   adjacent regions, and there may be multiple bundle routing "hops"
   within a region (particularly if intra-regional connectivity is

   Routing information exchange mechanisms may differ from one
   instantiation of the Delay Tolerant Network Architecture to another,
   reflecting different needs and constraints of the extreme
   environments.  Different routing protocols may be employed for intra-
   region routing and for inter-region routing.  We do not require that
   the *same* inter-region routing protocol run among all (inter-region)
   DTN gateways in a Delay Tolerant Network.  (It is required that
   necessary routing information be able to be propagated throughout the
   DTN, but the mechanisms for doing that are neither defined nor
   constrained by the DTN Architecture.)

3.8  Bundle Layer Reliability and Custodianship

   The bundle layer provides three types of reliability services: end-
   to-end acknowledgment of a bundle (and accompanying retransmission),
   custodial transmission (with in-network retransmission if required),
   and unacknowledged bundle transmission.  End-to-end acknowledgment
   and custodial transfers may be combined for additional reliability.

Cerf, et al.            Expires February 2003               [Page 19]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   Reliability services at the bundle layer are relatively simple
   because the bundle layer depends upon the reliability mechanisms in
   each of the underlying internets.  Bundles are delivered atomically
   and are, generally, independent of other bundles.  As a result,
   positive acknowledgment is possible, but partial or negative
   acknowledgment is not.  Since delays may be long and/or highly
   variable, retransmission timers must, in general, be coarse and may
   be strongly affected by local policy.

   Custody transfer, introduced in section 2.3.4, above, allows the
   source to delegate retransmission responsibility and recover its
   retransmission-related resources relatively soon after sending the
   bundle.  While not every node in a DTN must be capable of providing
   custodial services, all DTN gateways (that span two or more DTN
   regions) are expected to be able to be custodians.

3.9  Security

   Resource scarcity in delay-tolerant networks dictates that some form
   of access control is required.  It is not acceptable for an
   unauthorized user to be able to flood the network with high-priority
   traffic, possibly preventing the network's mission from being
   accomplished.  Implicit in this statement is a requirement for some
   form of admission control and/or in-network authentication that is
   sensitive to the class of service that a user has requested, and the
   means to verify that the user is authorized to make that request.  In
   a low delay environment, this would simply be tedious.  In a
   high/variable delay, low data rate environment, it is potentially
   much worse: access control lists are difficult to update, re-keying
   presents hard problems, and routers or end nodes might be

   Key management presents a number of dilemmas.  Keys will almost
   certainly require replacement in all but the most disposable of
   networks.  Re-keying and related operations on highly remote nodes is
   challenging, and proper approaches to key distribution in this
   environment remain active areas for research.

   The DTN must be able to support end-to-end user services that include
   verification of a message's integrity and the ability to provide
   confidentiality of a user's message.  It must do this in a way that
   does not require a great deal of interaction across long-delay (or
   resource-constrained) data links.  "Support" for these services may
   involve provision of a key management service to support user-level
   confidentiality and integrity, or it may involve providing these
   services as part of the bundle layer.  This is an area still under

Cerf, et al.            Expires February 2003               [Page 20]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

3.10 Time Synchronization

   The DTN architecture depends on time synchronization for two primary
   purposes: routing, and "time to live" computations.

   Routing that is based on schedules and dependent upon coordination of
   shared assets (such as directional antennas) creates a requirement
   for time synchronization.  As with many requirements, just how *much*
   time synchronization is required is an economic trade-off.  The cost
   to achieve a particular degree of time synchronization must be
   weighed against the cost of not having it.  The costs of not having
   time synchronization include reduced overall efficiency of a
   communication asset, potential loss of user data, power inefficiency,
   and many others.  For some communication assets, such as NASA's Deep
   Space Network, the operational cost is many thousands of dollars per
   hour, and the cost to achieve a modest level of time synchronization
   between peers is quite justifiable.  There is a study currently
   underway [DM02] to examine the time synchronization issues across
   NASA's Deep Space Network and to determine a particular time
   synchronization scheme suited to that network.

   Given that DTN routing depends to some degree on time
   synchronization, the DTN architects have made the decision to exploit
   its availability.  In particular, time to live computations may be
   done by including a source time stamp and an explicit time to live
   field (in time units after the time in the source time stamp).  This
   requires some level of time synchronization, but since its sole use
   is for purging data from the network, the synchronization
   requirements are not strict.  This approach allows a source time
   stamp to be used for a number of purposes, such as to provide replay
   protection and to identify individual messages.

4  Service Considerations:  Application Instances and Bundles

   This section describes the basic services available to applications
   using the DTN Architecture's Bundle Service.  The Bundle Service is
   the message-oriented communication service provided by the "Bundle
   Layer" described in section 3.6.

4.1  Networking Style Issues

   Before discussing specific Bundle Services, a note about application
   interaction style is in order.  This service is intended to operate
   in *delay-tolerant* networks.  That does not mean that there *must*
   be delays in the network, or that there *will* be delays, but that
   there *may* be delays.  The protocols providing the services exposed
   by the bundle layer are delay tolerant.  Applications using those
   services should be, too.

Cerf, et al.            Expires February 2003               [Page 21]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   Message-oriented communication differs from conversational
   communication.  In general, applications should attempt to include
   enough information in a bundle so that it may be treated as an
   independent unit of work by the remote entity (a "transaction",
   although transactions carry connotations of multi-phase commitment
   that we do not intend here).  The goal is to minimize conversation
   between application entities that are separated by a network that
   presents long and possibly highly variable delays, and to constrain
   any conversation that does occur to be asynchronous in nature.  A
   file transfer application, for example, might incorporate account
   information, file location information, file operation information,
   and file content within a bundle, allowing atomic operation on that
   bundle. Comparing this style of operation to a classic FTP transfer,
   one sees that the bundled model can complete in one round trip time,
   whereas an FTP file "put" operation can take as many as eight round
   trips to get to a point where file data can flow [WhynotTCP].

   Delay-tolerant applications must consider additional factors beyond
   the conversational implications of long delay paths.  Application
   instances may terminate (voluntarily or not) between the time that a
   bundle is sent and the time its response is received.  If an
   application has anticipated this possibility, it is possible to
   reinstantiate the application instance based on state information
   saved in persistent storage.  This is an implementation issue, but
   also an application design consideration.  Similarly, either host or
   application mobility may result in the application's address having
   changed by the time a response is received.  If applications embed
   physical addresses into their data, they will defeat the potential
   benefits that late binding provides.

   Some consideration of delay-tolerant application design can result in
   applications that work well in low-delay environments, and that do
   not suffer extraordinarily in high or highly-variable delay

4.2  Bundle Lifetime

   Applications are requested to provide an expiration time (actually, a
   time to live in seconds) for a bundle if the value of the data has a
   finite duration.  If not supplied, or if the user-supplied value is
   larger than local policy permits, the bundle layer will provide a
   value.  Note that this value is treated as an actual "time to live" -
   - it is added to the time that a bundle was submitted by the
   application to determine the time at which the bundle will be purged
   from the network.  Appropriate values depend on the network and the
   data, and may range from seconds to weeks.

Cerf, et al.            Expires February 2003               [Page 22]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

4.3  Classes of Service

   The DTN architecture provides for a range of classes of service.
   These classes of service are requests to the bundle layer to provide
   *relative* distinctions in the immediacy with which bundles are

   We have currently defined three classes of service in the DTN

   . Bulk - In bulk class, bundles are shipped on a "best effort"
     basis.  No bulk class bundle will be shipped until all bundles of
     other classes bound for the same next hop destination have been
   . Normal - Normal class bundles that are in queue and bound for the
     same next hop destination are shipped prior to any bulk class that
     are in queue.
   . Expedited - Expedited bundles, in general, are shipped prior to
     bundles of other classes.  However, the bundle layer *may*
     implement a queue service discipline that prevents starvation of
     the normal class, which may result in some normal bundles being
     shipped before expedited bundles bound for the same next hop
     destination as the normal class bundles.

4.4 Delivery Options

   The bundle layer offers a number of delivery options to users.  First
   and foremost is the option of custodial delivery.  Custodial delivery
   means that a bundle will be reliably transmitted by the bundle layer,
   and that the point of retransmission at the bundle layer will advance
   through the network from one custodian to another until the bundle
   reaches its destination.  The bundle layer depends on the underlying
   transport protocols of the networks that it operates over to provide
   the primary means of reliable transfer from one bundle layer entity
   to the next.  However, when custodial delivery is requested, the
   bundle layer additionally provides a coarse-grained timeout and
   retransmission mechanism and an accompanying acknowledgment
   mechanism.  When a bundle application does *not* request custodial
   delivery, this bundle layer timeout and retransmission mechanism is
   not employed, and the bundle depends solely on the reliability
   mechanisms of the underlying transport layers.

   A second delivery option is the option of a return receipt.  An
   application may request a return receipt for a bundle that is being
   transmitted.  When the subject bundle is received *by the destination
   application instance* (not simply the destination bundle layer
   entity), a return receipt is generated by the bundle layer entity and
   returned to the source or the source's designated alternate (reply-
   to) application instance, which would typically be located on a

Cerf, et al.            Expires February 2003               [Page 23]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   different host.  The return receipt uses the same custodial delivery
   option used in the bundle that caused the return receipt to be sent.
   (Return receipts are never issued with the return receipt delivery
   option requested.)

   The third and fourth delivery options are used for diagnostic
   purposes, and may not be available to all users, subject to local
   policy.  These two delivery options result in notifications being
   sent to the reply-to application.  The first causes a notification to
   be sent every time a bundle is forwarded.  The second option causes a
   notification to be sent every time a custody transfer occurs.

4.5  Naming and Addressing

   As described in section 3.4, bundle application instances refer to
   one another using tuples.  The tuple consists of two parts: a region
   identifier and an entity identifier.  DTN regions will be discussed
   in more detail in section 5.2, but they are named by applications
   using syntax identical to that used in the domain name system (DNS).
   (That is, hierarchical tree structure, dot-separated text node names,
   left to right progresses from leaf to root, sibling nodes must have
   different names.)  When a region is served by the Internet's Domain
   Name System distributed database, DTN region names may or may not be
   stored in that database.  The bundle layer may translate a region
   name to a bundle-layer-specific region address for transmission.  The
   scope of the region name space and region address space spans an
   entire DTN, and name-to-address translations of DTN regions must be
   consistent and reversible throughout the DTN.

   The other portion of a tuple (the entity identifier) is opaque
   outside the region specified by the tuple's region identifier (the
   "home region" for the entity).  In internet-served regions, an entity
   ID may be a hostname, protocol identifier, port (used to find the
   bundle service on the destination host) and potentially a token used
   for demultiplexing to a particular application using the bundle
   service.  These could potentially be combined into a URL (depending
   on the region's internal rules).  In diffusion-routed regions, they
   may be an expression of interest [IGE00]. In any case, no attempt
   shall be made by the bundle layer to bind the entity ID to an address
   from any location outside the entity's home region.

   Discovery of valid DTN region names and entity identifiers is out of
   the scope of this document.

5  Topological Considerations:  Nodes, Regions, and Gateways

   This section describes the three key topological elements in the DTN
   architecture and how they are related: DTN nodes, DTN regions, and
   DTN gateways.

Cerf, et al.            Expires February 2003               [Page 24]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

5.1  Nodes

   A DTN node (or "node" in this document) is an engine for sending and
   receiving DTN messages (bundles).  DTN nodes may act as sources,
   destinations, or forwarders of bundles.  There is a one-to-one
   mapping between DTN nodes and DTN entities.  Each node is uniquely
   identified by at least one tuple containing region identifier and
   entity identifier; a node that is reachable within multiple regions
   will have at least one identifying tuple for each region within which
   it is reachable.

   The identifier for a DTN node, meaning the bundle sending and
   receiving engine itself as opposed to an application *using* that
   engine, is specified in a region-specific manner using all of part of
   the entity identifier.

5.2  Regions

   The following list identifies the requirements for DTN regions:

   . Each DTN region shall have an identifier space that is shared by
     all DTN nodes within the region.  A region must specify the naming
     conventions to be employed within that region for entity

   . Each node that is a member of the region shall have a unique
     identifier drawn from that identifier space.  (Note that for some
     types of regions, a "node" may be made up of a collection of
     computational elements, possibly geographically distributed.  A
     single unique identifier may collectively refer to them.  Further,
     the unique identifier requirement only applies to nodes that are
     intended to receive data from other DTN nodes.)

   . To be considered a member of a region, each prospective member of
     the region shall have the ability to reach every other member of
     the region without depending on any DTN nodes outside the region.
     (Although a DTN node may not necessarily be *directly* reachable.
     This may require forwarding and/or store-and-forward operation by
     other DTN nodes inside the same region.)

   A DTN region is a generalization of the notion of a "network" that
   relaxes the assumption of real-time end-to-end connectivity.

   DTN regions may be created for a number of reasons.  Accommodation of
   tailored support for a particularly challenging networking
   environment is one reason for defining a DTN region.  Delimiting a
   particular security boundary may be another reason for defining a DTN
   region boundary.

Cerf, et al.            Expires February 2003               [Page 25]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

5.3  Gateways

   A gateway is an entity that has membership in two or more DTN regions
   and relays messages between regions.

   Entities that reside (only) in one region can only send messages to
   entities in other regions with the assistance of one or more DTN

5.4  Discussion

   DTN nodes may act as "hosts," meaning entities that send and receive
   bundles, but do not forward them.  DTN nodes may also act as
   "routers", meaning they perform the functions of a host, but also
   forward bundles within a single region.  Finally, DTN nodes may act
   as gateways, forwarding bundles across region boundaries.  A single
   DTN node may act in any subset of these roles simultaneously.

   As members in a store-and-forward chain, DTN nodes have the
   responsibility for resource allocation to support bundle transfers.
   These resources include, among other things, buffer space and
   transmission capacity.

   Additionally, DTN nodes have the responsibility of actually executing
   the bundle transfer.  Reliability requirements for bundle transfers
   are specified by the using application, and include both reliable and
   unreliable transfers.  The DTN nodes are responsible for using
   whatever reliability mechanisms exist in the underlying (transport-
   and-below) layers, and augmenting those mechanisms with bundle-layer
   mechanisms to effect the required reliability.

   We require that each DTN region have a unique identifier that is
   known among all regions in the DTN or that can be discovered as

   The entity identification rules for a particular DTN region have
   scope only within that region, and are opaque outside that region.
   However, all DTN nodes within a region must know and conform to the
   entity identification rules in order to successfully communicate.

6  Routing considerations

   Communication within a DTN can be either intra-regional or inter-
   regional.  While this architecture does not prescribe specific
   methods for the exchange of routing data, it acknowledges the fact
   that mechanisms for the exchange of intra-regional bundle routing
   data may well be quite different from the mechanisms to exchange
   inter-regional routing information.  Further, the architecture

Cerf, et al.            Expires February 2003               [Page 26]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   recognizes that mechanisms for inter-regional transfer in one portion
   of a DTN may be inappropriate for use in other portions of that DTN,
   and does not require homogeneity of inter-region routing protocol
   throughout the DTN.

6.1  Types of Routes

   DTNs may be required to function in the presence of any or all of the
   following types of routes.  These are presented as information rather
   than as requirements for all DTNs, although any of these may be
   required for a *particular* DTN.

6.1.1  Persistent Contacts

   Persistent contacts are sessions with a neighboring DTN node that are
   always available or that can be made available on demand.  In the IP
   world, Digital Subscriber Line (DSL) connections are an example of
   the former, while dial-up connections are an example of the latter.

6.1.2  Intermittent - Scheduled Contacts

   Scheduled routes are those where there is an agreement to establish a
   link between two points at a particular time, for a particular
   duration.  Attributes of that link, such as its data rate,
   probability of loss, and the routing metrics to destinations via that
   link, are routing protocol-specific.

   An example of a scheduled route is a link that uses a low-earth
   orbiting satellite.  A schedule of view times and durations can be
   constructed when next-hop neighbors are accessible via the satellite.

6.1.3  Intermittent - Opportunistic Contacts

   Opportunistic contacts are those that are not scheduled, but rather
   present themselves unexpectedly.  Such contacts could be with an
   aircraft flying overhead and beaconing, advertising its availability
   for communication.  Another type of opportunistic contacts might be
   via infrared communication link between a personal digital assistant
   (PDA) and a kiosk in an airport concourse offering bundle service as
   the PDA's owner walks by.  If the PDA's owner authorizes it, the PDA
   could communicate the owner's planned path and a desire for contacts
   with subsequent kiosks in the concourse, resulting in a set of
   probable contacts for which bundle transfers can be scheduled.

6.1.4  Intermittent - Predicted Contacts

Cerf, et al.            Expires February 2003               [Page 27]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   Predicted contacts are those that are based on no fixed schedule, but
   a history of opportunistic contacts that suggests that contact with
   an intermittently-connected neighbor will probably occur within a
   certain period of time and will probably last for a certain duration.
   Given a great enough certainty that the contact will occur, a DTN
   node may allocate bundles to that predicted contact period that would
   be allocated to different contacts otherwise.  In the previous
   example, the probable contacts in the airport concourse would result
   in the establishment of a set of predicted contacts of a given
   duration (perhaps based on the PDA owner's walking speed and the
   kiosk's coverage area).  The PDA could decide how to use those
   contacts for transmitting waiting bundles, as well as perhaps to
   request bundles that were awaiting delivery at any of a number of
   store-and-forward points to which the user had access.

   The algorithms for establishing the predicted time and duration of a
   contact, the degree of uncertainty in those estimates, the time at
   which to abandon the wait for a predicted contact, and the guidelines
   for allocating bundles to such contacts are all open research areas.

6.2  Store-and-forward operation

   While IP networks are based on "store-and-forward" operation, there
   is an assumption that the "storing" will not persist for more than a
   modest amount of queuing and transmission delay.  In contrast, the
   DTN architecture does not expect that an outbound link will be
   available when a bundle is received, and expects to store that bundle
   for some time until a link does become available.  We anticipate that
   most DTN nodes will use some form of persistent storage for this --
   disk, flash memory, etc.

   All DTN forwarding nodes ("DTN routers"), even those that do not
   provide custodial operations as described in section 3.8, must be
   able to queue bundles until outbound contacts are available.  Each
   DTN forwarding node and DTN gateway that also provides custody
   transfer operations must provide for longer-term storage of bundles,
   committing to store bundles for which it takes custody for the entire
   remainder of their lifetime, if necessary.  It is entirely possible
   that a custodian will need to "take a break" from further
   custodianship while it completes pending custodial operations and
   recovers persistent storage.  Two mechanisms support this: First, the
   node can simply forward incoming bundles without taking custody.  The
   fact that a node is a potential custodian is no guarantee that it
   will take custody of any given bundle that is routed to it.  Second,
   the node can revise its advertisement of custodial capability in
   routing updates (see section 6.4).  This is a longer-term solution,
   but has the attractive property that DTN nodes searching for a
   custodian do not route a bundle out of its way vainly in search of
   custodianship at the node in question.

Cerf, et al.            Expires February 2003               [Page 28]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

6.3  Contact-oriented routing

   Making routing decisions to allocate bundles to future contacts
   requires a DTN forwarding node to probabilistically estimate what the
   connectivity to other points in the network will be via that contact.
   We do not yet have a significant body of experience with this, but we
   suspect that maintaining a weighted average of previous routing
   metrics and developing an uncertainty metric will be useful.

   Based on the expected connectivity to the remainder of the network
   from each of these intermittent contacts, and similar
   characterizations of connectivity for persistently connected
   neighbors, a DTN forwarding node can make decisions about how to
   allocate bundles to contacts.

   One could think of this allocation of bundles to contacts as being
   made by creating an ordered list of references to the bundles in
   persistent storage and providing that list to the lower layer
   protocol (or to the convergence layer) for transmission at the start
   of a contact.  (We refer to such a list of bundles as a "contact
   list.")  Bundles not transmitted during the contact could be returned
   on the list for allocation to subsequent contacts. With such an
   approach, the bundle layer is certainly free to adjust the allocation
   of bundles to future contacts, based on the receipt of new, possibly
   high priority bundles.  It is also possible that the bundle layer
   could make changes to a contact list during the course of a contact,
   if the implementation permitted.

6.4  Routing protocols

   We have not yet developed routing protocols for this architecture,
   but envision the need for at least two: one to support inter-region
   routing, and at least one to support intra-region routing.
   Realistically, there will probably be more than two.  It is likely
   that an inter-region routing protocol for a network having one or
   more links with 40-minute one way delays and two-week link outages
   (such as might be encountered in a deep-space mission) would be
   wholly different than an inter-region routing protocol to support a
   sensor network connected to the Internet via a low-earth orbiting

   Our current thinking in routing protocols to support intermittent
   connectivity is to maintain one-hop link state information and a
   distance-vector type of aggregation beyond one hop.  We feel that
   attempting to maintain an "accurate" picture of network connectivity
   beyond one hop when routes are composed of scheduled or probabilistic
   connections of uncertain capacity will be futile.  However, we

Cerf, et al.            Expires February 2003               [Page 29]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   believe that a probabilistic view of connectivity can be built and
   advertised in a scalable way that permits effective use of these
   intermittent contacts.  The fact that there is no expectation of a
   contemporaneous end-to-end path makes the overall job seem more

   As previously mentioned, this area is teeming with interesting
   research topics.  We intend to address some of these in coming
   months, but we certainly solicit the participation of interested

7  Security considerations

   While the particular security problems presented by delay-tolerant
   networks do not necessarily differ from those presented by other
   networks, the environment has a significant effect on the available
   solution space.

   In addition to typical end-user oriented security, providing writer-
   to-reader services such as message confidentiality or end-to-end user
   authentication, protection of the network infrastructure and
   controlling access to that infrastructure tend to be important in
   delay-tolerant networks, which are typically resource-challenged and
   are critical components to mission success.  Along with high round-
   trip times and frequent partitioning, delay-tolerant networks often
   have low data rate links, making efficiency a necessary consideration
   in any solution.  It is not unusual for delay-tolerant networks to be
   deployed in places where a portion of the network might become
   compromised.  Such compromise, should it occur, must be limited in
   scope and the effect finite in duration.

7.1  User-oriented security services

   We have not received explicit user requirements for the following
   security services, but anticipate their need.  We are as yet
   undecided whether to provide confidentiality and user authentication
   as explicit bundle-layer services (a la IPSEC) or to simply make the
   key management and distribution mechanisms we require available to
   user applications for them to provide their own confidentiality and
   authentication (a la PGP, SSL, etc.).  Current discussions have
   focused on using a combination of public key and symmetric key
   cryptographic methods for the security services.

7.1.1  Confidentiality

   The confidentiality service attempts to insure that the information
   content of a bundle is not disclosed to anyone except the intended
   recipient.  If the bundle layer provides the confidentiality service
   (a la IPSec Encapsulating Security Payload) the intended recipient

Cerf, et al.            Expires February 2003               [Page 30]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   will be the bundle layer entity at the destination (rather than any
   specific application).  In a public key based system, this requires a
   high confidence that the sender has the public key of the intended
   recipient and that said key is unique.  The confidentiality service
   will render all affected portions of the bundle unreadable, so it
   cannot be applied to fields in the bundle header that are needed for
   delivery.  In general, the bundle user data is covered by the
   confidentiality service but none of the rest of the bundle header is

7.1.2  User Authentication

   User authentication is a capability to prove that the bundle has been
   sent by the entity that claims to have sent it.  Typically, an
   authentication service ensures that the content of the bundle is
   unaltered as a means to prevent replay attacks.  Authentication is
   accomplished in a public key system by the use of a digital
   signature.  A digital signature is created by computing a hash of the
   data to be covered by the authentication, and then encrypting that
   information with the private key of the sender.  If the sender's
   public/private key pair is uncompromised and the cryptographic
   algorithm is sufficiently strong, the recipient can apply the
   purported sender's public key to the encrypted data, recover the
   hash, and compare it to a hash of the corresponding information in
   the bundle.  This allows the recipient to determine if the sender's
   key is correct (authenticating the sender's identity), and if the
   data received is identical to the data that was sent (verifying the
   integrity of the data).  Note that this service is required to
   support the infrastructure security model described in section 7.2,
   and hence could be made available to end-users to authenticate them
   to the remote bundle entity at relatively low cost.  However, the
   remote bundle entity will have to request a trusted copy of the
   purported sender's public key, potentially adding considerable delay
   to the receipt of the first bundles until a signed copy of the
   sources public key is received, verified, and cached.

7.1.3  Key Management and Distribution

   The DTN Architecture requires that a key management service be
   available that can produce, upon request, a set of bytes that are
   useful to the recipient in verifying the cryptographic correctness of
   an information object.

   An additional service is required that allows the verifier to
   determine the status of the above-mentioned cryptographic
   verification information.

Cerf, et al.            Expires February 2003               [Page 31]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   This description is intentionally abstract, as these services may
   vary with the specific cryptographic mechanisms that are selected
   (e.g., Kerberos tickets, CRLs with X.509, etc.).

7.2  Infrastructure security services

   The following model describes an approach to provide infrastructure
   and data security for a delay tolerant network.  It is based on the
   sensitivity to the effects of long delays, constrained bandwidth,
   constrained data rates, and intermittent connectivity.  Like the rest
   of this architecture, this security model is preliminary and is still
   a work in progress.

   The security architecture is loosely based on the model of secure
   electronic mail.  In the secure electronic mail environment, a source
   sends secured electronic mail to a destination only knowing the
   destination's email address.  The security attributes of the
   electronic mail may be that the contents have been encrypted,
   digitally signed, or both encrypted and signed.

   Encryption provides "writer-to-reader" confidentiality.  That is,
   only the source and the destination will be able to read the contents
   of the message.  Digital signatures provide authentication of the
   source of the message as well as message content integrity.  In this
   way, the receiver of the message is assured that the message came
   from the claimed source and that the message contents have not been
   tampered with in transit ¡ what is received is exactly what was sent.

   In the secure electronic mail environment, few assumptions are made
   about the receiver of the mail.  When a message is digitally signed,
   the sender's private key is utilized.  In order to assure that the
   receiver will be able to authenticate the signature, the sender's
   authenticated public key is sent along with the message to the
   receiver.  The receiver is able to extract the sender's public key
   from the message, verify the public key's authenticity based on its
   being signed by a trusted-third party (e.g., a Certificate Authority
   (CA)), and then use the verified public key to verify the message
   digital signature.  The receiver needs to know nothing, security-
   wise, about the sender other than having a copy of the public key of
   the certificate authority (or equivalent).

   In order to encrypt the message contents, the sender must have the
   receiver's public key.  By virtue of public key cryptography, only
   the receiver, using its private key, will be able to decrypt what has
   been encrypted in the receiver's public key.  Typically, the message
   contents are encrypted in a randomly selected symmetric key.  The
   symmetric key is then encrypted using the public key of the receiver.
   The receiver is then able to use its private key to decrypt the
   symmetric key and then use the symmetric key to decrypt the message

Cerf, et al.            Expires February 2003               [Page 32]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   payload.  However, in this case, unlike the digital signature-only
   case, the sender must have some information about the receiver other
   than a destination address.  The sender must have previously obtained
   the receiver's public key via a previous exchange, or must have
   retrieved it from a key server.

   In a DTN, key servers may or may not exist.  In addition, bandwidth
   to communicate with key servers may be extremely limited or the delay
   to accomplish a round-trip may be intolerable.  Nevertheless, there
   still needs to be a means by which public keys may be obtained or

   For this architecture, we make the assumption that DTN systems that
   require public keys will have the means to acquire them.  They may be
   able to generate their own public/private key pairs (as in the PGP
   model) and then have the public key certified as belonging to the
   owner by a trusted third-party.  In this way, a DTN system may only
   need to expose its public key to the trusted third-party and does not
   have to trust the third party with its private key at all.  The
   downside to this is that the pedigree of the key generation software
   may be suspect.  That is, the key generators may exist on the
   specific DTN system but may have been compromised or corrupted.  As a
   result, the keys generated on directly on a DTN system might be weak
   or otherwise bad keys.

   Alternatively, a DTN system's keys may be generated and certified by
   a trusted third-party agent (e.g., Verisign, Thawte, Baltimore).
   Because the trusted third parties base their livelihood on the
   goodness of the keys they generate and the trust they provide through
   their certification of keys, the danger of a corrupted key generator
   producing weak keys is minimized.

   Key pairs are required by both end-systems (e.g., principal
   investigator workstations, space-based instruments, terrestrial-based
   instruments in a sensor web) as well as intermediate DTN systems
   (e.g., DTN routers).  The end-systems will use their key pairs for
   both admission control to a DTN as well as for writer-to-reader
   confidentiality, authentication, and integrity when required.  DTN
   routers will make use of their keys in order to provide assurance
   that only certified DTN systems have the ability to route data
   through a DTN.  In this manner, DTN systems implement "mutual
   suspicion."  That is, they do not trust each other without each
   providing proof of identity.  It is desirable that all DTN router
   interactions are authenticated, however that choice will have to be
   left up to the DTN implementer/operator.

   Four types of DTN elements have security mechanisms and services
   associated with them: an end-user system, a DTN router, a DTN

Cerf, et al.            Expires February 2003               [Page 33]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   interregional gateway (DTN gateway for short), and a DTN certificate
   authority (CA).

   As was already stated, each of the DTN elements will have a
   public/private key pair in place either self-generated and then
   certified by a DTN CA, or generated by a DTN CA and then populated as
   necessary in the DTN elements.  The final decisions about key
   management, key/certificate distribution, and key/certificate
   revocation have not been decided yet and are therefore left for
   future debates.  The key pairs may be pre-placed in the DTN elements.
   Alternatively, they may be exchanged during a "neighbor discovery"
   process.  Or finally, they may be exchanged in the same manner as
   with secure electronic mail.  That is, they may be sent in a
   certificate along with the payload.  The certificates should be
   cached by the recipient systems.  The "safe" method to ensure that
   recipients always have the most up-to-date keys would be to send the
   certificate with every payload.  However, because of bandwidth
   constraints, this may prove to be impractical and therefore an
   intelligent mode of operation should be employed where state
   information is maintained regarding communicating end-points to
   determine if the end-points already have the necessary certificates
   from previous communications.

   When an end-system requires admission to a DTN, that is, when it
   needs to inject data into a DTN, it first presents its credentials to
   a DTN "gatekeeper" which is analogous to a present day Radius (or
   other flavor) access control server.  The end-system's credentials,
   based on a certified public key are checked for authenticity and then
   an access control list is consulted to determine if the end-system
   should be allowed access.

   If the end-system is authenticated and access is allowed to a DTN,
   the end-system must decide whether or not end-to-end security
   services are required.  That is, the end-system must decide whether
   "write-to-reader" confidentiality, authentication, or integrity is
   required.  If end-to-end services are required, then the end-system
   may encrypt the payload with a randomly chosen key and then encrypt
   the key using the public key of the intended recipient of the
   payload.  This would be the equivalent of using Secure Sockets Layer
   (SSL) or its IETF-version, Transport Layer Security (TLS) at the
   application layer.  Alternatively, the end-system could request and
   end-to-end service from the bundle layer, if it is chosen to
   implement such a service at the bundle layer.  If this is done, the
   ends of the end-to-end services would be the sender and receiver
   bundle agents rather than the actual end-systems.  This is because
   the bundle agents have acted on behalf of the end-system to provide
   end-to-end security services.

Cerf, et al.            Expires February 2003               [Page 34]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   Regardless of whether end-to-end security services are required or
   not, the initial DTN router or gateway that receives the payload from
   an end-system applies its own digital signature to the bundle so that
   subsequent DTN routers and gateway can be assured of hop-by-hop
   authenticity.  Likewise, when DTN routers and gateways perform
   housekeeping chores (e.g., routing updates, etc.) those bundles are
   also digitally signed in order to assure that only secured
   transactions between the DTN entities occurs.

   When a DTN router or DTN gateway receives a bundle, and before it
   forwards it onward to a neighbor DTN router or gateway, it first
   validates the bundle by authenticating digital signature from its
   precursor hop.  As the bundle progresses through a DTN, each
   subsequent DTN device performs an authentication function and then
   forwards the bundle onwards with its own digital signature to its
   next hop.

8  Reliability Considerations

   The bundle layer depends on underlying transport services to provide
   hop-by-hop reliability.  In this context, "hop-by-hop" means from one
   DTN node to the next.  Those nodes may be separated by *many* hops in
   an underlying network, and the DTN architecture intends to exploit
   the reliability, flow control, and congestion control capabilities of
   those underlying networks.  However, some significant reliability
   issues remain at the bundle layer.  This section discusses the issues
   that we have currently identified.

8.1  Custodial Operation

   One of the fundamental tenets of the DTN architecture is that we must
   be able to provide reliable end-to-end message transfer via an
   infrastructure that may *never* be connected end-to-end

   Because no single transport-layer protocol operates end-to-end across
   the IPN, end-to-end reliability can only be assured at the bundle
   layer.  At each node along an end-to-end route, the bundle-layer
   protocol entity passes bundle data to the Transport layer for
   transmission.  Each bundle layer entity is highly confident that the
   transport layer will successfully convey the data entrusted to it to
   the next bundle-layer protocol entity (which may "take custody" of
   the data or merely relay it; a single hop).  But failures are
   possible (e.g., a host computer does an unplanned reboot).

   A bundle custodian makes a commitment to store the subject bundle
   until one of two events occurs: another DTN node accepts custody of
   the bundle, or the bundle's time to live expires.  The bundle layer
   attempts to deal with bundles atomically and independently of other

Cerf, et al.            Expires February 2003               [Page 35]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   bundles.  In this model, positive acknowledgement of a bundle is
   possible, but negative acknowledgment is not.  We consider the
   opportunity to *not* deal with partial bundles a significant plus.
   Since the underlying transport protocols are working on our behalf,
   we do not consider the lack of a negative acknowledgment capability a
   significant minus.  (Note that there actually *is* a circumstance in
   which a negative acknowledgment can be sent -- when the time to live
   for a bundle expires.  However, at that point, *all* copies of the
   bundle, including those stored at a custodian, are purged from the
   network.  A "timed-out" error can be sent to the source, which
   *might* be able to cause either an application layer retransmission
   or a bundle-layer retransmission (see section 8.2, below).)

   Without a negative acknowledgment mechanism, retransmission must be
   accomplished solely via timer expiration.  The bundle layer's
   confidence in the effectiveness of the underlying transport-layer
   protocols is reflected in the design of the timers for bundle-layer
   reliability.  These timers are highly optimistic ¡ that is, they
   expire as late as possible ¡ in order to give the transport protocols
   every opportunity to complete reliable transmission.  The effect of
   this optimism is to minimize the chance of unnecessary bundle-layer
   retransmission, which could seriously degrade DTN performance by
   consuming valuable bandwidth.

8.2  End-to-end Retransmission

   Even custody transfer operating on a bundle-hop by bundle-hop basis
   does not provide guaranteed end-to-end reliability.  This can only be
   achieved with an end-to-end reliability mechanism operating between
   source and destination.  In most cases, custodial reliability should
   suffice.  However, the architecture also provides for end-to-end
   reliability using retransmission mechanisms that are in addition to
   those used for custody transfers.

   The availability of timer-based retransmission and acknowledgment
   mechanisms to support custody transfers means that the mechanisms to
   support end-to-end retransmission at the bundle layer are essentially
   available for free.  If a DTN user has requested both custodial
   transfer and the return receipt delivery option, it is reasonable to
   assume that end-to-end retransmission would be appropriate if, for
   some reason, the custody transfer operation failed.  However, for
   timer-based retransmission, we must set the retransmission timer to
   some reasonable value to account for the diligent efforts and
   ultimate failure of the underlying transport protocols to succeed in
   either the end-to-end transfer or the end-to-end return receipt.

   For custody transfers, appropriate retransmission timer setting is
   less of an issue, since presumably the transfer between the current
   custodian and the next-hop custodian traverses fewer hops carried by

Cerf, et al.            Expires February 2003               [Page 36]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   underlying transport protocols.  One could reasonably expect better
   predictions of this round trip time than one could for an end-to-end

   The result is that a conservative (optimistic) retransmission time
   out value for an end-to-end transfer will likely be VERY large.  And
   that value plus the anticipated end-to-end delay for a retransmitted
   bundle must be less than the time to live for the user's data or else
   the retransmitted data will time out in transit.  If a bundle timed
   out in transit and the source received the indication of that error,
   the bundle layer could respond with an end-to-end retransmission,
   assuming that the remainder of the application's statement of the
   time-to-live for that bundle to have a reasonable chance of end-to-
   end transfer success.

   So, while we believe that the mechanisms are available to effect end-
   to-end retransmission, we are unsure whether it will be of value in
   all DTN environments.  It is possible that it may provide value in
   certain environments, and we will continue to consider whether it
   should be provided as an implicitly-requested capability when users
   request both custody transfer and return receipt.

8.3  Congestion and Flow Control at the Bundle Layer

   The subject of congestion control and flow control at the bundle
   layer is one on which the authors of this draft have not yet reached
   consensus.  We have unresolved concerns about the efficiency and
   efficacy of congestion and flow control schemes implemented across
   long and/or highly variable delay environments.

   One view of congestion control is as follows:  "Congestion control is
   concerned with allocating the resources in a network such that the
   network can operate at an acceptable performance level when the
   demand exceeds or is near the capacity of the network resources.
   These resources include bandwidths of links, buffer space (memory),
   and processing capacity at intermediate nodes.  Congestion occurs
   when the demand is greater than the available resources." [RJ90]

   For the purposes of this document, we define "flow control" as a
   means of assuring that the rate at which a sending node transmits
   data to a receiving node does not exceed the maximum rate at which
   the receiving node is prepared to receive data from that sender.
   (Note that this is a generalized notion of flow control, rather than
   one that applies only to end-to-end communication.  In particular,
   the concept of flow control between the two ends of a single link may
   be indispensable in such DTN regions as the "interplanetary
   backbone.")  We define "congestion control" as a means of assuring
   that the aggregate rate at which all traffic sources inject data into
   a network does not exceed the maximum aggregate rate at which the

Cerf, et al.            Expires February 2003               [Page 37]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   network can deliver data to destination nodes.  If flow control is
   propagated backward from destination nodes to routers and eventually
   back to traffic sources, then flow control can be at least a partial
   solution to the problem of congestion as well.

   DTN flow control decisions must be made within the bundling layer
   itself based on information about resources available within the
   bundle node.  However, the bundle layer *might* be able to delegate
   the implementation of those decisions to the underlying Transport
   protocol, as follows.

   We have not yet considered multipoint communication at or below the
   bundle layer, so each individual flow control problem involves just
   two nodes.  Since inter-region traffic must pass through inter-region
   gateways, any two nodes between which flow control is an issue must
   inhabit a common DTN region and be using a common Transport protocol
   below the bundle layer (otherwise they could not be communicating and
   there would be no flow to control).  Therefore, it may be possible to
   accomplish DTN flow control by invoking whatever flow control
   mechanisms are already provided within the region.

   Alternatively, a new, supplementary flow control protocol could be
   developed at the bundling layer; this would have the advantage of
   reducing DTN's reliance on capabilities provided by the underlying
   protocols.  At this time it's still unclear which approach is

   In either case, the problem of flow control between nodes separated
   by very large signal propagation times remains to be solved: TCP's
   flow control and congestion control facilities could be leveraged
   within regions characterized by small round-trip times, but at this
   time no comparable protocol exists for very long delay paths, such as
   the "interplanetary backbone".  [The Long-haul Transmission Protocol
   referenced below has yet to be developed.]

   It remains as an exercise for us to demonstrate that a hop-by-hop
   flow control scheme in long and/or highly variable delay environments
   can effect end-to-end congestion control in a fair and efficient

9  State Management Considerations

   An important aspect of any networking architecture is its management
   of state information.  This section describes the state information
   that is managed at the bundle layer and discusses the conditions
   under which that state information is established and how it is

Cerf, et al.            Expires February 2003               [Page 38]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

9.1  Application Interface State

   Although it is implementation-specific, it is very likely that
   application-programming interfaces (APIs) to the bundle layer will be
   stateful.  In long/variable delay environments, an asynchronous
   interface seems to be the only sensible approach, and such interfaces
   involve registration actions that create state information.  This
   information is typically created by explicit request of the
   application, and is removed by a separate explicit request (or,
   possibly, upon termination of the application).

   Since operation in a DTN environment means that delays might be quite
   long, it is reasonable to expect that an application instance might
   terminate (voluntarily or not) between the time that a bundle is sent
   and the time its response is received.  The application interface to
   the bundle layer will, in most cases, provide a mechanism to
   reinstantiate applications that have terminated, assuming the
   application has been written to take advantage of such a service.

   When a DTN node that is the destination of a bundle receives it, the
   bundle layer attempts to deliver it to the destination application
   instance indicated in the destination tuple. If the application
   instance is unavailable for immediate delivery (for example, it runs
   as a result of a "cron" job), then state is created pending delivery
   of the bundle.  That state is removed on successful delivery of the
   bundle to the application instance or on expiration of the time to
   live in the bundle.  Note that if the return receipt delivery option
   is enabled for a bundle, that return receipt is not generated until
   the bundle has been successfully delivered to the destination

   Note that in some DTN environments, delays might REALLY be long, and
   bundle layer operations may be required to operate across system
   reboots, host changes, or possibly even operating system upgrades.
   To facilitate this, some bundle layer implementations will employ
   various forms of persistent storage for their state information, and
   will avoid operating system attempts to "clean up" state information.

9.2  Bundle retransmission state

   State information to support bundle retransmission is created at a
   DTN node as a result of either of two events: First, state is created
   when a bundle is received from a DTN user application requesting
   custodial transmission (assuming that DTN node is capable of
   supporting custodial operation).  Second, state is created when a
   bundle is received from a peer DTN node and the receiving node
   intends to assume custody of the bundle.

Cerf, et al.            Expires February 2003               [Page 39]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   In the first case, recall that applications may indicate a bundle
   time to live that is greater than that which is allowed in the
   network.   (For example, a DTN application may legitimately feel that
   its data has value for a century or more, while the DTN that is
   carrying it doesn't want a particular bundle to potentially be in
   transit for that long.)  The bundle layer may reduce the time to live
   value in the bundle itself to a value that is consistent with network
   operation rules.  The bundle layer at the source *should* retain the
   information about the application's view of the bundle's valuable
   lifetime.  In the event that a bundle *transmission* times out before
   custody is assumed, the bundle layer *may* restart the bundle
   transmission procedure.  This highlights the need to be careful to
   ensure that bundle acknowledgments have a high probability of
   receipt, to avoid spurious retransmissions of bundles.  At the
   source, the state associated with a custodially transmitted bundle is
   removed when a custody transfer acknowledgment is received, or when a
   period of time subject to local policy has elapsed.

   For the second case (that of an in-network custodian), state
   information is removed when a custodial acknowledgment for the bundle
   is received or when the bundle's time to live has been exceeded.

9.3  Bundle routing state

   Routing tables, whether statically configured or maintained by
   routing protocols, introduce state information that must be managed.
   If the general approach to maintaining routing information is as
   described in section 6.4 (that is, maintaining link state information
   for next-hop neighbors, and distance vector information for points
   beyond), then we can make some basic statements about the amount of
   state information that must be maintained.  For persistent links, in
   the worst case the volume of routing information for inter-region
   routing scales by (n*R), where n is the number of next hop neighbors
   and R is the number of regions in the DTN.  For intermittent links,
   the volume of information scales by (c*n*R), where c is the number of
   contact periods maintained for planning purposes.  These values are
   worst-case.  Pruning and aggregation mechanisms can be used reduce
   the volume of information if necessary.  In particular, it is likely
   that the intermittent links may be reduced to order (n*R) by simply
   assuming that the distance vector information is constant for all
   contacts.  (There probably will not be enough useful information to
   do otherwise.)  We have not yet seriously considered the routing
   metrics that we will maintain, so we do not have an estimate for the
   size of each routing entry.  This remains work to be done.

   Additionally, we have not given significant consideration to intra-
   region routing, which will likely have different scaling properties.
   Intra-region routing information will be affected by the number of
   DTN gateways that are members of the region and by the routing

Cerf, et al.            Expires February 2003               [Page 40]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   approach used within the region.  (By routing approach we mean
   broadly either proactive, meaning that routes to all possible
   destinations are maintained, or reactive, meaning that routes are
   discovered only when necessary and discarded after some period of
   time.)  This also directly affects how and when routing state
   information is removed from a DTN forwarding node.

9.4  Transmission queue state

   Transmission queues are maintained within DTN nodes to manage bundles
   awaiting transmission.  Although implementation dependent, this state
   information will likely include a list of next hop destinations, and
   for intermittently connected next-hop destinations, a sublist of
   upcoming contacts.  These lists will likely contain lists of bundles
   for transmission on those contacts.  When a new contact is scheduled
   or predicted with an intermittently-connected next-hop neighbor, a
   new list is created and made available for population with bundles.
   Bundles are removed from these lists upon successful transmission.
   If a contact ends with bundles remaining on the list to be
   transmitted, those bundles are allocated to subsequent contact lists
   and the list for the completed contact is removed.

9.5  Receive queue state

   Incoming bundles are received from lower layer entities and either
   placed on a transmission queue (for bundles to be forwarded) or
   placed in a receive queue (for local delivery).  Receive queues are
   maintained to support demultiplexing of incoming bundles.  Again,
   although implementation-specific, receive queues are likely
   associated with particular local application entities.  A bundle is
   removed from its receive queue when it has been successfully
   delivered to its destination application entity.  The receive queue
   itself is removed when an application explicitly requests that its
   information be purged from the bundle layer (as part of an un-
   registration action), or when the application entity has terminated
   without leaving instructions for the disposition of remaining

9.6  Network management state

   We acknowledge that network management information is likely to be a
   necessary part of the operation of delay tolerant networks.  We have
   not, at this point, done a meaningful amount of work in identifying
   the network management requirements for DTNs or in defining the state
   to support such management.  This remains an item for future work.

Cerf, et al.            Expires February 2003               [Page 41]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

10 Convergence Layer Considerations for Use of Underlying Protocols

   The DTN Architecture provides for the existence of a convergence
   layer between the bundle layer and underlying transport protocols.
   The convergence layer manages the protocol-specific details of
   interfacing with a variety of underlying transport services and
   presents a consistent interface to the bundle layer.  The convergence
   layer also allows additional capability to be inserted as necessary
   to augment the services of the underlying transport protocols.

   The convergence layer provides an abstraction to the bundle layer in
   which bundles are delivered atomically.  Partial bundle delivery,
   especially at the end of a contact, is accommodated within the
   convergence layer.  Additionally, the convergence layer may provide
   performance enhancement services such as the ability to resume a
   partially-received bundle on a subsequent contact with the same next-
   hop neighbor (versus starting the bundle transmission over).

11 Bundle Header Information

   The bundle layer must carry some information end-to-end.  This
   section documents our current thinking on the information that must
   be carried end-to-end, and notes which of those data elements may be
   supplied by the application using the bundle service.

    * Version Identifier: this is small integer that indicates which
       version of the bundle protocol is being invoked.
    * Destination entity ID: this is the identifier of the destination
       bundle application instance, and is a tuple, as described above.
       It is supplied by the local application using the bundle
    * Source entity ID: this is the identifier of the source bundle
       application instance, and is a tuple.  It is supplied by the
       local bundle service, since a particular host may have multiple
       names and one may be chosen based on routing decisions or other
       criteria opaque to the application (as in multihomed hosts).
       The source entity ID may be returned to the application to
       support return receipt processing.
    * Reply-to entity ID (optional): a source may anticipate not being
       able to accept replies, and may use the reply-to entity ID to
       specify the destination for return receipts and delivery
    * Current custodian ID (optional): the bundle protocol supports
       store-and-forward operation in which the custody of a bundle
       (that is, the responsibility for ensuring reliable delivery) may
       transfer from one DTN node to another as the bundle progresses
       through the DTN.  There is not a requirement for *each* DTN node
       encountered to assume custody of a bundle.  As a result, it is
       necessary to identify the upstream node that currently has

Cerf, et al.            Expires February 2003               [Page 42]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

       custody of the bundle, in order to acknowledge correct receipt
       of the bundle and to accept custody of the bundle.
    * Class of service flags:
       . an indication that custody transfer is requested,
       . an indication that a return receipt is requested,
       . an indication that a delivery record for custody transfers is
       . an indication that a delivery record for all forwarding
          operations is requested,
       . an indication of the priority of the bundle (bulk, normal, or
          expedited), and
       . an indication of whether authentication information is present
          and its type.
    * Send timestamp: the time that a bundle was presented by the
       sending application to the bundle layer for transmission.
    * Time to live: an offset, in seconds, from the send timestamp
       that indicates when the bundle shall be purged from the DTN.
    * Authentication information (optional): access control
       information to prove that this bundle should be forwarded in the
    * User data: this is intended to be all of the data that the
       remote entity requires to perform whatever operation is
       requested.  Since the environmental characteristics of a DTN
       tend to make interactivity difficult, the notion is that all of
       the information that is required to perform a particular
       "transaction" would be provided in a single bundle or
       unidirectional sequence of bundles.

   Some bundles cause a response to be generated by the bundle layer.
   Bundle layer responses are sent as bundles with the user data portion
   replaced by a Status Report, consisting of the following information:

    * Source entity ID of the subject bundle: this field partially
       identifies the bundle that is the subject of the Status Report.
    * Send timestamp of the subject bundle: this field completes the
       identification of the bundle that is the subject of the Status
    * Status flags, indicating whether or not:
       . The subject bundle was received correctly by the source of the
          Status Report
       . The source of the Status Report has taken custody of the
          subject bundle
       . The source of the Status Report has forwarded the subject
       . The Status Report contains the time at which the source of the
          Status Report received the subject bundle
       . The Status Report contains the time at which the source of the
          Status Report forwarded the subject bundle

Cerf, et al.            Expires February 2003               [Page 43]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

    * Time of Receipt (optional): the time at which the sender of the
       Status Report received the bundle.
    * Time of Forward (optional): the time at which the sender of the
       Status Report forwarded the bundle.

12 An Example Bundle Transfer

   We provide the following example of an end-to-end bundle transfer
   across a Delay-Tolerant Network.  This particular example is based on
   the Interplanetary Internet: A host on Earth sends a bundle to a
   destination on Mars.  Figure 2 illustrates the network that is the
   subject of this example ¡ the Interplanetary Internet in this example
   has five distinct regions interconnected by four DTN Gateways.  This
   example highlights the actions taken by the bundle layer and the
   interactions of the bundle layer with applications and with
   underlying transport protocols.

12.1 Rules for forming tuples in the example network

   As noted in section 4.5, application instance ID tuples consist of
   two parts: a region ID, an entity ID.  Section 5.2 requires that each
   DTN region have an entity identifier space shared by all DTN nodes
   within that region.  Section 5.4 requires that each DTN region have a
   unique identifier that is known (or discernable) by all regions in
   the DTN.  This particular delay-tolerant network is the
   Interplanetary Internet.  In this section we will establish the
   naming rules that permit interoperation within this network.  Note
   that this is only an example, and that not all DTNs must use this
   particular region identifier space (subject to the requirements of
   section 4.5).

12.1.1 Region identification

   All regions in *this* DTN (the example Interplanetary Internet) must
   share a region identifier space.  Per section 4.5, the DTN region
   *name* space is hierarchical, dot-separated text, structured such
   that left to right is leaf-to-root in the tree.  The *identifier*
   space may be exactly this name space, or a DTN may define a
   translation from the name to a particular identifier, in the same way
   that names in the DNS may be translated to Internet addresses.  For
   this example, we will use the *names* as the *identifiers*.

   The DTN region name space is shown in Figure 1:

Cerf, et al.            Expires February 2003               [Page 44]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

        Root                             .
        The Interplanetary Internet     int
        The Solar System                sol
                           |     |     |     |     |
        Region          jupiter mars earth venus  ipn

     Figure 1. An Example Interplanetary Internet Region Name Space

12.1.2 Intra-region identification

   For simplicity in this example, we will assume that all regions use a
   well-known TCP port in combination with DNS host names as the portion
   of their entity identifier that locates the application providing the
   bundle service.  The bundle layer application resides above this
   well-known port (which we arbitrarily choose to be TCP port 6769, a
   currently unassigned port in the registered port space). The
   interplanetary backbone region, labeled "ipn" in Figure 1, will
   certainly NOT use TCP as its underlying transport protocol.  For the
   sake of simplicity in this example, let us assume that the protocol
   used within the interplanetary backbone region uses the same entity
   identifier space (IP addresses and port numbers) that the remainder
   of the network uses.

   The mechanism used to demultiplex bundles received by a bundle node
   is entity-specific, and for the sake of simplicity, we will assume
   that this is a process ID that has been incorporated into the entity
   ID.  [Note:  this is a simple, but quite limiting decision, as it has
   implications on process reinstantiation and process
   portability/migration from one entity to the next.  We choose it only
   for simplicity.]

12.2 Example Network Topology at the Region Level

   It is important to have some understanding of the routing that is
   required at the DTN Gateways, so it is important to understand the
   topology of the network at the region level.  Unlike typical
   terrestrial communications, the Interplanetary Internet's (IPN's)
   *long-haul* communication links are directional, mobile, and highly
   scheduled.  This is important, because directionality combined with
   mobility means that a transmitter and receiver must track each other
   in order to establish and maintain a communication link.  In the IPN,
   much of the mobility is due to orbital mechanics and is therefore
   relatively predictable.  However, this means that nodes that we would
   normally consider to be fixed, such as antennas on the surface of the
   Earth, are actually highly mobile as a result of the Earth's rotation
   and its revolution around the Sun.  (In this example, we confine

Cerf, et al.            Expires February 2003               [Page 45]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   ourselves to our local solar system, and do not consider the motion
   of our sun relative to celestial bodies outside our solar system.)

   We can describe the predictable aspects of node mobility with an
   ephemeris, a table of the positions of celestial bodies at specified
   intervals of time.  Both a directional sender and receiver must each
   know the other's position in order to establish a link between the

                       |     Earth's Internet    |
                       |  DTN Region:  earth.sol |
                       |            +---+        |
                       +-----------/   /|--------+
                                  +---+ |
                                  |G/W| +
                       +----------| 1 |/---------+
                       |          +---+          |
                       |     The "Backbone"      |
                       |  DTN Region:  ipn.sol   |
                      +---+       +---+        +---+
                     /   /|------/   /|-------/   /|
                    +---+ |     +---+ |      +---+ |
                    |G/W| +     |G/W| +      |G/W| +
   +----------------| 3 |/  +---| 4 |/-----+ | 2 |/-------------------+
   |                +---+   |   +---+      | +---+                    |
   | Venus's Internet |     | Jupiter's    |   |    Mars's Internet   |
   | DTN Region       |     |  Internet    |   |    DTN region:       |
   |  venus.sol       |     | DTN Region   |   |      mars.sol        |
   +------------------+     |  jupiter.sol |   +----------------------+

         Figure 2.  An Interplanetary Internet of Five IPN Regions

   In addition, these communication resources are highly scheduled.  It
   is not sufficient for a receiver to point at a prospective target and
   just wait ¡ for example, a terrestrial node will typically have to
   point at several targets sequentially, and an interplanetary node
   will typically not have enough power to just wait for incoming
   messages.  Rather, a schedule of communication *opportunities* must
   be calculated and then refined with planned communication *contacts*.
   A communication opportunity establishes that the endpoints could
   establish a link if they were pointing at each other at the proper
   times.  We refer to a planned communication contact as an agreement
   (although not irrevocable) between the two parties to establish
   contact and communicate for a defined period of time.

Cerf, et al.            Expires February 2003               [Page 46]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   The protocols for establishing the schedule of communication contacts
   between all pairs of possible communicants will evolve -- from
   something primarily done manually to something more automated as the
   real Interplanetary Internet grows.

   The scheduled nature of connectivity in the Interplanetary Internet,
   particularly across the deep-space links, means that at the time of a
   bundle's arrival at a DTN Gateway, some or all of the possible
   outbound routes may be "down."  The gateway must store the bundle
   until the appropriate link is available and then transmit the bundle
   over that link.  One of the fundamental differences between the
   Interplanetary Internet and the terrestrial Internet is this inherent
   use of store-and-forward mechanisms in routing bundles.   With that
   in mind, let us consider the routing decisions made at some of the
   DTN Gateways in Figure 3.

12.3 DTN Gateway routing

   Bundle routing at a DTN gateway will typically have to deal with a
   mix of continuously available links and intermittently available
   links. Routing across a continuously available link is a relatively
   straightforward activity.  Routing in the presence of intermittently
   available links can be significantly more complex, as the gateway
   will need to decide not only the next hop destination but also the
   time at which to send the bundle.  Conditions elsewhere in the
   network may make it more desirable to send a bundle to a next-hop
   destination that is not yet accessible, even when an alternative
   route is currently available.  This decision process is discussed in
   section 6.3.

   The intermittent links in this example provide connectivity among the
   DTN Gateways that are part of the ipn.sol.int region.  The contacts
   among gateways in this region are scheduled.  As is currently the
   case, Gateway 1 (the Earth gateway) has scheduled contacts with each
   of the other DTN gateways in the ipn.sol.int region (the "backbone"),
   but there is no direct contact among any of the DTN gateways on
   Venus, Jupiter, or Mars.  Recall that this communication uses
   directional antennas, so it is generally not possible to communicate
   with two different entities at once unless they are in the same field
   of view.  Power availability on the remote planets is an issue, so
   transmitters and receivers are turned on only during a contact.
   Further, the speed of light delays are nontrivial, skewing transmit
   and receive times between remote gateways.  In Table 1, a schedule of
   contacts is presented that will be used in the example.  All times
   are referenced to Gateway 1.  The information in this table is
   completely hypothetical, and the speed of light delays are plausible,
   but completely back-of-the-envelope.  One-way light times between
   Gateway 1 and Gateways 3, 4, and 2 are 4 minutes, 32 minutes, and 10

Cerf, et al.            Expires February 2003               [Page 47]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   minutes, respectively.  Those details are perhaps interesting but not
   the point of the example.

   Note that any bundles sent by GW3 after 1156 GW1 time cannot be
   acknowledged before the next contact, because the bundle will arrive
   at GW1 after the end of GW1's transmission time (1200).  Since the
   next contact between GW1 and GW3 might be the subsequent Monday, the
   acknowledgement delay might be VERY long.  Bundles sent by GW4 after
   1358 cannot be acknowledged during the current contact, because they
   will not be received before GW1's transmission time ends at 1430.
   Similarly, bundles sent by GW2 after 1150 cannot be acknowledged
   during the contact.

   Table 1.  Contact schedule for Gateway 1.

   | Day     |Destination | GW1 Send  | GW1 Receive | Destination Send |
   |         |            |   Time    |    Time     |   /Receive Time  |
   | Monday  |   GW3      | 0900-1200 |  0908-1208  |  0904-1204       |
   | Tuesday |   GW4      | 1300-1430 |  1404-1534  |  1332-1502       |
   |Wednesday|   GW2      | 1000-1200 |  1020-1220  |  1010-1210       |

   DTN Gateways 2, 3, and 4 have entries in their contact schedules for
   the contacts with Gateway 1.

12.4 Systems participating in example bundle data transfer

   Figure 3 is a revision of Figure 2 that highlights those portions of
   the Interplanetary Internet that participate directly in the bundle
   transfer example.  Also shown in Figure 3 are the source and
   destination for the transfer and the Domain Name System equivalents
   in the terrestrial Internet (DNS 1), in the IPN "Backbone" (DNS 2),
   and in the Mars Internet (DNS 3).  This figure will serve as the
   basis for the bundle data transfer example.

   Table 2 provides the full host names of each of the primary elements
   in Figure 4.  Recall that in this example all bundle layer

Cerf, et al.            Expires February 2003               [Page 48]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

                                       /   /|
             +------------------------+---+ |
           +---+   Earth's Internet   |DNS| +
          /   /|IPN Region:  earth.sol| 1 |/
         +---+ |          +---+       +---+
         |SRC| +---------/   /|--------+
         |   |/         +---+ |
         +---+          |G/W| +
             +----------| 1 |/---------+
             |          +---+          |
             |     The "Backbone"      |
             |  IPN Region:  ipn.sol   |
            +---+                    +---+              +---+
           /   /|-------------------/   /|             /   /|
          +---+ |                  +---+ |            +---+ |
          |DNS| +                  |G/W| +            |DST| +
          | 2 |/                   | 2 |/-------------|   |/+
          +---+                    +---+              +---+ |
                                     |    Mars's Internet   |
                                  +---+   IPN region:       |
                                 /   /|     mars.sol        |
                                +---+ |---------------------+
                                |DNS| +
                                | 3 |/

       Figure 1.  Interplanetary Internet showing principal systems.

   Table 1.  Host name tuples (entity ID demultiplexing info omitted).
   | Host       | IPN Regions      |   Host name tuples        |
   | SRC        | earth.sol        |{src.jpl.nasa.gov:6769,    |
   |            |                  |    earth.sol}             |
   | IPN G/W1   | earth.sol        |{ipngw1.jpl.nasa.gov:6769, |
   |            |                  |     earth.sol}            |
   |            | ipn.sol          |{ipngw1.jpl.nasa.gov:6769, |
   |            |                  |     ipn.sol}              |
   | IPN G/W2   | ipn.sol          |{ipngw2.nasa.mars.org:6769,|
   |            |                  |     ipn.sol}              |
   |            | mars.sol         |{ipngw2.nasa.mars.org:6769,|
   |            |                  |     mars.sol}             |
   | DST        | mars.sol         |{dst.jpl.nasa.gov:6769,    |
   |            |                  |     mars.sol}             |

Cerf, et al.            Expires February 2003               [Page 49]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   applications reside on TCP port 6769.  The portion of the entity
   identifier used to demultiplex to the application using the bundle
   service has been omitted until the applications are discussed in
   section 12.5.1.

12.5 End-to-end Transfer

12.5.1 Step 0:  Application instance registration

   Before the beginning of this example, all of the bundle nodes in the
   network exchange their signed key material and credentials with next
   hop neighbors.  These materials are cached for subsequent use.  The
   applications at the source and destination register with their local
   bundle layers, providing similar key material and credentials to the
   bundle layer, and receiving in return their application instance
   identifiers.  The destination has registered its IPN-standardized
   well-known demultiplexing id of "8," which becomes part of the entity
   ID used to refer to this particular application. The source has
   registered a demultiplexing identifier "0x1763421A" (a hexadecimal
   number) that (arbitrarily) corresponds to its process ID.

12.5.2 Step 1:  Bundle creation and first-hop transmission

   The application on the source host in Figure 3 has data that it
   wishes to send to the destination on Mars.  The exact content of this
   data is opaque to the bundle transfer, but assume that it contains
   all of the information necessary to accomplish some desired function.
   That is, assume that application-specific instruction for storage,
   handling, error processing, and disposal accompanies whatever data
   object is to be operated upon.  The application invokes the bundle
   layer, supplying it the information shown in Table 2.

   The bundle agent verifies the signature, then creates a bundle and
   stores it in persistent storage (on disk or other non-volatile
   memory).  There are some fields of the bundle header that the bundle
   agent must supply: the Sending Timestamp, the Source Host name tuple,
   the Custodian name tuple, and the time to live.  (The application may
   state a time after which the data are obsolete, but the actual time-
   to-live field in the bundle header uses the application's data in
   combination with network restrictions on time-to-live to initialize
   this field properly.)  The bundle layer consults routing tables notes
   that its next-hop destination to reach the mars.ipn.sol region within
   the earth.ipn.sol region is ipngw1.jpl.nasa.gov:6769.  (Since a host
   may reside in multiple IPN Regions, the source host name tuple is a
   function of the outbound route selected.  The bundle layer uses this
   information to complete the Source Host and Custodian name tuples
   prior to transmission.)  Having verified the application's signature,
   it incorporates this into the authentication information of the

Cerf, et al.            Expires February 2003               [Page 50]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   bundle header and appends its own credentials, as described in
   section 7.2.  It further notes that it has a continuous connection to
   that gateway, and transmits the bundle to it, retaining a copy for
   custodial storage.  The bundle layer starts a retransmission timer
   for the bundle and awaits a custodial transfer acknowledgment.

   Table 2.  Information supplied by source application.

   | Item        | Value               | Description                  |
   | Destination | {mars.sol,          | IPN Name tuple of the        |
   | DTN         |  dst.jpl.nasa.gov,  | destination.  Note that appl.|
   | tuple       |  8}                 | demuxing ID 8 is supplied.   |
   | Source      | 0x1763421A          | Value used to identify the   |
   | application |                     | appropriate instance of the  |
   | demultiplex |                     | source application for       |
   | identifier  |                     | response processing (becomes |
   |             |                     | part of the source entity ID)|
   | Class of    | Custodial delivery, | The services requested from  |
   | service     | normal priority,    | the bundle layer.            |
   |             | data obsolete in 36 |                              |
   |             | hours.              |                              |
   | Signature   | Opaque              | Digital signature of the     |
   |             |                     | app.-supplied information    |
   | User data   | N/A                 |                              |

12.5.3 Step 2:  Bundle processing at first-hop destination

   When the IPN Gateway {ipngw1.jpl.nasa.gov, earth.sol} receives the
   bundle via TCP, it checks the signature of the previous router and
   determines that the bundle has been forwarded by a legitimate source.
   Further, since this is the security boundary for the Interplanetary
   Internet, it verifies the signature of the sending application,
   requesting a copy of the application's public key from a secure
   distribution service if it does not already have one cached.  Having
   verified that the application is authorized to access the deep-space
   portion of the Interplanetary Internet, the bundle layer replaces the
   signature block of the bundle layer entity with its own, leaving the
   application's signature block intact.  It stores the received bundle
   on persistent storage (disk).  The bundle layer consults the contact
   schedule and determines that the appropriate next-hop destination is
   in the "ipn.sol" IPN Region:  {ipngw2.nasa.mars.org, ipn.sol}, and

Cerf, et al.            Expires February 2003               [Page 51]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   that the next contact is the following Wednesday.  The bundle layer
   manifests the bundle on the contact for that Wednesday.

   In considering alternative contacts for the bundle, the dispatcher
   checks the time-to-live in the bundle, which was 36 hours from the
   time of initial submission to the bundle agent at the source, to
   ensure that the route selected is consistent with the time-to-live
   requirements of the bundle.  The bundle transport functionality of
   the bundle agent in ipngw1 accepts custody of the bundle, updates
   this information in the bundle header, and informs the source that
   has done so.  The sources bundle agent deletes its custodial copy of
   the bundle. When the time arrives for contact with the
   ipngw2.nasa.mars.org DTN gateway to be established, the convergence
   layer establishes that contact via the Long Haul Transport Protocol
   (LTP).  When informed that the contact is available, the bundle layer
   posts its bundles to the convergence layer for transmission via LTP.

12.5.4 Step 3:  Bundle processing at gateway to destination IPN Region

   The Mars gateway, {ipngw2.nasa.mars.org, mars.sol}, receives the
   bundle from the Earth gateway via LTP.  It verifies the signature
   block of the Earth gateway, then replaces that signature block with
   its own.  It stores the bundle in persistent storage and accepts
   custody of the bundle, signaling back to the Earth gateway that it
   has done so.  When the notification of acceptance reaches the Earth
   gateway, ipngw1 deletes its custodial copy.  The Mars gateway
   consults its routing tables to find an outbound contact to forward
   the bundle over.  It determines that the appropriate next hop is the
   destination itself, that the proper transport protocol is TCP, and
   that the destination is accessible immediately.  The gateway verifies
   that the time-to-live has not expired, and forwards the bundle to the

12.5.5 Step 4:  Bundle processing at destination

   The bundle layer at the destination entity receives the bundle via
   TCP, verifies the signature block of the IPN G/W2, stores it on its
   own persistent storage, and accepts custody of the bundle from IPN
   G/W2.  The bundle agent "awakens" the destination application process
   identified by the Destination Application demultiplexing portion of
   the entity ID.  This may involve creating a new instance of a server
   from a daemon process, signaling an idle running process, or
   reinstantiating a process that has been suspended with its state
   stored on persistent storage.  The bundle agent deletes the copy of
   the bundle from persistent storage when the application has received
   it.  The destination application may generate an application-layer
   acknowledgment in a new bundle and send it to the source.

Cerf, et al.            Expires February 2003               [Page 52]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

12.6 Error Conditions at the Bundle Layer

   This section describes the error conditions that may arise at the
   bundle layer during bundle creation and transport.  When these errors
   occur within the sender's DTN region, it may be possible to conduct a
   near-real-time dialog to correct them before the bundle is forwarded.
   We say 'may be possible' because even if two nodes are in the same
   IPN domain, they may not have real-time connectivity.  An example of
   such a situation would be if a lander were on the opposite side of
   the planet from its DTN gateway, and used bundles to communicate with
   the gateway through a low altitude orbiter, with the orbiter itself
   serving as a bundle agent.

   Table 3: Error conditions at the bundle layer.
   | Error | Description               | Places where Error can Occur |
   | 1*    | Unknown destination region| Source Bundle Node           |
   | 2*    | Invalid Source App.       | Source Bundle Node           |
   | 3*    | Bundle Parameter Syntax   | Source Bundle Node           |
   |       | Error                     |                              |
   | 4*    | Bundle Parameter Semantic | Source Bundle Node           |
   |       | Error                     |                              |
   | 5     | Insufficient buffer space | Any bundle node              |
   | 6     | DNS unreachable           | Any bundle node              |
   | 7*    | Time exceeded             | Any bundle node other than   |
   |       |                           | the source                   |
   | 8*    | Source Entity Access      | Any bundle node other than   |
   |       | Denied                    | the source                   |
   | 9 *   | Invalid Entity ID in      | IPN gateway serving          |
   |       | Destination Name          | destination DTN region       |
   | 11*   | Invalid Destination App.  | Destination                  |
   | 12*   | End-to-end Access Denied  | Destination                  |

   The errors that can occur at the bundle layer are shown in Table 3.
   Error numbers marked with an asterisk (*) are reported back to the
   sending application.

Cerf, et al.            Expires February 2003               [Page 53]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   * Unknown Destination Region: This error occurs when the source
     bundle node is directed to create a bundle destined for an DTN
     Region that is not recognized.  Note that only the DTN Region part
     of the destination name has to be interpretable outside the
     destination's DTN Region.  In particular, the entity identifier of
     the destination name need not be interpretable to the source
     (assuming the source and destination are in different DTN
     Regions), so it cannot necessarily be checked when the bundle is
   * Invalid Source Application: If the source authentication
     information fails, the source bundle layer responds with an
     Invalid Source Application error.
   * Bundle Parameter Syntax Error: The source bundle layer may check
     the syntax of some of the bundle-creation parameters (i.e. it may
     ensure that the end-to-end and DTN access security certificates
     are well-formed, etc.)  If a parameter is found to be
     syntactically incorrect or obviously and definitely erroneous, the
     bundle layer will report a Bundle Parameter Syntax Error back to
     the source that includes, at a minimum, the parameter that caused
     the error.
   * Bundle Parameter Semantic Error: If the source bundle layer can
     identify a particular bundle creation parameter as being well-
     formed but unserviceable, it will report a Bundle Parameter
     Semantic Error to the source application that includes, at a
     minimum, the parameter that caused the error.
   * Insufficient Buffer Space: If a bundle node does not have
     sufficient buffer space to accept a bundle, it drops the bundle
     and generates an Insufficient Buffer Space error.  Note that a
     bundle node may choose to drop lower priority bundles in order to
     make room for higher priority ones. This error is not propagated
     back to the source.
   * DNS Unreachable: If a bundle node needs access to its DNS (or DNS-
     equivalent) and cannot obtain information from it, it generates a
     DNS Unreachable error.  This information is not propagated back to
     the source application.
   * Time Exceeded: If the time-to-live field (either the source-
     provided TTL or the internal bundle TTL) expires, the source is
     notified with a Time Exceeded message.  These errors are
     propagated to the source application.
   * Source Entity Access Denied: This error indicates that the source
     entity does not have access to a needed resource at a DTN node.
     The source might not be authorized to use the node at all, or it
     might not have authorization to use a particular interface
     required by the bundle.  Source Entity Access Denied errors
     indicate that the source is not *authorized* to use a particular
     resource; other errors (e.g. Insufficient Buffer Space) indicate
     that a particular resource is *unavailable* to a bundle.  For
     example, an entity on the surface of a planet might be authorized
     to communicate, using the bundle protocol, with another entity on

Cerf, et al.            Expires February 2003               [Page 54]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

     the other side of the planet via a low-altitude orbiter that is
     also an IPN gateway.  The sender might not, however, be authorized
     to send bundles across interplanetary space.  In this case bundles
     sent to the orbiter destined for the other side of the planet
     would not cause errors, while any bundles with off-planet
     destination addresses would.  Source Entity Access Denied errors
     are propagated back to the source application.
   * Invalid Entity ID in Destination Name: Once a bundle has reached
     its destination DTN Region, the Entity ID part of the destination
     name can be parsed.  If the Entity ID of the destination name is
     not valid, the source is notified with an Invalid Administrative
     Destination Name error message.
   * Invalid Destination Application: If the destination bundle layer
     cannot instantiate the destination application (based on the
     demultiplexing information the region-specific entity ID of the
     bundle), it notifies the source application with an Invalid
     Destination Application error message.
   * End-to-End Access Denied: If the bundle destination does not
     accept the bundle due to an authentication or access-control
     error, the source is notified with an End-to-End Access Denied

13 Summary

   This architecture addresses many of the problems of networks that
   must operate in extreme environments.  It is message based, and uses
   as a model of service classes those offered by the postal mail
   system.  It accommodates many different forms of connectivity,
   including scheduled, predicted, and opportunistically connected
   links.  It introduces a novel approach to end-to-end reliability
   across frequently partitioned and unreliable networks.  It also
   proposes a scheme for securing the network infrastructure against
   unauthorized access.

   It is our belief that this architecture is applicable to many
   different types of extreme environments.  In subsequent documents, we
   intend to specify profiles of this architecture that address specific
   environments in detail.  We plan to develop profiles of this
   architecture for Interplanetary Internetworking (the genesis of the
   architecture), for Sensor Internetworking, for Military Tactical
   Internetworking, and for "Snowmobile" Internetworking.  In addition
   to these profiles, our upcoming tasks include precise specification
   of the constituent protocols in concert with their prototyping and
   testing.  These activities are currently under way.

Cerf, et al.            Expires February 2003               [Page 55]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

14 References

   [KP97] K. Park, G. Kim, and M. Crovella, "On the Effect of Traffic
   Self-Similarity on Network Performance," Proceedings of the 1997 SPIE
   International Conference on Performance and Control of Network

   [BLMR98] J. Byers, M. Luby, M. Mitzenmacher, A. Rege, "A  Digital
   Fountain  Approach  to  Reliable Distribution  of  Bulk
   Data", SIGCOMM  1998

   [ARINC] Presentation  by  ARINC, see  page  at http://www.arinc.com

   [DMT96] R. Durst, G. Miller, E. Travis, "TCP  Extensions  for  Space
   Communications", Wireless Networks 3(5):389-403, 1997.

   [MLS00] U. Madhow, T. V. Lakshman, B. Suter, "TCP/IP  Performance
   with  Random  Loss  and  Bidirectional  Congestion", IEEE/ACM
   Transactions  on Networking, 8(5), Oct  2000.

   [PILC-ASYM] H. Balakrishnan, V. Padmanabhan, G. Fairhurst, M.
   Sooriyabandara, "TCP  Performance  Implications of  Network  Path
   Asymmetry", IETF  draft-ietf-pilc-asym-07, (Work  in  Progress), Nov

   [M95] J. Mogul, "Fragmentation  Considered  Harmful", SIGCOMM  1995

   [IGE00] C. Intanagonwiwat, R. Govindan, D. Estrin, "Directed
   Diffusion: A  scalable  and  robust  communication  paradigm for
   sensor  networks", MobiCOM  2000, Aug  2000

   [NEWARCH]  NewArch Project: Future-Generation Internet Architecture.

   [META]  Wroclawski, John T., "The Metanet", Workshop on Research
   Directions for the Next Generation Internet, May 12-14, 1997, Vienna,
   VA. http://www.cra.org/Policy/NGI/papers/wroklawWP.

   [CK74] V. Cerf, R. Kahn, "A  Protocol  for  Packet  Network
   Intercommunication", IEEE  Trans. on  Comm., COM-22(5), May  1974

   [DM02] D. Mills, "Timekeeping in the Interplanetary Internet", July
   2002, http://www.eecis.udel.edu/~mills/database/brief/ipin/ipin.pdf

   [WhynotTCP] "Why not use the Standard Internet Suite for the
   Interplanetary Internet?"  http://www.ipnsig.org/reports/TCP_IP.pdf

   [RJ90] R. Jain, "Congestion Control in Computer Networks:  Issues and
   Trends," IEEE Network Magazine, May 1990.

Cerf, et al.            Expires February 2003               [Page 56]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

15 Security Considerations

   Security is an integral concern of the Delay Tolerant Network
   Architecture.  Section 7 of this document, also titled "Security
   considerations" is devoted to examining the issues involved in
   securing the DTN architecture and providing basic security services
   to DTN applications.

Cerf, et al.            Expires February 2003               [Page 57]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

16 Authors' Addresses

   Dr. Vinton G. Cerf
   MCI WorldCom
   22001 Loudoun County Parkway
   Building F2, Room 4115, ATTN: Vint Cerf
   Ashburn, VA 20147
   Telephone +1 (703) 886-1690
   FAX  +1 (703) 886-0047
   Email vcerf@mci.net

   Scott C. Burleigh
   Jet Propulsion Laboratory
   4800 Oak Grove Drive
   M/S: 179-206
   Pasadena, CA 91109-8099
   Telephone +1 (818) 393-3353
   FAX  +1 (818) 354-1075
   Email Scott.Burleigh@jpl.nasa.gov

   Robert C. Durst
   The MITRE Corporation
   1820 Dolley Madison Blvd.
   M/S W650
   McLean, VA 22102
   Telephone +1 (703) 883-7535
   FAX +1 (703) 883-7142
   Email durst@mitre.org

   Dr. Kevin Fall
   Intel Research, Berkeley
   2150 Shattuck Ave., #1300
   Berkeley, CA 94704
   Telephone +1 (510) 495-3014
   FAX +1 (510) 495-3049
   Email kfall@intel-research.net

   Adrian J.  Hooke
   Jet Propulsion Laboratory
   4800 Oak Grove Drive
   M/S: 303-400
   Pasadena, CA 91109-8099
   Telephone +1 (818) 354-3063
   FAX  +1 (818) 393-3575
   Email Adrian.Hooke@jpl.nasa.gov

   Dr. Keith L. Scott
   The MITRE Corporation
   1820 Dolley Madison Blvd.

Cerf, et al.            Expires February 2003               [Page 58]

Internet Draft       draft-irtf-ipnrg-arch-01.txt          August 2002

   M/S W650
   McLean, VA 22102
   Telephone +1 (703) 883-6547
   FAX +1 (703) 883-7142
   Email kscott@mitre.org

   Leigh Torgerson
   Jet Propulsion Laboratory
   4800 Oak Grove Drive
   M/S: T1710-
   Pasadena, CA 91109-8099
   Telephone +1 (818) 393-0695
   FAX  +1 (818) 354-9068
   Email Leigh.Torgerson@jpl.nasa.gov

   Howard S. Weiss
   SPARTA, Inc.
   9861 Broken Land Parkway
   Columbia, MD 21046
   Telephone +1 (410) 381-9400 x201
   FAX  +1 (410) 381-5559
   Email hsw@sparta.com

   Please refer comments to ipn-team@mailman.ipnrg.org

Html markup produced by rfcmarkup 1.129d, available from https://tools.ietf.org/tools/rfcmarkup/