[Docs] [txt|pdf] [Tracker] [Email] [Nits]

Versions: 00 01

IPN Research Group                                               V. Cerf
Internet Draft                        Worldcom/Jet Propulsion Laboratory
May 2001                                                     S. Burleigh
Expires November 2001                                           A. Hooke
                                                            L. Torgerson
                                          NASA/Jet Propulsion Laboratory
                                                                R. Durst
                                                                K. Scott
                                                   The MITRE Corporation
                                                               E. Travis
                                           Global Science and Technology
                                                                H. Weiss
                                                            SPARTA, Inc.

        Interplanetary Internet (IPN):  Architectural Definition

                     draft-irtf-ipnrg-arch-00.txt

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that other
   groups may also distribute working documents as Internet-Drafts.
   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


Abstract

   This document describes the Interplanetary Internet: a communication
   system to provide Internet-like services across interplanetary
   distances in support of deep space exploration.  Our approach, which
   we refer to as bundling, builds a store-and-forward overlay network
   above the transport layers of underlying networks.  Bundling uses
   many of the techniques of electronic mail, but is directed toward
   interprocess communication, and is designed to operate in
   environments that have very long speed-of-light delays.  We partition
   the Interplanetary Internet into IPN Regions, and discuss the
   implications that this has on naming and routing.  We discuss the way
   that bundling establishes dialogs across intermittently connected
   internets, and go on to discuss the types of bundle nodes that exist
   in the interplanetary internet, followed by a discussion of security
   in the IPN, a discussion of the IPN backbone network, and a
   discussion of remote deployed internets.

Cerf, et al.                                                  [Page 1]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001




Table of Contents

   Status of this Memo................................................1
   Abstract...........................................................1
   Table of Contents..................................................2
   Copyright Notice...................................................2
   Desiderata of Interplanetary Internetworking.......................3
   Acknowledgments....................................................3
   Foreward...........................................................4
   Executive Summary..................................................5
   1. Introduction....................................................8
         1.1. Preliminary Considerations .............................9
         1.2. The IPN Operating Environment .........................11
         1.3. A "Postal" Communications Model .......................14
   2. IPN Architectural Overview.....................................14
   3. Inter-Internet Dialogs.........................................15
         3.1. Principles of Design ..................................15
         3.2. Information Carried by the Bundle Layer ...............19
         3.3. Reliability at the Bundle Layer .......................21
         3.4. Bandwidth Allocation via Market Mechanisms:
              "Starbucks" ...........................................21
   4. IPN Nodes......................................................23
         4.1. Types of IPN Nodes ....................................24
         4.2. Example end-to-end transfer ...........................24
         4.3. Error Conditions at the Bundle Layer ..................32
         4.4. Support of existing Internet applications .............35
   5. Security in the IPN............................................36
         5.1. Assumptions Regarding Required IPN Security
              Mechanisms ............................................36
         5.2. Secure Email Technology ...............................38
         5.3. Application of Secure Email Technology to the IPN .....40
         5.4. Protecting IPN Data and the IPN Backbone
              Infrastructure ........................................41
   6. Building a Stable Backbone for the IPN.........................42
         6.1. Backbone Design Considerations ........................43
   7. Deployed Internets in the IPN..................................46
         7.1. Applications of deployed internets in the IPN .........47
         7.2. Characteristics of remote deployed internets
              in the IPN ............................................48
         7.3. Effects of environmental characteristics on
              protocols for the IPN RDIs ............................49
         7.4. Summary ...............................................53
   8. Working Conclusions............................................53
   9. Security Considerations........................................56
   10. Authors' Addresses............................................57


Copyright Notice

   Copyright (C) The Internet Society (2001).  All Rights Reserved.


Cerf, et al.             Expires November 2001                  [Page 2]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001




              Desiderata of Interplanetary Internetworking

   Go thoughtfully in the knowledge that all interplanetary
   communication derives from the modulation of radiated energy, and
   sometimes a planet will be between the source and the destination.
   Therefore rely not on end-to-end connectivity at any time, for the
   universe does not work that way.

   Neither rely on ample bandwidth, for power is scarce out there and
   the bit error rates are high.  Know too that signal strength drops
   off by the square of the distance, and there is a lot of distance.

   Consider the preciousness of interplanetary communication links, and
   restrict access to them with all your heart.  Protect also the
   confidentiality of application data or risk losing your customers.

   Remember always that launch mass costs money.  Think not, then, that
   you may require all the universe to adopt at once the newest
   technologies.  Be backward compatible.

   Never confuse patience with inaction.  By waiting for acknowledgement
   to one message before sending the next, you squander tracking pass
   time that will never come to you again in this life.  Send as much as
   you can, as early as you can, and meanwhile confidently await
   responses for as long as they may take to find their way to you.

   Therefore be at peace with physics, and expect not to manage the
   network in closed control loops -- neither in the limiting of
   congestion nor in the negotiation of connection parameters nor even
   in on-demand access to transmission bands.  Each node must make its
   own operating choices in its own understanding, for all the others
   are too far away to ask.  Truly the solar system is a large place and
   each one of us is on his or her own.  Deal with it.

                                        S. Burleigh






Acknowledgments

   Robert Braden, of USC ISI, Deborah Estrin, of UCLA, and Craig
   Partridge, of BBN all contributed useful thoughts and criticisms to
   this document.

   This work was performed under DOD Contract DAA-B07-00-CC201, DARPA AO
   H912; JPL Task Plan No. 80-5045, DARPA AO H870; and NASA Contract
   NAS7-1407.


Cerf, et al.             Expires November 2001                  [Page 3]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



Foreward

   This document presents the current state of our team's efforts to
   define an end-to-end architecture for the Interplanetary Internet.
   This is a "living" document, and is frequently updated.  In this
   version of the document, we introduce a new construct, a tuple
   consisting of a topologically significant routing handle and the
   administrative name.  In this model, the routing handle identifies an
   "IPN Region," an area of the Interplanetary Internet in which A) the
   administrative name is resolvable, and in which B) a route can be
   formed from anywhere within the region to the address returned when
   the administrative name is resolved.

   On the presumption that the exploration of space will eventually lead
   to the need for communication among planets, satellites, asteroids,
   robotic spacecraft and crewed vehicles, our project has a heavy focus
   on advocating the development of a stable interplanetary backbone
   network. We have concluded that simply extending the Internet suite
   to operate end-to-end over interplanetary distances is not feasible.
   Rather, we envision a "network of internets" _ ordinary internets
   that are interconnected by a system of gateways that cooperate to
   form the stable backbone across interplanetary space.  Each
   internet's protocols are terminated at its local gateway, which then
   uses a specialized long-haul transport protocol to communicate with
   peer gateways.  An end-to-end "bundle" protocol will operate above
   the transport layer to carry necessary information from one internet
   to another.  Bundling will, to the extent possible, remove any
   "chattiness" from the local protocols, forming atomic units that will
   be shipped across the backbone.  Bundles may be protected from
   unauthorized access and unauthorized modification. The IPN will have
   a global namespace that is broken into a number of name-to-address
   binding regions, referred to as IPN regions.  Names carried in the
   bundles consist of a tuple, one element identifying the destination
   IPN region and used for routing and a second element that carries the
   administrative name of the destination in the namespace that is
   relevant within the destination IPN region.  The administrative name
   will be bound to an address that is routable within the destination
   IPN region.  Finally, strong authentication and strict access
   controls at several levels will protect the IPN from tampering.

   In summary, the best way to envision the fundamental architecture of
   the Interplanetary Internet is to picture a network of internets.
   Ordinary internets (many being wireless in nature) are placed on the
   surface of moons and planets as well as in free-flying spacecraft.
   These remotely deployed internets run ordinary Internet protocols. A
   system of Interplanetary Gateways connected by deep-space
   transmission links form a backbone communication infrastructure that
   provides connectivity for each of the deployed internets. New long-
   haul protocols, some confined to the backbone network and some
   operating end-to-end, allow the deployed internets to communicate
   with each other.


Cerf, et al.             Expires November 2001                  [Page 4]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



Executive Summary

   This document describes the Interplanetary Internet: a communication
   system to provide Internet-like services across interplanetary
   distances in support of deep space exploration.  The communications
   environment is characterized by high bandwidth-delay products
   resulting from very long signal propagation delays, intermittent
   connectivity that results in long periods of network partitioning,
   and discontinuities in the capabilities of adjacent networks.  Many
   of these characteristics are similar to those facing emerging
   communication services in the terrestrial Internet. For example,
   terabit networks exhibit very high bandwidth-delay products, mobility
   can result in partitioning of both nodes and subnetworks, and
   interconnecting different physical layer technologies can result in
   "impedance mismatches."  It is possible to build an end-to-end
   solution to address any of these, but difficult to build one that is
   adequate for all of them simultaneously.

   The long bandwidth-delay products shared by the IPN and very high-
   speed terrestrial networks argue in favor of non-chatty communication
   protocols.  In the IPN, the long delays mean that protocols that use
   many round-trips to accomplish some task pay a significant time
   penalty.  In terrestrial terabit networks where switches can forward
   data very fast but take a (relatively) long time to reconfigure, the
   penalty is one of lost efficiency in the use of the resources.  Both
   environments benefit from protocols that pack as much as possible
   into each transmission and minimize the number of round-trips needed.
   Thus a file transfer protocol that can place the entire file and
   associated control information together in a single atomic
   transaction completes faster within the IPN and makes more efficient
   use of terrestrial high-speed networks.

   Most of the problems cited above have existed and been considered,
   albeit separately, during the evolution of what is now the Internet.
   Many of the solutions, however, represent branches of the Internet's
   evolutionary tree that have withered and died as a result of the
   availability of infrastructure to mitigate environmental differences.
   This effective homogeneity is diminishing, however, with the rapid
   deployment of new technologies with fundamentally different
   characteristics, such as wireless communication and Dense Wavelength
   Division Multiplexing (DWDM) networks.  One solution, electronic
   mail, provides a means to span very different networks that are not
   necessarily always connected.  The electronic mail approach has a
   number of attractions.  First, there is no expectation of continuous
   or instantaneous connectivity.  Second, electronic mail embodies the
   concept of indirection as a means of providing store-and-forward
   traversal of different, sometimes-disconnected networks.  Finally,
   the electronic mail concept is generally considered to be a non-
   interactive communications mechanism, potentially well suited to long
   delay environments.  The electronic mail approach has limitations,
   though.  Without and end-to-end retransmission mechanism, electronic
   mail does not provide true end-to-end reliability.  Additionally,

Cerf, et al.             Expires November 2001                  [Page 5]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   electronic mail is oriented toward human use rather than interprocess
   communication.  Further, the protocol that typically provides
   electronic mail services in the Internet, SMTP, is highly interactive
   in its control traffic, even though the electronic mail concept is
   only minimally interactive.

   Our approach, which we refer to as bundling, builds a store-and-
   forward overlay network above the transport layers of underlying
   networks.  Thus two nodes that are adjacent in bundle space may be
   many hops apart in the context of the underlying network topology.
   We see the bundle layer as a second "thin waist of the hourglass"
   that allows applications built on top of it to communicate across
   discontinuities in connectivity, and to communicate efficiently over
   a multitude of underlying transport technologies.  This discontinuity
   in connectivity may result from fluctuations in link availability or
   from artificial discontinuities, such as are imposed by firewalls.
   For efficient communication, the bundle layer attempts to minimize
   interactivity of its control traffic, and expects applications to do
   likewise.  The bundle layer also provides a level of indirection
   between applications and the specific services of the underlying
   network protocols.  Bundle applications can specify requested
   "handling instructions," such as reliability and quality of service
   requests that are mapped into the most appropriate mechanisms
   available in the underlying networks.

   Bundling uses many of the techniques of electronic mail, but is
   directed toward interprocess communication.  Bundle nodes use the
   capabilities of the underlying networks, including transport layer
   retransmission protocols, to effect the transfer of bundles between
   bundle nodes.  Optional end-to-end reliability at the bundle layer
   facilitates end-to-end reliability at the application layer.  In
   addition, the bundle layer allows any bundle node in the path to take
   custody of a bundle.  When custody is transferred, the receiving
   bundle node assumes responsibility for delivering the bundle
   according to its handling instructions, and the previous bundle
   custodian is allowed to recover its storage resources.  The bundle
   protocol is designed to function over simplex and half-duplex links,
   and custody transfers may occur between non-adjacent bundle nodes.
   Finally, because the bundle layer operates over networks that are
   often disconnected, the bundle-layer reliability mechanisms adapt the
   operation of timers to accommodate this episodic connectivity.

   To illustrate how bundling allows connectivity across intermittently
   connected networks, one could envision a "strobe light" that
   illuminates the portions of the network's topology that are connected
   at a given time.  Lit portions of the network are available for
   bundle forwarding.  If one imagines a high-persistence CRT capturing
   an ordered sequence of illuminations that proceed from source to
   destination, one can envision the available routes for bundle
   forwarding.  This environment, therefore, requires a mechanism to
   route based on both current and expected connectivity.  This routing
   mechanism exploits predictability, such as that provided by orbital

Cerf, et al.             Expires November 2001                  [Page 6]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   mechanics or by scheduled network events (possibly including rate
   changes, such as "5c Sundays").  It is important to note that this
   routing mechanism may choose to defer communication even though a
   current path toward the destination exists, if a substantially more
   attractive path is expected to become available.

   We believe that the bundle layer functionality has utility within the
   context of the terrestrial Internet as well.  The Internet is
   evolving to encompass very different networking technologies,
   architectures, and applications.  Some of these, such as firewalls,
   result in logical partitioning of the Internet, while others, such as
   extreme rate mismatches, may result in inefficient use of network
   resources.  The bundle layer extends the Internet architecture to
   provide consistent end-to-end communication in the current and
   emerging partitioned environments, facilitating the deployment of new
   applications that can operate reliably while allowing networks to
   operate efficiently.

  Readers may find more about the project at http://www.ipnsig.org/,
  the Interplanetary Internet Special Interest Group of the Internet
  Society (http://www.isoc.org).


Cerf, et al.             Expires November 2001                  [Page 7]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001





1. Introduction

   The exploration of space began with naked-eye observations of the
   stars and moon. In more recent centuries, this exploration was aided
   by new Earth-based technologies such as optical and radio telescopes.
   Even more recently, we have placed observing resources into near
   Earth orbit, such as the Hubble Space Telescope. We have also sent
   robotic missions to other parts of the Solar system for a closer
   look.

   It is the latter activity -- the current exploration of the Solar
   System by robotic means and possibly later by missions crewed by
   people -- that motivates our interest in an Interplanetary Internet.
   The new technology of the terrestrial Internet needs to be extended
   into space. We believe that the creation and adoption of Internet-
   friendly standards for space communication will enhance our ability
   to build a common interplanetary communication infrastructure.  We
   think this infrastructure will be needed to support he expansion of
   human intelligence throughout the Solar System.  The current
   terrestrial Internet and its technology provide a robust basis to
   support these missions in an efficient and scalable manner.

   Although many missions during the last 40 years of space exploration
   have by necessity provided a substantial portion of their own
   communication resources, a significant amount of shared
   infrastructure is already in place. For example, the multi-mission
   Deep Space Network (DSN) provides a complex of large-diameter (up to
   70-meter) tracking and data acquisition dishes at three points on the
   Earth's surface, each about 120 degrees from each other. Similarly,
   the Tracking and Data Relay Satellite System (TDRSS) and a global
   network of small ground tracking stations are used to relay data to
   and from many near-Earth missions.

   In terms of the communications protocols that support space
   exploration, the Consultative Committee for Space Data Systems
   (CCSDS) has for almost twenty years been developing internationally
   agreed standards for the physical and link layers that interconnect
   remote spacecraft with their ground control systems. NASA, through
   CCSDS, has also been working since 1993 on general application of
   terrestrial Internet or Internet-like protocols for space data use.
   Such standardization opens up the possibility of re-purposing and re-
   using existing and planned communication facilities for multiple
   subsequent missions.

   Early in 1998, it became apparent to our team that the space
   communications research community and the Internet research and
   development community were pursuing technology paths that can
   potentially lead to a kind of convergence. A few members of the
   Internet community began thinking about adaptation of the terrestrial
   Internet to deep space communications. The space communications

Cerf, et al.             Expires November 2001                  [Page 8]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   research community was already trying out variants of the Internet's
   TCP/IP protocol suite to support space-based applications. Mutual
   recognition led to the formation of a program of work that was aimed
   at extending the notion of Internet to interplanetary scale. The
   heretofore-independent "rivers" of evolving space technology and
   Internet technology are converging in this program.

   To realize this convergence, the effort to define the IPN
   architecture is being undertaken at the Jet Propulsion Laboratory
   (JPL) that is operated by the California Institute of Technology
   (Caltech) for the US National Aeronautics and Space Administration
   (NASA). Partial funding for the effort comes from the US Defense
   Advanced Research Projects Agency (DARPA) that sponsored the original
   Internet design work starting in 1973 and its predecessors, such as
   the ARPANET, in 1969. NASA supplies in-kind support and staffing
   through its standardization program with CCSDS as well as program
   involvement from the Mars exploration enterprise.

   An Interplanetary Internet Research Group (IPNRG) has been formed
   under the auspices of the Internet Research Task Force of the
   Internet Society and an Interplanetary Internet Special Interest
   Group has also been created as a means of keeping the public informed
   as to progress.

   Early protocol design phases are underway now and prototype testing
   of candidate designs is anticipated within a year. Demonstration of
   these protocols in terrestrial environments will likely occur during
   2001 and plans for their use in interplanetary contexts are under
   consideration as part of the NASA Mars mission in the years beyond
   2003. The Mars exploration program is particularly interesting as a
   "reference model" for our work, since it includes a "Mars Network"
   project that hopes to deploy a series of remote communications
   satellites into orbit around the planet. These satellites will be
   dedicated to servicing the local communication and positioning
   requirements of the other in-orbit and surface observation and
   exploration missions that are in the vicinity of the planet. By 2010,
   as many as seven communications and navigation satellites could be in
   orbit around Mars, most in low Mars orbit (LMO) but with perhaps at
   least one in an "areo" synchronous orbit that is analogous to a
   geosynchronous orbit around the Earth.

1.1.   Preliminary Considerations

   The remarkable success and growth of the Earth-bound TCP/IP protocols
   of the Internet illustrate the power of communication
   standardization. The simplicity of the Internet architecture, with
   its layered structure, contributes to its ability to adapt to almost
   any underlying communication capability. As with the terrestrial
   Internet, the ultimate test of the IPN technology is whether it
   successfully supports commercial applications that have a space
   component. We expect that space will eventually be commercialized -
   not only for communication services, but also for mining the

Cerf, et al.             Expires November 2001                  [Page 9]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   asteroids, for the operation of space-based hotels, for manufacturing
   and medical treatments, and for general tourism. While such
   developments may still lie decades in the future, the potential
   investment and benefits can be appreciated as we contemplate the
   explosion of new markets associated with the commercialization of the
   Internet that began only ten years ago, in 1990. We will therefore
   architect the Interplanetary Internet in anticipation of possibly
   rapid commercialization.

   In terms of technologies, the current Internet capabilities work well
   on Earth where the propagation delay of light-speed signals is short.
   The packets exchanged according to the TCP protocol reach their
   destination in fractions of a second, for the most part. The TCP/IP
   protocols (a system of over 150 related communication standards), are
   therefore expected to work just as well on the surface of other
   planets or moons, on space craft and orbiting space stations, all of
   which involve data exchange over fairly short distances, subject to
   the availability of sufficient power to maintain good signal-to-noise
   ratios. However tempting it is to employ similar concepts in
   extending the Internet into deep space, there are problems - deep
   space communications really still are "rocket science." The distances
   between the planets are, well, astronomical. For example, the round-
   trip propagation delays - at the speed of light - between Earth and
   Mars range from about 8 minutes to over 40 minutes. This makes
   "chatty" protocols like TCP relatively unattractive because of their
   heavy dependence on near real-time exchanges between the
   communicating parties.

   These large distances also impair the data rates that can be
   sustained because of radio signal degradation and attenuation.
   Moreover, the celestial mechanics of the solar system mean that the
   distances between the planets change with time. While these changes
   are essentially calculable, they still cause variations in delay, in
   transmission capacity and occasionally in connectivity due to
   occultation of satellites as they orbit a planet, or of ground-based
   facilities as planets rotate.

   Size, weight and - most of all - power are supreme challenges for
   space-based communication systems, as they are for ground-based
   mobile systems. Launching mass into interplanetary trajectories,
   injecting mass into orbit, and landing mass into the gravity well of
   another planet is currently very expensive. Mass translates directly
   into the local availability of power. Efficient use of the
   communications channel allows more information to be carried per unit
   of transmitted power. But power limitations introduce asymmetries in
   the communication capacities available between Earth, for example,
   and the remote spacecraft and planets. There can be factors of ten or
   more differences between the data rate that can be received on Earth
   and the rates of reception of off-Earth resources. It is quite common
   to be able to receive transmissions from Mars at 100 kilobits/second
   while the Mars-based systems may only receive from Earth at 1
   kilobits/second.

Cerf, et al.             Expires November 2001                 [Page 10]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001




   All of these effects combine to make the design of an interplanetary
   backbone communication system a considerable challenge. The Deep
   Space Network - the current interplanetary backbone - uses its three
   terrestrial communication complexes to communicate with spacecraft,
   orbiting satellites and ground-based resources on moons or other
   planets of the Solar System.  Because these resources must be shared
   among many missions, it is necessary to schedule them to be aimed in
   particular directions at particular times. Time synchronization is
   needed among the various parts of such a system. For example, a
   signal from Mars may take 20 minutes to reach Earth, at which time,
   the appropriate antenna of the Deep Space network must be aimed
   properly to receive the transmission, 20 minutes after it was sent.
   This same antenna might then have to be repositioned to send data to
   another spacecraft elsewhere in the Solar System, and the receiving
   system must be ready to receive that transmission at the right time.
   In some ways, this problem is somewhat like the problem of scheduling
   trains on railroad tracks; since multiple trains use the tracks, they
   must be scheduled to avoid collisions.

1.2.     The IPN Operating Environment

   There are a number of fundamental differences between the
   environments for terrestrial communications and those we envision for
   the IPN.  These differences include delay, low and asymmetric
   bandwidth, intermittent connectivity, and a relatively high bit error
   rate.  Taking these into account affects the entire model for
   communicating; shifting us from the "telephony" model implicit in
   current Internet communications to the "Postal", or "Pony Express,"
   model. We will first therefore describe the environmental differences
   between terrestrial communications and the IPN and gives a brief
   accounting of why the standard Internet protocol for reliable
   transport, TCP, is unsuitable for end-to-end communications in the
   IPN.

   The most obvious difference between communicating between points on
   Earth and communicating between planets is the delay.  While round-
   trip times in the terrestrial Internet range from milliseconds to a
   few seconds, round-trip times to Mars range from 8 to 40 minutes,
   depending on the planet's position, and round-trip times between
   Earth and Europa run between 66 and 100 minutes.  In addition to the
   propagation delay, communicating over interplanetary distances
   currently requires special equipment (large antennas, high-
   performance receivers, etc.).  For most deep-space missions, even
   non-NASA ones, these are currently provided by NASA's Deep Space
   Network (DSN). The communication resources of the DSN are currently
   oversubscribed and will probably continue to be so in the future.
   While studies have been done as to the feasibility of upgrading or
   replacing the current DSN, the number of deep space missions will
   probably continue to grow faster than the terrestrial infrastructure
   needed to support them, making over-subscription a persistent
   problem.

Cerf, et al.             Expires November 2001                 [Page 11]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001




   This over-subscription means that the round-trip times experienced by
   packets will be affected not only by the propagation delay, but also
   by the scheduling and queuing delays imposed by the Earth-based
   resources.  Thus packets to a given destination may have to be queued
   until the next scheduled contact period, which may be hours, days, or
   even weeks away.  While queuing and scheduling delays are generally
   known well in advance except when missions need emergency service
   (such as during landings and maneuvers), the long and highly variable
   delays make the design of timers, and retransmission timers in
   particular, quite difficult.  This again forms a point of departure
   from the current Internet model, as IPN-aware applications will
   probably need ways to track the status of a communication and to
   apprise users of the expected delay before a response can be
   expected.  This will be complicated once the IPN moves from its
   initial Earth-centric approach to a peer-to-peer network, since
   notifying users of the progress of their communications will itself
   consume precious bandwidth within the network.

   The combined effects of large distances, the expense and difficulty
   of deploying large antennas to distant planets, and the difficulty in
   generating power in space all mean that the available bandwidth for
   communications in the IPN will likely be modest compared to
   terrestrial systems.  Data rates on the order of hundreds of kilobits
   per second to a few megabits per second will probably be the norm for
   the next few decades.  Another characteristic prevalent in today's
   deep-space missions is bandwidth asymmetry, where data is transmitted
   at different rates in different directions.  Current missions are
   usually designed with a much higher data return rate (from space to
   Earth) than command rate.  The reason for the asymmetry is simple:
   nobody ever wanted a high-rate command channel, and, all else being
   equal, it was deemed better to have a more reliable command channel
   than a faster one.  This design choice has led to data rate
   asymmetries in excess of 100:1, sometimes approaching 1000:1.  A
   strong desire for a very robust command channel will probably remain,
   so that any transport protocol designed for use in the IPN will need
   to function with a relatively low bandwidth outbound channel to
   spacecraft / landers.

   The difficulties of generating power on and around other planets will
   also result in relatively high bit error rates.  Current deep-space
   missions operate with very high bit error rates (on the order of 10e-
   1, or one error in ten bits) that are then improved using heavy
   coding.  The tradeoffs between coding, bit error rate, and
   reliability requirements will need to be reexamined in the context of
   the IPN.

   Finally, interplanetary communications will, at least in the near
   future, be characterized by intermittent connectivity between nodes.
   As satellites or moons pass behind planets, and as planets pass
   behind the sun as seen from Earth, we lose the ability to communicate
   with them.  This effect adds to the delays experienced by packets,

Cerf, et al.             Expires November 2001                 [Page 12]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   and could push queuing delays to several weeks or a month if the
   source and destination are in opposition (on opposite sides of the
   sun).  Inter-layer signaling, especially from the link layer to
   provide notifications of such breaks in connectivity, will probably
   be required.

   We see the IPN growing outwards from Earth as we explore more and
   more planets, moons, asteroids, and possibly other stars.  Thus there
   will always be a fringe to the fabric of the IPN, an area without a
   rich communications infrastructure.  As a result, the data rate,
   connectivity, and error characteristics mentioned above will probably
   always be an issue somewhere in the IPN.  For the more highly
   developed core areas of the IPN, it is interesting to note that delay
   is the only truly immutable characteristic that differentiates the
   IPN from terrestrial communications.  Data rates, intermittent
   connectivity, and bit error rate can all be mitigated or eliminated
   by adding additional infrastructure, in theory if not in practice.
   Additional infrastructure can also mitigate the scheduling and
   queuing delays mentioned above, but the propagation delays will
   remain unless and until we find a way to transmit information faster
   than the speed of light.

   These environmental characteristics: long delays, low and asymmetric
   bandwidth, intermittent connectivity, and relatively high error rate
   make using unmodified TCP/IP for end to end communications in the IPN
   infeasible.  Using the equations from Mathis, et al [ref:
   http://www.psc.edu/networking/papers/model_ccr97.ps], we can
   calculate an upper bound on the sustainable throughput of a TCP
   connection, taking into account TCP's congestion avoidance
   mechanisms.  Even if only 1 in 100 million packets are lost, a TCP
   connection to Mars is limited to just under 250kbps.  If we assume
   that 1 in 5000 packets is lost (this figure was reported by Paxson as
   the packet corruption rate in the Internet ref:
   ftp://ftp.ee.lbl.gov/papers/vp-thesis/dis.ps.gz caution: very large
   file) then that number falls to around 1,600bps.  These values are
   upper bounds on steady-state throughput; since the number of packets
   in a connection will generally be under 10,000, TCP performance would
   be dominated by its behavior during slow-start.  Even when Mars is at
   its closest approach to Earth, this means that it would take a TCP
   nearly 100 minutes to ramp up to a transmission rate of 20kbps.  Lab
   experiments using a channel emulator and standard applications show
   that even if TCP could be pushed to work efficiently at such
   distances, many applications either rely on several rounds of
   handshaking or have built-in timers that render them non-functional
   when the round-trip-time is pushed over a couple of minutes.  It
   typically takes eight round trips for FTP to get to a state where
   data can begin flowing, for example, and an FTP server may time out
   and reset the connection after 5 minutes of inactivity.  This means
   that a conformant standard FTP server could be unusable for
   communicating even with the closest planets.


Cerf, et al.             Expires November 2001                 [Page 13]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



1.3.     A "Postal" Communications Model

   We have concluded that the standard Internet protocols should be
   essentially "terminated" at the Interplanetary Gateways and the
   information payloads conveyed through a new set of protocols better
   suited to the long distances, variable delays and asymmetric data
   rates of the interplanetary backbone network.  In essence, the design
   is analogous to a kind of postal relay system in which messages are
   delivered to the intermediate Interplanetary Gateways, extracted from
   their standard Internet protocols, and encapsulated in new link and
   transport protocols to be forwarded to the next IPN gateway and
   ultimately into the target internet.

   Internet electronic mail already works in this fashion but the
   transfer of files, the operation of the World Wide Web, and remote
   interactive applications do not fit into this model directly. There
   are circumstances under which researchers on planet Earth do need an
   ability to interact with remote devices, for example to steer wheeled
   robotic vehicles around on the surface. But because of the enormous
   round-trip delays, such systems must work very indirectly. For
   example, to steer the Mars Pathfinder rover, one sends instructions
   about intermediate points that the robot must steer past. This is
   analogous to automatic airplane pilots that are given a series of
   coordinates through which to pass. In effect, a planned itinerary is
   sent and the robot vehicle executes the plan, dealing with local
   conditions as required.

   The "store-and-forward" nature of this communication method is
   reminiscent of bucket brigades, except that the contents of the
   buckets are actually the payloads (i.e. data) of the applications
   that utilize the network. The concept of "custody" is important in
   such a system. A sender does not relinquish a copy of a transmission
   until it is sure that the next in line has successfully received it.

2. IPN Architectural Overview

   We now consider five broad areas that represent areas of significant
   research in the Interplanetary Internet architecture:

    *  The communication conducted between independent internets
       (termed "Inter-internet dialogs")
    *  The architecture and functions of the unique nodes of the
       Interplanetary Internet
    *  A security architecture for meeting anticipated data and
       infrastructure protection needs
    *  The issues in developing a stable backbone network for the
       Interplanetary Internet
    *  The issues in deploying internets on remote planets, asteroids,
       and spacecraft

   The following five sections of this document consider each of these
   areas.  Following these sections we present our conclusions to-date.

Cerf, et al.             Expires November 2001                 [Page 14]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001





3. Inter-Internet Dialogs

   This section first presents four principles that guide the design of
   the Interplanetary Internet.  In doing so, we introduce the concept
   of the "bundle" layer, a protocol layer providing end-to-end service
   in the IPN.  The section continues by discussing the information
   carried by the bundle layer, and concludes by discussing reliability
   at the bundle layer.

3.1.   Principles of Design

3.1.1. Name Tuples Consisting of Administrative and Routing Parts are
       the Means of Reference

   In the (terrestrial) Internet, names are administrative in nature,
   and are hierarchically organized.  The Domain Name System (DNS) uses
   a highly distributed database to translate the name to a numeric
   address, and addresses are the common medium used throughout the
   network for reference.  This scheme has worked well in the Internet,
   but the emergence of network address translators and other
   partitioning mechanisms have begun to cause some problems with this
   scheme.

   One of the problems involved with using DNS names across
   interplanetary space is the distributed nature of the DNS database.
   This means that it is entirely possible for the portion of the
   database that can resolve a name to its address to be far (very far,
   in the Interplanetary Internet) from the host requesting resolution.
   This means that the times required to resolve names to addresses can
   become impossibly high, especially when the issues of intermittent
   connectivity come into play.  The DNS has a mechanism to replicate
   portions of its database, using a technique known as zone transfers.
   However, this is not a good solution in the Interplanetary Internet,
   either.  One could easily spend all of the available communication
   time transferring the ".com" zone to another planet, rather than
   actually transferring data.  Clearly, another approach is indicated.

   In our initial designs, we considered creating a new top-level
   domain, e.g. ".sol", and assigning to it topological significance.
   We constructed a scheme by which we routed on the "most significant"
   portions of the domain name, such as ".mars.sol", and essentially bet
   that these portions would be topologically significant, rather than
   administratively significant.  This scheme, while attractive from the
   standpoint of making use of existing infrastructure, makes use of
   that infrastructure in a bad way.  We were forced to "grandfather"
   existing top-level domain names to be bound to Earth's Internet, so
   that ".com" meant ".com ON EARTH."  This spawned philosophical
   debates within the group regarding sovereignty and the current top-
   level domain structure within the internet, but also had the
   potential of creating serious technical problems.  For example, some

Cerf, et al.             Expires November 2001                 [Page 15]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   organizations encode geographic information at fairly low levels of
   their DNS names (consider "zurich.ibm.com", and then consider
   "mars.ibm.com").  If all ".com's" are shipped off to Earth, the
   "mars.ibm.com" data is going to take a serious detour.  We eventually
   concluded that this was not the right approach.

   We have concluded that names in the Interplanetary Internet should
   consist of a tuple that contains the administrative part plus a
   routing part.  These names must be carried end-to-end throughout the
   Interplanetary Internet.  The routing part serves the purpose of the
   new top-level domain described above, except that it is not required
   to conform to the naming conventions of the Domain Name System (i.e.,
   it may be numeric rather than textual), it may be hierarchically
   organized, and it must be routed.  This requires us to develop the
   means for computing and distributing routing information, but
   relieves us from a dependence on the relationship between
   administrative names and network topology.

3.1.2. The Routing Part of an IPN Tuple Identifies an Internet

   The routing portion of the name identifies an IPN Region.  We
   envision the IPN as a "network of Internets", and the IPN Region, to
   some extent, allows us to route to a particular Internet.  The use of
   an IPN Region is an explicit form of aggregation that is not
   otherwise possible using administrative names.

   It is necessary and sufficient that the administrative portion of an
   IPN name be resolvable to a useful address within its IPN Region.  We
   are currently treating the structure of the routing part of the name
   in a manner similar to the structure of DNS names:  the name space of
   the routing part is a tree of text labels separated by "dots," with
   the root node of the space having a null label.  We denote a name in
   the IPN in the following manner:  {administrative part, routing
   part}.  So, if "earth.sol" were an IPN Region encompassing the entire
   Earth, the web site for the Internet Society's IPN Special Interest
   Group would be { www.ipnsig.org, earth.sol}.

   In order that any node in the IPN be able to send data to any other
   node, the routing part of the name must be interpretable everywhere.
   That is, any IPN node, when confronted with any valid IPN Region
   name, must be able to identify a transport layer destination that
   moves the data toward that region.

3.1.3. The "Bundle Layer" Terminates Local Transport Protocols and
       Operates End-to-End

   In the IPN, we cannot ever assume that there is direct connectivity
   between source and destination.  That is, we cannot assume that bits
   emitted by a source can travel, delayed only by routing and
   transmission delays, to the destination.  There may be any number of
   reasons for this, ranging from physical (the destination is on the
   far side of a distant planet and can't communicate with anything

Cerf, et al.             Expires November 2001                 [Page 16]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   right now), to schedule-related (a required IPN gateway is currently
   serving other customers), to administrative (the source is only on
   during the day, the destination only during the night).  For the
   long-haul links of the backbone, information will almost certainly
   have to be stored for some amount of time as the antennas used for
   the long-haul links will almost surely be highly directional.

   Thus depending on the schedules of the nodes involved and the
   possibility of high-priority interrupt traffic, the nodes that make
   up the IPN may have to buffer data for hours, days, or weeks before
   it can be forwarded.  Also, the highly varying communications
   environments that will make up the IPN, ranging from optical fiber on
   Earth to wireless communications around Mars, to the long-haul links
   of the backbone, suggest that different transport protocols will be
   needed for the different environments. It makes sense, therefore, for
   the IPN nodes to terminate the transport-layer protocols used in the
   respective IPN regions, holding data at a higher layer before
   forwarding it on, possibly using a different transport-layer
   protocol.

   We call this higher layer the "bundle layer," and the protocol used
   to send data between the various nodes of the IPN the "bundle
   protocol."  The term "bundle" is used to connote the store-and-
   forward aspect of communications where as much interactivity as
   possible has been distilled out of the communication.  A bundle file
   transfer request, for example, might contain the user's
   authentication (login/password, e.g.), the location of the file to
   get, and where that file should be delivered in the requester's IPN
   domain.  All of this information would be transmitted as one atomic
   "bundle," and the requested file would be returned. We use the term
   "bundle" rather than "transaction" to avoid notions of two-phase and
   three-phase commitment that are commonly associated with transaction
   processing.

   In traditional networking terminology it is generally the transport-
   layer protocol that operates end-to-end.  Since IPN nodes terminate
   transport layer protocols in order to buffer data and to enable them
   to use a transport protocol appropriate to the IPN region through
   which data will be sent, it is the bundle layer in the IPN that
   operates end-to-end.

   It should be noted that terminating the transport protocols at the
   IPN nodes decouples the internets in different IPN regions to a
   significant degree.  This has the desirable effect of also decoupling
   the evolutionary rates of those internets: changes in the Earth's
   Internet do not necessarily dictate changes in other internets.  This
   is important in an environment in which resources are and will
   continue to be severely constrained.

   Figure 1 illustrates the progression of a bundle of data through the
   Interplanetary Internet, from its source at host "A" to the
   destination at host "E."  Custody transfers are indicated by

Cerf, et al.             Expires November 2001                 [Page 17]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   asterisks (*), and occur at B, C, D, and E.  Host "A" initiates the
   bundle transfer, and the bundle is transferred using internet
   protocols (i.e., TCP and IP) to bundle node "B", which serves as the
   gateway between the left-hand internet and the interplanetary
   backbone.  The box icon indicates that custody of the bundle is
   transferred to that gateway.  When conditions permit, the bundle is
   forwarded on to the next hop in the store-and-forward chain.  In this
   case the next hop is another host within the interplanetary backbone
   network: host "C".

   Also illustrated in Figure 1 is the notion of a "return receipt" sent
   by the ultimate destination of the data back to the source.  This is
   an optional service, much like a return receipt within the postal
   system. If the source desires notification of delivery, that is
   accomplished by a separate return receipt, which is transmitted as
   its own bundle, and is subject to the same custody transfers as the
   original transmission (similar to the fact that a postal return
   receipt is, in itself, a postcard).  It is shown figuratively as
   bypassing the forwarding path (E to D to C to B to A), but this is
   simply for clarity.  The return receipt is forwarded in exactly the
   same manner as the original data is.



      An Internet         The IPN Backbone          An Internet
   +----------------+ +------------------------+ +----------------+
   |                | |                        | |                |
   |  +---+        +---+        +---+         +---+        +---+  |
   | /   /|       /   /|       /   /|        /   /|       /   /|  |
   |+---+ |      +---+ |      +---+ |       +---+ |      +---+ |  |
   ||   |=+=====>|   |=+=====>|   |=+======>|   |=+=====>|   | +  |
   || A |/       | B*|/       | C*|/        | D*|/       | E*|/   |
   |+---+        +---+|       +---+         +---+|       +---+    |
   |  /\           |  |                       |  |         ||     |
   |  ||           |  |                       |  |         ||     |
   |  ||           |  |                       |  |         ||     |
   +--||-----------+  +-----------------------+  +---------||-----+
      ||                                                   ||
      +=====================================================+
                            Return Receipt

                       Figure 1.  Custody Transfers

   Many aspects of this mode of data transfer resemble the postal
   system, or even the Pony Express.  The source depends on the store-
   and-forward nodes in the chain to operate on its behalf to deliver
   data that may not be retained at the source.  Source notification
   that the destination has received the data is optional, and there is
   no guarantee or even any firm expectation on the part of the source
   about when the data will be delivered.

   Conceptually, the bundle protocol resides above the transport

Cerf, et al.             Expires November 2001                 [Page 18]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   layer(s), and operates end-to-end between the ultimate source and the
   ultimate destination of the data.  The bundle layer provides a number
   of services to applications using it. For example the bundle layer
   carries the source and destination name tuples end-to-end to support
   the late binding of the destination name's administrative part to an
   address.  The bundle layer also carries the users' specifications for
   reliability, quality of service, and security.  Because the
   communication resources are precious, it is desirable to provide
   error recovery mechanisms that do not necessitate discarding of
   bundles that contain errors.  We are considering adding to the
   information carried in a bundle some information to assist in
   transfer-related error handling.  We are thinking in terms of active
   networking, using a combination of active packets and active nodes as
   a means of specifying appropriate actions to take in the event of
   problems in completing the transfer.  Additionally, the bundle layer
   provides invoking applications with a transfer identifier that is
   carried with the bundle, and is used in "return receipts." [Ref:
   http://www.comsoc.org/pubs/surveys/1q99issue/psounis.html]

3.2.   Information Carried by the Bundle Layer

   To effect the end-to-end transfers necessary in the IPN, the bundle
   layer must carry some information end-to-end.  This section documents
   our current thinking on the information that must be carried end-to-
   end, and notes which of those data elements may be supplied by the
   application using the bundle service.

    *  Bundle Identifier: this is a monotonically increasing number
       that is carried in the bundle, and also returned to the
       application to support return receipt processing.  It is not
       necessarily a sequence number, and as a result, there is no
       requirement that the value of Bundle Identifiers increase
       consecutively.  The requirement for monotonicity derives from a
       need to provide robustness against system crashes, and therefore
       must be persistent across system crashes.
    *  Remote entity name: this is the IPN name of the remote bundle
       agent, and is a tuple, as described above.  It is supplied by
       the local application using the bundle service.
    *  Source entity name: this is the IPN name of the local bundle
       agent, and is a tuple.  It is supplied by the local bundle
       service, since a particular host may have multiple names and one
       may be chosen based on routing decisions or other criteria
       opaque to the application (as in multihomed hosts).  The source
       name may be returned to the application to support return
       receipt processing.
    *  Authentication information: this is information, such as a
       digital signature, that is passed by the application to the
       bundle layer to unambiguously identify the source of the bundle.
       (Just what the source of the bundle is, person, place, address,
       etc. is still undecided.)  This information is checked for
       validity at both the source bundle agent and the destination
       bundle agent.  The authentication information may also be used

Cerf, et al.             Expires November 2001                 [Page 19]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



       for access control purposes within the network.
    *  Source application instance handle: this is similar in nature to
       a source port number in that it identifies the sending
       application.  Since bundles are inherently non-interactive, the
       typical use for this handle is to "reanimate" the source
       application when a return receipt arrives.  This could be hours,
       days, or weeks after the initial transmission, so the handle may
       well be a reference to a structure that allows the application
       to be reinstantiated with known state.  The source application
       instance handle may be used at the destination as an identifier,
       but may also be redundant with the end-to-end authentication
       information for this purpose.  This handle is supplied by the
       source application.
    *  Destination application instance handle: this is essentially the
       destination port identifier.   As with ports, these must be
       known to and supplied by the source application.  We have not
       yet fully explored the implications of port advertisement across
       the IPN.
    *  Size of data: this is a statement of the size of the bundle, in
       bytes.  It is supplied by the source application, and is used
       initially to ensure that sufficient space is available to store
       the bundle for its initial transmission.  Nodes receiving
       transmitted bundles use this information in the same manner: as
       a means of making a determination about the availability of
       storage early in the bundle handling process.  In forming the
       initial bundle, the bundle layer at the source may use the size
       of data parameter as a consistency check on the amount of data
       actually delivered.
    *  Handling instructions: these are parameters supplied by the user
       with the bundle that convey the user's preferences to the
       network.  Our thoughts on just exactly what these parameters
       look like are not yet firm, however, our thoughts are that they
       include some or all of the following: priority, quality of
       service (although we are still debating what this means in the
       context of the IPN), elapsed time after which the content of
       this bundle is meaningless (time to live as specified by the
       user), reliability requirements, and any error handling
       information.  For the most part, these are requests, and we
       perceive that the bundle layer may either override some or all
       of these requests or fail the request if local policy does not
       permit that particular user to make that particular request at
       that particular time.
    *  Data Descriptor: This is a reservation token that is generated
       by the bundle layer.  Its detailed definition and use are still
       to be determined.
    *  Time to Live: This is the time after which this bundle is to be
       discarded from the network.  It is present to facilitate the
       recovery of network resources and to terminate routing loops,
       should they occur.
    *  Loose/Strict Source Route and Record: This information is
       provided by the source application to facilitate debugging of
       the network.  It consists of a list of names of IPN nodes

Cerf, et al.             Expires November 2001                 [Page 20]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



       through which the bundle must pass on its way to the
       destination, and our intent is that it should behave similarly
       to the corresponding option within IP.
    *  Current bundle custodian: the bundle protocol supports store-
       and-forward operation in which the custody of a bundle (that is,
       the responsibility for ensuring reliable delivery) may transfer
       from one IPN node to another as the bundle progresses through
       the IPN.  There is not a requirement for each IPN node
       encountered to assume custody of a bundle.  As a result, it is
       necessary to identify the upstream node that has custody of the
       bundle, in order to either request retransmissions or to accept
       custody of the bundle.
    *  User data: this is intended to be all of the data that the
       remote entity requires to perform whatever operation is
       requested.  Since the environmental characteristics of the IPN
       make interactivity difficult, the notion is that all of the
       information that is required to perform a particular
       "transaction" would be provided in a single bundle.

3.3.   Reliability at the Bundle Layer

   Because no single transport-layer protocol operates end-to-end across
   the IPN, end-to-end reliability can only be assured at the bundle
   layer.  At each node along an end-to-end route, the bundle-layer
   protocol entity passes bundle data to the Transport layer for
   transmission.  Each bundle layer entity is highly confident that the
   transport layer will successfully convey the data entrusted to it to
   the next bundle-layer protocol entity (which may "take custody" of
   the data or merely relay it; a single hop).  But failures are
   possible (e.g., a host computer does an unplanned reboot).  Just in
   case the highly unlikely happens and a Transport-layer transmission
   fails, the first subsequent node that detects the failure and is
   capable of taking custody of the bundle will request that the prior
   custodian re-transmit any missing data (again using Transport-level
   transmission services across, potentially, one or more relay nodes).

   The bundle layer's confidence in the effectiveness of the underlying
   Transport-layer protocols is reflected in the design of the timers
   for bundle-layer reliability.  These timers are highly optimistic _
   that is, they expire as late as possible _ in order to give the
   Transport protocols every opportunity to complete reliable
   transmission.  The effect of this optimism is to minimize the chance
   of unnecessary bundle-layer retransmission, which could seriously
   degrade IPN performance by consuming valuable bandwidth.

3.4.   Bandwidth Allocation via Market Mechanisms:  "Starbucks"

   To promote effective and efficient use of the IPN's scarce
   transmission resources, some sort of sophisticated and adaptable
   bandwidth allocation system will probably be needed.  The scheme
   described below is based on a free-market notion of "fare-paying
   packets", where at initial transmission each bundle is issued a

Cerf, et al.             Expires November 2001                 [Page 21]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   "draft" on some small percentage of IPN resources.  These resources
   might well map back to actual monetary funds provided by the
   originator of the bundle to the provider(s) of the IPN service.  The
   bundle in effect pays its own way across the various legs of end-to-
   end transmission.  As it traverses each hop, the bundle spends funds
   from its original draft until either it is received or else its funds
   are exhausted.  If a bundle is dropped due to insufficient funds then
   the hope is that all available transmission resources were allocated
   to bundles that were allotted more funds and therefore were
   presumably more important (to somebody).

   At the initial Send-bundle service request, the source application
   (bundle sender) would specify total funds allocated to getting the
   bundle delivered to the destination.  Total funds allocation needs to
   be a function of (a) the total number of bytes of data and metadata
   to be sent, and (b) the prices charged for each "transmission class,"
   (something like First Class, Business Class, Economy Class, Steerage,
   Overhead Bin; etc.).  The allocation should cover the anticipated
   costs of traversing all the bundle hops along the anticipated route
   to destination.  If the bundle must be split up into multiple packets
   (bindles, segments), the bundle agent that performs the split also
   distributes the bundle's total funds among individual packets.  This
   distribution will probably be on a prorated basis.

   Also, the source application would supply the bundle (and, by
   implication, each of its packets) with traveling instructions:

   For each time epoch that elapses while awaiting transmission from any
   single bundle transmission agent:

    *  The length of the epoch in units of time (seconds?)
    *  The queue (transmission class) to wait in over the course of the
       epoch

   For example: get in the Steerage queue, but if 30 minutes pass and
   you still haven't been transmitted then pay for an upgrade to
   Business Class; if you still haven't been transmitted after 20
   minutes in Business Class, then give up.

   At each bundle transmission agent, each packet is handled as follows:

    *  For each of the packet's authorized time epochs until the packet
       is transmitted (that is, initially handed to the underlying
       communication layer; retention until successful custody
       acquisition by downstream agent is provided at no charge):

       - If (epoch duration * packet length * agent's price per unit of
         time per byte for this epoch's transmission class, i.e. queue)
         is greater than packet's residual funds, then discard the
         packet.  The packet's residual funds remain unexpended.

       - Else, append the packet to the requested queue.  Then:

Cerf, et al.             Expires November 2001                 [Page 22]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001





       - If epoch expires before packet is de-queued for transmission,
         then charge the source application an amount equal to (epoch
         duration * packet length * price per unit of time per byte for
         queue), reduce packet's residual funds by this amount, and
         start the next epoch; if no more authorized epochs, give up.

       - Else, upon de-queuing the packet for transmission, charge the
         source application an amount equal to (length of time spent in
         queue * packet length * price per unit of time per byte for
         queue) and reduce packet's residual funds by this amount.


    *  Transmission classes _ queues _are of varying maximum length
       (the more expensive queues are shorter) but packets in all
       transmission classes are at the same level of priority; packets
       are de-queued in round-robin fashion to ensure that no class is
       altogether starved for service.  Because First Class is a
       shorter queue than Business Class, packets that pay for First
       Class spend less time waiting.

    *  Separately, an Emergency queue is provided for system-critical
       packets.  Everything in the Emergency queue has higher priority
       than everything else, so nothing else gets transmitted until the
       Emergency queue is emptied.

   Periodically, each bundle protocol agent reports aggregate charge
   amounts back to the source applications and also to some central
   accounting authority ["the bank", nominally based on Earth]; this is
   a separate application-layer protocol.  When the central authority
   determines that a source application is out of funds, it reports the
   source application's bankruptcy to all bundle agents; from that time
   on, all service requests and packets received from that source
   application are rejected.

   Although this particular scheme may ultimately prove too complex to
   be workable, we think the general principle of fine-grained bandwidth
   allocation could contribute significantly to the viability of the
   IPN.

4. IPN Nodes

   Nodes within the IPN have a number of responsibilities.  As members
   in a store-and-forward chain, they have the responsibility for
   resource allocation to support bundle transfers.  These resources
   include, among other things, buffer space and transmission capacity.

   Additionally, IPN nodes have the responsibility of actually executing
   the bundle transfer.  Reliability requirements for bundle transfers
   are specified by the using application, and include both reliable and
   unreliable transfers (possibly with some intermediate, or partial,

Cerf, et al.             Expires November 2001                 [Page 23]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   reliability services).  The IPN nodes are responsible for using
   whatever reliability mechanisms exist in the underlying (transport-
   and-below) layers, and augmenting those mechanisms as necessary to
   effect the required reliability.

   Finally, IPN nodes are responsible for routing bundles between IPN
   domains.  IPN nodes may depend upon the services of the local
   internets (or the IPN backbone) for intra-domain routing.

   In this section, we first briefly state the types of IPN nodes that
   we have identified, and then we provide a number of exemplary end-to-
   end data transfer descriptions.  Finally, we list the error
   conditions that we have identified that may occur at the bundle layer
   during the course of end-to-end data transfer.

4.1.   Types of IPN Nodes

   We identify three grades of IPN functional capability.  In order of
   increasing scope, they are: agent capability, relay capability, and
   gateway capability.  All IPN nodes are able to act as bundle agents;
   some bundle agents are additionally able to act as IPN relays; some
   IPN relays are additionally able to act as IPN gateways.

   * Bundle agents build and consume bundles.  These could be stand-
     alone proxy devices or could be an operating system service
     collocated with an application.  The endpoints of an end-to-end
     bundle transmission need not be any more than bundle agents,
     though they may additionally have relay or even gateway
     capability.

   * IPN Relays receive bundles and forward them toward their
     destinations, either within or between IPN regions.

   * IPN Gateways reside at the interface between IPN regions.  The IPN
     gateways perform routing between the IPN regions.

   Orthogonal to these grades of capability is the ability to take
   custody of a bundle.  The endpoints of a bundle transaction are
   typically the initial and final custodians of the bundle.  Non-
   gateway relays may take custody of the bundles they receive;
   alternatively, they may simply provide mitigation of R2 effects
   (signal strength losses due to the extreme distances of
   interplanetary communication), but provide no custody transfer
   capabilities.  (In the latter case they are essentially "repeaters.")
   Gateways normally take custody but are not required to do so.

4.2.   Example end-to-end transfer

   We provide the following example of an end-to-end transfer that
   crosses multiple IPN Regions:  A host on Earth sends a bundle to a
   destination on Mars.  Figure 2 illustrates the network that is the
   subject of this example _ the Interplanetary Internet in this example

Cerf, et al.             Expires November 2001                 [Page 24]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   has five distinct regions interconnected by four IPN Gateways.  This
   example highlights the actions taken by the bundle layer and the
   interactions of the bundle layer with applications and with
   underlying transport protocols.

4.2.1. Backbone connectivity

   It is important to have some understanding of the routing that is
   required at the IPN Gateways.  Unlike terrestrial communications, the
   IPN's long-haul communication links are directional, mobile, and
   highly scheduled.  This is important, because directionality combined
   with mobility means that a transmitter and receiver must track each

                       +-------------------------+
                       |     Earth's Internet    |
                       |  IPN Region:  earth.sol |
                       |            +---+        |
                       +-----------/   /|--------+
                                  +---+ |
                                  |G/W| +
                       +----------| 1 |/---------+
                       |          +---+          |
                       |     The "Backbone"      |
                       |  IPN Region:  ipn.sol   |
                      +---+       +---+        +---+
                     /   /|------/   /|-------/   /|
                    +---+ |     +---+ |      +---+ |
                    |G/W| +     |G/W| +      |G/W| +
   +----------------| 3 |/  +---| 4 |/-----+ | 2 |/-------------------+
   |                +---+   |   +---+      | +---+                    |
   | Venus's Internet |     | Jupiter's    |   |    Mars's Internet   |
   | IPN Region       |     |  Internet    |   |    IPN region:       |
   |  venus.sol       |     | IPN Region   |   |      mars.sol        |
   +------------------+     |  jupiter.sol |   +----------------------+
                            +--------------+

         Figure 2.  An Interplanetary Internet of Five IPN Regions

   other in order to establish and maintain a communication link.  In
   the IPN, much of the mobility is due to orbital mechanics and is
   therefore relatively predictable.  However, this means that nodes
   that we would normally consider to be fixed, such as antennas on the
   surface of the Earth, are actually highly mobile as a result of the
   Earth's rotation and its revolution around the Sun.  (In this
   example, we confine ourselves to our local solar system, and do not
   consider the motion of our sun relative to celestial bodies outside
   our solar system.)  We can describe the predictable aspects of node
   mobility with an ephemeris, a table of the positions of celestial
   bodies at specified intervals of time.  Both a directional sender and
   receiver must know the other's position in order to establish a link
   between the pair.  In addition, these communication resources are
   highly scheduled.  It is not sufficient for a receiver to point at a

Cerf, et al.             Expires November 2001                 [Page 25]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   prospective target and just wait _ for example, a terrestrial node
   will typically have to point at several targets sequentially, and an
   interplanetary node will typically not have enough power to just wait
   for incoming messages.  Rather, a schedule of communication
   opportunities must be calculated and then refined with planned
   communication instances.  A communication opportunity establishes
   that the endpoints could establish a link if they were pointing at
   each other at the proper times.  We refer to a planned communication
   instance as an agreement (although not irrevocable) between the two
   parties to establish contact and communicate for a defined period of
   time.  The protocols for establishing the schedule of communication
   instances between all pairs of possible communicants will evolve --
   from something primarily done manually to something more automated as
   the Interplanetary Internet grows.

   The scheduled nature of connectivity in the Interplanetary Internet,
   particularly across the deep-space links, means that at the time of a
   bundle's arrival at an IPN Gateway, some or all of the possible
   outbound routes may be "down."  The gateway must store the bundle
   until the appropriate link is available and then transmit the bundle
   over that link.  One of the fundamental differences between the
   Interplanetary Internet and the terrestrial Internet is this inherent
   use of store-and-forward mechanisms in routing bundles.   With that
   in mind, let us consider the routing decisions made at some of the
   IPN Gateways in Figure 2.

4.2.2. IPN Gateway routing

   Bundle routing at an IPN gateway will typically have to deal with a
   mix of continuously available links and intermittently available
   links. Routing across a continuously available link is a relatively
   straightforward activity.  Routing in the presence of intermittently
   available links can be significantly more complex, as the gateway
   will need to decide not only the next hop destination but also the
   time at which to send the bundle.  Conditions elsewhere in the
   network may make it more desirable to send a bundle to a next-hop
   destination that is not yet accessible, even when an alternative
   route is currently available.  The routing function in IPN Gateways
   and other IPN nodes that have intermittent links has three distinct
   parts:  the contact scheduler, the route evaluation algorithm, and
   the dispatcher algorithm.

   Figure 3 illustrates the relationship of these functions with each
   other and with other elements of the communication protocol suite.
   The contact scheduling application establishes the schedule for
   communicating with next-hop neighbors.  The implications of this
   scheduling activity are broad _ orbital mechanics, resource
   management onboard the spacecraft, prospective communications loads,
   and so forth all play a role.  Because spacecraft resources must be
   coordinated among all functions (not just communications), this is
   typically going to be part of a larger resource management
   application (interactions with functions such as pointing and power

Cerf, et al.             Expires November 2001                 [Page 26]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   control are not shown in Figure 3, but are present nonetheless).  It
   is very likely that the contact scheduler will *not* be local to the
   bundle node, but rather a centralized function that distributes a
   contact schedule to IPN nodes that perform routing.  This may change
   in the future to a distributed contact scheduling algorithm, but for
   the foreseeable future, this will not be the case.  The product of
   the contact scheduler is a schedule of planned contacts, durations,
   expected data rates, etc.  It is intrinsically link-state in nature.



            +-------------------------------+
            |        User Applications      |
            +-------------------------------+
            +-------------------------------+   +--------+
            |    +-----------------------+  |   |        |
+---------+ |    |    Bundle Transport   |  |   | Policy |
| Contact | |    +-----------------------+  |   |        |
|Scheduler| |    +-----------------------+  |   +--------+
+---------+ |    |                       |  |       |
     |      |    |    +---------------+  |  |       |
     V      |    |    |  Dispatcher   |<------------+
 /-------\  |    |    |  Algorithm    |  |  |            +----------+
<Schedules>----->|    |               |  |  | /------\   |Route     |
 \-------/  |    |    +---------------+  |<--< Routes ><-|Evaluation|
            |    |   Bundle Routing      |  | \------/   |Algorithm |
            |    +-----------------------+  |            +----------+
            | Bundle Agent Functions        |
            +-------------------------------+
            +-------------------------------+
            |  +------+  +------+  +------+ |
            |  |  TCP |  |  UDP |  | LTP  | |
            |  +------+  +------+  +------+ |
            |  +----------------+  +------+ |
            |  |       IP       |  |  -   | |
            |  +----------------+  +------+ |
            +-------------------------------+
            +-------------------------------+
            |  Link and Physical Interfaces |
            +-------------------------------+

    Figure 3.  IPN routing applications and their relation to other
                        communication functions.

   The route evaluation application exchanges information with all
   first-hop neighbors (we hope this occurs infrequently) to build a
   general picture of the IPN beyond the first-hop neighbors.  We
   currently view this in terms of a distance-vector representation, but
   with metrics such as the expected delay to the destination IPN region
   and average aggregate data rate to the destination IPN region.  The
   mechanics for building these metrics are still in development.


Cerf, et al.             Expires November 2001                 [Page 27]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   The dispatcher algorithms accepts routing requests from the bundle
   transport layer and builds a "manifest" for each next-hop contact
   that is subsequently consumed by the bundle routing layer.  The
   dispatcher application consumes some or all of the following in
   deciding how to schedule bundle transmissions:  the contact schedule,
   the routing information, local policy information, and the per-bundle
   specifics provided by the bundle transport layer (such as bundle
   destination, length, priority, time-to-live, and possibly
   "Starbucks"-related information).  We envision a family of different
   dispatcher algorithms, operating as "plug-ins," that provide
   different levels of sophistication in the scheduling function to suit
   different needs.  The simplest versions of the dispatcher might just
   consider destination to schedule the bundle on the soonest-possible
   outbound contact.  More sophisticated versions might consider
   priority, bundle length (to preserve atomicity), and time-to-live
   requirements.  Others could support the Starbucks model or apply
   optimization techniques to attempt to improve the use of each
   contact.

   The following is a conceptual description of what happens:  when a
   bundle arrives at the bundle layer and needs to be routed, the bundle
   routing function posts a request to the dispatcher, noting the
   destination, length, priority, time to live, etc. of the bundle to be
   routed.  The dispatcher integrates this bundle into its manifest and,
   at the bundle's transmission time, informs the bundle routing
   function to send the bundle to the appropriate next-hop destination.

4.2.3. Systems participating in example bundle data transfer

   Figure 4 is a revision of Figure 2 to highlight those portions of the
   Interplanetary Internet that participate directly in the bundle
   transfer example.  Also shown in Figure 4 are the source and
   destination for the bundle data transfer, and the Domain Name System
   equivalents in the terrestrial Internet (DNS 1), in the IPN
   "Backbone" (DNS 2), and in the Mars Internet (DNS 3).  This figure
   will serve as the basis for the following bundle data transfer
   example.

   Table 1 provides the full host names of each of the primary elements
   in Figure 4.

Cerf, et al.             Expires November 2001                 [Page 28]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001




                                        +---+
                                       /   /|
             +------------------------+---+ |
           +---+   Earth's Internet   |DNS| +
          /   /|IPN Region:  earth.sol| 1 |/
         +---+ |          +---+       +---+
         |SRC| +---------/   /|--------+
         |   |/         +---+ |
         +---+          |G/W| +
             +----------| 1 |/---------+
             |          +---+          |
             |     The "Backbone"      |
             |  IPN Region:  ipn.sol   |
            +---+                    +---+              +---+
           /   /|-------------------/   /|             /   /|
          +---+ |                  +---+ |            +---+ |
          |DNS| +                  |G/W| +            |DST| +
          | 2 |/                   | 2 |/-------------|   |/+
          +---+                    +---+              +---+ |
                                     |    Mars's Internet   |
                                  +---+   IPN region:       |
                                 /   /|     mars.sol        |
                                +---+ |---------------------+
                                |DNS| +
                                | 3 |/
                                +---+

       Figure 4.  Interplanetary Internet showing principal systems.

   Table 1.  Host name tuples.
   +------------+------------------+---------------------------+
   | Host       | IPN Regions      |   Host name tuples        |
   +------------+------------------+---------------------------+
   | SRC        | earth.sol        |   {src.jpl.nasa.gov,      |
   |            |                  |    earth.sol}             |
   +------------+------------------+---------------------------+
   | IPN G/W1   | earth.sol        |    {ipngw1.jpl.nasa.gov,  |
   |            |                  |     earth.sol}            |
   |            | ipn.sol          |    {ipngw1.jpl.nasa.gov,  |
   |            |                  |     ipn.sol}              |
   +------------+------------------+---------------------------+
   | IPN G/W2   | ipn.sol          |    {ipngw2.nasa.mars.org, |
   |            |                  |     ipn.sol}              |
   |            | mars.sol         |    {ipngw2.nasa.mars.org, |
   |            |                  |     mars.sol}             |
   +------------+------------------+---------------------------+
   | DST        | mars.sol         |    {dst.jpl.nasa.gov,     |
   |            |                  |     mars.sol}             |
   +------------+------------------+---------------------------+



Cerf, et al.             Expires November 2001                 [Page 29]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



4.2.4. Step 1:  Bundle creation and first-hop transmission

   An application on the source host in Figure 4 has data that it wishes
   to send to the destination on Mars.  The exact content of this data
   is opaque to the bundle transfer, but assume that it contains all of
   the information necessary to accomplish some desired function.  That
   is, assume that application-specific instructions for storage,
   handling, error processing, and disposal accompany whatever data
   object is to be operated upon.  The application invokes the bundle
   agent, supplying it the information shown in Table 2.

   Table 2.  Information passed from source application to bundle
   agent.

   +-------------+---------------------+------------------------------+
   | Item        | Value               | Description                  |
   +-------------+---------------------+------------------------------+
   | Destination | {dst.jpl.nasa.gov,  |  IPN Name tuple of the       |
   | host name   |  mars.sol}          |  destination                 |
   +-------------+---------------------+------------------------------+
   | Destination | 0x00000008          |  Similar to port number.  Can|
   | application |                     |  be "well-known" (i.e.,      |
   | instance    |                     |  identify a daemon) or       |
   | handle      |                     |  "ephemeral" (i.e., identify |
   |             |                     |  a running, suspended, or    |
   |             |                     |  hibernating process         |
   +-------------+---------------------+------------------------------+
   | Source      | 0x1763421A          |  Value used to identify the  |
   | application |                     |  appropriate instance of the |
   | instance    |                     |  source application for      |
   | handle      |                     |  response processing         |
   +-------------+---------------------+------------------------------+
   | Handling    | Reliable delivery,  |  The services requested from |
   | instructions| normal priority,    |  the bundle layer.           |
   |             | data obsolete in 36 |                              |
   |             | hours.              |                              |
   +-------------+---------------------+------------------------------+
   | User data   | N/A                 |                              |
   +-------------+---------------------+------------------------------+

   The bundle agent creates a bundle and stores it in persistent storage
   (on disk or other non-volatile memory).  There are some fields of the
   bundle header that the bundle agent must supply:  the Bundle
   Identifier, the Source Host name tuple, the Custodian name tuple, and
   the time to live.  (The application may state a time after which the
   data are obsolete, but the actual time-to-live field in the bundle
   header uses the application's data in combination with network
   restrictions on time-to-live to initialize this field properly.)  The
   bundle agent's routing function requests a route from the dispatcher
   application, and receives next-hop destination information and source
   information.  (Since a host may reside in multiple IPN Regions, the
   source host name tuple is a function of the outbound route selected.

Cerf, et al.             Expires November 2001                 [Page 30]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   The bundle agent uses this information to complete the Source Host
   and Custodian name tuples prior to transmission.)

   Note:  For hosts that have continuously available communication
   resources, it is likely that an optimization would be implemented to
   reduce the required interaction with the dispatcher application.  In
   this instance, the bundle routing function might consult a local
   routing table before contacting the dispatcher, and only requesting
   routing information from the dispatcher if there is no appropriate
   entry in the routing table.

   The bundle in this example is destined for the "mars.sol" IPN Region
   (per Table 1).  The dispatcher (or routing table) determines that the
   proper next-hop destination for the mars.sol region is
   {ipngw1.jpl.nasa.gov, earth.sol}, and that the appropriate transport
   protocol to use is TCP.  The bundle agent consults the Terrestrial
   Domain Name Server to resolve ipngw1.jpl.nasa.gov to an IP address,
   establishes a TCP connection with ipngw1, and transfers the bundle to
   it.  The bundle agent at the source retains a custodial copy of the
   bundle in persistent storage.


4.2.5. Step 2:  Bundle processing at first-hop destination

   When the IPN Gateway {ipngw1.jpl.nasa.gov, earth.sol} receives the
   bundle via TCP, it stores it on persistent storage (disk).  The
   bundle agent consults the dispatcher, and is informed that the
   appropriate next-hop destination is in the "ipn.sol" IPN Region:
   {ipngw2.nasa.mars.org, ipn.sol}.  The dispatcher provides the time at
   which the bundle should be transmitted, at which time the contact
   scheduler and its associated functionality will ensure that there is
   a contact with ipngw2 via the Long Haul Transport Protocol (LTP).  In
   considering alternative contacts for the bundle, the dispatcher
   checks the time-to-live in the bundle, which was 36 hours from the
   time of initial submission to the bundle agent at the source, to
   ensure that the route selected is consistent with the time-to-live
   requirements of the bundle.  The bundle transport functionality of
   the bundle agent in ipngw1 accepts custody of the bundle, updates
   this information in the bundle header, and informs the source that
   has done so.  The sources bundle agent deletes its custodial copy of
   the bundle.  When the time indicated by the dispatching function
   arrives, the bundle is transmitted via LTP to the IPN Gateway that
   connects the ipn.sol Region with the mars.sol Region:
   {ipngw2.nasa.mars.org, ipn.sol}.

4.2.6. Step 3:  Bundle processing at gateway to destination IPN Region

   The Mars gateway, {ipngw2.nasa.mars.org, mars.sol}, receives the
   bundle from the Earth gateway via LTP.  It stores the bundle in
   persistent storage and accepts custody of the bundle, signaling back
   to the Earth gateway that it has done so.  When the notification of
   acceptance reaches the Earth gateway, ipngw1 deletes its custodial

Cerf, et al.             Expires November 2001                 [Page 31]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   copy.  The Mars gateway consults its dispatcher application to find
   an outbound contact to forward the bundle over.  The dispatcher
   returns an indication that the appropriate next hop is the
   destination itself, that the proper transport protocol is TCP, and
   that the destination is accessible immediately.  The gateway verifies
   that the time-to-live has not expired, and forwards the bundle to the
   destination.

4.2.7. Step 4:  Bundle processing at destination

   The destination bundle agent receives the bundle via TCP, stores it
   on its own persistent storage, and accepts custody of the bundle from
   IPN G/W2.  The bundle agent "awakens" the destination application
   process identified by the Destination Application Instance Handle.
   This may involve creating a new instance of a server from a daemon
   process, signaling an idle running process, or reinstantiating a
   process that has been suspended with its state stored on persistent
   storage.  (The specifics of this are system-dependent, and may have
   to be robust against system restarts, upgrades, and migration of
   processes from one host to another.)  The bundle agent deletes the
   copy of the bundle from persistent storage when the application has
   received it.  The destination application may generate an
   application-layer acknowledgment in a new bundle and send it to the
   source's application instance identifier (0x1763421A from Table 2).

4.3.   Error Conditions at the Bundle Layer

   This section describes the error conditions that may arise at the
   bundle layer during bundle creation and transport.  When these errors
   occur within the sender's IPN domain, it may be possible to conduct a
   near-real-time dialog to correct them before the bundle is forwarded.
   We say `may be possible' because even if two nodes are in the same
   IPN domain, they may not have real-time connectivity.  An example of
   such a situation would be if a lander were on the opposite side of
   the planet from its IPN gateway, and used bundles to communicate with
   the gateway through a low altitude orbiter, with the orbiter itself
   serving as a bundle agent.


Cerf, et al.             Expires November 2001                 [Page 32]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   Table 3: Error conditions at the bundle layer.
   +-------+---------------------------+------------------------------+
   | Error | Description               | Places where Error can Occur |
   +-------+---------------------------+------------------------------+
   | 1*    | Unknown destination region| Source Bundle Agent          |
   +-------+---------------------------+------------------------------+
   | 2*    | Invalid Source App.       | Source Bundle Agent          |
   +-------+---------------------------+------------------------------+
   | 3*    | Bundle Parameter Syntax   | Source Bundle Agent          |
   |       | Error                     |                              |
   +-------+---------------------------+------------------------------+
   | 4*    | Bundle Parameter Semantic | Source Bundle Agent          |
   |       | Error                     |                              |
   +-------+---------------------------+------------------------------+
   | 5*    | Invalid Node Name in LSRR | Any bundle node              |
   |       | or SSRR                   |                              |
   +-------+---------------------------+------------------------------+
   | 6     | Insufficient buffer space | Any bundle node              |
   +-------+---------------------------+------------------------------+
   | 7     | DNS unreachable           | Any bundle node              |
   +-------+---------------------------+------------------------------+
   | 8*    | Time exceeded             | Any bundle node other than   |
   |       |                           | the source agent             |
   +-------+---------------------------+------------------------------+
   | 9*    | Source Entity Access      | Any bundle node other than   |
   |       | Denied                    | the source agent             |
   +-------+---------------------------+------------------------------+
   | 10*   | Invalid Administrative    | IPN gateway serving          |
   |       | Destination Name          | destination IPN domain       |
   +-------+---------------------------+------------------------------+
   | 11*   | Invalid Destination App.  | Destination                  |
   +-------+---------------------------+------------------------------+
   | 12*   | End-to-end Access Denied  | Destination                  |
   +-------+---------------------------+------------------------------+

   The errors that can occur at the bundle layer are shown in Table 3.
   Error numbers marked with an asterisk (*) are reported back to the
   sending application.

   * Unknown Destination Region: This error occurs when the source
     bundle agent is directed to create a bundle destined for an IPN
     Region that is not recognized (i.e. one for which there is no
     applicable route known to the Dispatcher Application).  Note that
     only the IPN Region part of the destination name has to be
     interpretable outside the destination's IPN Region.  In
     particular, the administrative part of the destination name need
     not be interpretable to the source DNS (assuming the source and
     destination are in different IPN Regions), so it cannot
     necessarily be checked when the bundle is created.

   * Invalid Source Application: If the source application instance
     handle supplied by the source application is invalid, the source

Cerf, et al.             Expires November 2001                 [Page 33]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



     bundle agent responds with an Invalid Source Application error.
     This might be the case, for instance, if the source application
     provided an instance handle referencing a system process for which
     the application didn't have privileges.

   * Bundle Parameter Syntax Error: The source bundle agent may check
     the syntax of some of the bundle-creation parameters (i.e. it may
     ensure that the end-to-end and IPN access security certificates
     are well-formed, etc.)  If a parameter is found to be
     syntactically incorrect or obviously and definitely erroneous, the
     bundle agent will report a Bundle Parameter Syntax Error back to
     the source that includes, at a minimum, the parameter that caused
     the error.

   * Bundle Parameter Semantic Error: If the source bundle agent can
     identify a particular bundle creation parameter as being well-
     formed but unserviceable, it will report a Bundle Parameter
     Semantic Error to the source application that includes, at a
     minimum, the parameter that caused the error.

   * Invalid Node Name in LSRR/SSRR: If an invalid node name is
     discovered in the loose or strict source route & record, the
     bundle agent that detects the error will propagate it back to the
     source application.  Note that it may be advantageous to have
     bundle agents check the validity not only of the next hop in the
     source route but as many entries as they can.  The value of
     checking multiple entries would be in detecting errors as soon as
     possible, preferably before the bundle traversed any of the long-
     haul links.

   * Insufficient Buffer Space: If a bundle agent does not have
     sufficient buffer space to accept a bundle, it drops the bundle
     and generates an Insufficient Buffer Space error.  Note that a
     bundle node may choose to drop lower priority bundles in order to
     make room for higher priority ones. This error is not propagated
     back to the source.

   * DNS Unreachable: If a bundle agent needs access to its DNS (or
     DNS-equivalent) and cannot obtain information from it, it
     generates a DNS Unreachable error.  This information is not
     propagated back to the source application.

   * Time Exceeded: If the time-to-live field (either the source-
     provided TTL or the internal bundle TTL) expires, the source is
     notified with a Time Exceeded message.  These errors are
     propagated to the source application.

   * Source Entity Access Denied: This error indicates that the source
     entity does not have access to a needed resource at an IPN node.
     The source might not be authorized to use the node at all, or it
     might not have authorization to use a particular interface
     required by the bundle.  Source Entity Access Denied errors

Cerf, et al.             Expires November 2001                 [Page 34]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



     indicate that the source is not authorized to use a particular
     resource; other errors (e.g. Insufficient Buffer Space) indicate
     that a particular resource is unavailable to a bundle.  For
     example, an entity on the surface of a planet might be authorized
     to communicate, using the bundle protocol, with another entity on
     the other side of the planet via a low-altitude orbiter that is
     also an IPN gateway.  The sender might not, however, be authorized
     to send bundles across interplanetary space.  In this case bundles
     sent to the orbiter destined for the other side of the planet
     would not cause errors, while any bundles with off-planet
     destination addresses would.  Source Entity Access Denied errors
     are propagated back to the source application.

   * Invalid Administrative Destination Name: Once a bundle has reached
     its destination IPN Region, the administrative part of the
     destination name can be verified.  If the administrative part of
     the destination name is not valid, the source is notified with an
     Invalid Administrative Destination Name error message.  As with
     LSRR/SSRR, it would probably be advantageous to check the
     administrative part of the destination name as soon as possible to
     avoid propagating misnamed bundles and error messages across the
     backbone.

   * Invalid Destination Application: If the destination bundle agent
     cannot instantiate the destination application (based on the
     destination application instance handle in the bundle), it
     notifies the source application with an Invalid Destination
     Application error message.

   * End-to-End Access Denied: If the bundle destination does not
     accept the bundle due to an authentication or access-control
     error, the source is notified with an End-to-End Access Denied
     Message.

4.4.   Support of existing Internet applications

   There is no clean way to support "legacy" applications in the IPN.
   One might think that application-layer proxies could protect
   applications from the rigors of interplanetary communications, but it
   turns out that this is not the case, even if the appropriate timers
   (at both application and transport layers) could be scaled correctly.
   As an example, consider a legacy FTP client on Earth trying to get a
   file from an FTP server on Mars.  If the client's connection request
   to {foo.bar.com, mars.sol} is somehow coerced into binding to an
   Earth-resident application proxy (as can be done with nonstandard
   modifications to the terrestrial DNS) the proxy then has to negotiate
   enough information out of the client to form the bundle that will
   travel to Mars.  This bundle needs to include at least:
   * The client's IPN domain name
   * The server's name tuple on Mars  (Note that if wildcarded A-
     records are used to cause the off-planet server name to bind to
     the proxy, the proxy will not have a notion of the destination

Cerf, et al.             Expires November 2001                 [Page 35]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



     name.  The proxy will have to explicitly request the source name
     from the source.)
   * The file to be retrieved
   * Where to deliver the retrieved file to, since it probably won't be
     coming back in this session

   While all of this information could in principle be elicited from a
   command-line client via appropriate server messages and extensive use
   of the `quote' command, there is no guarantee that other clients
   (specifically GUI clients) will be able to accomplish this.  SMTP (e-
   mail) is perhaps the only application that could possibly be tuned to
   work over interplanetary distances, but even SMTP has embedded timers
   that would have to be altered significantly before it would work.

5. Security in the IPN

   We do not have a detailed list of security requirements for the
   Interplanetary Internet, primarily because there currently aren't any
   "users" of the IPN and few, if any, of the potential users have given
   enough thought to security to commit to a set of security
   "requirements".  However, we know that the Interplanetary Internet's
   bandwidth resources will be precious. We can also safely assume that
   the IPN will be a prized hacker's target (at least from Earth).  We
   can also envision that there will be precious, private data flowing
   across the IPN.  As a result, we assume that various security
   mechanisms and services will be required to provide protection for
   the bundles traversing the IPN and for the IPN infrastructure itself.

   There are two aspects to IPN security: protection of the IPN
   infrastructure and protection of the data traversing the IPN. The
   protection of the IPN infrastructure is not unlike the protection
   required for the Earth-based Internet infrastructure.  There will be
   a need to exchange routing information securely among the IPN nodes
   as well as to securely manage them.  For the IPN infrastructure to
   protect itself, the IPN nodes need to be mutually suspicious of one
   another.  That is, the IPN nodes will authenticate themselves to each
   other to ensure that they are not being spoofed by an untrustworthy
   entity.  One might ask if this is overkill if we believe that there
   will always be a small, controlled number of IPN nodes (a la the
   original ARPANET). However, one could equally envision that there
   could be many IPN nodes that are sponsored and controlled by multiple
   organizations (a la the current Internet).  Since we would like the
   IPN to easily scale, we want to build mutual suspicion and security
   mechanisms into the IPN architecture from the outset. It should be
   noted that the same mechanisms could be used to provide security for
   both the IPN infrastructure and the data flowing through it.

5.1.   Assumptions Regarding Required IPN Security Mechanisms

   The security mechanisms assumed to be required for the IPN are:
   * Access control
   * Authentication

Cerf, et al.             Expires November 2001                 [Page 36]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   * Data integrity
   * Data privacy

   Access controls will be required because the IPN's space-based assets
   will have limited resources available and because it will be a likely
   target from a hacking perspective.  By limiting access to the IPN
   resources, we limit the availability of the IPN to only those users
   who require its services and do not allow it to be overburdened by
   those who are not authorized to access the IPN services.

   Authentication of identity will be required to perform access control
   mediation.  To allow or disallow access to the IPN, an assured
   identity of the source of the network traffic will be needed.  The
   identity might be of an individual (e.g., a payload principal
   investigator) or a location (e.g., a science payload control center
   or a spacecraft control center).

   Data integrity will be required to ensure that data received across
   the IPN is the same data that was originally sent.  This is true from
   both an IPN infrastructure perspective (e.g., management data) as
   well as from a user's perspective (e.g., science data, command
   sequences). Data integrity assures that any unauthorized modification
   of the data will be detectable by the receiver.

   Data privacy will be required to ensure that only those who are
   authorized to obtain data traversing the IPN or destined to/from the
   IPN infrastructure will be able to do so.  This mechanism is also
   known as confidentiality.

   In the network security community, there are two well-known security
   paradigms: hop-by-hop security (also known as link security) and end-
   to-end security.  In the hop-by-hop paradigm, the data to be
   transmitted over the network is protected on a hop-by-hop basis.
   That is, the data is protected at its source, but in order to be
   routed to its final destination, it must be unprotected at a trusted
   routing point (e.g., a ground-based gateway, an IPN gateway) in order
   to be examined for onward routing.  The trusted routing point must
   then re-protect the data and forward it on to its next routing point.
   Each successive hop point must unprotect and re-protect the data
   until it reaches its ultimate destination.  A negative aspect of this
   security paradigm is that, depending on how the protection is
   applied; the data might be completely exposed while in the gateway
   (hence the term trusted gateway).  This means that the data is
   potentially vulnerable to unauthorized modification and unauthorized
   disclosure.

   The end-to-end paradigm does not employ trusted gateways.  Rather, it
   assumes that the path between the source of the data and its
   destination is hostile and cannot be trusted.  As a result, the data
   is protected at its source and is not unprotected until it reaches
   its destination.  However, in order for this scheme to work, routing
   information must remain unprotected so that the intermediate gateways

Cerf, et al.             Expires November 2001                 [Page 37]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   are able to determine how to forward the data without having the
   opportunity to read or change the data.

   One problem with the end-to-end paradigm is that it can only work
   across a network where there are end-to-end protocols (e.g., TCP).
   There has to be a protocol below the data that provides the ability
   to route.  One example of this is the Internet Engineering Task
   Force's (IETF) Internet Protocol Security Protocol (IPSEC).  The
   IPSEC Encapsulating Security Payload (ESP) protocol provides an end-
   to-end security service between the IP and TCP protocol layers.

   However, in the IPN, as has already been discussed earlier, IP and
   TCP are not necessarily carried end-to-end protocols.  Rather, those
   protocols can/will be terminated in a local internet (e.g. a ground
   segment on earth or another celestial body) and not carried end-to-
   end through the IPN.  The data will be carried end-to-end via the
   bundle protocol that would in turn, be transported by other IPN
   protocols (e.g., LTP, TCP) using the pony express model.

   Since the IPN must be structured on a store-and-forward basis, and
   since users may not trust the IPN gateways, solutions such as IPSEC
   cannot be employed.  In order to provide an end-to-end like security
   solution, security mechanisms can only be applied to the data and not
   any other protocol layer(s) below the bundle protocol.  This is the
   way the Secure Sockets Layer (SSL) (now being specified within the
   IETF as the Transport Layer Security (TLS) protocol) and secure email
   techniques such as S/MIME and OpenPGP work.  In essence, security
   services are only applied to the data portion of a packet, leaving
   all of the packet's protocol headers in the open and available for
   use by intermediate systems.  For example, to transmit the string
   "hello world" across the Internet would entail encapsulating the
   string into a TCP header, an IP header, and a media access (MAC)
   layer header.  To provide end-to-end security services, only the
   payload ("hello world") would have security services applied to it
   (e.g., it might be encrypted to provide privacy/confidentiality).
   However the TCP, IP, and MAC headers would all remain without
   security services applied to them.  In comparison, when using IPSEC
   ESP, the TCP header would have security services applied to it to
   protect it as well as the payload data.  And if it were operating in
   "tunnel" mode, ESP would encapsulate the entire payload - the TCP and
   IP headers - inside of an IP packet.

   Given that data will be transported across the IPN in bundles using
   an email-like paradigm, borrowing technology from email security is a
   solution for the IPN.

5.2.   Secure Email Technology

   Since secure electronic mail operates in a non-interactive mode, and
   since both communicating parties have not necessarily communicated
   with one another previously, a technique different than used by IPSEC
   was developed.  One way in which email could be securely wrapped

Cerf, et al.             Expires November 2001                 [Page 38]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   (e.g., encrypted and/or digitally signed) is via through the use of
   public key cryptography.  Using this technology, an email sender
   would use the public key of the intended recipient to encrypt the
   payload (i.e., the message) and the recipient would use its private
   key to decrypt the payload.  Likewise, the sender could digitally
   sign the message (in order to authenticate the message) using its
   private key and the recipient would verify the message's authenticity
   using the sender's public key.   However, there are two problems with
   this scheme for the IPN.

   The first problem is that public key cryptography is quite
   consumptive of processor power.  Therefore, the common practice is to
   use symmetric key (i.e., shared secret) algorithms for large amounts
   of data.  With changes in technology, this problem may disappear, at
   least for earth-based systems.  However, it is not clear that space-
   based assets will catch up as quickly.  The second problem is that
   public key cryptography is consumptive of bandwidth.  A public key
   exchange technology that allows two communicating entities to derive
   a shared symmetric key by virtue of knowing each other's public keys
   and exchanging some random information is known as a Diffie-Hellman
   exchange.  However, the exchange of information must be performed in
   a near-real-time environment so that the shared key can be used to
   encrypt transmissions.  This is not practical in the IPN since there
   are no real-time exchanges of data on an end-to-end basis.

   However, the secure email community has realized that they also
   operate under a pony-express model. The secure email community has
   also realized that secure email may be sent to entities with which
   one has not previously communicated.  Therefore, there needed to be a
   base-level expectation of a common, minimal set of security services
   that both sides could use.

   The general technique used by the secure email community is that the
   sending entity decides what security mechanism(s) to employ (e.g.,
   encryption for confidentiality, digital signature for authentication
   and integrity, or both).  If the data to be sent via email is to be
   encrypted, the sender generates a random key that is used to encrypt
   the data and the data is encrypted.  The sender then either has in
   its possession the public key of the receiver, or queries a public
   key server (e.g., a public key infrastructure or PKI) to obtain the
   receiver's public key, which would be contained in a digital
   certificate. At the expense of additional bandwidth, the sender can
   transmit its digital certificate in the email message, which the
   receiver can verify as genuine based on the well-known certificate
   authority's signature on the certificate.  The key used to encrypt
   the data is encrypted using the public key of the receiver and the
   message is sent.  The receiver uses its private key to first decrypt
   the key used to encrypt the message and then uses the decrypted key
   to decrypt the data.

   When a digital signature is used, the sender calculates a hash of the
   message using a digest algorithm such as MD5 or the Secure Hash

Cerf, et al.             Expires November 2001                 [Page 39]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   Algorithm (SHA-1).  The resulting hash is then encrypted using the
   sender's private key.  The encrypted hash is sent along with the
   message data.  The receiver uses its copy of the sender's public key
   to authenticate the fact that the message was sent by the sender.

5.3.   Application of Secure Email Technology to the IPN

   Given that the IPN and electronic mail share the same operational
   paradigm, the IPN's notion of "bundle-space" is directly analogous to
   a secure email MIME body part.  As was previously explained, in
   secure email the contents of the message are encrypted using a
   symmetric key.  The actual message contents are "containerized"
   (which can be analogized to the contents being "bundled") into a MIME
   body part.  Additional MIME body parts are also attached which would
   contain the encrypted symmetric key and additional information needed
   for decryption.

   Therefore, the same security techniques can be applied to the IPN's
   notion of "bundle-space." It can be seen how bundle payload data
   carried through the IPN is equivalent to an email message.
   Therefore, the bundle payload data, like the email message, can have
   security services applied to it before being sent as a protected
   entity via bundling across the IPN.

   In essence, the really difficult part of deploying an IPN security
   solution based on secure email is the problem of deploying a public
   key infrastructure (PKI) in the IPN. Deploying a PKI on Earth is
   equally difficult as can be seen by the continuing problems faced in
   the various PKI pilot programs currently underway.

   On Earth, the problem is how do competing PKIs cross certify public
   keys and who acts as the root certification authority? (PKI is based
   on a tree hierarchy where a root certificate authority's public key
   is well known in order to certify public key certificates).  On
   Earth, when a user does not have the necessary public keys, the user
   can go off to a public key certificate server to obtain the needed
   certificates which contain digitally signed (i.e., certified and
   authenticated) public keys. In the case of email, the sender can
   attach its public key certificate as a MIME body part.  The receiver
   then validates the digital certificate and uses it to obtain the
   information contained in the message.  Eventually, a cache of digital
   certificates is formed containing the information needed to securely
   communicate among a cadre of entities without having to resort to the
   use of a key/certificate server.

   The IPN, however, will not have the luxury of being able to query on-
   line PKI certificate servers (at least not in the same sense as is
   being contemplated on Earth).  The problem is very much analogous to
   the one with the Domain Name System (DNS).  When a user sends a
   secured "bundle" to an IPN entity on another celestial body, we would
   not want the receiver to have to first query a PKI for a public key
   certificate on Earth to obtain the receiver's public key.  A local

Cerf, et al.             Expires November 2001                 [Page 40]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   PKI might exist. However there is the issue of the dynamic nature of
   such a server and the high likelihood that it would quickly drop out
   of synch.  It would seem reasonable that the public key certificates
   of the various space-based entities would not change often, but this
   could not be said about the Earth-based entities.

   An earth-based sender may be able to query a certificate authority to
   obtain a certificate for an IPN entity not located on earth.
   However, an IPN entity in space sending a bundle would not be able to
   perform such a query.  Therefore, it would appear that a set of pre-
   placed, cached certificates containing the public keys of those IPN
   entities that are expected to be communicated with would make sense.
   In addition, an IPN sender should always include its public key
   certificate as part of the bundled transmission across the IPN as is
   done using secure email.  This would "cost" us in terms of bandwidth
   (a typical X.509 public key certificate is about 1K bytes in size but
   can be larger when certificate chains are included).  However, by
   including the certificate, we can be assured that the receiver will
   not have to use any additional bandwidth, or incur any additional
   delays in making use of the transmitted data.

5.4.   Protecting IPN Data and the IPN Backbone Infrastructure

   The IPN will have few and precious resources.  Therefore, not only
   the user data flowing across the IPN will require security services -
   the backbone infrastructure itself will require similar services.  In
   order for the IPN infrastructure to be self-protecting, it must be
   built using the paradigm of mutual suspicion.  Mutual suspicion
   requires each entity of the IPN to not assume that another IPN entity
   with which it is communicating is a "friendly" entity.  That is, it
   should not assume that the IPN entity is who it claims to be without
   a verified means of identification.  Based on a verified identity,
   access controls can be performed to allow or disallow communications
   between entities.

   Infrastructure information such as routing updates, node management
   information, distribution of digital certificates, and certificate
   revocation lists need to be protected from unauthorized modification
   and potentially from unauthorized disclosure.  The IPN nodes would
   use the same mechanisms as are used to provide protection for user
   data - namely the security services available to a bundle aware
   application (e.g., a "bundle-agent").

   A bundle aware application would encrypt and/or digitally sign the
   IPN infrastructure payload to be transmitted to another IPN node.
   The signed and/or encrypted payload would then be presented to a
   bundle-agent that would prepare the payload data to be carried across
   the IPN in a bundle.  Potentially, the bundle-agent might also sign
   and/or encrypt for hop-by-hop security protection, which would allow
   each bundle-agent receiving the bundle to authenticate the identity
   of the transmitting bundle-agent.  This mechanism provides the
   ability to ensure that no other entity can masquerade as a rogue

Cerf, et al.             Expires November 2001                 [Page 41]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   bundle-agent.

   The bundle-agent residing at a ground-based IPN gateway should check
   the signature of the bundle payload to perform an access control
   check to limit access to the IPN only to those who are authorized to
   use it.  The access control check can be accomplished via an access
   control list.

   Finally, the receiving application would make the final security
   checks before accepting the data payload from the final receiving
   bundle-agent.  Should all the final checks succeed, the receiving
   application would then be free to consume the data.

6. Building a Stable Backbone for the IPN

   Just as the performance and capability of the terrestrial Internet
   are largely determined by the capacity and stability of its backbone
   links, so will the performance and capability of the Interplanetary
   Internet depend in large part on the capacity and stability of the
   interplanetary backbone.

   By "backbone" we mean a set of high-capacity, high-availability links
   between points of access to high-activity subnets.  In the
   terrestrial Internet, backbone links are typically between the high-
   activity subnets for cities such as Houston and Chicago.  In the
   Interplanetary Internet the backbone links will be between the high-
   activity "subnets" for planets such as Earth and Mars.

   But the IPN backbone will differ from familiar terrestrial backbones
   in several important ways.

   * The media by which information is transmitted obviously differ.
     Terrestrial backbone links used to be copper wire and are now
     optical fiber.  In the Interplanetary Internet, all information
     will be via radiation _ either RF or optical _ through empty
     space.
   * The nature of the connectivity between backbone points of presence
     (POPs) will be fundamentally different.  Terrestrial Internet
     connectivity is structural and relatively static: nodes are
     physically attached to fiber.  Interplanetary Internet
     connectivity will be operational, directed, and highly dynamic:
     radiation must be directed at the right moment toward nodes that
     are prepared to detect it, and the transmitting and receiving
     entities will be in continuous movement relative to one another.
   * The costs of deploying, repairing, and upgrading infrastructure
     will be much higher for the interplanetary backbone than for
     terrestrial backbones.
   * The costs of configuring, operating, and managing the
     interplanetary backbone will likely be somewhat higher than for
     terrestrial backbones.  One factor in this is the scarcity and
     high cost of electrical power at IPN nodes deployed off Earth.
   * Perhaps most important of all, the distances between nodes of the

Cerf, et al.             Expires November 2001                 [Page 42]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



     interplanetary backbone will be vastly greater than the distances
     between nodes of terrestrial backbones _ so great that the speed
     of light becomes a very significant constraint on backbone
     operations.

6.1.   Backbone Design Considerations

   The differences described above imply two general constraints on the
   design of the interplanetary backbone.

   1.
     Bandwidth is not free, or even cheap.  Reliable delivery cost per
     bit will be far higher across the interplanetary backbone than
     across any terrestrial backbone, so bandwidth must be thoughtfully
     allocated.  Both retransmission and forward error correction
     coding contribute to reliable delivery, but both cost bandwidth;
     we must seek a balance between the two that minimizes overall
     overhead.
   2.
     Interactive protocols don't work, at least not well.  Negotiation
     _ connection establishment, flow control, congestion control _ can
     easily take so long as to consume all available transmission time.
     Reliable, in-order stream delivery can be so severely delayed by
     retransmission latency as to be operationally useless.

   These design constraints, in turn, must be accommodated at four
   layers of the protocol stack.

   At the physical layer, the relevant research is in radiant RF or
   optical communication.  The physical infrastructure of the
   interplanetary backbone consists mainly of antennae, many of them
   mounted aboard orbiting or landed spacecraft, rather than cable in
   buried or undersea conduit.  In the near to medium term, the
   principal elements of this infrastructure will be Earth-based
   tracking stations, such as NASA's Deep Space Network, and the planned
   "Marsnet" of low-Mars orbiters plus a possible areostationary relay
   satellite.  In the long term these assets might be augmented by
   optical communication satellites orbiting Earth and/or other
   planetary bodies, and perhaps by additional relay satellites
   positioned at the LaGrange points of planetary orbits (possibly using
   solar sailing technology for autonomous station-keeping).

   Although this infrastructure will not be subject to some of the
   problems of terrestrial backbones _ for example, we needn't worry
   about backhoes cutting through underground fiber _ there will be
   other challenging operational issues.  Accuracy in pointing and
   transmission scheduling at the backbone antennae will be critical to
   assuring the most efficient use of these assets.  This means that all
   elements of the interplanetary backbone infrastructure must share a
   common understanding of (a) one another's orbital dynamics and (b)
   the current time.  The latter argues for the importance of reliable
   space-borne clocks, together with a protocol for frequent correction
   of clock drift based on current navigation data.


Cerf, et al.             Expires November 2001                 [Page 43]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   Moreover, there will be times at which connectivity between a given
   pair of backbone antennae will be impossible due to the interposition
   of large, radiant planetary bodies _ or worse, the sun.  These
   occultations will be predictable, given good models of orbital
   dynamics, but will nonetheless necessarily constrain the manner in
   which data flow through the backbone.

   At the link layer, the design issues are somewhat less exotic.  The
   interplanetary backbone will require link protocols that minimize
   overhead and that have no inherent expectations of low noise or low
   point-to-point transmission latency.  CCSDS protocol standards such
   as Proximity-1 are plausible candidates. They support robust encoding
   to reduce bit error rates, at some cost in bandwidth consumption.

   Our current thinking is that no interplanetary backbone functionality
   will be required at the network layer.  The reason for this is that
   we expect the endpoints of transport-layer protocol on the backbone
   to be in direct line-of-sight contact as discussed below; since there
   will be no intervening nodes to route through, there will be no need
   for routing functionality at the network layer.  [Selection of
   endpoints for point-to-point transport-layer communication does
   indeed imply the need for routing and network functionality, but this
   requirement is addressed at the Bundling layer above Transport.]

   Finally, the general constraints on the design of the interplanetary
   backbone mandate constraints on the protocol used at the transport
   layer.  The TCP transport protocol used in both the backbones and the
   subnets of the terrestrial Internet _ and of other planetary subnets
   of the IPN _ will not be suitable: connection negotiation and in-
   order stream delivery are incompatible with the enormous distances
   between nodes of the interplanetary backbone.

   As noted earlier, the bundle protocol residing just above the
   transport layer is by nature relatively optimistic about transmission
   success but it must have transport-layer-like properties, i.e., it
   must be able to recover from transmission failure at lower layers.
   The capacity for timeout detection and custodian-to-custodian
   retransmission has to be built into it.

   However, the bundle protocol's structural optimism would result in
   poor performance if such a capacity were exercised frequently.  That
   optimism has to be justified by the general trustworthiness of lower
   layers:

   * When the bundle protocol runs over TCP in a deployed internet,
     TCP's own retransmission regime automatically recovers from errors
     in the network and link layers, and only a failure in TCP itself
     will trigger retransmission at the bundle layer.

   * When the bundle protocol is operating over interplanetary
     distances, a similar level of trustworthiness must be provided by
     a new interplanetary transport protocol.

Cerf, et al.             Expires November 2001                 [Page 44]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001




   This protocol will necessarily be very different in operation from
   TCP.  For example, connection time within the interplanetary backbone
   will be too rare and valuable to squander in waiting for reciprocal
   traffic of any kind.  Consequently the basic mode of operation in the
   interplanetary transport protocol will be asynchronous: data will be
   issued continuously throughout each episode of connectivity, with
   return traffic _ acknowledgements and coordination messages _ matched
   up to the corresponding original data as it arrives.  Data will
   arrive at the final destination out of order (because retransmitted
   data will arrive late); in many cases, even out-of-order delivery to
   applications maybe preferable to waiting the minutes, hours, or even
   days required for complete, error-free acquisition of data.

   Fortunately, out-of-order delivery due to retransmission delay may be
   made relatively rare by forward error correction at the link layer.
   Only a failure in that error correction need trigger retransmission
   at the transport layer.

   Accurate timeout detection will be critical to the success of this
   retransmission regime.  Premature timeouts would result in
   unnecessary retransmission, consuming precious bandwidth and
   degrading link utilization; late detection of timeouts would delay
   both bundle delivery and the recovery of retransmission processing
   and storage resources.  We believe interplanetary transmission
   timeouts can be detected accurately, but only when round-trip times
   can be calculated from firm operational data rather than estimated
   from past experience: the extreme variability in round-trip time
   introduced by intermittent connectivity means that the total round-
   trip time for transmission N might be hundreds of times greater than
   that for transmission N _ 1.  The required operational data could in
   theory be provided regardless of the topological relationship between
   the Transport-layer endpoints, but we believe support for any model
   other than point-to-point, direct line-of-sight transmission between
   the endpoints will not scale.  [Note that accurate clock
   synchronization across the interplanetary backbone is as important
   for implementation of timeouts at the transport layer as it is for
   resource scheduling at the physical layer.]

   Any number of blocks of transport-layer data traffic might
   concurrently be in various stages of transmission and retransmission,
   so the aggregate storage space allocated to transmission state data
   and retransmission buffers might be very large.  Moreover, loss of
   this information due to (for example) an unplanned power cycle could
   abort any number of transmissions _ unlike an unplanned reboot of a
   TCP host, which will crash only the messages that are currently being
   transmitted on all open streams.  For these reasons, it may well be
   desirable to retain interplanetary transport protocol data in non-
   volatile storage rather than dynamic memory.

   In some cases a set of transport protocol agents _ e.g., the set of
   relay satellites orbiting a planet _ may be most useful if they can

Cerf, et al.             Expires November 2001                 [Page 45]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   operate in concert as a single "aggregate" entity.  By distributing
   the retransmission workload across multiple agents as they
   successively acquire connectivity to a common peer agent, this
   collective behavior might reduce the total elapsed time required to
   effect reliable completion of a large block.  If so, we'll
   additionally need some sort of application-layer coordination
   protocol operating among the members of this aggregate entity.

   Flow control and congestion control are possibly the least tractable
   interplanetary backbone design issues at the transport layer.  TCP's
   strategies are based on reciprocal message exchange that the large
   delays in interplanetary communication make generally impractical.
   Alternative approaches remain to be discovered.

7. Deployed Internets in the IPN

   We differentiate between two types of networks in the IPN: the long
   haul backbone and deployed internets.  The long haul backbone,
   described in the previous section, is characterized by long
   propagation times, due to the great physical separation among the
   communicating elements.  Its protocols are designed to operate
   efficiently in long-delay, intermittent-connectivity environments.
   As discussed in [ref: Why not TCP], as delays increase, the
   efficiency of the Internet suite steadily decreases until
   applications fail altogether due to embedded time outs.

   We designate a network to be a "deployed internet" if it meets the
   following criteria:

   * It has connectivity to an interplanetary gateway, and through that
     gateway can reach the interplanetary backbone or other deployed
     networks.
   * It has a communication environment that doesn't inherently
     preclude the use of (possibly enhanced) internet protocols.
   * It uses the Domain Name space as a common means of referencing
     objects and systems across deployed internets.

   This designation covers a wide range of possible configurations.  The
   following list provides examples of deployed internets:

   * A single lander that hosts an interplanetary gateway and a (real
     or virtual) lander-internal network;
   * A small number of cooperating robots on a planetary surface, such
     as a single lander and a single rover;
   * An orbiter, a lander, and a sample-return vehicle communicating
     among themselves and via interplanetary gateways hosted in the
     orbiter and/or the lander;
   * Multiple "cells" of robots on a planet surface communicating
     beyond line of sight via low-orbiting satellites that serve as
     relays and as interplanetary gateways;
   * Similar scenarios as above, but with one or more planet-stationary
     satellites serving as interplanetary gateways;

Cerf, et al.             Expires November 2001                 [Page 46]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   * Spacecraft-onboard networks that communicate with remote endpoints
     via interplanetary gateways;
   * The earth's Internet is the initial instantiation of a "deployed
     internet."

7.1.   Applications of deployed internets in the IPN

   It is appropriate to consider the types of activities that will
   exercise the deployed internets.  Clearly, we do not yet know all of
   the types of applications that will be used in deployed internets,
   but we do have a basic idea of some of the types of applications that
   will be used in the relatively near term.  In developing this notion
   of applications within deployed internets, we want to eventually
   develop the ability to characterize their use of the IPN in terms of
   arrival rates, data volume, latency and reliability requirements,
   number and location of producers, number and location of consumers
   (within the IPN), etc.  At this moment, we have only highly
   qualitative notions of these kinds of data.  Over time, we will
   develop a more quantitative representation of this traffic as an
   adjunct for testing.  The model that we develop will probably never
   accurately reflect the actual use of the IPN, because the use of the
   IPN will be affected by its availability:  users will adapt their
   usage patterns as their understanding of the network's capabilities
   develops.  We don't ever expect to hit this moving target, but we
   hope to come close enough to discover most of the usage-related
   issues that might exist.

   One of the primary uses of a deployed internet is to return science
   data from the point where it is collected to the point where it will
   be processed.  The processing point could be on a remote internet
   (say, on earth) or at some point in the local deployed internet.
   Depending on the resource availability at the gathering site versus
   the availability at the processing point, the transfer could be
   largely unprocessed data, shipped as a formatted stream or as a file.
   Alternatively, the data could be processed to reduce its transmission
   size, which would likely result in a file transfer operation.  Sizes,
   frequency, precise reliability requirements, and so forth remain to
   be determined.  However, in general, this data is not particularly
   time-sensitive (although the equipment may be quite sensitive to the
   amount of time that it takes to complete a transfer due to power
   considerations, which will be discussed later).

   Another primary application, the reporting of health and status
   telemetry, is typically accomplished via repetitive, unreliable
   transmission in today's spacecraft.  There is underway a slow
   evolution toward data-summarization on board spacecraft.  But there
   is, understandably, some resistance to filtering or summarizing data
   at the spacecraft, since in the event of a spacecraft catastrophe,
   data not previously transmitted is not available for post-mortem
   analysis.  An important aspect of current telemetry systems is its
   delivery characteristics, which are either "stream-oriented" or
   periodically delivered.  Mission operators perceive that "heartbeat"-

Cerf, et al.             Expires November 2001                 [Page 47]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   style information that is ancillary to a periodic or "continuous"
   stream of telemetry is an important characteristic that will not
   displace event-driven telemetry for the foreseeable future.

   A third type of application is the command and control of in-situ
   elements.  Command and control refers to the closed-loop control of
   remote systems.  The endpoints of the control loop could be separated
   by interplanetary space, as in the case of terrestrial commanding of
   a rover.  Alternatively, the endpoints could be in reasonably close
   proximity, but resident on physically separate platforms connected by
   wireless media.  This is the case when a lander controls a rover.
   These types of applications must be designed to accommodate the
   delays intrinsic in the network, but the network can permit more
   responsive control operations by providing qualities of service that
   mitigate those delays.  With command and control applications, data
   volumes tend to be fairly low, and the commands tend to be followed
   by responses (although in some instances, command responses are
   inferred from returned telemetry, described above).

   The final type of application we consider is telescience and the
   development of a "virtual presence."  This type of application is
   intended to synthesize great volumes of information about the local
   environment in order to allow earth-resident scientists, in-situ
   controlling robots, or eventually in-situ astronauts to interact with
   high-fidelity models of a portion of a remote environment. To an
   extent, this technology has already been used to assist in planning
   the Mars Pathfinder/Sojourner excursions (ref
   http://www.piercorp.com/Projects/mars/mars2.htm). However, the
   technology for fully exploiting this type of adjunct to exploration
   is still in development.  Terrestrial research into telepresence and
   tele-immersion (ref:
   http://www.advanced.org/tele-immersion/publications.html) may provide
   insight into the workload that this type of application might
   represent for the IPN.

7.2.   Characteristics of remote deployed internets in the IPN

   Although we consider the Earth's Internet to be the first instance of
   a deployed internet, the environmental characteristics affecting
   internets deployed in remote locations may differ significantly from
   those affecting Earth's Internet.  In examining these
   characteristics, we must differentiate between two different types of
   environments in the Earth's Internet: the traditional, "wired"
   environments; and the emerging mobile, ad hoc wireless environments.

   First and foremost among the environmental characteristics of note is
   that of power availability.  In the "wired" Internet on Earth, power
   is cheap and plentiful.  Power availability is more of an issue in
   terrestrial mobile, ad hoc networks (MANETs), because the mobile
   nodes operate using portable power sources, typically batteries.
   However, in terrestrial MANETs, the mobile nodes eventually have
   access to the same cheap and abundant sources of power that the

Cerf, et al.             Expires November 2001                 [Page 48]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   "wired" nodes use.  This is not the case for remote deployed
   internets (RDIs) in the IPN.  For the foreseeable future, the primary
   source of power for RDIs in the IPN will be the sun.  Solar power
   conversion is currently relatively inefficient, but even if dramatic
   improvements in conversion efficiency occur, the fact remains that as
   one moves away from the sun, the available power diminishes.  In the
   orbit of Mars, the average solar intensity is 590 W/m2, less than
   half of the 1370 W/m2 in Earth orbit [ref:
   http://powerweb.lerc.nasa.gov/pv/marspower.html ].  On the surface of
   a planet, the solar intensity is by seasonal variations, by dust
   build-up on solar panels, and by dust erosion of the solar panels
   over time.   As a result, power availability is and will be of
   overriding importance to all aspects of communication in RDIs, and
   will dictate a need for efficiency at all protocol layers.

   The signal-to-noise ratios (SNRs) experienced in the terrestrial
   wired Internet are currently very high, making loss due to data
   corruption a rare experience.  In terrestrial MANETs, SNRs are lower,
   due to both power availability and to node density.  In remote
   deployed internets, SNRs will be very low, due primarily to power
   availability.

   In the terrestrial wired Internet, the bulk of the routing
   infrastructure is fixed in terms of its location.  Conversely, in
   terrestrial MANETs, the infrastructure is deployed on an as-needed
   basis, and may be mobile.  In RDIs, this infrastructure is also
   deployable and mobile, but will have a large component of satellite-
   based resources.

   If we examine the transmission media in use, the terrestrial wired
   Internet uses primarily copper cable and optical fiber.  Terrestrial
   MANETs use free-space propagation in the radio frequency (RF) range
   or the infrared range of the electromagnetic spectrum.  In RDIs, we
   anticipate that free-space RF will be the primary communication
   medium, even for many permanent, immobile installations as they
   develop, due to the cost of landing, deploying, and maintaining cable
   or fiber.

   The cost characteristics of deployment, operations, repair of RDIs is
   not currently well understood.  However, deployment cost of anything
   that must be landed on the surface of another planetary body is quite
   high.  Correspondingly, repair or replacement costs of components of
   the RDIs are also high.  We do not yet have a clear understanding of
   how the operations costs of the RDIs will compare to the operating
   costs of terrestrial MANETs or of the terrestrial wired Internet.

7.3.   Effects of environmental characteristics on protocols for the
       IPN RDIs

   In considering how to deal with the anticipated operating environment
   for RDIs, we are encouraged that a significant amount of relevant
   research is currently under way to support protocols for terrestrial

Cerf, et al.             Expires November 2001                 [Page 49]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   MANETs.  This section briefly reviews some of the areas in which work
   is required, and is organized by protocol layer.

   At the physical layer, spectrum management is an issue that requires
   coordination.  There is currently no "Federal Communications
   Commission" or International Telecommunications Union that regulates
   use of radio-frequency spectrum outside of earth.  However,
   coordination of this sort is needed sooner rather than later, due to
   the attractiveness of certain frequency bands as a result of their
   free-space propagation loss and refractive characteristics [ref:
   private communication].

   The cost of landing equipment on remote planetary surfaces motivates
   designers to keep as much communication infrastructure in orbit as
   possible.  Low-orbiting satellites will be used extensively to
   support surface-to-surface communication and RDI-to-RDI
   communication.  Antenna design to support wideband communication
   between surface-based mobile nodes and these low-orbiting satellites
   is an area where research might yield great benefit.  Although we
   have heard of some research in these areas to support military on-
   the-move applications, we have seen nothing in detail and feel that
   it is an area that is ripe for additional study.

   At the link layer, management of low SNRs is of significant
   importance, with many areas of relevant research.  While many
   different coding schemes are available, it is not yet clear whether
   one of these is best for all RDI use or whether the QOS requirements
   of different types of traffic will dictate the use of different
   coding schemes on a per-packet basis.  Many codes are currently being
   considered, including convolutional coding, concatenated codes (such
   as Reed Solomon codes), and the emerging Turbo codes.  Each of these
   has different properties related to delay, residual error rate, and
   link acquisition characteristics.

   Resource reservation schemes at the media access level may be of
   significant importance in supporting closed-loop control operations.
   Some of these schemes have the benefits of providing bounded delays
   and of avoiding interference, which is important in power-constrained
   environments.

   Link-layer status detection and signaling (to upper layers) is
   important in resource-constrained environments. We believe that the
   link layers must detect and signal link availability, link capacity
   and congestion status, and current error conditions.  As the
   terrestrial cellular telephone industry becomes internet-enabled,
   some of these issues are beginning to be addressed in single-hop
   wireless environments.  There is also ongoing research of interest in
   MANETs and in DARPA-sponsored research [ref.
   ftp://ftp.rooftop.com/pub/apis/link_api.pdf ].

   At the network layer, there is a need for routing protocols to
   support both fast- and slow-moving mobile nodes.  The routing

Cerf, et al.             Expires November 2001                 [Page 50]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   protocols must be able to constitute and maintain networks comprised
   of a combination of fixed and mobile nodes.  Significant research in
   this area is currently being performed, and is being coordinated by
   the IETF's MANET working group [ref.
   http://www.ietf.org/html.charters/manet-charter.html ].

   Another area of relevant network-layer research concerns "vertical
   handoffs," which allow nodes to adapt to changes in available links
   or the resources available via particular links.  This work [ref.
   http://daedalus.cs.Berkeley.edu/publications/monet97.ps.gz ] is
   focused on "last-hop" use.  Its potential for use within wireless
   routers requires further research.

   As link layers enhance their capabilities to monitor and report their
   status internally to the network layer, network layer control
   protocols and algorithms must be enhanced to appropriately propagate
   this information to affected parties within the RDI.  This topic is
   an active research area, with some applicable work having been done
   within the SCPS project [ref: http://www.scps.org ].

   Just as resource reservation schemes within the link layer may be
   important mechanisms for providing bounded link-layer delays,
   resource allocation schemes at the network layer may also be
   important for providing such bounds on end-to-end communication
   within the RDI.  However, resource allocation and resource
   reservation within mobile wireless networks is still a subject of
   ongoing research.  Should wireless networks use the integrated
   services model of resource allocation?  Or should they use the
   differentiated services model?  It is relatively clear that
   guarantees of network capacity are not feasible in a highly dynamic
   environment.  However, the specification of operating ranges may be
   more tractable, and may provide useful services
   * To applications (by giving them feedback on available network
     capacity),
   * To the transport layer (by providing dynamic rate control
     information), and
   * To the network layer (by ensuring that resources are not over
     allocated in the non-bottleneck portions of the network).

   The applicability to MANETs of the current integrated services versus
   differentiated services models is the subject of ongoing research
   [ref: http://comet.ctr.columbia.edu/insignia/,
   http://www.cs.ucla.edu/~terzis/mobile.html].

   The networks that will be deployed remotely are truly without any
   fixed infrastructure.  Therefore, the elements of this network will
   need to establish and maintain the set of services necessary to
   bootstrap the network.  Self-configuration has relevance here in the
   areas of addresses allocation and management, name-to-address
   binding, and dynamic hierarchical organization.  There is ongoing
   research in all of these areas that is of potential utility, ref:
   http://www.ietf.org/html.charters/zeroconf-charter.html

Cerf, et al.             Expires November 2001                 [Page 51]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   http://www.ietf.org/html.charters/dhc-charter.html
   http://www.ietf.org/html.charters/svrloc-charter.html
   http://www.ietf.org/html.charters/dnsind-charter.html
   http://gershwin.utdallas.edu/publications].

   At the transport layer, there is a need for development of new
   protocols or extensions to existing protocols that are capable of
   participating in power efficient and mobility-aware communication
   schemes [ref:
   http://www.isi.edu/workshop/public_html/wmcw97/nsf4.html ].

   These enhanced transport protocols must be able to adapt to changing
   network conditions.  This applies in particular to adaptation of the
   protocol's behavior in order to meet application Quality of Service
   requirements.  Additionally, support is required for explicit
   signaling of and responses to changing network conditions such as
   link outages, congestion, and corruption trends.

   For the foreseeable future, some links in the RDIs will exhibit
   significant data rate asymmetry (on the order of 100s: 1).  The
   effective asymmetries may be even higher, since the low-capacity link
   may not necessarily be dedicated to supporting the movement of data
   across the high capacity link.  Accordingly, accommodation of
   significant data rate asymmetries will be required while still
   maintaining a high degree of power efficiency in the presence of
   errors.

   At the application layer, service location in mobile ad hoc networks
   is an issue that is closely related to self-configuration at the
   network layer, and is of current research interest, ref:
   http://www.ietf.org/html.charters/zeroconf-charter.html,
   http://www.ietf.org/html.charters/svrloc-charter.html ).
   This is important both in initial network startup, and also in the
   (potentially frequent) case that RDIs with low connectivity become
   partitioned, either expectedly or unexpectedly.

   Efficient, autonomous network management and control in RDIs is an
   area of interest.  There is some ongoing research in the area, ref:
   http://www.ietf.org/html.charters/disman-charter.html,
   ftp://ftp.ece.orst.edu:/pub/users/singh/papers/anmp.ps.gz
    that is of potential interest, but this is an area in which further
   research is required.

   Finally, monitoring the health and status of mobile nodes (not
   necessarily just the networking components) is of significant
   importance in RDIs.  The cost to land new elements on a planetary
   surface is extremely high.  Therefore, we are motivated to perform
   preemptive maintenance and to repair, rather than replace, failed
   parts.  The ability to perform these activities autonomously is an
   area where a significant amount of research is required (and sounds
   really fun).


Cerf, et al.             Expires November 2001                 [Page 52]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



7.4.   Summary

   The effects of power limitations in RDIs are significant and will be
   present in at least some portions of the IPN for the foreseeable
   future.  These effects strongly suggest the likelihood of some
   divergence from the standard internet suite of protocols that is in
   current use on Earth.  Whether these changes become the standard
   internet suite on Earth, as a result of the significant increase in
   the use of mobile, wireless internet nodes, remains to be determined.
   It is also likely that some of the RDIs that are deployed will use a
   model of organization that is substantially different than that
   employed in the Internet.  It may be that the IPN treats these
   networks as single, distributed nodes, but further investigation is
   required.

8. Working Conclusions

   With the increasing pace of space exploration, Earth will distribute
   large numbers of robotic vehicles, landers, and possibly even humans,
   to asteroids and other planets in the coming decades.  Possible
   future missions include lander/rover/orbiter sets, sample return
   missions, aircraft communicating with orbiters, and outposts of
   humans or computers remotely operating rovers.  All of these missions
   involve clusters of entities in relatively close proximity
   communicating with each other in challenging environments. These
   clusters, in turn, will be in intermittent contact with one another,
   possibly across interplanetary space.  This dual-mode communications
   environment: relatively cheap round-trips and more constant
   connectivity when communicating with "local" elements coupled with
   the very long-delay and intermittent connectivity with non-local
   elements has led us to the architecture just described, with the
   following working conclusions.

We need to use a "Pony-Express" model and bundles for interplanetary
communication

   For communicating between IPN domains such as Earth and Mars, the
   standard Internet protocols fail, both at the application and
   transport layers, mainly due to the large delays. Intermittent
   connectivity and bit error rate are also significant issues, but
   delay cannot be mitigated by any current means. Communicating over
   these distances requires a fundamentally different model that does
   not assume that round-trip-times are cheap.  To combat this we
   propose a "pony-express" model, where senders submit bundles
   describing entire transactions to the network, which then uses
   custody transfers to move bundles from node to node.  Under this
   model the sender has little or no expectation of real-time delivery
   of the bundle, which may be stored at intermediate nodes for
   arbitrary periods of time.  In time, the communications links
   connecting the various deployed internets will grow to form a stable
   interplanetary backbone network.


Cerf, et al.             Expires November 2001                 [Page 53]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



We can't support most legacy applications, even with application-layer
proxies

   SMTP (e-mail) is perhaps the only application that could possibly be
   tuned to work over interplanetary distances, but even SMTP has
   embedded timers that would have to be altered significantly before it
   would work.

Name tuples consisting of a routing part and an administrative part are
the means of reference

   To assist in interplanetary routing, we have introduced the notion of
   an IPN name tuple.  These tuples, each consisting of a topology-
   related routing part and an administratively controlled
   administrative part are the analog of IP addresses in the terrestrial
   Internet.  Just as with IP addresses, bundles carry these name tuples
   (source and destination) end-to-end.  The routing part of an IPN name
   tuple serves as an identifier of the destination internet and is
   consulted to find the next bundle-hop destination whenever the bundle
   is NOT in its destination IPN region.  The administrative part of an
   IPN name tuple serves the same function as DNS names in the
   terrestrial Internet, allowing symbolic names to be used in place of
   network-layer addresses.  For the IPN region that includes Earth, and
   possibly others, we envision using actual DNS names as the
   administrative parts of the tuples.

   This two-part naming approach has a number of advantages.  First, it
   explicitly allows routing information to be encoded in the IPN name
   tuple (in the routing part) without interfering with the
   administrative part.  Second, since the administrative parts of name
   tuples are only resolved in the destination IPN region,
   administrative parts of IPN names do not have to be exchanged between
   IPN regions for the purpose of routing.  Third, while all IPN nodes
   must be able to interpret the routing part of a name tuple, different
   name-to-address binding mechanisms for the administrative part can be
   used in the different regions.  Finally, decoupling the
   administrative parts of the various IPN regions from one another
   allows autonomy of the administrative naming in different regions.

IPN Nodes will be used as "impedance-matchers" between the (relatively)
rich environment of local communications and the long-delay
interplanetary environment

   We refer to the nodes that perform the store-and-forward operations
   on bundles simply as interplanetary nodes, and the bundle-agents on
   these nodes are responsible for routing bundles through the IPN.

We need flexible and bandwidth-efficient security (particularly
authentication and access control)

   For the foreseeable future, the Interplanetary Internet will be an
   expensive resource, and an irresistible lure to hackers.  For these

Cerf, et al.             Expires November 2001                 [Page 54]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   reasons, access to the IPN will need to be strictly controlled and
   IPN gateways and resources will need to authenticate commands sent to
   them.  In many cases, data privacy will be required by scientists to
   ensure that they get first access to the data from their instruments,
   and integrity will of course be required for command sequences and
   some telemetry return.  At the same time, the goal of the IPN is to
   allow scientists, researchers, and on occasion the general public,
   easy access to information from outside of Earth's domain.  These
   competing goals will require a flexible security scheme.

   In addition to being flexible, the security mechanisms used over the
   deep-space links will also need to be bit efficient.  Standard
   Diffie-Hellman exchanges cannot be used to generate traffic keys, as
   they require the exchange of several rounds of random information in
   near-real time.  Something similar to secure email may be a better
   approach, where the sender applies the appropriate security to the
   payload of a bundle and then attaches its public key certificate to
   the bundle.  This approach, while costly in terms of bandwidth, will
   allow the receiver to immediately authenticate/verify/decode the
   contents of received bundles.

The long-haul transport protocol used to carry bundles in interplanetary
space will be very different from TCP

   The transport layer that moves these bundles across interplanetary
   space will necessarily be very different from TCP, the predominant
   transport protocol in the terrestrial Internet.  Since bandwidth is a
   precious commodity in the IPN, the LTP must be "connectionless" in
   that it cannot wait a round-trip to establish a connection before
   beginning to send data.  It is quite possible that the LTP will
   provide partial data delivery where data with "holes" or errors is
   delivered to the application out of order instead of in order, as
   with TCP.  A major challenge of the LTP is how to perform flow and
   congestion control without timely feedback.

Deployed internets may or may not use internet protocols.

   For the "local" communications, between nodes on a spacecraft LAN,
   elements on the surface of a planet, or between elements on the
   surface and in orbit, for example, round-trip times are comparable to
   those in the terrestrial Internet.  In these cases, it is feasible
   and indeed very reasonable to use protocols like the standard
   Internet Protocols (TCP/IP).  Using an internet model, and perhaps
   even versions of the standard internet protocols themselves, will
   allow deployed devices to easily communicate with one another and to
   share a common communications infrastructure.  A common, shared
   infrastructure will free each mission from having to design, build,
   test, and operate its own custom communications system, and will pay
   off in terms of development time, development cost, mass,
   interoperability (reliability), and efficiency. Thus we envision the
   deployments of fragments that are similar to Earth's Internet to
   remote locations of interest.  Current advances such as Mobile IP and

Cerf, et al.             Expires November 2001                 [Page 55]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   SCPS that extend the internet model to include mobile nodes and
   stressed environments will be useful in these environments.

   At the same time, it may be advantageous to use protocols entirely
   foreign to the internet suite in an "RDI."  For example, it might be
   preferable in some instances to deploy a self-organizing sensor
   network in which only data sets, not individual nodes, are
   addressable.  We believe that the naming method under consideration,
   where each entity name is divided into an IPN domain part and a
   domain-specific part, would support these types of RDIs.  The
   mechanics of how to interface such a network with the rest of the IPN
   and whether/how such a network could be integrated with an RDI using
   internet protocols are topics of current discussion.

Terrestrial advances in Mobile Ad Hoc networking are necessary but
possibly not sufficient for RDIs

   There is a large and growing body of ongoing work in the IETF,
   universities, and government and private agencies to develop
   protocols for mobile ad hoc networks.  While much of this work will
   be directly applicable to the remotely deployed internets, RDIs will
   remain a breed apart from terrestrial wireless networks.  The main
   differences are in power and node density.  Terrestrial MANETs, while
   they may be power-starved, still have the prospect of recharging at a
   wall socket; elements of an RDI have no such expectation.  Secondly,
   the density of elements in an RDI may be low compared to a typical
   terrestrial MANET (although it is too soon to know that a "typical"
   terrestrial MANET will look like).  The node density may be so low in
   fact that real-time communications between members of the RDI may not
   be possible.  This will be the case if many landed elements
   communicate with one another via a small number of low-altitude
   orbiters, for example.

9. Security Considerations

   Security is an integral concern of the Interplanetary Internet.
   Section 5 of this document is devoted to examining the security
   requirements in the IPN, some potential approaches for securing data
   in the IPN, and some approaches for securing the backbone
   infrastructure of the IPN.


Cerf, et al.             Expires November 2001                 [Page 56]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



10. Authors' Addresses

   Dr. Vinton G. Cerf
   MCI WorldCom
   22001 Loudoun County Parkway
   Building F2, Room 4115, ATTN: Vint Cerf
   Ashburn, VA 20147
   Telephone +1 (703) 886-1690
   FAX  +1 (703) 886-0047
   Email vcerf@mci.net

   Scott C. Burleigh
   Jet Propulsion Laboratory
   4800 Oak Grove Drive
   M/S: 179-206
   Pasadena, CA 91109-8099
   Telephone +1 (818) 393-3353
   FAX  +1 (818) 354-1075
   Email Scott.Burleigh@jpl.nasa.gov

   Adrian J.  Hooke
   Jet Propulsion Laboratory
   4800 Oak Grove Drive
   M/S: 303-400
   Pasadena, CA 91109-8099
   Telephone +1 (818) 354-3063
   FAX  +1 (818) 393-3575
   Email Adrian.Hooke@jpl.nasa.gov

   Leigh Torgerson
   Jet Propulsion Laboratory
   4800 Oak Grove Drive
   M/S: T1710-
   Pasadena, CA 91109-8099
   Telephone +1 (818) 393-0695
   FAX  +1 (818) 354-9068
   Email Leigh.Torgerson@jpl.nasa.gov

   Robert C. Durst
   The MITRE Corporation
   1820 Dolley Madison Blvd.
   M/S W650
   McLean, VA 22102
   Telephone +1 (703) 883-7535
   FAX +1 (703) 883-7142
   Email durst@mitre.org


Cerf, et al.             Expires November 2001                 [Page 57]


Internet Draft       draft-irtf-ipnrg-arch-00.txt        May 2001



   Dr. Keith L. Scott
   The MITRE Corporation
   1820 Dolley Madison Blvd.
   M/S W650
   McLean, VA 22102
   Telephone +1 (703) 883-6547
   FAX +1 (703) 883-7142
   Email kscott@mitre.org

   Eric J. Travis
   Global Science and Technology, Inc.
   6411 Ivy Lane
   Suite 300
   Greenbelt, MD  20770
   Telephone  +1 (301) 474-9696
   FAX  +1 (301) 474-5970
   Email travis@gst.com

   Howard S. Weiss
   SPARTA, Inc.
   9861 Broken Land Parkway
   Columbia, MD 21046
   Telephone +1 (410) 381-9400 x201
   FAX  +1 (410) 381-5559
   Email hsw@columbia.sparta.com


Cerf, et al.             Expires November 2001                 [Page 58]


Html markup produced by rfcmarkup 1.128, available from https://tools.ietf.org/tools/rfcmarkup/