[Docs] [txt|pdf] [Tracker] [WG] [Email] [Nits]

Versions: 00 01 02 03 04 05 06 07 08 09 10 11 12 RFC 7094

INTERNET-DRAFT                               Danny McPherson
                                              Arbor Networks
                                                   Dave Oran
                                               Cisco Systems
Expires: August 2010                        February 9, 2010
Intended Status: Informational

               Architectural Considerations of IP Anycast
              <draft-iab-anycast-arch-implications-00.txt>



Status of this Memo


   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Copyright (c) 2010 IETF Trust and the persons identified as the document
   authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal Provisions
   Relating to IETF Documents (http://trustee.ietf.org/license-info)
   in effect on the date of publication of this document.  Please
   review these documents carefully, as they describe your rights and
   restrictions with respect to this document.  Code Components
   extracted from this document must include Simplified BSD License
   text as described in Section 4.e of the Trust Legal Provisions and
   are provided without warranty as described in the Simplified BSD
   License.

   Internet-Drafts are working documents of the Internet Engineering Task
   Force (IETF), its areas, and its working groups. Note that other groups
   may also distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference material
   or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at



McPherson, Huston, Kolkman                                      [Page 1]


INTERNET-DRAFT            Expires: August 2010             February 2010


   http://www.ietf.org/shadow.html



Copyright Notice

   Copyright (C) (2010) The IETF Trust and the persons identified as the
   document authors.  All rights reserved.

                                Abstract


   This memo discusses architectural implications of IP anycast.






































McPherson, Huston, Kolkman                                      [Page 2]


INTERNET-DRAFT            Expires: August 2010             February 2010



Table of Contents


   1. Specification of Requirements. . . . . . . . . . . . . . . . .   4
   2. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . .   4
   3. Anycast History. . . . . . . . . . . . . . . . . . . . . . . .   4
   4. Use of Anycast in RFCs . . . . . . . . . . . . . . . . . . . .   5
   5. Anycast in IPv6. . . . . . . . . . . . . . . . . . . . . . . .   6
   6. DNS Anycast. . . . . . . . . . . . . . . . . . . . . . . . . .   7
   7. BCP 126 Revisited. . . . . . . . . . . . . . . . . . . . . . .   7
   8. Layering and Resiliency. . . . . . . . . . . . . . . . . . . .   8
   9. Anycast Addresses as Destinations. . . . . . . . . . . . . . .   8
   10. Anycast Addresses as Sources. . . . . . . . . . . . . . . . .   9
   11. Regarding Widespread Anycast Use. . . . . . . . . . . . . . .   9
   12. Service Discovery . . . . . . . . . . . . . . . . . . . . . .   9
   13. Middleboxes and Anycast . . . . . . . . . . . . . . . . . . .  10
   14. Transport Implications. . . . . . . . . . . . . . . . . . . .  10
   15. Security Considerations . . . . . . . . . . . . . . . . . . .  11
   16. Deployment Considerations . . . . . . . . . . . . . . . . . .  11
   17. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  12
   18. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  13
   19. References. . . . . . . . . . . . . . . . . . . . . . . . . .  14
    19.1. Normative References . . . . . . . . . . . . . . . . . . .  14
    19.2. Informative References . . . . . . . . . . . . . . . . . .  14
   20. Authors' Addresses. . . . . . . . . . . . . . . . . . . . . .  16

























McPherson, Huston, Kolkman                                      [Page 3]


INTERNET-DRAFT            Expires: August 2010             February 2010


1.  Specification of Requirements


   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC 2119].



2.  Overview


   IP anycast is used for at least one critical Internet service, that
   of the Domain Name System [RFC 1035] root servers.  As of early 2009,
   at least 10 of the 13 root name servers were using IP anycast [RSSAC
   29].  Use of IP anycast is growing for other applications as well.
   It has been deployed for over a decade for DNS resolution services
   and is currently used by several DNS Top Level Domain (TLD)
   operators.  IP anycast is also used for other services in operational
   environments, to include Network Time Protocol (NTP) [RFC 1305].

   Anycast addresses are syntactically indistinguishable from unicast
   addresses.  Allocation of anycast addresses typically follows a model
   similar to that of unicast allocation policies.  Anycast addressing
   is largely equivalent to that of unicast in multiple locations, and
   leverages unicast destination routing to deliver a packet to either
   zero or one interface among the interfaces asserting the address.
   The expectation of delivery is to the "closest" instance as
   determined by unicast routing topology metric(s).  There is also an
   expectation of load-balancing that exists among equal cost routes.

   Unlike IP unicast, it is not considered an error to assert the same
   anycast address on multiple interfaces within the same or multiple
   systems.

   Some consider anycast a "deceptively simple idea".  That is, many
   pitfalls and subtleties exist with application and transport, as well
   as for routing configuration and operation, when IP anycast is
   employed.  In this document, we aim to capture many of the
   architectural implications of IP anycast.



3.  Anycast History


   As of this writing, the term "anycast" appears in 126 RFCs, and ~360
   Internet-Drafts (since 2006).



McPherson, Huston, Kolkman                          Section 3.  [Page 4]


INTERNET-DRAFT            Expires: August 2010             February 2010


   The first formal specification of anycast was provided in "Host
   Anycasting Service" [RFC 1546].  The authors of this document did a
   good job of capturing most of the issues that exist with IP anycast
   today.

   One of the first documented uses of anycast was in 1994 for a "Video
   Registry" experiment [IMR 9401].  In the experiment, a UDP query was
   transmitted to an anycast address to locate a server, and TCP was
   used by the client to query the server, and then multicast was used
   to distribute the server database.  There is also discussion that
   ISPs began using anycast for DNS resolution services around the same
   time, although no public references to support this are available.

   In 1997 the IAB clarified that IPv4 anycast addresses were pure
   "locators", and could never serve as an "identifier" (of a host, an
   interface, or anything else) [RFC 2101].

   In 1998 the IAB conducted a routing workshop [RFC 2902].  Of the
   conclusions and output action items from the report, an Anycast
   section is contained in S 2.10.3.  Specifically called out in the
   conclusions section is the need to describe the advantages and
   disadvantages of anycast, and the belief that local-scoped well-known
   anycast addresses will be useful to some applications.  In the
   subsequent section, an action item was outlined that suggested a BOF
   should be held to plan work on progress, and if a working group
   forms, a paper on the advantages and the disadvantages of anycast
   should be included as part of the charter.



4.  Use of Anycast in RFCs


   SNTPv4 [RFC 2030] defined how to use anycast for server discovery.
   This was extended in [RFC 4330] as an NTP-specific "manycast"
   service, in which anycast was used for the discovery part.

   IPv6 defined some reserved subnet anycast addresses [RFC 2526] and
   assigned one to "Mobile IPv6 Home-Agents" [RFC 3775].

   The original IPv6 transition mechanism [RFC 2893] made use of IPv4
   anycast addresses as tunnel endpoints for IPv6 encapsulated in IPv4,
   but this was later removed [RFC 4213].  Carpenter's Relay Router [RFC
   3056] scheme was augmented by a 6to4 relay anycast prefix [RFC 3068]
   aiming to simplify the configuration of 6to4 routers.  Incidentally,
   6to4 deployment has shown a fair number of operational and security
   issues [RFC 3964] that result from using anycast as a discovery
   mechanism.  Specifically, one inference is that operational



McPherson, Huston, Kolkman                          Section 4.  [Page 5]


INTERNET-DRAFT            Expires: August 2010             February 2010


   consideration is needed to ensure that anycast addresses get
   advertised and/or filtered in a way that produces intended scope
   (e.g., only advertise a route for your 6to4 relay to ASes that
   conform to your own acceptable usage policy), an attribute that can
   easily become quite resource (e.g., manpower) intensive.

   DNS use of anycast was first specified in "Distributing Authoritative
   Name Servers via Shared Unicast Addresses" [RFC 3258].  Is is
   noteable that it used the term "shared unicast address" rather than
   "anycast address" for the service.

   Anycast was used for routing to rendezvous points (RPs) for MSDP and
   PIM [RFC 4610].

   "Operation of Anycast Services" [RFC 4786] deals with how the routing
   system interacts with anycast services, and the operation of anycast
   services.

   "Requirements for a Mechanism Identifying a Name Server Instance"
   [RFC 4892] cites the use of anycast with DNS as a motivation to
   identify individual nameserver instances, and the NSID option was
   defined for this purpose [RFC 5001].

   "Reflections on Internet Transparency" [RFC 4924] briefly mentions
   how violating transparency can also damage global services that use
   anycast.



5.  Anycast in IPv6


   Originally, the IPv6 addressing architecture [RFC 1884] [RFC 2373]
   [RFC 3513] severly restricted the use of anycast addresses.  In
   particular, they provided that anycast addresses MUST NOT be used as
   a source address, and MUST NOT be assigned to an IPv6 host (i.e.,
   only routers).  These restrictions were later lifted in 2006 [RFC
   4291].

   In fact, the recent "IPv6 Transition/Co-existence Security
   Considerations" [RFC 4942] overview now recommends:

     "To avoid exposing knowledge about the internal structure of
     the network, it is recommended that anycast servers now take
     advantage of the ability to return responses with the anycast
     address as the source address if possible."





McPherson, Huston, Kolkman                          Section 5.  [Page 6]


INTERNET-DRAFT            Expires: August 2010             February 2010


6.  DNS Anycast


   "Distributed Authoritative Name Servers via Shared Unicast Addresses"
   [RFC 3258] described how to reach authoritative name servers using
   anycast.  It made some interesting points:

     o it asserted (as an advantage) that no routing changes were needed

     o it recommended stopping DNS processes, rather than withdrawing
       routes, to deal with fail-over.

     o it argued that failure modes involving state were not serious,
       because:

       - the vast majority of DNS queries are UDP
       - large routing metric disparity among authoritative server
         instances would localize queries to a single instance for
         most clients
       - when the resolver tries TCP and it breaks, the resolver
         will move to a different server instance (where presumably
         it doesn't break)




7.  BCP 126 Revisited


   "Operation of Anycast Services" (BCP 126) [RFC 4786] was a product of
   the IETF's GROW working group.  The primary design constraint
   considered was that routing "be stable" for significantly longer than
   a "transaction time", where "transaction time" is loosely defined as
   "a single interaction between a single client and a single server".
   It takes no position on what applications are suitable candidates for
   anycast usage.

   Furthermore, it views anycast service disruptions as an operational
   problem, "Operators should be aware that, especially for long running
   flows, there are potential failure modes using anycast that are more
   complex than a simple 'destination unreachable' failure using
   unicast."

   The document primary deals with global Internet-wide services
   provided by anycast.  Where internal topology issues are discussed
   they're mostly regarding routing implications, not application design
   implications.  BCP 126 also views networks employing per-packet load
   balancing on equal cost paths as "pathological".



McPherson, Huston, Kolkman                          Section 7.  [Page 7]


INTERNET-DRAFT            Expires: August 2010             February 2010


8.  Layering and Resiliency


   Preserving the integrity of a modular layered design for IP protocols
   on the Internet is critical to its continued success and flexibility.
   One such consideration is that of whether an application should have
   to adapt to changes in the routing system.

   Higher layer protocols should make minimal assumptions about lower
   layer protocols.  E.g., applications should make minimal assumptions
   about routing stability, just as they should make minimal assumptions
   about congestion and packet loss.  When designing applications, it
   would perhaps be safe to assume that the routing system may deliver
   each packet to a different service instance, in any pattern, with
   termporal re-ordering being a not-so-rare phenomenon.

   Stateful transport protocols (TCP, DCCP, SCTP), without modification,
   do not understand the properties of anycast and hence will fail
   probabilistically, but possibly catastrophically, when using anycast
   addresses in the presence of "normal" routing dynamics.



9.  Anycast Addresses as Destinations


   Anycast addresses are "safe" to use as destination addresses for an
   application if:

     o A request message or "one shot" message is self-contained in a
       single transport packet

     o A stateless transport (e.g., UDP) is used for the above

     o Replies are always sent to a unicast address; these can be
       multi-packet since the unicast destination is "stable"

       * Note: this constrains the use of anycast as source addresses
         in request messages, since reply messages sent back to that
         address may reach a device that was not the source that
         initially triggered it.

     o The server side of the application keeps no hard state across
       requests

     o Retries are idempotent; in addition to not assuming server state,
       they do not encode any assumptions about loss of requests versus
       loss of replies.



McPherson, Huston, Kolkman                          Section 9.  [Page 8]


INTERNET-DRAFT            Expires: August 2010             February 2010


10.  Anycast Addresses as Sources


   Anycast addresses are "safe" to use as source addresses for an
   application if:

     o No reflexive (response) message is generated by the receiver
       with the anycast source used as a destination

       * unless the application has some private state synchronization
       that allows for the reflexive message arriving at a different
       instance

     o The source anycast address is a bona fide interface address if
       reverse path forwarding (RPF) checking is on, or a service
       address explicitly provisioned to bypass RPF



11.  Regarding Widespread Anycast Use


   Widespread use of anycast for global Internet-wide services or inter-
   domain services has some scaling challenges.  Similar in ways to
   multicast, each service generates at least one unique route in the
   global BGP routing system.  As a result, additional anycast instances
   result in additional paths for a given prefix, which scales super-
   linearly as a function of denseness of inter-domain interconnection
   within the routing system (i.e., more paths result in more resources,
   more network interconnections result in more paths)..



12.  Service Discovery


   Applications able to tolerate an extra round trip time (RTT) to learn
   a unicast destination address for multi-packet exchanges can safely
   use anycast destination addresses for service instance discovery.

     o "Instance discovery" message sent to anycast destination address

     o Reply sent from unicast source address of the interface that
       received the discovery message OR reply sent from anycast source
       address of the interface that received the message, containing
       the unicast address to be used to invoke the service - this will
       avoid potential NAT binding issues for the relay packet.




McPherson, Huston, Kolkman                         Section 12.  [Page 9]


INTERNET-DRAFT            Expires: August 2010             February 2010


     o Subsequent exchanges use the unicast address




13.  Middleboxes and Anycast


   Middleboxes (e.g., NATs, firewalls) may cause problems when used in
   conjunction with anycast.  In particular, a switch from anycast to
   unicast may require a new binding, and this may not exist in the
   middlebox.




14.  Transport Implications


   UDP is the "lingua franca" for anycast today.  Stateful transports
   could be enhanced to be more anycast friendly.  It seems as though
   this was anticipated in Host Anycasting Services [RFC 1546],
   specifically:

     "The solution to this problem is to only permit anycast addresses
      as the remote address of a TCP SYN segment (without the ACK bit
      set).  A TCP can then initiate a connection to an anycast address.
      When the SYN-ACK is sent back by the host that received the
   anycast
      segment, the initiating TCP should replace the anycast address of
      its peer, with the address of the host returning the SYN-ACK.
   (The
      initiating TCP can recognize the connection for which the SYN-ACK
      is destined by treating the anycast address as a wildcard address,
      which matches any incoming SYN-ACK segment with the correct
      destination port and address and source port, provided the SYN-
   ACK's
      full address, including source address, does not match another
      connection and the sequence numbers in the SYN-ACK are correct.)
      This approach ensures that a TCP, after receiving the SYN-ACK is
      always communicating with only one host."

   Multi-address transports (e.g., SCTP) might be more amenable to such
   extensions than TCP.

   Some similarities exist between what is needed for anycast and what
   is needed for address discovery when doing multi-homing in the
   transport layer.  **NEED TO EXPAND ON THIS***



McPherson, Huston, Kolkman                        Section 14.  [Page 10]


INTERNET-DRAFT            Expires: August 2010             February 2010


15.  Security Considerations


   Anycast is often employed to mitigate or at least localize the
   effects of distributed denial of service (DDOS) attacks.  For
   example, with the Netgear NTP fiasco [RFC 4085] anycast was used in a
   distributed sinkhole model to mitigate the effects of embedded
   globally-routed Internet addresses in network elements.

   "Internet Denial-of-Service Considerations" [RFC 4732] notes that "A
   number of the root nameservers have since been replicated using
   anycast to further improve their resistance to DoS".

   "Operation of Anycast Services" [RFC 4786] cites DoS mitigation,
   constraining DoS to localized regions, and identifying attack sources
   using spoofed addresses as some motivations to deploy services using
   anycast.  Multiple anycast service instances such as those used by
   the root name servers also add resiliency when network partitioning
   occurs (e.g., as the result of transoceanic fiber cuts or natural
   disasters).

   It should be noted that there is a significant man in the middle
   (MITM) exposure in either variant of anycast discovery (see Section
   12: Service Discovery) so the need for server authentication should
   be considered.

   Furthermore, as provided in Section 4, operational consideration
   needs to be given to ensure that anycast addresses get advertised
   and/or filtered in a way that produces intended scope (for example,
   only advertise a route to your 6to4 relay to ASes that conform to
   your own AUP). This seems to be manpower intensive, and is often
   vulnerable to errors outside of the local routing domain.

   Additional security considerations are scattered throughout the list
   of references provided herein.



16.  Deployment Considerations


   This document covers issues associated with the architectural
   implications of anycast.  Operators should heed these considerations
   when evaluating the use of anycast in their specific environments.







McPherson, Huston, Kolkman                        Section 16.  [Page 11]


INTERNET-DRAFT            Expires: August 2010             February 2010


17.  IANA Considerations


   No IANA actions are required.















































McPherson, Huston, Kolkman                        Section 17.  [Page 12]


INTERNET-DRAFT            Expires: August 2010             February 2010


18.  Acknowledgments

   Many thanks for Dave Thaler and Kurtis Lindqvist for their early
   review and feedback on this document.  Brian Carpenter, ....

   Your name could be here....













































McPherson, Huston, Kolkman                        Section 18.  [Page 13]


INTERNET-DRAFT            Expires: August 2010             February 2010


19.  References




19.1.  Normative References





19.2.  Informative References


   [IMR 9401] "INTERNET MONTHLY REPORT", January 1994,
    http://mirror.facebook.com/rfc/museum/imr/imr9401.txt

   [RSSAC 29] "RSSAC 29 Meeting Minutes", December 2, 2007,
    http://www.rssac.org/meetings/04-08/rssac29.pdf

   [RFC 1035] Mockapetris, P., "DOMAIN NAMES - IMPLEMENTATION
              AND SPECIFICATION", RFC 1035, November 1987.

   [RFC 1305] Mills, D., "Network Time Protocol (Version 3)
              Specification, Implementation and Analysis", RFC
              1305, March 1992.

   [RFC 1546] Partridge, C., Mendez, T., Milliken, W., "Host
              Anycasting Service", RFC 1546, November 1993.

   [RFC 1884] Hinden, R., Deering, S., "IP Version 6 Addressing
              Architecture", RFC 1884, December 1995.

   [RFC 2030] Mills, D., "Simple Network Time Protocol (SNTP)
              Version 4 for IPv4, IPv6 and OSI", RFC 2030,
              October 1996.

   [RFC 2101] Carpenter, B., Crowcroft, J., Rekhter, Y., "IPv4
              Address Behaviour Today", RFC 2101, February 1997.

   [RFC 2119] Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", RFC 2119, March 1997.

   [RFC 2373] Hinden, R., Deering, S., "IP Version 6 Addressing
              Architecture", RFC 2373, July 1998.

   [RFC 2526] Johnson, D., Deering, S., "Reserved IPv6 Subnet
              Anycast Addresses", RFC 2526, March 1999.



McPherson, Huston, Kolkman                      Section 19.2.  [Page 14]


INTERNET-DRAFT            Expires: August 2010             February 2010


   [RFC 2893] Gilligan, R., Nordmark, E., "Transition Mechanisms
              for IPv6 Hosts and Routers", RFC 2893, August 2000.

   [RFC 2902] Deering, S., Hares, S., Perkins, C., Perlman, R.,
              "Overview of the 1998 IAB Routing Workshop", RFC
              2902, August 2000.

   [RFC 3056] Carpenter, B., "Connection of IPv6 Domains via IPv4
              Clouds", RFC 3056, February 2001.

   [RFC 3068] Huitema, C., "An Anycast Prefix for 6to4 Relay
              Routers", RFC 3068, June 2001.

   [RFC 3258] Hardie, R., "Distributing Authoritative Name Servers
              via Shared Unicast Addresses", RFC 3258, April 2002.

   [RFC 3513] Hinden, R., Deering, S., "Internet Protocol Version
              6 (IPv6) Addressing Architecture", RFC 3513, April
              2003.

   [RFC 3775] Johnson, D., Perkins, C., Arkko, J., "Mobility
              Support in IPv6", RFC 3775, June 2004.

   [RFC 3964] Savola, P., "Security Considerations for 6to4", RFC
              3964, December 2004.

   [RFC 4085] Plonka, D., "Embedding Globally-Routable Internet
              Addresses Considered Harmful", RFC 4085, June 2005.

   [RFC 4213] Normark, E., Gilligan, R., "Basic Transition
              Mechanisms for IPv6 Hosts and Routers", RFC 4213,
              October 2005.

   [RFC 4291] Hinden, R., Deering, S., "IP Version 6 Addressing
              Architecture", RFC 4291, February 2006.

   [RFC 4330] Mills, D., "Simple Network Time Protocol (SNTP)
              Version 4 for IPv4, IPv6 and OSI", RFC 4330,
              January 2006.

   [RFC 4610] Farinacci, D., Cai, Y., "Anycast-RP Using Protocol
              Independent Multicast (PIM)", RFC 4610, August 2006.

   [RFC 4732] Handley, M., Rescorla, E., IAB, "Internet Denial-of-
              Service Considerations", RFC 4732, November 2006.

   [RFC 4786] Abley, J., Lindqvist, K., "Operation of Anycast
              Services", RFC 4786, December 2006.



McPherson, Huston, Kolkman                      Section 19.2.  [Page 15]


INTERNET-DRAFT            Expires: August 2010             February 2010


   [RFC 4892] Woolf, S., Conrad, D., "Requirements for a Mechanism
              Identifying a Name Server Instance", RFC 4892, June
              2007.

   [RFC 4924] Aboba, B., Davies, E., " Reflections on Internet
              Transparency", RFC 4924, July 2007.

   [RFC 4942] Davies, E., Krishnan, S., Savola, P., "IPv6
              Transition/Coexistence Security Considerations",
              RFC 4942, September 2007.

   [RFC 5001] Austein, R., "DNS Name Server Identifier (NSID)
              Option", RFC 5001, August 2007.











20.  Authors' Addresses



   Danny McPherson
   Arbor Networks, Inc.
   Email:  danny@arbor.net

   Dave Oran
   Cisco Systems
   Email: oran@cisco.com




Copyright Statement

   Copyright (C) (2010) The IETF Trust and the persons
   identified as the document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents



McPherson, Huston, Kolkman                        Section 20.  [Page 16]


INTERNET-DRAFT            Expires: August 2010             February 2010


   carefully, as they describe your rights and restrictions with
   respect to this document.

















































McPherson, Huston, Kolkman                        Section 20.  [Page 17]


Html markup produced by rfcmarkup 1.129d, available from https://tools.ietf.org/tools/rfcmarkup/