[Docs] [txt|pdf] [Tracker] [Email] [Nits]

Versions: 00

MPLS Working Group                                          L. Dunbar
Internet Draft                                                 Huawei
Intended status: Standard Track                              S. Hares
Expires: January 2011                                          Huawei
                                                          July 2, 2010


    Scalable Address Resolution for Large Data Center Problem Statements
           draft-dunbar-arp-for-large-dc-problem-statement-00.txt


Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on January 2, 2009.

Copyright Notice

   Copyright (c) 2010 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the BSD License.




Dunbar                 Expires January 2, 2011                [Page 1]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


Abstract

   Virtual machines, or virtualized servers, basically allow one
   physical server to support multiple hosts (20, 30, or hundreds of).
   As virtual machines are introduced to Data center, the number of
   hosts within one data center can grow dramatically, which could
   create tremendous impact to networks and hosts.

   This document describes reasons why it is still desirable to have
   virtual machines in Data Center to be in one Layer 2 network and
   potential problems this type of Layer 2 network will face. The goal
   is to justify why it is necessary for IETF to create a working group
   to work on interoperable and scalable solutions for data center(s)
   with large number of virtual machines.

Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC-2119 0.

Table of Contents


   1. Introduction................................................3
   2. Terminology.................................................4
   3. Reasons for Virtual Machines in Data Center to stay in Layer 24
      3.1. Load balance requires group of hosts on same Layer 2.....4
      3.2. Redundancy requires both active and standby VM on same Layer
      2...........................................................5
      3.3. VM mobility requires them on same Layer 2...............5
   4. Cloud Computing Service......................................6
   5. Problems facing Layer 2 with large number of hosts...........7
      5.1. Address Resolution creates significant burden to hosts...8
      5.2. Large amount of MAC addresses to be learnt by intermediate
      switches....................................................9
      5.3. More chances of unknown broadcast......................10
   6. Why dividing one Layer 2 into many smaller subnets is not enough?
    ..............................................................10
   7. Why IETF needs to develop solutions instead of relying on IEEE802
    ..............................................................11
   8. Conclusion and Recommendation...............................11
   9. Manageability Considerations................................12
   10. Security Considerations....................................12
   11. IANA Considerations........................................12
   12. Acknowledgments...........................................12
   13. References................................................13


Dunbar                 Expires January 2, 2011                [Page 2]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


   Authors' Addresses............................................13
   Intellectual Property Statement................................13
   Disclaimer of Validity........................................14



1. Introduction

   Virtual machines are created by server virtualization, which allows
   the sharing of the underlying physical machine (server) resources
   among different virtual machines, each running its own operating
   system. Server virtualization is the key enabler for Cloud Computing
   services, such as Amazon's EC2 service. Virtual Machine also makes it
   possible for virtual desktop services, which allow servers in data
   center(s) to provide virtual desktops to millions of end users.

   Servers virtualization provides numerous benefits, including higher
   utilization rates, improved IT productivity, increased data security,
   reduced user downtime, and even significant power conservation, and
   the promise of a more flexible and dynamic computing environment. As
   a result, many organizations are highly motivated to incorporate
   server virtualization technologies into their data centers. In fact,
   ESG research indicates that virtualization is being widely adopted in
   production environments.

   While Servers Virtualization is a great technology for flexible
   management of server resources, it does impose great challenges to
   networks which interconnect all the servers in data center(s).

   For a typical tree structured Layer 2 network, with one or two
   aggregation switches connected to a group of Top of Rack (ToR)
   switches and each ToR switch connected to a group of physical servers
   (hosts), the number of servers connected in this network is limited
   to the switches' port count. If ToR switch has 20 downstream ports,
   there are only 20 servers or hosts connected to the ToR switch. If
   the Aggregation Switch has 256 ports connecting to ToR switch, there
   could be up to 20*256=5120 hosts connected to one aggregation switch
   when the servers are not virtualized.

   When Virtual Machines are introduced to servers, one server can
   support hundreds of VMs. Hypothetically, if one server supports up to
   100 VMs, the same ToR switches and Aggregation switch as above can
   support up to 512000 hosts. Even if there is enough bandwidth on
   links to support the traffic volume from all those VMs, other issues
   associated with Layer 2, like frequent ARP broadcast by hosts,
   unknown broadcast, are creating a lot of challenges to the network
   and hosts.


Dunbar                 Expires January 2, 2011                [Page 3]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


2. Terminology

   Bridge:  IEEE802.1Q compliant device. In this draft, Bridge is used
             interchangeably with Layer 2 switch.

   FDB:    Filtering Database for Bridge or Layer 2 switch

   ToR:    Top of Rack Switch. It is also known as access switch.

   Aggregation switch: a Layer 2 switch which connects a group of ToR
             switches. It is also known as End of Row switch in data
             center.

   VM:     Virtual Machines



3. Reasons for Virtual Machines in Data Center to stay in Layer 2

   Two application scenarios for Virtual Machines deployment are
   considered here:

      - One Data Center with large amount of Virtual Machines

      - Cloud Computing Service (Infrastructure as a Service) which
        requires a large amount of virtual hosts.

3.1. Load balance requires group of hosts on same Layer 2

   Server load balancing is a technique to distribute workload evenly
   across two or more servers, in order to get optimal resource
   utilization, minimize response time, and avoid overload. Using
   multiple servers with load balancing, instead of a single one, also
   increase reliability through redundancy. One of the most common
   applications of load balancing is to provide a single Internet
   service from multiple servers, sometimes known as a server farm.
   Commonly, load-balanced systems include popular web sites, large
   Internet Relay Chat networks, high-bandwidth File Transfer Protocol
   sites, NNTP servers and DNS servers.

   The load balancer typically sits in-line between the client and the
   hosts that provide the services the client wants to use. Some load
   balancer requires hosts to have a return route that points back to
   the load balancer so that return traffic will be processed through it
   on its way back to the client.




Dunbar                 Expires January 2, 2011                [Page 4]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


   However, for applications with relative smaller amount of traffic
   going into server and relative large amount of traffic from server,
   it is desirable to allow reply data from servers go directly to
   clients without going through the Load Balancer. In this kind of
   design, called Direct Server Return, all servers in the cluster have
   same IP addresses as the load balancer. External requests from
   clients are directly sent to the Load Balancer, which distributes the
   request to appropriate host among the cluster based on their load.
   Any of those servers send reply directly out to clients, who see the
   same IP address regardless which server handles the requests.  Under
   this design it is necessary for Load Balancer and the cluster of
   hosts to be on same Layer 2 network so that they communicate with
   each other via their MAC addresses.

3.2. Redundancy requires both active and standby VM on same Layer 2

   For redundant servers (or VMs) serving same applications, both Active
   and Standby servers (VMs) need to have keep-alive messages between
   them. Since both Active and Standby servers (VMs) might have same IP
   address, the only way to achieve this is via Layer 2 keep-alive
   because the Active and Stand-by will have different MAC Addresses.

   The VRRP (virtual router redundancy protocol) (RFC 3768) is designed
   to increase the availability of the default gateway servicing hosts
   on the same subnet. Two or more physical routers are then configured
   to stand for the virtual router, with only one doing the actual
   routing at any given time. It is necessary for the group of physical
   routers serving one virtual router to be on same subnet as their
   hosts.

3.3. VM mobility requires them on same Layer 2

   VM mobility is referring to moving virtual machines from one server
   to another. To fully realize the benefits of a virtualized server
   environment, the ability to seamlessly move VMs within a resource
   pool and still guarantee application performance and security is a
   must. Mobility adds tremendous value because it enables organizations
   to:

      - Rapidly scale out new applications - Creating golden copies of a
        VM allows organizations to introduce new applications into a
        resource pool in dramatically less time. This accelerates the
        time to market for new applications.

      - Balance workloads - The ability to dynamically adjust workloads
        within a resource pool will enable companies to optimize



Dunbar                 Expires January 2, 2011                [Page 5]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


        performance and minimize power and cooling costs during off
        hours by dynamically adjusting VM locations.

      - Deliver high levels of availability - Mobility guarantees that
        even in the event of physical infrastructure failure,
        applications can be quickly moved to another physical resource
        within that pool, dramatically minimizing downtime and
        eliminating the need for dedicated redundant infrastructure.

      - Recover from a site disaster - This involves the ability to
        quickly migrate VMs to a remote secondary location in order for
        operations to resume. Recovering from VMs significantly reduces
        the amount of time required vs. manually reloading servers from
        bare metal.

   One of the key requirements for VM mobility is the VM maintaining the
   same IP address after moving to the new location so that its
   operation can be continued in the new location.

   For a VM to maintain the same IP address when moving from Server A to
   Server B, it is necessary for both servers to be on the same subnet.
   If Server A and Server B are on different subnets, they will have
   different gateway routers. In the subnet where Server A is in, e.g.
   Subnet A, the VM sends ARP broadcast requests for target IP addresses
   in the same subnet. For the target IP address not in the Subnet A,
   the VM sends data frame to default gateway router. When this VM is
   moved to Server B, if server B is in a different subnet than Server
   A, e.g. Subnet B, then this VM couldn't even forward data to its
   default gateway and can't find MAC addresses for other hosts in
   subnet A.

   That is why most VM mobility systems, such as VMware's vMotion,
   require all hosts in one Layer 2.

4. Cloud Computing Service

   Cloud Computing service, like Amazon's Elastic Compute Cloud (Amazon
   EC2), allows users (clients) to create their own virtual servers and
   virtual subnets. There are many potential services which Cloud
   Computing services could offer to their clients. Here are just some
   examples of those services:

      - A client can request a group of virtual servers and their
        associated virtual subnet,

      - A client can specify policies among multiple subnets they
        purchased,


Dunbar                 Expires January 2, 2011                [Page 6]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


      - A client can specify preferred geographic locations for some of
        the virtual servers they purchased,

      - A client can specify redundancy criteria, like two virtual
        servers on two different physical servers or in two different
        locations, etc.

   In order to efficiently support those services, network has to
   support virtual subnet, i.e. one Layer 2 network spanning across
   multiple locations, in addition to large amount of hosts in Layer 2.

   It is possible for Cloud Computing service to have a network design
   that each virtual subnet is mapped one independent Layer 2 network.
   But this kind of design would require huge amount of administrative
   and planning work to properly partition servers, switches to
   appropriate Layer 2 network. The problem is that virtual servers and
   virtual subnets purchased by clients change all the time. It is a lot
   of administrative work to change Layer 2 network partition each time
   there is a client request. Having a large amount of virtual machines
   in one Layer 2 network can simplify some aspects of Cloud Computing
   service design and management.

5. Problems facing Layer 2 with large number of hosts

   In Layer 2 network, hosts can be attached and re-attached at any
   location on the network. Hosts use ARP (Address Resolution Protocol)
   to find the corresponding MAC address of a target host. ARP is a
   protocol that uses the Ethernet broadcast service for discovering a
   host's MAC address from its IP address. For host A to find the MAC
   address of a host B on the same subnet with IP Address B-IP, host A
   broadcasts an ARP query packet containing B  as well as its own IP                                                  IP
   address (A-IP) on its Ethernet interface. All hosts in the same
   subnet receive the packet. Host B, whose IP address is B-IP, replies
   (via unicast) to inform A of its MAC address. A will also record the
   mapping between B-IP and B-MAC.

   Even though all hosts maintain the MAC to target IP address mapping
   locally to avoid repetitive ARP broadcast message for the same target
   IP address, all hosts age out their learnt MAC to IP mapping very
   frequently. For Microsoft Windows (versions XP and server 2003), the
   default ARP cache policy is to discard entries that have not been
   used in at least two minutes, and for cache entries that are in use,
   to retransmit an ARP request every 10 minutes. So hosts send out ARP
   very frequently.

   In addition to broadcast messages sent from hosts, Layer 2 switches
   also broadcast received packet if the destination address is unknown.


Dunbar                 Expires January 2, 2011                [Page 7]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


   All Layer 2 switches learn MAC address of data frames which traverse
   through the switches. Layer 2 switches also age out their learnt MAC
   addresses in order to limit the number of entries in their Filtering
   Database (FDB). When a switch receives packet with an unknown MAC
   address, it broadcast this packet to all ports which are enabled for
   the corresponding VLAN.

   The flooding and broadcast have worked well in the past when the
   Layer 2 network is limited to a smaller size. Most Layer 2 networks
   limit the number of hosts to be less then 200, so that broadcast
   storm and flooding can be kept in a smaller domain.

5.1. Address Resolution creates significant burden to hosts

   When a Layer 2 network has tens of thousands of hosts, the frequent
   ARP broadcast messages from all those hosts clearly present a
   significant burden, especially to hosts. Many of today's layer 2
   switches, even with hundreds of ports, can forward 10G traffic at
   line rate. But they don't need to process ARP requests, they just
   forward them. It is the host who needs to process every ARP message
   that circulates in the network.

   [Scaling Ethernet] of Carnegie Mellon did a study on the number of
   ARP queries received at a workstation on CMU's School of Computer
   Science LAN over a 12 hour period on August 9, 2004. At peak, the
   host received 1150 ARPs per second, and on average, the host received
   89 ARPs per second. During the data collection, 2,456 hosts were
   observed sending ARP queries. [Scaling Ethernet] expects that the
   amount of ARP traffic will scale linearly with the number of hosts on
   the LAN. For 1 million hosts, it is expected to have 468,240 ARPs per
   second or 239 Mbps of ARP traffic to arrive at each host at peak,
   which is more than enough to overwhelm a standard 100 Mbps LAN
   connection. Ignoring the link capacity, forcing hosts to handle an
   extra half million packets per second to inspect each ARP packet
   would impose a prohibitive computational burden.

   To detect address conflict and refresh hosts address in a Layer 2
   network, many types of hosts and almost all Virtual Machines send out
   gratuitous ARP on regular basis. The Gratuitous ARP could mean either
   gratuitous ARP request or gratuitous ARP reply. Gratuitous in this
   case means a request/reply that is not normally needed according to
   the ARP specification (RFC 826) but could be used in some cases. A
   gratuitous ARP request is an Address Resolution Protocol request
   packet where the source and destination IP are both set to the IP of
   the machine issuing the packet and the destination MAC is the
   broadcast address ff:ff:ff:ff:ff:ff. Ordinarily, no reply packet will



Dunbar                 Expires January 2, 2011                [Page 8]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


   occur. A gratuitous ARP reply is a reply to which no request has been
   made.

   All the Gratuitous ARP messages also need to be processed by all
   hosts in the Layer 2 domain.

   Handling up to 1000~2000 ARP requests per second is almost the high
   limit for any hosts. With more than 20K hosts in one Layer 2 domain,
   the amount of ARP broadcast messages, plus other broadcast messages
   such as DHCP, can create too much burden to be handled by hosts.

5.2. Large amount of MAC addresses to be learnt by intermediate switches

   Ethernet's non-hierarchical flat layer 2 MAC addressing makes it not
   possible for any types for address summarization.  MAC addresses,
   plus their VLAN IDs, have to be placed in switch's FDB without any
   abbreviation, not like IP addresses which only need proper prefix to
   be stored in router's forwarding table.

   One advantage of Ethernet switches is that it can forward much more
   addresses than its FDB entries. When a data frame's destination
   address is not present in a switch's FDB entry, the switch just flood
   this data frame to all ports which are enabled for the corresponding
   VLAN. That is why Ethernet switches can have a relative small FDB
   size, which is one of the key reasons that Ethernet switches can be
   built at much lower cost than routers. To improve efficiency of the
   FDB, Ethernet switches frequently age out learnt MAC addresses which
   haven't been in use for a while and always replace older MAC entries
   with newly learnt MACs when the FDB is full.

   When servers in data center are virtualized, each server can host
   tens or hundreds of virtual machines. Each virtual machine can be a
   host to an application, which has its own IP address and MAC address.
   With the same type and number of network equipments, i.e. ToR
   switches and Aggregation switches, the number of hosts can grow
   dramatically in this network. When the number of hosts grows, the
   number of MAC addresses to be learnt by Layer 2 switches grows too.
   For an example of tree shaped Layer 2 network with one core switch
   connected to 3 aggregation switches, each aggregation switch
   connected to 25 ToR switches, and each ToR switch connected to 25
   physical servers, if each server supports 50 virtual machines, there
   will be 50*25*25*3= 93750 hosts in this network.

   Typical bridges support in the range of 16 to 32K MAC Addresses, with
   some supporting 64K. With external memory (TCAM), bridges can support
   up to 512K to 1M MAC addresses. But TCAM is expensive, which will
   defeat the low cost advantage of Layer 2 switches. This problem is


Dunbar                 Expires January 2, 2011                [Page 9]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


   especially severe for Top of Rack switches, which are supposed to be
   very low cost.

   In summary, the low cost ToR switches usually don't have enough FDB
   entries for all VM's MAC addresses in the domain.

5.3. More chances of unknown broadcast

   When the number of hosts, MAC addresses, are above the switches FDB
   size, learnt MAC addresses in the FDB are aged out faster, which will
   increase the chances of switch's FDB not having the entry for the
   received packet's destination address, which then causes the packet
   being flooded.

   When the spanning tree topology changes (e.g. when a link fails), a
   bridge clears its cached station location information because a
   topology change could lead to a change in the spanning tree, and
   packets for a given source may arrive on a different port on the
   bridge. As a result, during periods of network convergence, network
   capacity drops significantly as the bridges fall back to flooding for
   all hosts.

6. Why dividing one Layer 2 into many smaller subnets is not enough?

   Subnet (VLAN) can partition one Layer 2 network into many virtual
   Layer 2 domains. All the broadcast messages are confined within one
   subnet (VLAN). Subnet (VLAN) has worked well when each server serving
   one single host. The server will not receive broadcast messages from
   hosts in other subnets (VLANs).

   When one physical server is supporting 100 plus Virtual Machines,
   i.e. 100 plus hosts, most likely the virtual hosts on one server are
   on different subnets (VLANs). If there are 50 subnets (VLANs) enabled
   on the switch port to the server, the server has to handle all the
   ARP broadcast messages on all 50 subnets (VLANs). When virtual hosts
   are added or deleted from a server, the switch port to the server may
   end up enabling more VLANs than the number of subnets actually active
   on the server.  Therefore, the amount of ARP messages to be processed
   by each server is still too much.

   For Cloud Computing Service, the number of virtual hosts and virtual
   subnets can be very high. It might not be possible to limit the
   number of virtual hosts in each subnet.






Dunbar                 Expires January 2, 2011               [Page 10]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


7. Why IETF needs to develop solutions instead of relying on IEEE802

   Here are the reasons that IETF need to develop solutions:

      - Client of Cloud Computing services may request redundancy across
        two geographical locations. They may want two VMs in one Virtual
        Subnet to be in two locations -> Most likely it is the IP/MPLS
        networks which interconnect multiple locations

      - The two redundant VMs may have same IP address with one being
        Active and other one being Standby. The Active and Stand-by need
        to have keep-alive messages between them. The only way to
        achieve this is via Layer 2 keep-alive because the Active and
        Stand-by will have different MAC Addresses -> Require the two
        VMs on same Layer 2

      - It is desirable for all hosts (VMs) of one Virtual Subnet to be
        in one Layer 2 network for efficient multicast and broadcast
        among them.

        -Client may request hosts (VMs) in one Virtual Subnet to be on
         different locations to have faster response for their
         applications. -> Require IP/MPLS to interconnect

        -Hosts can be added to one Virtual Subnet at different time. It
         is possible that newly added hosts have to be placed at a
         different site due to computing & storage resource availability
         -> Require IP/MPLS to interconnect.

8. Conclusion and Recommendation

    When there are tens of thousands of VMs in one Data Center, we have
    concluded that:

      - It is necessary to restrain the ARP storm and broadcast storm
         initiated by (unpredictable) servers and applications into a
         confined domain.

      - It is necessary to have a way to restrain Layer 2 switches from
         learning tens of thousands of MAC addresses.

      - It is necessary for reduce the amount of un-known addresses
         arriving at any Layer 2 switches to prevent large amount of un-
         known broadcast in one Layer 2.

    For Cloud services which offer virtual hosts and virtual subnets, we
    have concluded that:


Dunbar                 Expires January 2, 2011               [Page 11]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


      - It is necessary to have a more scalable address resolution
         protocol for Layer 2 Network which spans across multiple
         locations.

      - It is desirable to constrain MAC addresses in each site from
         being learnt by other sites. This is to allow traditional Layer
         2 switches, which have limited amount of address space for
         forwarding table, to function properly and to minimize unknown
         broadcast by those switches.

    Therefore, we recommend IETF to create a working group:

      - To develop scalable address resolution protocols for data
         center with large amount of hosts and Layer 2 spanning across
         multiple locations,

      - To develop mechanism to scope the broadcast messages, beyond
         ARP and DHCP, to minimize impact to each layer 2 domain by
         broadcast storms from other layer 2 domains,

      - To have a scalable inter Layer 2 domain protocol, like BGP, for
         each domain's gateways to exchange hosts' reachability
         information among each other, and

      - To identify mechanisms for proper handling of multicast
         messages among hosts in one Virtual Subnet which spans across
         multiple locations.

9. Manageability Considerations

   This document does not add additional manageability considerations.

10. Security Considerations

   This document has no additional requirement for a change to the
   security models of MPLS-Ping and MPLS-Ping-Enhanced.

11. IANA Considerations

   A future revision of this document will present requests to IANA for
   codepoint allocation.



12. Acknowledgments

   This document was prepared using 2-Word-v2.0.template.dot.


Dunbar                 Expires January 2, 2011               [Page 12]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


13. References

   [ARP]   D.C. Plummer, "An Ethernet address resolution protocol."
             RFC826, Nov 1982.

   [Microsoft Window] "Microsoft Windows Server 2003 TCP/IP
             implementation details."
             http://www.microsoft.com/technet/prodtechnol/windowsserver2
             003/technologies/networking/tcpip03.mspx, June 2003.

   [Scaling Ethernet] Myers, et. al., " Rethinking the Service Model:
             Scaling Ethernet to a Million Nodes", Carnegie Mellon
             University and Rice University








Authors' Addresses

   Linda Dunbar
   Huawei Technologies
   1700 Alma Drive, Suite 500
   Plano, TX 75075, USA
   Phone: (972) 543 5849
   Email: ldunbar@huawei.com


   Sue Hares
   Huawei Technologies
   2330 Central Expressway,
   Santa Clara, CA 95050, USA
   Phone:
   Email: shares@huawei.com


Intellectual Property Statement

   The IETF Trust takes no position regarding the validity or scope of
   any Intellectual Property Rights or other rights that might be
   claimed to pertain to the implementation or use of the technology
   described in any IETF Document or the extent to which any license
   under such rights might or might not be available; nor does it



Dunbar                 Expires January 2, 2011               [Page 13]


Internet-Draft   ARP for Large DC Problem Statement          July 2010


   represent that it has made any independent effort to identify any
   such rights.

   Copies of Intellectual Property disclosures made to the IETF
   Secretariat and any assurances of licenses to be made available, or
   the result of an attempt made to obtain a general license or
   permission for the use of such proprietary rights by implementers or
   users of this specification can be obtained from the IETF on-line IPR
   repository at http://www.ietf.org/ipr

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   any standard or specification contained in an IETF Document. Please
   address the information to the IETF at ietf-ipr@ietf.org.

Disclaimer of Validity

   All IETF Documents and the information contained therein are provided
   on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
   REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE
   IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL
   WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY
   WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE
   ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
   FOR A PARTICULAR PURPOSE.

Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

















Dunbar                 Expires January 2, 2011               [Page 14]


Html markup produced by rfcmarkup 1.129d, available from https://tools.ietf.org/tools/rfcmarkup/