[Docs] [txt|pdf|xml] [Tracker] [WG] [Email] [Nits]

Versions: 00

P2P Research Group                                       V. Gurbani, Ed.
Internet-Draft                                 Bell Labs, Alcatel-Lucent
Intended status: Informational                                   A. Basu
Expires: December 30, 2011            Tokai University and University of
                                                                  Sussex
                                                              T. Schmidt
                                                             HAW Hamburg
                                                              S. Fleming
                                                    University of Sussex
                                                              M. Kolberg
                                                  University of Stirling
                                                            M. Waehlisch
                                                    link-lab & FU Berlin
                                                           June 28, 2011


              Peer-to-peer simulation frameworks: a survey
                 draft-irtf-p2prg-simulation-survey-00

Abstract

   Peer-to-peer (p2p) protocols, like all distributed protocols, are
   complex, and therefore harder to debug and study in the wild.  This
   is more true of existing p2p protocols, where changing the behaviour
   of the protocol --- however minor the change may be --- may result in
   unknown manifestations on the dynamics of the swarm using that
   protocol.  In lieu of the unintended consequences of perturbing a
   live swarm, researchers have resorted to simulation frameworks.
   However, simulation results obtained from one simulator are often
   hard to reproduce when using another simulation framework.  This
   document surveys existing simulator frameworks prevalent in
   simulating p2p protocols today in order to quantify any assumptions
   and characteristics inherent in the simulator.  This, we hope, will
   aid future researchers in choosing the right simulation framework for
   their abstraction.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any



Gurbani, et al.         Expires December 30, 2011               [Page 1]


Internet-Draft            P2P simulation survey                June 2011


   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on December 30, 2011.

Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.
































Gurbani, et al.         Expires December 30, 2011               [Page 2]


Internet-Draft            P2P simulation survey                June 2011


Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 4
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 4
   3.  Criteria for evaluating simulation  frameworks  . . . . . . . . 5
   4.  List of simulation frameworks . . . . . . . . . . . . . . . . . 5
   5.  Security Considerations . . . . . . . . . . . . . . . . . . . . 6
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 6
   7.  References  . . . . . . . . . . . . . . . . . . . . . . . . . . 6
     7.1.  Normative References  . . . . . . . . . . . . . . . . . . . 6
     7.2.  Informative References  . . . . . . . . . . . . . . . . . . 6
   Appendix A.  Acknowledgments  . . . . . . . . . . . . . . . . . . . 7
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . . . 7






































Gurbani, et al.         Expires December 30, 2011               [Page 3]


Internet-Draft            P2P simulation survey                June 2011


1.  Introduction

   Peer-to-peer (p2p) protocols, like all distributed protocols, are
   complex, and therefore harder to debug and study in the wild.  This
   is more true of existing p2p protocols, where changing the behaviour
   of the protocol --- however minor the change may be --- may result in
   unknown manifestations on the dynamics of the swarm using that
   protocol.

   Researchers contemplating on changing the behavior of an existing p2p
   protocol have to be careful still, least they inadvertently do more
   harm than good by introducing their changes.  Furthermore, any
   changes to an existing p2p protocol or a newly developed p2p protocol
   must be tested and evaluated for validity and reproducibility by the
   research community.  While analytical and mathematical modeling
   (fluid models, optimization and linear programming) is easily
   validated, it is harder to validate empirical experiments due to the
   dynamic nature of the networks, hosts, and interconnections between
   them.  Simulation frameworks are attractive since they provide a
   controlled environment under which new behavior of p2p protocols can
   be studied and quantified.

   The good news is that there is a plethora of simulation frameworks
   for p2p protocols available today, some of them are surveyed in
   Naicken et al. [naicken].  However, that survey is dated and does not
   include simulation frameworks like ns-3 [ns-3] and ProtoPeer
   [protopeer] that have become available since the survey was
   published.

   The aim of this document is to update the state-of-art with respect
   to p2p simulation frameworks available today.  We will survey
   simulator frameworks prevalent --- and actively used --- in
   simulating p2p protocols today in order to quantify any assumptions
   and characteristics inherent in the simulator.  This, we hope, will
   aid future researchers in choosing the right simulation framework for
   their abstraction.  This document can also serve as a guidance for
   those researchers who are interested in developing P2P simulators
   from scratch.


2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].






Gurbani, et al.         Expires December 30, 2011               [Page 4]


Internet-Draft            P2P simulation survey                June 2011


3.  Criteria for evaluating simulation  frameworks

   This is a non-exhaustive list of all criteria that we should evaluate
   when surveying a simulation framework.

   o  Type of simulator: flow-level, message-level, or packet- level.
      Advantages and disadvantages of each.
   o  Does the simulator specifically target p2p networks?  Some like
      ns-3 are general purpose simulators, but p2p models can be
      constructed and evaluated over a general-purpose simulator.
   o  Level of documentation (APIs, wiki, etc.)
   o  Support for building models: script level, compiled language,
      through a visualization editor, etc.
   o  System limitations imposed by the simulator framework, if any.
   o  Learning curves associated with the simulator framework.
   o  Support for trace-driven simulation (i.e., using live traces to
      inject events in the simulator queue.
   o  Scalability of the simulator.
   o  Whether or not the simulator framework supports distributed
      simulations synchronized on a common time source or event queue.
   o  Support for transitioning from a simulation environment to actual
      system implementation (or, can the code developed for a simulator
      be used with minimal or no modifications in a real host)?  See
      Galuba et al. [protopeer].
   o  Support for modeling link-level (delay, latency, loss, data rate)
      and host-level characteristics (i.e., simulate both low-level
      events and application PDUs).
   o  Support for interfacing real hosts that inject events into the
      simulator.
   o  Support for collecting statistics and measurements from the
      models.
   o  Visualization tools for creating topologies, viewing the
      simulation in action, etc.
   o  Support for importing existing topologies (GT-ITM) and others.
   o  Support for exporting topologies in a standard graph markup
      language.
   o  Should we focus on only academic and research simulators or
      commercial simulators as well?
   o  ...


4.  List of simulation frameworks

   A list of simulation frameworks that we can survey appears below
   (original list is in Naicken et al. [naicken], I have added a couple
   more simulators).  This is a rather exhaustive list, however, going
   forward, we should focus on those frameworks that are: newer,
   actively in use today, and those frameworks that are actively used



Gurbani, et al.         Expires December 30, 2011               [Page 5]


Internet-Draft            P2P simulation survey                June 2011


   today and have been surveyed before, but could stand to be looked at
   again in light of hardware and software advances in the last few
   years (multi-cores, parallel programming, etc.):

   o  ns-3 [ns-3].
   o  ProtoPeer [protopeer].
   o  GPS.
   o  PeerSim.
   o  P2PSim.
   o  OverSim.
   o  DHTSim.
   o  PlanetSim.
   o  VPDNS.
   o  Narses.
   o  Neurogrid.
   o  GnutellaSim.
   o  myNS --- we could probably drop this in favor of ns-3.
   o  Overlay Weaver.
   o  Query-cycle Sim.
   o  GTNetS [gtnets] --- seems to be abandoned.
   o  PeerfactSim [peerfactsim].
   o  ...


5.  Security Considerations

   This document does not introduce any new security considerations in
   p2p protocols.


6.  IANA Considerations

   This document does not require any IANA considerations.


7.  References

7.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

7.2.  Informative References

   [gtnets]   "The Georgia Tech Network Simulator (GTNetS)",
               http://www.ece.gatech.edu/research/labs/MANIACS/GTNetS/.

   [naicken]  Naicken, S., Basu, A., Livingston, B., and S. Rodhetbhai,



Gurbani, et al.         Expires December 30, 2011               [Page 6]


Internet-Draft            P2P simulation survey                June 2011


              "A Survey of Peer-to-Peer Network Simulators",
               Proceedings of the Seventh Annual Postgraduate Symposium,
              Liverpool, UK, 2006.

   [ns-3]     "The ns-3 network simulator",  http://www.nsnam.org.

   [peerfactsim]
              "PeerfactSim.KOM - A Large Scale Simulation Framework for
              Peer-to-Peer System",
               http://peerfact.kom.e-technik.tu-darmstadt.de/.

   [protopeer]
              Galuba, W., Aberer, K., Despotovic, Z., and W. Kellerer,
              "ProtoPeer: A p2p toolkit bridging the gap between
              simulation and live deployment",  Proceedings of
              SIMUTools, Rome, Italy, 2009.


Appendix A.  Acknowledgments


Authors' Addresses

   Vijay K. Gurbani (editor)
   Bell Labs, Alcatel-Lucent
   1960 Lucent Lane, Rm 9C-533
   Naperville, IL  60563
   USA

   Email: vkg@bell-labs.com


   Anirban Basu
   Tokai University and University of Sussex

   Email: a.basu@sussex.ac.uk


   Thomas Schmidt
   HAW Hamburg

   Email: schmidt@informatik.haw-hamburg.de









Gurbani, et al.         Expires December 30, 2011               [Page 7]


Internet-Draft            P2P simulation survey                June 2011


   Simon Fleming
   University of Sussex

   Email: S.Fleming@sussex.ac.uk


   Mario Kolberg
   University of Stirling

   Email: mko@cs.stir.ac.uk


   Matthias Waehlisch
   link-lab & FU Berlin

   Email: mw@link-lab.net



































Gurbani, et al.         Expires December 30, 2011               [Page 8]


Html markup produced by rfcmarkup 1.124, available from https://tools.ietf.org/tools/rfcmarkup/