< draft-dreibholz-vnfpool-rserpool-applic-07.txt   draft-dreibholz-vnfpool-rserpool-applic-08.txt >
Network Working Group T. Dreibholz Network Working Group T. Dreibholz
Internet-Draft Simula@OsloMET Internet-Draft SimulaMet
Intended status: Informational M. Tuexen Intended status: Informational M. Tuexen
Expires: September 7, 2018 Muenster Univ. of App. Sciences Expires: August 5, 2019 Muenster Univ. of Appl. Sciences
M. Shore M. Shore
No Mountain Software No Mountain Software
N. Zong N. Zong
Huawei Technologies Huawei Technologies
March 6, 2018 February 1, 2019
The Applicability of Reliable Server Pooling (RSerPool) for Virtual The Applicability of Reliable Server Pooling (RSerPool) for Virtual
Network Function Resource Pooling (VNFPOOL) Network Function Resource Pooling (VNFPOOL)
draft-dreibholz-vnfpool-rserpool-applic-07.txt draft-dreibholz-vnfpool-rserpool-applic-08
Abstract Abstract
This draft describes the application of Reliable Server This draft describes the application of Reliable Server
Pooling (RSerPool) for Virtual Network Function Resource Pooling (RSerPool) for Virtual Network Function Resource
Pooling (VNFPOOL). Pooling (VNFPOOL).
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 7, 2018. This Internet-Draft will expire on August 5, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
skipping to change at page 2, line 37 skipping to change at page 2, line 37
3.8. Protocol Stack . . . . . . . . . . . . . . . . . . . . . 8 3.8. Protocol Stack . . . . . . . . . . . . . . . . . . . . . 8
3.9. Extensions . . . . . . . . . . . . . . . . . . . . . . . 9 3.9. Extensions . . . . . . . . . . . . . . . . . . . . . . . 9
3.10. Reference Implementation and Deployment . . . . . . . . . 9 3.10. Reference Implementation and Deployment . . . . . . . . . 9
4. Usage of Reliable Server Pooling . . . . . . . . . . . . . . 9 4. Usage of Reliable Server Pooling . . . . . . . . . . . . . . 9
5. Security Considerations . . . . . . . . . . . . . . . . . . . 10 5. Security Considerations . . . . . . . . . . . . . . . . . . . 10
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
7. Testbed Platform . . . . . . . . . . . . . . . . . . . . . . 10 7. Testbed Platform . . . . . . . . . . . . . . . . . . . . . . 10
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 10
9.1. Normative References . . . . . . . . . . . . . . . . . . 10 9.1. Normative References . . . . . . . . . . . . . . . . . . 10
9.2. Informative References . . . . . . . . . . . . . . . . . 11 9.2. Informative References . . . . . . . . . . . . . . . . . 12
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15
1. Introduction 1. Introduction
1.1. Abbreviations 1.1. Abbreviations
o PE: Pool Element o PE: Pool Element
o PR: Pool Registrar o PR: Pool Registrar
o PU: Pool User o PU: Pool User
o RSerPool: Reliable Server Pooling o RSerPool: Reliable Server Pooling
o SCTP: Stream Control Transmission Protocol o SCTP: Stream Control Transmission Protocol
o VNFPOOL: Virtual Network Function Resource Pooling o VNFPOOL: Virtual Network Function Resource Pooling
2. Virtual Network Function Resource Pooling 2. Virtual Network Function Resource Pooling
Virtualised Network Function (VNF) (e.g. vFW, vLB) -- as introduced Virtualised Network Function (VNF) (e.g. vFW, vLB) -- as introduced
in more detail in [12] -- provides the same function as the in more detail in [I-D.zong-vnfpool-problem-statement] -- provides
equivalent network function (e.g. FW, LB), but is deployed as the same function as the equivalent network function (e.g. FW, LB),
software instances running on general purpose servers via but is deployed as software instances running on general purpose
virtualisation platform. The main features of VNF include the servers via virtualisation platform. The main features of VNF
following aspects: include the following aspects:
1. A service consists of a sequence of topologically distributed VNF 1. A service consists of a sequence of topologically distributed VNF
instances where the data connections are preferably directly instances where the data connections are preferably directly
established between the instances. established between the instances.
2. There are potentially more factors that cause VNF instance 2. There are potentially more factors that cause VNF instance
transition or even failure; VNF pool refers to a group of VNF transition or even failure; VNF pool refers to a group of VNF
instances providing same network function. instances providing same network function.
Virtualisation technology allows network function virtualisation Virtualisation technology allows network function virtualisation
operators to build a reliable VNF by pooling the underlying operators to build a reliable VNF by pooling the underlying
resources, such as CPU, storage, networking, etc. to form a cluster resources, such as CPU, storage, networking, etc. to form a cluster
of VNF instances. VNF pool refers to a cluster or group of VNF of VNF instances. VNF pool refers to a cluster or group of VNF
instances providing same network function. Each VNF pool has a Pool instances providing same network function. Each VNF pool has a Pool
Manager (PM) to manage the VNF instance such as instance selection, Manager (PM) to manage the VNF instance such as instance selection,
monitoring, etc. There will be a redundancy mechanism for a reliable monitoring, etc. There will be a redundancy mechanism for a reliable
PM to achieve reliable VNF. More details on VNF pool can be found in PM to achieve reliable VNF. More details on VNF pool can be found in
[12]. [I-D.zong-vnfpool-problem-statement].
3. Reliable Server Pooling 3. Reliable Server Pooling
3.1. Introduction 3.1. Introduction
+---------------+ +---------------+
| Pool User | | Pool User |
+---------------+ +---------------+
^ ^
| ASAP | ASAP
V V
skipping to change at page 4, line 26 skipping to change at page 4, line 26
+------------------------------------------------------------+ +------------------------------------------------------------+
| +--------------+ +--------------+ +--------------+ | | +--------------+ +--------------+ +--------------+ |
| | Pool Element | | Pool Element | ... ... | Pool Element | | | | Pool Element | | Pool Element | ... ... | Pool Element | |
| +--------------+ +--------------+ +--------------+ | | +--------------+ +--------------+ +--------------+ |
| Server Pool | | Server Pool |
+------------------------------------------------------------+ +------------------------------------------------------------+
Figure 1 Figure 1
An overview of the RSerPool framework -- which is defined as RFC in An overview of the RSerPool framework -- which is defined as RFC in
[2] -- is provided in Figure 1. There are three types of components: [RFC5351] -- is provided in Figure 1. There are three types of
components:
o Pool Element (PE) denotes a server in a pool. PEs in the same o Pool Element (PE) denotes a server in a pool. PEs in the same
pool provide the same service. pool provide the same service.
o Pool User (PU) denotes a client using the service of a pool. o Pool User (PU) denotes a client using the service of a pool.
o Pool Registrar (PR) is the management component for the pools. o Pool Registrar (PR) is the management component for the pools.
The set of all pools within an operation scope (for example: an The set of all pools within an operation scope (for example: an
organisation, a company or a department) is denoted as handlespace. organisation, a company or a department) is denoted as handlespace.
Clearly, a single PR would be a single point of failure. Therefore, Clearly, a single PR would be a single point of failure. Therefore,
PRs also have to be redundant. Within the handlespace, each pool is PRs also have to be redundant. Within the handlespace, each pool is
identified by a unique pool handle (PH). identified by a unique pool handle (PH).
3.2. Registrar Operations 3.2. Registrar Operations
The PRs of an operation scope synchronise their view of the The PRs of an operation scope synchronise their view of the
handlespace by using the Endpoint haNdlespace Redundancy handlespace by using the Endpoint haNdlespace Redundancy
Protocol (ENRP, defined as RFCs in [4], [5]). In contrast to for Protocol (ENRP, defined as RFCs in [RFC5353], [RFC5354]). In
instance the Domain Name System (DNS), an operation scope is contrast to for instance the Domain Name System (DNS), an operation
restricted to a single administrative domain. That is, all of its scope is restricted to a single administrative domain. That is, all
components are under the control of the same authority (for example: of its components are under the control of the same authority (for
a company). This property leads to small management overhead, which example: a company). This property leads to small management
also allows for RSerPool usage on devices having only limited memory overhead, which also allows for RSerPool usage on devices having only
and CPU resources (for example: telecommunications equipment). limited memory and CPU resources (for example: telecommunications
Nevertheless, PEs may be distributed globally to continue their equipment). Nevertheless, PEs may be distributed globally to
service even in case of localised disasters (like for example an continue their service even in case of localised disasters (like for
earthquake). Each PR in the operation scope is identified by a PR example an earthquake). Each PR in the operation scope is identified
ID, which is a randomly chosen 32-bit number. by a PR ID, which is a randomly chosen 32-bit number.
3.3. Pool Element Operations 3.3. Pool Element Operations
Within their operation scope, the PEs may choose an arbitrary PR to Within their operation scope, the PEs may choose an arbitrary PR to
register into a pool by using the Aggregate Server Access register into a pool by using the Aggregate Server Access
Protocol (ASAP, defined as RFCs in [3], [5]). The registration is Protocol (ASAP, defined as RFCs in [RFC5352], [RFC5354]). The
performed by using an ASAP_REGISTRATION message. Within its pool, a registration is performed by using an ASAP_REGISTRATION message.
PE is characterised by its PE ID, which is a randomly chosen 32-bit Within its pool, a PE is characterised by its PE ID, which is a
number. Upon registration at a PR, the chosen PR becomes the Home- randomly chosen 32-bit number. Upon registration at a PR, the chosen
PR (PR-H) of the newly registered PE. A PR-H is responsible for PR becomes the Home-PR (PR-H) of the newly registered PE. A PR-H is
monitoring the availability of its PEs by ASAP_ENDPOINT_KEEP_ALIVE responsible for monitoring the availability of its PEs by
messages (to be acknowledged by a PE via an ASAP_ENDPOINT_KEEP_ALIVE messages (to be acknowledged by a PE via an
ASAP_ENDPOINT_KEEP_ALIVE_ACK message within a configured timeout). ASAP_ENDPOINT_KEEP_ALIVE_ACK message within a configured timeout).
The PR-H propagates the information about its PEs to the other PRs of The PR-H propagates the information about its PEs to the other PRs of
the operation scope via ENRP_UPDATE messages. the operation scope via ENRP_UPDATE messages.
PEs re-register regularly in an interval denoted as registration PEs re-register regularly in an interval denoted as registration
lifetime and for information updates. Similar to the registration, a lifetime and for information updates. Similar to the registration, a
re-registration is performed by using another ASAP_REGISTRATION re-registration is performed by using another ASAP_REGISTRATION
message. PEs may intentionally deregister from the pool by using an message. PEs may intentionally deregister from the pool by using an
ASAP_DEREGISTRATION message. Also like for the registration, the ASAP_DEREGISTRATION message. Also like for the registration, the
PR-H makes the deregistration known to the other PRs within the PR-H makes the deregistration known to the other PRs within the
skipping to change at page 5, line 44 skipping to change at page 5, line 45
As soon as a PE detects the failure of its PR-H (that is: its request As soon as a PE detects the failure of its PR-H (that is: its request
is not answered within a given timeout), it simply tries another PR is not answered within a given timeout), it simply tries another PR
of the operation scope for its registration and deregistration of the operation scope for its registration and deregistration
requests. However, as a double safeguard, the remaining PRs also requests. However, as a double safeguard, the remaining PRs also
negotiate a takeover of the PEs managed by a dead PR. This ensures negotiate a takeover of the PEs managed by a dead PR. This ensures
that each PE again gets a working PR-H as soon as possible. The PRs that each PE again gets a working PR-H as soon as possible. The PRs
of an operation scope monitor the availability of each other PR by of an operation scope monitor the availability of each other PR by
using ENRP_PRESENCE messages, which are transmitted regularly. If using ENRP_PRESENCE messages, which are transmitted regularly. If
there is no ENRP_PRESENCE within a given timeout, the peer is assumed there is no ENRP_PRESENCE within a given timeout, the peer is assumed
to be dead and a so-called takeover procedure (see also [21] for more to be dead and a so-called takeover procedure (see also [AINA2009]
details) is initiated for the PEs managed by the dead PR: from all for more details) is initiated for the PEs managed by the dead PR:
PRs having started this takeover procedure, the PR with the highest from all PRs having started this takeover procedure, the PR with the
PR ID takes over the ownership of these PEs. The PEs are informed highest PR ID takes over the ownership of these PEs. The PEs are
about being taken over by their new PR-H via an informed about being taken over by their new PR-H via an
ASAP_ENDPOINT_KEEP_ALIVE with Home-flag set. The PEs are requested ASAP_ENDPOINT_KEEP_ALIVE with Home-flag set. The PEs are requested
to adopt the sender of this Home-flagged message as their new PR-H. to adopt the sender of this Home-flagged message as their new PR-H.
3.5. Pool User Operations 3.5. Pool User Operations
3.5.1. Handle Resolution and Response 3.5.1. Handle Resolution and Response
In order to access the service of a pool given by its PH, a PU In order to access the service of a pool given by its PH, a PU
requests a PE selection from an arbitrary PR of the operation scope, requests a PE selection from an arbitrary PR of the operation scope,
again by using ASAP. This selection procedure is denoted as handle again by using ASAP. This selection procedure is denoted as handle
resolution. Upon reception of a so-called ASAP_HANDLE_RESOLUTION resolution. Upon reception of a so-called ASAP_HANDLE_RESOLUTION
message the PR selects the requested list of PE identities and message the PR selects the requested list of PE identities and
returns them in an ASAP_HANDLE_RESOLUTION_RESPONSE message. returns them in an ASAP_HANDLE_RESOLUTION_RESPONSE message.
3.5.2. Pool Member Selection Policies 3.5.2. Pool Member Selection Policies
The pool-specific selection rule is denoted as pool member selection The pool-specific selection rule is denoted as pool member selection
policy or shortly as pool policy. Two classes of load distribution policy or shortly as pool policy. Two classes of load distribution
policies are supported: non-adaptive and adaptive strategies (a policies are supported: non-adaptive and adaptive strategies (a
detailed overview is provided by [16], [18], [23], [19]). While detailed overview is provided by [Dre2006], [LCN2005], [IJIIDS2010],
adaptive strategies base their selections on the current PE state [IJHIT2008]). While adaptive strategies base their selections on the
(which requires up-to-date information), non-adaptive algorithms do current PE state (which requires up-to-date information), non-
not need such data. A basic set of adaptive and non-adaptive pool adaptive algorithms do not need such data. A basic set of adaptive
policies is defined as RFC in [7]. and non-adaptive pool policies is defined as RFC in [RFC5356].
Defined in [7] are the non-adaptive policies Round Robin (RR), Defined in [RFC5356] are the non-adaptive policies Round Robin (RR),
Random (RAND) and Priority (PRIO) as well as the adaptive policies Random (RAND) and Priority (PRIO) as well as the adaptive policies
Least Used (LU) and Least Used with Degradation (LUD). While RR/RAND Least Used (LU) and Least Used with Degradation (LUD). While RR/RAND
select PEs in turn/randomly, PRIO selects one of the PEs having the select PEs in turn/randomly, PRIO selects one of the PEs having the
highest priority. PRIO can for example be used to realise a master/ highest priority. PRIO can for example be used to realise a master/
backup PE setup. Only if there are no master PEs left, a backup PE backup PE setup. Only if there are no master PEs left, a backup PE
is selected. Round-robin selection is applied among PEs having the is selected. Round-robin selection is applied among PEs having the
same priority. LU selects the least-used PE, according to up-to-date same priority. LU selects the least-used PE, according to up-to-date
application-specific load information. Round robin selection is application-specific load information. Round robin selection is
applied among multiple least-loaded PEs. LUD, which is evaluated by applied among multiple least-loaded PEs. LUD, which is evaluated by
[20], furthermore introduces a load decrement constant that is added [ICDS2008-LUD], furthermore introduces a load decrement constant that
to the actual load each time a PE is selected. It is used to is added to the actual load each time a PE is selected. It is used
compensate inaccurate load states due to delayed updates. An update to compensate inaccurate load states due to delayed updates. An
resets the load to the actual load value. update resets the load to the actual load value.
3.5.3. Handle Resolution and Response 3.5.3. Handle Resolution and Response
PE may fail, for example due to hardware or network failures. Since PE may fail, for example due to hardware or network failures. Since
there is a certain latency between the actual failure of a PE and the there is a certain latency between the actual failure of a PE and the
removal of its entry from the handlespace -- depending on the removal of its entry from the handlespace -- depending on the
interval and timeout for the ASAP_ENDPOINT_KEEP_ALIVE monitoring -- interval and timeout for the ASAP_ENDPOINT_KEEP_ALIVE monitoring --
the PUs may report unreachable PEs to a PR by using an the PUs may report unreachable PEs to a PR by using an
ASAP_ENDPOINT_UNREACHABLE message. A PR locally counts these reports ASAP_ENDPOINT_UNREACHABLE message. A PR locally counts these reports
for each PE and when reaching the threshold MAX-BAD-PE-REPORT for each PE and when reaching the threshold MAX-BAD-PE-REPORT
(default is 3, as defined in the RFC [3]), the PR may decide to (default is 3, as defined in the RFC [RFC5352]), the PR may decide to
remove the PE from the handlespace. The counter of a PE is reset remove the PE from the handlespace. The counter of a PE is reset
upon its re-registration. More details on this threshold and upon its re-registration. More details on this threshold and
guidelines for its configuration can be found in [22]. guidelines for its configuration can be found in [IJAIT2009].
3.6. Automatic Configuration 3.6. Automatic Configuration
RSerPool components need to know the PRs of their operation scope. RSerPool components need to know the PRs of their operation scope.
While it is of course possible to configure a list of PRs into each While it is of course possible to configure a list of PRs into each
component, RSerPool also provides an auto-configuration feature: PRs component, RSerPool also provides an auto-configuration feature: PRs
may send so-called announces, that is, ASAP_ANNOUNCE and may send so-called announces, that is, ASAP_ANNOUNCE and
ENRP_PRESENCE messages which are regularly sent over UDP via IP ENRP_PRESENCE messages which are regularly sent over UDP via IP
multicast. Unlike broadcasts, multicast messages can also be multicast. Unlike broadcasts, multicast messages can also be
transported over routers (at least, this is easily possible within transported over routers (at least, this is easily possible within
skipping to change at page 7, line 30 skipping to change at page 7, line 30
3.7. State Synchronisation 3.7. State Synchronisation
RSerPool has been explicitly designed to be application-independent. RSerPool has been explicitly designed to be application-independent.
Therefore, RSerPool has not intended to define special state Therefore, RSerPool has not intended to define special state
synchronisation mechanisms for RSerPool-based applications. Such synchronisation mechanisms for RSerPool-based applications. Such
state synchronisation mechanisms are considered as tasks of the state synchronisation mechanisms are considered as tasks of the
applications themselves. However, RSerPool defines two mechanisms to applications themselves. However, RSerPool defines two mechanisms to
at least support the implementation of more sophisticated strategies: at least support the implementation of more sophisticated strategies:
Cookies and Businesss Cards. Details on these mechanisms can also be Cookies and Businesss Cards. Details on these mechanisms can also be
found in Subsection 3.9.5 of [16]. found in Subsection 3.9.5 of [Dre2006].
3.7.1. Cookies 3.7.1. Cookies
ASAP provides the mechanism of Client-Based State Sharing as ASAP provides the mechanism of Client-Based State Sharing as
introduced in [17]. Whenever useful, the PE may package its state in introduced in [LCN2002]. Whenever useful, the PE may package its
form of a state cookie and send it -- by an ASAP_COOKIE message -- to state in form of a state cookie and send it -- by an ASAP_COOKIE
the PU. The PU stores the latest state cookie received from the PE. message -- to the PU. The PU stores the latest state cookie received
Upon PE failure, this stored cookie is sent in an ASAP_COOKIE_ECHO to from the PE. Upon PE failure, this stored cookie is sent in an
the newly chosen PE. This PE may then restore the state. A shared ASAP_COOKIE_ECHO to the newly chosen PE. This PE may then restore
secret known by all PEs of a pool may be used to protect the state the state. A shared secret known by all PEs of a pool may be used to
from being manipulated or read by the PU. protect the state from being manipulated or read by the PU.
While Client-Based State Sharing is very simple, it may be While Client-Based State Sharing is very simple, it may be
inefficient when the state changes too frequently, is too large (the inefficient when the state changes too frequently, is too large (the
size limit of an ASAP_COOKIE/ASAP_COOKIE_ECHO is 64 KiB) or if it size limit of an ASAP_COOKIE/ASAP_COOKIE_ECHO is 64 KiB) or if it
must be prevented that a PU sends a state cookie to multiple PEs in must be prevented that a PU sends a state cookie to multiple PEs in
order to duplicate its sessions. order to duplicate its sessions.
3.7.2. Businesss Cards 3.7.2. Businesss Cards
Depending on the application, there may be constraints restricting Depending on the application, there may be constraints restricting
skipping to change at page 8, line 47 skipping to change at page 8, line 47
3.8. Protocol Stack 3.8. Protocol Stack
The protocol stack of a PR provides ENRP and ASAP services to PRs and The protocol stack of a PR provides ENRP and ASAP services to PRs and
PEs/PUs respectively. But between PU and PE, ASAP provides a Session PEs/PUs respectively. But between PU and PE, ASAP provides a Session
Layer protocol in the OSI model. From the perspective of the Layer protocol in the OSI model. From the perspective of the
Application Layer, the PU side establishes a session with a pool. Application Layer, the PU side establishes a session with a pool.
ASAP takes care of selecting a PE of the pool, initiating and ASAP takes care of selecting a PE of the pool, initiating and
maintaining the underlying transport connection and triggering a maintaining the underlying transport connection and triggering a
failover procedure when the PE becomes unavailable. failover procedure when the PE becomes unavailable.
The Transport Layer protocol is by default SCTP (as defined in [1]) The Transport Layer protocol is by default SCTP (as defined in
-- except for the UDP-based automatic configuration announces (see [RFC4960]) -- except for the UDP-based automatic configuration
Section 3.6) -- over possibly multi-homed IPv4 and/or IPv6. SCTP has announces (see Section 3.6) -- over possibly multi-homed IPv4 and/or
been chosen due to its support of multi-homing and its reliability IPv6. SCTP has been chosen due to its support of multi-homing and
features (see also [26]). its reliability features (see also [Dre2012]).
3.9. Extensions 3.9. Extensions
A couple of extensions to RSerPool are existing: Handle Resolution A couple of extensions to RSerPool are existing: Handle Resolution
Option defined in [9] improves the PE selection by letting the PU Option defined in [I-D.dreibholz-rserpool-asap-hropt] improves the PE
tell the PR its required number of PEs to be selected. ENRP Takeover selection by letting the PU tell the PR its required number of PEs to
Suggestion introduced in [11] ensures load balancing among PRs. [10] be selected. ENRP Takeover Suggestion introduced in
defines a delay-sensitive pool policy. [8] defines an SNMP MIB for [I-D.dreibholz-rserpool-enrp-takeover] ensures load balancing among
RSerPool. PRs. [I-D.dreibholz-rserpool-delay] defines a delay-sensitive pool
policy. [RFC5525] defines an SNMP MIB for RSerPool.
3.10. Reference Implementation and Deployment 3.10. Reference Implementation and Deployment
RSPLIB is the Open Source reference implementation of RSerPool. It RSPLIB is the Open Source reference implementation of RSerPool. It
is currently -- as of February 2016 -- available for Linux, FreeBSD, is currently -- as of February 2016 -- available for Linux, FreeBSD,
MacOS and Solaris. It is actively maintained. Particularly, it is MacOS and Solaris. It is actively maintained. Particularly, it is
also included in Ubuntu Linux as well as in the FreeBSD ports also included in Ubuntu Linux as well as in the FreeBSD ports
collection. RSPLIB can be downloaded from [14]. Further details on collection. RSPLIB can be downloaded from [RSerPoolPage]. Further
the implementation are available in [16], [24]. details on the implementation are available in [Dre2006],
[Globecom2010-Demo].
RSerPool with RSPLIB is deployed in a couple of Open Source projects, RSerPool with RSPLIB is deployed in a couple of Open Source projects,
including the SimProcTC Simulation Processing Tool-Chain for including the SimProcTC Simulation Processing Tool-Chain for
distributing simulation runs in a compute pool (see [25] as well as distributing simulation runs in a compute pool (see
the simulation run distribution project explained in [26] for a [OMNeTWorkshop2008] as well as the simulation run distribution
practical example) as well as for service infrastructure management project explained in [Dre2012] for a practical example) as well as
in the NorNet Core research testbed (see [27], [28]). for service infrastructure management in the NorNet Core research
testbed (see [ComNets2013-Core], [PAMS2013-NorNet]).
4. Usage of Reliable Server Pooling 4. Usage of Reliable Server Pooling
**** TO BE DISCUSSED! **** **** TO BE DISCUSSED! ****
The following features of RSerPool can be used for VNFPOOL: The following features of RSerPool can be used for VNFPOOL:
o Pool management. o Pool management.
o PE seclection with pool policies. o PE seclection with pool policies.
o Session management with help of ASAP_BUSINESS_CARD. o Session management with help of ASAP_BUSINESS_CARD.
The following features have to be added to RSerPool itself: The following features have to be added to RSerPool itself:
o Support of TCP including MPTCP as additional/alternative transport o Support of TCP including MPTCP as additional/alternative transport
protocols. protocols.
o Possibly add some special pool policies? o Possibly add some special pool policies?
o See also [I-D.dreibholz-rserpool-nextgen-ideas] for ideas on a
o See also [13] for ideas on a next generation of RSerPool. next generation of RSerPool.
The following features have to be provided outside of RSerPool: The following features have to be provided outside of RSerPool:
o State synchronisation for VNFPOOL. o State synchronisation for VNFPOOL.
o Pool Manager functionality as an RSerPool-based service. o Pool Manager functionality as an RSerPool-based service.
5. Security Considerations 5. Security Considerations
Security considerations for RSerPool can be found in [6]. Security considerations for RSerPool can be found in [RFC5355].
Furthermore, [23] examines the robustness of RSerPool systems against Furthermore, [IJIIDS2010] examines the robustness of RSerPool systems
attacks. against attacks.
6. IANA Considerations 6. IANA Considerations
This document introduces no additional considerations for IANA. This document introduces no additional considerations for IANA.
7. Testbed Platform 7. Testbed Platform
A large-scale and realistic Internet testbed platform with support A large-scale and realistic Internet testbed platform with support
for Reliable Server Pooling and the underlying SCTP protocol is for Reliable Server Pooling and the underlying SCTP protocol is
NorNet. A description of and introduction to NorNet is provided in NorNet. A description of and introduction to NorNet is provided in
[28], [29], [30]. Further information can be found on the project [PAMS2013-NorNet], [NICTA2016-Presentation],
website [15] at https://www.nntb.no. [UCLM2017-NorNet-Tutorial]. Further information can be found on the
project website [NorNet-Website] at https://www.nntb.no.
8. Acknowledgments 8. Acknowledgments
The authors would like to thank Xing Zhou for the friendly support. The authors would like to thank Xing Zhou for the friendly support.
9. References 9. References
9.1. Normative References 9.1. Normative References
[1] Stewart, R., Ed., "Stream Control Transmission Protocol", [I-D.dreibholz-rserpool-asap-hropt]
Dreibholz, T., "Handle Resolution Option for ASAP", draft-
dreibholz-rserpool-asap-hropt-23 (work in progress),
September 2018.
[I-D.dreibholz-rserpool-delay]
Dreibholz, T. and X. Zhou, "Definition of a Delay
Measurement Infrastructure and Delay-Sensitive Least-Used
Policy for Reliable Server Pooling", draft-dreibholz-
rserpool-delay-22 (work in progress), September 2018.
[I-D.dreibholz-rserpool-enrp-takeover]
Dreibholz, T. and X. Zhou, "Takeover Suggestion Flag for
the ENRP Handle Update Message", draft-dreibholz-rserpool-
enrp-takeover-20 (work in progress), September 2018.
[I-D.dreibholz-rserpool-nextgen-ideas]
Dreibholz, T., "Ideas for a Next Generation of the
Reliable Server Pooling Framework", draft-dreibholz-
rserpool-nextgen-ideas-10 (work in progress), September
2018.
[I-D.zong-vnfpool-problem-statement]
Zong, N., Dunbar, L., Shore, M., Lopez, D., and G.
Karagiannis, "Virtualized Network Function (VNF) Pool
Problem Statement", draft-zong-vnfpool-problem-
statement-06 (work in progress), July 2014.
[RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol",
RFC 4960, DOI 10.17487/RFC4960, September 2007, RFC 4960, DOI 10.17487/RFC4960, September 2007,
<https://www.rfc-editor.org/info/rfc4960>. <https://www.rfc-editor.org/info/rfc4960>.
[2] Lei, P., Ong, L., Tuexen, M., and T. Dreibholz, "An [RFC5351] Lei, P., Ong, L., Tuexen, M., and T. Dreibholz, "An
Overview of Reliable Server Pooling Protocols", RFC 5351, Overview of Reliable Server Pooling Protocols", RFC 5351,
DOI 10.17487/RFC5351, September 2008, <https://www.rfc- DOI 10.17487/RFC5351, September 2008,
editor.org/info/rfc5351>. <https://www.rfc-editor.org/info/rfc5351>.
[3] Stewart, R., Xie, Q., Stillman, M., and M. Tuexen, [RFC5352] Stewart, R., Xie, Q., Stillman, M., and M. Tuexen,
"Aggregate Server Access Protocol (ASAP)", RFC 5352, "Aggregate Server Access Protocol (ASAP)", RFC 5352,
DOI 10.17487/RFC5352, September 2008, <https://www.rfc- DOI 10.17487/RFC5352, September 2008,
editor.org/info/rfc5352>. <https://www.rfc-editor.org/info/rfc5352>.
[4] Xie, Q., Stewart, R., Stillman, M., Tuexen, M., and A. [RFC5353] Xie, Q., Stewart, R., Stillman, M., Tuexen, M., and A.
Silverton, "Endpoint Handlespace Redundancy Protocol Silverton, "Endpoint Handlespace Redundancy Protocol
(ENRP)", RFC 5353, DOI 10.17487/RFC5353, September 2008, (ENRP)", RFC 5353, DOI 10.17487/RFC5353, September 2008,
<https://www.rfc-editor.org/info/rfc5353>. <https://www.rfc-editor.org/info/rfc5353>.
[5] Stewart, R., Xie, Q., Stillman, M., and M. Tuexen, [RFC5354] Stewart, R., Xie, Q., Stillman, M., and M. Tuexen,
"Aggregate Server Access Protocol (ASAP) and Endpoint "Aggregate Server Access Protocol (ASAP) and Endpoint
Handlespace Redundancy Protocol (ENRP) Parameters", Handlespace Redundancy Protocol (ENRP) Parameters",
RFC 5354, DOI 10.17487/RFC5354, September 2008, RFC 5354, DOI 10.17487/RFC5354, September 2008,
<https://www.rfc-editor.org/info/rfc5354>. <https://www.rfc-editor.org/info/rfc5354>.
[6] Stillman, M., Ed., Gopal, R., Guttman, E., Sengodan, S., [RFC5355] Stillman, M., Ed., Gopal, R., Guttman, E., Sengodan, S.,
and M. Holdrege, "Threats Introduced by Reliable Server and M. Holdrege, "Threats Introduced by Reliable Server
Pooling (RSerPool) and Requirements for Security in Pooling (RSerPool) and Requirements for Security in
Response to Threats", RFC 5355, DOI 10.17487/RFC5355, Response to Threats", RFC 5355, DOI 10.17487/RFC5355,
September 2008, <https://www.rfc-editor.org/info/rfc5355>. September 2008, <https://www.rfc-editor.org/info/rfc5355>.
[7] Dreibholz, T. and M. Tuexen, "Reliable Server Pooling [RFC5356] Dreibholz, T. and M. Tuexen, "Reliable Server Pooling
Policies", RFC 5356, DOI 10.17487/RFC5356, September 2008, Policies", RFC 5356, DOI 10.17487/RFC5356, September 2008,
<https://www.rfc-editor.org/info/rfc5356>. <https://www.rfc-editor.org/info/rfc5356>.
[8] Dreibholz, T. and J. Mulik, "Reliable Server Pooling MIB [RFC5525] Dreibholz, T. and J. Mulik, "Reliable Server Pooling MIB
Module Definition", RFC 5525, DOI 10.17487/RFC5525, April Module Definition", RFC 5525, DOI 10.17487/RFC5525, April
2009, <https://www.rfc-editor.org/info/rfc5525>. 2009, <https://www.rfc-editor.org/info/rfc5525>.
[9] Dreibholz, T., "Handle Resolution Option for ASAP", draft-
dreibholz-rserpool-asap-hropt-21 (work in progress),
August 2017.
[10] Dreibholz, T. and X. Zhou, "Definition of a Delay
Measurement Infrastructure and Delay-Sensitive Least-Used
Policy for Reliable Server Pooling", draft-dreibholz-
rserpool-delay-20 (work in progress), August 2017.
[11] Dreibholz, T. and X. Zhou, "Takeover Suggestion Flag for
the ENRP Handle Update Message", draft-dreibholz-rserpool-
enrp-takeover-18 (work in progress), August 2017.
[12] Zong, N., Dunbar, L., Shore, M., Lopez, D., and G.
Karagiannis, "Virtualized Network Function (VNF) Pool
Problem Statement", draft-zong-vnfpool-problem-
statement-06 (work in progress), July 2014.
[13] Dreibholz, T., "Ideas for a Next Generation of the
Reliable Server Pooling Framework", draft-dreibholz-
rserpool-nextgen-ideas-08 (work in progress), August 2017.
9.2. Informative References 9.2. Informative References
[14] Dreibholz, T., "Thomas Dreibholz's RSerPool Page", [AINA2009]
Online: https://www.uni-due.de/~be0001/rserpool/, 2017, Zhou, X., Dreibholz, T., Fa, F., Du, W., and E. Rathgeb,
<https://www.uni-due.de/~be0001/rserpool/>. "Evaluation and Optimization of the Registrar Redundancy
Handling in Reliable Server Pooling Systems", Proceedings
of the IEEE 23rd International Conference on Advanced
Information Networking and Applications (AINA) Pages
256-262, ISBN 978-0-7695-3638-5, DOI 10.1109/AINA.2009.25,
May 2009, <https://www.wiwi.uni-
due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/AINA2009.pdf>.
[15] Dreibholz, T., "NorNet -- A Real-World, Large-Scale Multi- [ComNets2013-Core]
Homing Testbed", Online: https://www.nntb.no/, 2017, Gran, E., Dreibholz, T., and A. Kvalbein, "NorNet Core - A
<https://www.nntb.no/>. Multi-Homed Research Testbed", Computer Networks, Special
Issue on Future Internet Testbeds Volume 61, Pages 75-87,
ISSN 1389-1286, DOI 10.1016/j.bjp.2013.12.035, March 2014,
<https://www.simula.no/file/simulasimula2236pdf/download>.
[16] Dreibholz, T., "Reliable Server Pooling - Evaluation, [Dre2006] Dreibholz, T., "Reliable Server Pooling - Evaluation,
Optimization and Extension of a Novel IETF Architecture", Optimization and Extension of a Novel IETF Architecture",
March 2007, <https://duepublico.uni-duisburg- March 2007, <https://duepublico.uni-duisburg-
essen.de/servlets/DerivateServlet/Derivate-16326/ essen.de/servlets/DerivateServlet/Derivate-16326/
Dre2006_final.pdf>. Dre2006_final.pdf>.
[17] Dreibholz, T., "An Efficient Approach for State Sharing in [Dre2012] Dreibholz, T., "Evaluation and Optimisation of Multi-Path
Server Pools", Proceedings of the 27th IEEE Local Computer Transport using the Stream Control Transmission Protocol",
Networks Conference (LCN) Pages 348-349, Habilitation Treatise, March 2012,
ISBN 0-7695-1591-6, DOI 10.1109/LCN.2002.1181806, November <https://duepublico.uni-duisburg-
2002, <https://www.wiwi.uni-due.de/fileadmin/fileupload/I- essen.de/servlets/DerivateServlet/Derivate-29737/
TDR/ReliableServer/Publications/StateSharing-Paper- Dre2012_final.pdf>.
ShortVersion.pdf>.
[18] Dreibholz, T. and E. Rathgeb, "On the Performance of
Reliable Server Pooling Systems", Proceedings of the IEEE
Conference on Local Computer Networks (LCN) 30th
Anniversary Pages 200-208, ISBN 0-7695-2421-4,
DOI 10.1109/LCN.2005.98, November 2005,
<https://www.wiwi.uni-due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/LCN2005.pdf>.
[19] Dreibholz, T. and E. Rathgeb, "An Evaluation of the Pool [Globecom2010-Demo]
Maintenance Overhead in Reliable Server Pooling Systems", Dreibholz, T. and M. Becke, "The RSPLIB Project - From
SERSC International Journal on Hybrid Information Research to Application", Demo Presentation at the IEEE
Technology (IJHIT) Number 2, Volume 1, Pages 17-32, Global Communications Conference (GLOBECOM), December
ISSN 1738-9968, April 2008, <https://www.wiwi.uni- 2010, <https://www.wiwi.uni-due.de/fileadmin/fileupload/I-
due.de/fileadmin/fileupload/I- TDR/ReliableServer/Publications/Globecom2010-Demo.pdf>.
TDR/ReliableServer/Publications/IJHIT2008.pdf>.
[20] Zhou, X., Dreibholz, T., and E. Rathgeb, "A New Server [ICDS2008-LUD]
Zhou, X., Dreibholz, T., and E. Rathgeb, "A New Server
Selection Strategy for Reliable Server Pooling in Widely Selection Strategy for Reliable Server Pooling in Widely
Distributed Environments", Proceedings of the 2nd IEEE Distributed Environments", Proceedings of the 2nd IEEE
International Conference on Digital Society (ICDS) Pages International Conference on Digital Society (ICDS) Pages
171-177, ISBN 978-0-7695-3087-1, DOI 10.1109/ICDS.2008.12, 171-177, ISBN 978-0-7695-3087-1, DOI 10.1109/ICDS.2008.12,
February 2008, <https://www.wiwi.uni- February 2008, <https://www.wiwi.uni-
due.de/fileadmin/fileupload/I- due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/ICDS2008-LUD.pdf>. TDR/ReliableServer/Publications/ICDS2008-LUD.pdf>.
[21] Zhou, X., Dreibholz, T., Fa, F., Du, W., and E. Rathgeb, [IJAIT2009]
"Evaluation and Optimization of the Registrar Redundancy Dreibholz, T. and E. Rathgeb, "Overview and Evaluation of
Handling in Reliable Server Pooling Systems", Proceedings
of the IEEE 23rd International Conference on Advanced
Information Networking and Applications (AINA) Pages
256-262, ISBN 978-0-7695-3638-5, DOI 10.1109/AINA.2009.25,
May 2009, <https://www.wiwi.uni-
due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/AINA2009.pdf>.
[22] Dreibholz, T. and E. Rathgeb, "Overview and Evaluation of
the Server Redundancy and Session Failover Mechanisms in the Server Redundancy and Session Failover Mechanisms in
the Reliable Server Pooling Framework", International the Reliable Server Pooling Framework", International
Journal on Advances in Internet Technology (IJAIT) Number Journal on Advances in Internet Technology (IJAIT) Number
1, Volume 2, Pages 1-14, ISSN 1942-2652, June 2009, 1, Volume 2, Pages 1-14, ISSN 1942-2652, June 2009,
<https://www.wiwi.uni-due.de/fileadmin/fileupload/I- <https://www.wiwi.uni-due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/IJAIT2009.pdf>. TDR/ReliableServer/Publications/IJAIT2009.pdf>.
[23] Dreibholz, T., Zhou, X., Becke, M., Pulinthanath, J., [IJHIT2008]
Dreibholz, T. and E. Rathgeb, "An Evaluation of the Pool
Maintenance Overhead in Reliable Server Pooling Systems",
SERSC International Journal on Hybrid Information
Technology (IJHIT) Number 2, Volume 1, Pages 17-32,
ISSN 1738-9968, April 2008, <https://www.wiwi.uni-
due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/IJHIT2008.pdf>.
[IJIIDS2010]
Dreibholz, T., Zhou, X., Becke, M., Pulinthanath, J.,
Rathgeb, E., and W. Du, "On the Security of Reliable Rathgeb, E., and W. Du, "On the Security of Reliable
Server Pooling Systems", International Journal on Server Pooling Systems", International Journal on
Intelligent Information and Database Intelligent Information and Database
Systems (IJIIDS) Number 6, Volume 4, Pages 552-578, Systems (IJIIDS) Number 6, Volume 4, Pages 552-578,
ISSN 1751-5858, DOI 10.1504/IJIIDS.2010.036894, December ISSN 1751-5858, DOI 10.1504/IJIIDS.2010.036894, December
2010, <https://www.wiwi.uni-due.de/fileadmin/fileupload/I- 2010, <https://www.wiwi.uni-due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/IJIIDS2010.pdf>. TDR/ReliableServer/Publications/IJIIDS2010.pdf>.
[24] Dreibholz, T. and M. Becke, "The RSPLIB Project - From [LCN2002] Dreibholz, T., "An Efficient Approach for State Sharing in
Research to Application", Demo Presentation at the IEEE Server Pools", Proceedings of the 27th IEEE Local Computer
Global Communications Conference (GLOBECOM), December Networks Conference (LCN) Pages 348-349,
2010, <https://www.wiwi.uni-due.de/fileadmin/fileupload/I- ISBN 0-7695-1591-6, DOI 10.1109/LCN.2002.1181806, November
TDR/ReliableServer/Publications/Globecom2010-Demo.pdf>. 2002, <https://www.wiwi.uni-due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/
StateSharing-Paper-ShortVersion.pdf>.
[25] Dreibholz, T. and E. Rathgeb, "A Powerful Tool-Chain for [LCN2005] Dreibholz, T. and E. Rathgeb, "On the Performance of
Reliable Server Pooling Systems", Proceedings of the IEEE
Conference on Local Computer Networks (LCN) 30th
Anniversary Pages 200-208, ISBN 0-7695-2421-4,
DOI 10.1109/LCN.2005.98, November 2005,
<https://www.wiwi.uni-due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/LCN2005.pdf>.
[NICTA2016-Presentation]
Dreibholz, T., "NorNet at NICTA - An Introduction to the
NorNet Testbed", Invited Talk at National Information
Communications Technology Australia (NICTA), January 2016,
<https://www.simula.no/file/nicta2016-presentationpdf/
download>.
[NorNet-Website]
Dreibholz, T., "NorNet -- A Real-World, Large-Scale Multi-
Homing Testbed", Online: https://www.nntb.no/, 2017,
<https://www.nntb.no/>.
[OMNeTWorkshop2008]
Dreibholz, T. and E. Rathgeb, "A Powerful Tool-Chain for
Setup, Distributed Processing, Analysis and Debugging of Setup, Distributed Processing, Analysis and Debugging of
OMNeT++ Simulations", Proceedings of the 1st ACM/ICST OMNeT++ Simulations", Proceedings of the 1st ACM/ICST
International Workshop on OMNeT++ ISBN 978-963-9799-20-2, International Workshop on OMNeT++ ISBN 978-963-9799-20-2,
DOI 10.4108/ICST.SIMUTOOLS2008.2990, March 2008, DOI 10.4108/ICST.SIMUTOOLS2008.2990, March 2008,
<https://www.wiwi.uni-due.de/fileadmin/fileupload/I- <https://www.wiwi.uni-due.de/fileadmin/fileupload/I-
TDR/ReliableServer/Publications/OMNeTWorkshop2008.pdf>. TDR/ReliableServer/Publications/OMNeTWorkshop2008.pdf>.
[26] Dreibholz, T., "Evaluation and Optimisation of Multi-Path [PAMS2013-NorNet]
Transport using the Stream Control Transmission Protocol", Dreibholz, T. and E. Gran, "Design and Implementation of
Habilitation Treatise, March 2012,
<https://duepublico.uni-duisburg-
essen.de/servlets/DerivateServlet/Derivate-29737/
Dre2012_final.pdf>.
[27] Gran, E., Dreibholz, T., and A. Kvalbein, "NorNet Core - A
Multi-Homed Research Testbed", Computer Networks, Special
Issue on Future Internet Testbeds Volume 61, Pages 75-87,
ISSN 1389-1286, DOI 10.1016/j.bjp.2013.12.035, March 2014,
<https://www.simula.no/file/simulasimula2236pdf/download>.
[28] Dreibholz, T. and E. Gran, "Design and Implementation of
the NorNet Core Research Testbed for Multi-Homed Systems", the NorNet Core Research Testbed for Multi-Homed Systems",
Proceedings of the 3nd International Workshop on Protocols Proceedings of the 3nd International Workshop on Protocols
and Applications with Multi-Homing Support (PAMS) Pages and Applications with Multi-Homing Support (PAMS) Pages
1094-1100, ISBN 978-0-7695-4952-1, 1094-1100, ISBN 978-0-7695-4952-1,
DOI 10.1109/WAINA.2013.71, March 2013, DOI 10.1109/WAINA.2013.71, March 2013,
<https://www.simula.no/file/ <https://www.simula.no/file/
threfereedinproceedingsreference2012-12-207643198512pdf/ threfereedinproceedingsreference2012-12-207643198512pdf/
download>. download>.
[29] Dreibholz, T., "NorNet at NICTA - An Introduction to the [RSerPoolPage]
NorNet Testbed", Invited Talk at National Information Dreibholz, T., "Thomas Dreibholz's RSerPool Page",
Communications Technology Australia (NICTA), January 2016, Online: https://www.uni-due.de/~be0001/rserpool/, 2017,
<https://www.simula.no/file/nicta2016-presentationpdf/ <https://www.uni-due.de/~be0001/rserpool/>.
download>.
[30] Dreibholz, T., "An Experiment Tutorial for the NorNet Core [UCLM2017-NorNet-Tutorial]
Dreibholz, T., "An Experiment Tutorial for the NorNet Core
Testbed at the the Universidad de Castilla-La Mancha", Testbed at the the Universidad de Castilla-La Mancha",
Tutorial at the Universidad de Castilla-La Mancha, Tutorial at the Universidad de Castilla-La Mancha,
Instituto de Investigacion Informatica de Albacete, Instituto de Investigacion Informatica de Albacete,
February 2017, <https://www.simula.no/file/uclm2017- February 2017, <https://www.simula.no/file/
nornet-tutorialpdf/download>. uclm2017-nornet-tutorialpdf/download>.
Authors' Addresses Authors' Addresses
Thomas Dreibholz Thomas Dreibholz
Simula Centre for Digital Engineering Simula Metropolitan Centre for Digital Engineering
Martin Linges vei 17 Martin Linges vei 17
1364 Fornebu, Akershus 1364 Fornebu, Akershus
Norway Norway
Phone: +47-6782-8200 Phone: +47-6782-8200
Fax: +47-6782-8201 Fax: +47-6782-8201
Email: dreibh@simula.no Email: dreibh@simula.no
URI: https://www.uni-due.de/~be0001/ URI: https://www.uni-due.de/~be0001/
Michael Tuexen Michael Tuexen
Muenster University of Applied Sciences Muenster University of Applied Sciences
Stegerwaldstrasse 39 Stegerwaldstrasse 39
48565 Steinfurt, Nordrhein-Westfalen 48565 Steinfurt, Nordrhein-Westfalen
Germany Germany
Email: tuexen@fh-muenster.de Email: tuexen@fh-muenster.de
URI: https://www.fh-muenster.de/fb2/personen/professoren/tuexen/
Melinda Shore Melinda Shore
No Mountain Software No Mountain Software
PO Box 16271 PO Box 16271
Two Rivers, Alaska 99716 Two Rivers, Alaska 99716
U.S.A. U.S.A.
Phone: +1-907-322-9522 Phone: +1-907-322-9522
Email: melinda.shore@nomountain.net Email: melinda.shore@nomountain.net
URI: https://www.linkedin.com/pub/melinda-shore/9/667/236 URI: https://www.linkedin.com/pub/melinda-shore/9/667/236
 End of changes. 57 change blocks. 
188 lines changed or deleted 214 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/