draft-ietf-lmap-use-cases-06.txt   rfc7536.txt 
INTERNET-DRAFT Marc Linsner Internet Engineering Task Force (IETF) M. Linsner
Intended Status: Informational Cisco Systems Request for Comments: 7536 Cisco Systems
Expires: August 15, 2015 Philip Eardley Category: Informational P. Eardley
Trevor Burbridge ISSN: 2070-1721 T. Burbridge
BT BT
Frode Sorensen F. Sorensen
Nkom Nkom
February 11, 2015 May 2015
Large-Scale Broadband Measurement Use Cases Large-Scale Broadband Measurement Use Cases
draft-ietf-lmap-use-cases-06
Abstract Abstract
Measuring broadband performance on a large scale is important for Measuring broadband performance on a large scale is important for
network diagnostics by providers and users, as well as for public network diagnostics by providers and users, as well as for public
policy. Understanding the various scenarios and users of measuring policy. Understanding the various scenarios and users of measuring
broadband performance is essential to development of the Large-scale broadband performance is essential to development of the Large-scale
Measurement of Broadband Performance (LMAP) framework, information Measurement of Broadband Performance (LMAP) framework, information
model and protocol. This document details two use cases that can model, and protocol. This document details two use cases that can
assist to developing that framework. The details of the measurement assist in developing that framework. The details of the measurement
metrics themselves are beyond the scope of this document. metrics themselves are beyond the scope of this document.
Status of this Memo Status of This Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months This document is not an Internet Standards Track specification; it is
and may be updated, replaced, or obsoleted by other documents at any published for informational purposes.
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This document is a product of the Internet Engineering Task Force
http://www.ietf.org/1id-abstracts.html (IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
The list of Internet-Draft Shadow Directories can be accessed at Information about the current status of this document, any errata,
http://www.ietf.org/shadow.html and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc7536.
Copyright and License Notice Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the Copyright (c) 2015 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction ....................................................3
2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Use Cases .......................................................3
2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3 2.1. Internet Service Provider (ISP) Use Case ...................3
2.2 Regulator Use Case . . . . . . . . . . . . . . . . . . . . . 4 2.2. Regulator Use Case .........................................4
3 Details of ISP Use Case . . . . . . . . . . . . . . . . . . . . 5 3. Details of ISP Use Case .........................................5
3.1 Understanding the quality experienced by customers . . . . . 5 3.1. Understanding the Quality Experienced by Customers .........5
3.2 Understanding the impact and operation of new devices and 3.2. Understanding the Impact and Operation of New Devices
technology . . . . . . . . . . . . . . . . . . . . . . . . . 6 and Technology .............................................6
3.3 Design and planning . . . . . . . . . . . . . . . . . . . . 6 3.3. Design and Planning ........................................6
3.4 Monitoring Service Level Agreements . . . . . . . . . . . . 7 3.4. Monitoring Service Level Agreements ........................7
3.5 Identifying, isolating and fixing network problems . . . . . 7 3.5. Identifying, Isolating, and Fixing Network Problems ........7
4 Details of Regulator Use Case . . . . . . . . . . . . . . . . . 8 4. Details of Regulator Use Case ...................................8
4.1 Providing transparent performance information . . . . . . . 8 4.1. Providing Transparent Performance Information ..............8
4.2 Measuring broadband deployment . . . . . . . . . . . . . . . 9 4.2. Measuring Broadband Deployment .............................9
4.3 Monitoring traffic management practices . . . . . . . . . . 9 4.3. Monitoring Traffic Management Practices ...................10
6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5. Implementation Options .........................................10
7 Security Considerations . . . . . . . . . . . . . . . . . . . . 13 6. Conclusions ....................................................12
8 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 14 7. Security Considerations ........................................13
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 8. Informative References .........................................15
Informative References . . . . . . . . . . . . . . . . . . . . . . 14 Contributors ......................................................17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17 Authors' Addresses ................................................17
1 Introduction 1. Introduction
This document describes two use cases for the Large-scale Measurement This document describes two use cases for the Large-scale Measurement
of Broadband Performance (LMAP). The use cases contained in this of Broadband Performance (LMAP). The use cases contained in this
document are (1) the Internet Service Provider Use Case and (2) the document are (1) the Internet Service Provider Use Case and (2) the
Regulator Use Case. In the first, a network operator wants to Regulator Use Case. In the first, a network operator wants to
understand the performance of the network and the quality experienced understand the performance of the network and the quality experienced
by customers, whilst in the second, a regulator wants to provide by customers, while in the second, a regulator wants to provide
information on the performance of the ISPs in their jurisdiction. information on the performance of the ISPs in their jurisdiction.
There are other use cases that are not the focus of the initial LMAP There are other use cases that are not the focus of the initial LMAP
work, for example end users would like to use measurements to help work (for example, end users would like to use measurements to help
identify problems in their home network and to monitor the identify problems in their home network and to monitor the
performance of their broadband provider; it is expected that the same performance of their broadband provider); it is expected that the
mechanisms are applicable. same mechanisms are applicable.
Large-scale measurements raise several security concerns, including Large-scale measurements raise several security concerns, including
privacy issues. These are summarized in Section 7 and considered in privacy issues. These are summarized in Section 7 and considered in
further detail in [framework]. further detail in [Framework].
2 Use Cases 2. Use Cases
From the LMAP perspective, there is no difference between fixed From the LMAP perspective, there is no difference between fixed
service and mobile (cellular) service used for Internet access. service and mobile (cellular) service used for Internet access.
Hence, like measurements will take place on both fixed and mobile Hence, like measurements will take place on both fixed and mobile
networks. Fixed services include technologies like Digital networks. Fixed services include technologies like Digital
Subscriber Line (DSL), Cable, and Carrier Ethernet. Mobile services Subscriber Line (DSL), Cable, and Carrier Ethernet. Mobile services
include all those advertised as 2G, 3G, 4G, and Long-Term Evolution include all those advertised as 2G, 3G, 4G, and Long Term Evolution
(LTE). A metric defined to measure end-to-end services will execute (LTE). A metric defined to measure end-to-end services will execute
similarly on all access technologies. Other metrics may be access similarly on all access technologies. Other metrics may be access
technology specific. The LMAP architecture covers both IPv4 and IPv6 technology specific. The LMAP architecture covers both IPv4 and IPv6
networks. networks.
2.1 Internet Service Provider (ISP) Use Case 2.1. Internet Service Provider (ISP) Use Case
A network operator needs to understand the performance of their A network operator needs to understand the performance of their
networks, the performance of the suppliers (downstream and upstream networks, the performance of the suppliers (downstream and upstream
networks), the performance of Internet access services, and the networks), the performance of Internet access services, and the
impact that such performance has on the experience of their impact that such performance has on the experience of their
customers. Largely, the processes that ISPs operate (which are based customers. Largely, the processes that ISPs operate (which are based
on network measurement) include: on network measurement) include:
o Identifying, isolating and fixing problems, which may be in the o Identifying, isolating, and fixing problems, which may be in the
network, with the service provider, or in the end user equipment. network, with the service provider, or in the end-user equipment.
Such problems may be common to a point in the network topology Such problems may be common to a point in the network topology
(e.g. a single exchange), common to a vendor or equipment type (e.g., a single exchange), common to a vendor or equipment type
(e.g. line card or home gateway) or unique to a single user line (e.g., line card or home gateway), or unique to a single user line
(e.g. copper access). Part of this process may also be helping (e.g., copper access). Part of this process may also be helping
users understand whether the problem exists in their home network users understand whether the problem exists in their home network
or with a third party application service instead of with their or with a third-party application service instead of with their
broadband (BB) product. broadband (BB) product.
o Design and planning. Through monitoring the end user experience o Design and planning. Through monitoring the end-user experience,
the ISP can design and plan their network to ensure specified the ISP can design and plan their network to ensure specified
levels of user experience. Services may be moved closer to end levels of user experience. Services may be moved closer to end
users, services upgraded, the impact of QoS assessed or more users, services upgraded, the impact of QoS assessed, or more
capacity deployed at certain locations. Service Level Agreements capacity deployed at certain locations. Service Level Agreements
(SLAs) may be defined at network or product boundaries. (SLAs) may be defined at network or product boundaries.
o Understanding the quality experienced by customers. The network o Understanding the quality experienced by customers. The network
operator would like to gain better insight into the end-to-end operator would like to gain better insight into the end-to-end
performance experienced by its customers. "End-to-end" could, for performance experienced by its customers. "End-to-end" could, for
instance, incorporate home and enterprise networks, and the impact instance, incorporate home and enterprise networks, and the impact
of peering, caching and Content Delivery Networks (CDNs). of peering, caching, and Content Delivery Networks (CDNs).
o Understanding the impact and operation of new devices and o Understanding the impact and operation of new devices and
technology. As a new product is deployed, or a new technology technology. As a new product is deployed, or a new technology
introduced into the network, it is essential that its operation introduced into the network, it is essential that its operation
and its impact is measured. This also helps to quantify the and its impact are measured. This also helps to quantify the
advantage that the new technology is bringing and support the advantage that the new technology is bringing and support the
business case for larger roll-out. business case for larger roll-out.
2.2 Regulator Use Case 2.2. Regulator Use Case
A regulator may want to evaluate the performance of the Internet A regulator may want to evaluate the performance of the Internet
access services offered by operators. access services offered by operators.
While each jurisdiction responds to distinct consumer, industry, and While each jurisdiction responds to distinct consumer, industry, and
regulatory concerns, much commonality exists in the need to produce regulatory concerns, much commonality exists in the need to produce
datasets that can be used to compare multiple Internet access service datasets that can be used to compare multiple Internet access service
providers, diverse technical solutions, geographic and regional providers, diverse technical solutions, geographic and regional
distributions, and marketed and provisioned levels and combinations distributions, and marketed and provisioned levels and combinations
of broadband Internet access services. of broadband Internet access services.
Regulators may want to publish performance measures of different ISPs Regulators may want to publish performance measures of different ISPs
as background information for end users. They may also want to track as background information for end users. They may also want to track
the growth of high-speed broadband deployment, or to monitor the the growth of high-speed broadband deployment, or to monitor the
traffic management practices of Internet providers. traffic management practices of Internet providers.
A regulator's role in the development and enforcement of broadband A regulator's role in the development and enforcement of broadband
Internet access service policies requires that the measurement Internet access service policies requires that the measurement
approaches meet a high level of verifiability, accuracy and provider- approaches meet a high level of verifiability, accuracy, and
independence to support valid and meaningful comparisons of Internet provider-independence to support valid and meaningful comparisons of
access service performance. Standards can help regulators' shared Internet access service performance. Standards can help regulators'
needs for scalable, cost-effective, scientifically robust solutions shared needs for scalable, cost-effective, scientifically robust
to the measurement and collection of broadband Internet access solutions to the measurement and collection of broadband Internet
service performance information. access service performance information.
3 Details of ISP Use Case 3. Details of ISP Use Case
3.1 Understanding the quality experienced by customers 3.1. Understanding the Quality Experienced by Customers
Operators want to understand the quality of experience (QoE) of their Operators want to understand the quality of experience (QoE) of their
broadband customers. The understanding can be gained through a broadband customers. The understanding can be gained through a
"panel", i.e. measurement probes deployed to several customers. A "panel", i.e., measurement probes deployed to several customers. A
probe is a device or piece of software that makes measurements and probe is a device or piece of software that makes measurements and
reports the results, under the control of the measurement system. reports the results, under the control of the measurement system.
Implementation options are discussed in Section 5. The panel needs to Implementation options are discussed in Section 5. The panel needs
include a representative sample of the operator's technologies and to include a representative sample of the operator's technologies and
broadband speeds. For instance it might encompass speeds ranging from broadband speeds. For instance, it might encompass speeds ranging
sub 8Mbps to over 100Mbps. The operator would like the end-to-end from below 8 Mbps to over 100 Mbps. The operator would like the
view of the service, rather than just the access portion. This end-to-end view of the service, rather than just the access portion.
involves relating the pure network parameters to something like a This involves relating the pure network parameters to something like
'mean opinion score' [MOS] which will be service dependent (for a 'mean opinion score' [MOS], which will be service dependent (for
instance web browsing QoE is largely determined by latency above a instance, web-browsing QoE is largely determined by latency above a
few Mb/s). few Mbps).
An operator will also want compound metrics such as "reliability", An operator will also want compound metrics such as "reliability",
which might involve packet loss, DNS failures, re-training of the which might involve packet loss, DNS failures, retraining of the
line, video streaming under-runs etc. line, video streaming under-runs, etc.
The operator really wants to understand the end-to-end service The operator really wants to understand the end-to-end service
experience. However, the home network (Ethernet, WiFi, powerline) is experience. However, the home network (Ethernet, Wi-Fi, powerline)
highly variable and outside its control. To date, operators (and is highly variable and outside its control. To date, operators (and
regulators) have instead measured performance from the home gateway. regulators) have instead measured performance from the home gateway.
However, mobile operators clearly must include the wireless link in However, mobile operators clearly must include the wireless link in
the measurement. the measurement.
Active measurements are the most obvious approach, i.e., special Active measurements are the most obvious approach, i.e., special
measurement traffic is sent by - and to - the probe. In order not to measurement traffic is sent by -- and to -- the probe. In order not
degrade the service of the customer, the measurement data should only to degrade the service of the customer, the measurement data should
be sent when the user is silent, and it shouldn't reduce the only be sent when the user is silent, and it shouldn't reduce the
customer's data allowance. The other approach is passive measurements customer's data allowance. The other approach is passive
on the customer's ordinary traffic; the advantage is that it measures measurements on the customer's ordinary traffic; the advantage is
what the customer actually does, but it creates extra variability that it measures what the customer actually does, but it creates
(different traffic mixes give different results) and especially it extra variability (different traffic mixes give different results)
raises privacy concerns. RFC6973] discusses privacy considerations and, in particular, it raises privacy concerns. [RFC6973] discusses
for Internet protocols in general, whilst [framework] discusses them privacy considerations for Internet protocols in general, while
specifically for large-scale measurement systems. [Framework] discusses them specifically for large-scale measurement
systems.
From an operator's viewpoint, understanding customer experience From an operator's viewpoint, understanding customer experience
enables it to offer better services. Also, simple metrics can be more enables it to offer better services. Also, simple metrics can be
easily understood by senior managers who make investment decisions more easily understood by senior managers who make investment
and by sales and marketing. decisions and by sales and marketing.
3.2 Understanding the impact and operation of new devices and technology 3.2. Understanding the Impact and Operation of New Devices and
Technology
Another type of measurement is to test new capabilities before they Another type of measurement is to test new capabilities before they
are rolled out. For example, the operator may want to: are rolled out. For example, the operator may want to:
o Check whether a customer can be upgraded to a new broadband o Check whether a customer can be upgraded to a new broadband
option option.
o Understand the impact of IPv6 before it is made available to o Understand the impact of IPv6 before it is made available to
customers. Questions such as these could be assessed: will v6 customers. Questions such as these could be assessed: Will v6
packets get through? what will the latency be to major websites? packets get through? What will the latency be to major websites?
what transition mechanisms will be most appropriate? What transition mechanisms will be most appropriate?
o Check whether a new capability can be signaled using TCP options o Check whether a new capability can be signaled using TCP options
(how often it will be blocked by a middlebox? - along the lines of (how often it will be blocked by a middlebox -- along the lines of
the experiments described in [Extend TCP]); the experiments described in [Extend-TCP]).
o Investigate a quality of service mechanism (e.g. checking o Investigate a QoS mechanism (e.g., checking whether Diffserv
whether Diffserv markings are respected on some path); and so on. markings are respected on some path).
3.3 Design and planning 3.3. Design and Planning
Operators can use large scale measurements to help with their network Operators can use large-scale measurements to help with their network
planning - proactive activities to improve the network. planning -- proactive activities to improve the network.
For example, by probing from several different vantage points the For example, by probing from several different vantage points the
operator can see that a particular group of customers has performance operator can see that a particular group of customers has performance
below that expected during peak hours, which should help capacity below that expected during peak hours, which should help with
planning. Naturally operators already have tools to help this - a capacity planning. Naturally, operators already have tools to help
network element reports its individual utilization (and perhaps other with this -- a network element reports its individual utilization
parameters). However, making measurements across a path rather than (and perhaps other parameters). However, making measurements across
at a point may make it easier to understand the network. There may a path rather than at a point may make it easier to understand the
also be parameters like bufferbloat that aren't currently reported by network. There may also be parameters like bufferbloat that aren't
equipment and/or that are intrinsically path metrics. currently reported by equipment and/or that are intrinsically path
metrics.
With information gained from measurement results, capacity planning With information gained from measurement results, capacity planning
and network design can be more effective. Such planning typically and network design can be more effective. Such planning typically
uses simulations to emulate the measured performance of the current uses simulations to emulate the measured performance of the current
network and understand the likely impact of new capacity and network and understand the likely impact of new capacity and
potential changes to the topology. Simulations, informed by data from potential changes to the topology. Simulations, informed by data
a limited panel of probes, can help quantify the advantage that a new from a limited panel of probes, can help quantify the advantage that
technology brings and support the business case for larger roll-out. a new technology brings and support the business case for larger
roll-out.
It may also be possible to use probes to run stress tests for risk It may also be possible to use probes to run stress tests for risk
analysis. For example, an operator could run a carefully controlled analysis. For example, an operator could run a carefully controlled
and limited experiment in which probing is used to assess the and limited experiment in which probing is used to assess the
potential impact if some new application becomes popular. potential impact if some new application becomes popular.
3.4 Monitoring Service Level Agreements 3.4. Monitoring Service Level Agreements
Another example is that the operator may want to monitor performance Another example is that the operator may want to monitor performance
where there is a service level agreement (SLA). This could be with where there is a Service Level Agreement (SLA). This could be with
its own customers, especially enterprises may have an SLA. The its own customers; in particular, enterprises may have an SLA. The
operator can proactively spot when the service is degrading near to operator can proactively spot when the service is degrading near the
the SLA limit, and get information that will enable more informed point of the SLA limit and get information that will enable more
conversations with the customer at contract renewal. informed conversations with the customer at contract renewal.
An operator may also want to monitor the performance of its An operator may also want to monitor the performance of its
suppliers, to check whether they meet their SLA or to compare two suppliers, to check whether they meet their SLA or to compare two
suppliers if it is dual-sourcing. This could include its transit suppliers if it is dual-sourcing. This could include its transit
operator, CDNs, peering, video source, local network provider (for a operator, CDNs, peering, video source, or local network provider for
global operator in countries where it doesn't have its own network), a global operator in countries where it doesn't have its own network.
even the whole network for a virtual operator. A virtual operator may monitor the whole underlying network.
Through a better understanding of its own network and its suppliers, Through a better understanding of its own network and its suppliers,
the operator should be able to focus investment more effectively - in the operator should be able to focus investment more effectively --
the right place at the right time with the right technology. in the right place at the right time with the right technology.
3.5 Identifying, isolating and fixing network problems 3.5. Identifying, Isolating, and Fixing Network Problems
Operators can use large scale measurements to help identify a fault Operators can use large-scale measurements to help identify a fault
more rapidly and decide how to solve it. more rapidly and decide how to solve it.
Operators already have Test and Diagnostic tools, where a network Operators already have Test and Diagnostic tools, where a network
element reports some problem or failure to a management system. element reports some problem or failure to a management system.
However, many issues are not caused by a point failure but something However, many issues are not caused by a point failure but something
wider and so will trigger too many alarms, whilst other issues will wider and so will trigger too many alarms, while other issues will
cause degradation rather than failure and so not trigger any alarm. cause degradation rather than failure and so not trigger any alarm.
Large-scale measurements can help provide a more nuanced view that Large-scale measurements can help provide a more nuanced view that
helps network management to identify and fix problems more rapidly helps network management to identify and fix problems more rapidly
and accurately. The network management tools may use simulations to and accurately. The network management tools may use simulations to
emulate the network and so help identify a fault and assess possible emulate the network and so help identify a fault and assess possible
solutions. solutions.
An operator can obtain useful information without measuring the An operator can obtain useful information without measuring the
performance on every broadband line. By measuring a subset, the performance on every broadband line. By measuring a subset, the
operator can identify problems that affect a group of customers. For operator can identify problems that affect a group of customers. For
example, the issue could be at a shared point in the network topology example, the issue could be at a shared point in the network topology
(such as an exchange), or common to a vendor, or equipment type; for (such as an exchange), or common to a vendor, or equipment type; for
instance, [IETF85-Plenary] describes a case where a particular home instance, [IETF85-Plenary] describes a case where a particular home
gateway upgrade had caused a (mistaken!) drop in line rate. gateway upgrade had caused a (mistaken!) drop in line rate.
A more extensive deployment of the measurement capability to every A more extensive deployment of the measurement capability to every
broadband line would enable an operator to identify issues unique to broadband line would enable an operator to identify issues unique to
a single customer. Overall, large-scale measurements can help an a single customer. Overall, large-scale measurements can help an
operator help an operator fix the fault more rapidly and/or allow the operator fix the fault more rapidly and/or allow the affected
affected customers to be informed what's happening. More accurate customers to be informed of what's happening. More accurate
information enables the operator to reassure customers and take more information enables the operator to reassure customers and take more
rapid and effective action to cure the problem. rapid and effective action to cure the problem.
Often customers experience poor broadband due to problems in the home Often, customers experience poor broadband due to problems in the
network - the ISP's network is fine. For example they may have moved home network -- the ISP's network is fine. For example, they may
too far away from their wireless access point. Anecdotally, a large have moved too far away from their wireless access point.
fraction of customer calls about fixed BB problems are due to in-home Anecdotally, a large fraction of customer calls about fixed BB
wireless issues. These issues are expensive and frustrating for an problems are due to in-home wireless issues. These issues are
operator, as they are extremely hard to diagnose and solve. The expensive and frustrating for an operator, as they are extremely hard
operator would like to narrow down whether the problem is in the home to diagnose and solve. The operator would like to narrow down
(with the home network or edge device or home gateway), in the whether the problem is in the home (a problem with the home network,
operator's network, or with an application service. The operator edge device, or home gateway), in the operator's network, or with an
would like two capabilities. Firstly, self-help tools that customers application service. The operator would like two capabilities:
use to improve their own service or understand its performance firstly, self-help tools that customers use to improve their own
better, for example to re-position their devices for better WiFi service or understand its performance better -- for example, to
coverage. Secondly, on-demand tests that can the operator can run reposition their devices for better Wi-Fi coverage; and secondly,
instantly - so the call center person answering the phone (or e-chat) on-demand tests that the operator can run instantly, so that the call
could trigger a test and get the result whilst the customer is still center person answering the phone (or e-chat) could trigger a test
in an on-line session. and get the result while the customer is still in an online session.
4 Details of Regulator Use Case 4. Details of Regulator Use Case
4.1 Providing transparent performance information 4.1. Providing Transparent Performance Information
Some regulators publish information about the quality of the various Some regulators publish information about the quality of the various
Internet access services provided in their national market. Quality Internet access services provided in their national market. Quality
information about service offers could include speed, delay, and information about service offers could include speed, delay, and
jitter. Such information can be published to facilitate end users' jitter. Such information can be published to facilitate end users'
choice of service provider and offer. Regulators may also check the choice of service provider and offer. Regulators may check the
accuracy of the marketing claims of Internet service providers, and accuracy of the marketing claims of Internet service providers and
may also encourage ISPs all to use the same metrics in their service may also encourage ISPs to all use the same metrics in their service
level contracts. The goal with these transparency mechanisms is to level contracts. The goal of these transparency mechanisms is to
promote competition for end users and potentially also help content, promote competition for end users and potentially also help content,
application, service and device providers develop their Internet application, service, and device providers develop their Internet
offerings. offerings.
The published information needs to be: The published information needs to be:
o Accurate - the measurement results must be correct and not o Accurate - the measurement results must be correct and not
influenced by errors or side effects. The results should be influenced by errors or side effects. The results should be
reproducible and consistent over time. reproducible and consistent over time.
o Comparable - common metrics should be used across different ISPs o Comparable - common metrics should be used across different ISPs
and service offerings, and over time, so that measurement results and service offerings, and over time, so that measurement results
can be compared. can be compared.
o Meaningful - the metrics used for measurements need to reflect o Meaningful - the metrics used for measurements need to reflect
what end users value about their broadband Internet access what end users value about their broadband Internet access
service. service.
o Reliable - the number and distribution of measurement agents, o Reliable - the number and distribution of measurement agents, and
and the statistical processing of the raw measurement data, needs the statistical processing of the raw measurement data, need to be
to be appropriate. appropriate.
In practical terms, the regulators may measure network performance In practical terms, the regulators may measure network performance
from users towards multiple content and application providers, from users towards multiple content and application providers,
including dedicated test measurement servers. Measurement probes are including dedicated test measurement servers. Measurement probes are
distributed to a 'panel' of selected end users. The panel covers all distributed to a 'panel' of selected end users. The panel covers all
the operators and packages in the market, spread over urban, suburban the operators and packages in the market, spread over urban,
and rural areas, and often includes both fixed and mobile Internet suburban, and rural areas, and often includes both fixed and mobile
access. Periodic tests running on the probes can for example measure Internet access. Periodic tests running on the probes can, for
actual speed at peak and off-peak hours, but also other detailed example, measure actual speed at peak and off-peak hours, but can
quality metrics like delay and jitter. Collected data goes afterwards also measure other detailed quality metrics like delay and jitter.
through statistical analysis, deriving estimates for the whole Collected data goes afterwards through statistical analysis, deriving
population. Summary information, such as a service quality index, is estimates for the whole population. Summary information, such as a
published regularly, perhaps alongside more detailed information. service quality index, is published regularly, perhaps alongside more
detailed information.
The regulator can also facilitate end users to monitor the The regulator can also facilitate end users to monitor the
performance of their own broadband Internet access service. They performance of their own broadband Internet access service. They
might use this information to check that the performance meets that might use this information to check that the performance meets that
specified in their contract or to understand whether their current specified in their contract or to understand whether their current
subscription is the most appropriate. subscription is the most appropriate.
4.2 Measuring broadband deployment 4.2. Measuring Broadband Deployment
Regulators may also want to monitor the improvement through time of Regulators may also want to monitor the improvement over time of
actual broadband Internet access performance in a specific country or actual broadband Internet access performance in a specific country or
a region. The motivation is often to evaluate the effect of the a region. The motivation is often to evaluate the effect of the
stimulated growth over time, when government has set a strategic goal stimulated growth over time, when government has set a strategic goal
for high-speed broadband deployment, whether in absolute terms or for high-speed broadband deployment, whether in absolute terms or
benchmarked against other countries. An example of such an initiative benchmarked against other countries. An example of such an
is [DAE]. The actual measurements can be made in the same way as initiative is [DAE]. The actual measurements can be made in the same
described in Section 4.1. way as described in Section 4.1.
4.3 Monitoring traffic management practices 4.3. Monitoring Traffic Management Practices
A regulator may want to monitor traffic management practices or A regulator may want to monitor traffic management practices or
compare the performance of Internet access service with specialized compare the performance of Internet access service with specialized
services offered in parallel to but separate from Internet access services offered in parallel to, but separate from, Internet access
service (for example IPTV). A regulator could monitor for service (for example, IPTV). A regulator could monitor for
departures from application agnosticism such as blocking or departures from application agnosticism such as blocking or
throttling of traffic from specific applications, or preferential throttling of traffic from specific applications, or preferential
treatment of specific applications. A measurement system could send, treatment of specific applications. A measurement system could send,
or passively monitor, application-specific traffic and then measure or passively monitor, application-specific traffic and then measure
in detail the transfer of the different packets. Whilst it is in detail the transfer of the different packets. While it is
relatively easy to measure port blocking, it is a research topic how relatively easy to measure port blocking, how to detect other types
to detect other types of differentiated treatment. The paper, of differentiated treatment is a research topic in itself. The
"Glasnost: Enabling End Users to Detect Traffic Differentiation" [M- "Glasnost: Enabling End Users to Detect Traffic Differentiation"
Labs NSDI 2010] and follow-on tool "Glasnost" [Glasnost] is an paper [M-Labs_NSDI-2010] and follow-on tool "Glasnost" [Glasnost]
example of work in this area. provide an example of work in this area.
A regulator could also monitor the performance of the broadband A regulator could also monitor the performance of the broadband
service over time, to try and detect if the specialized service is service over time, to try and detect if the specialized service is
provided at the expense of the Internet access service. Comparison provided at the expense of the Internet access service. Comparison
between ISPs or between different countries may also be relevant for between ISPs or between different countries may also be relevant for
this kind of evaluation. this kind of evaluation.
The motivation for a regulator monitoring such traffic management The motivation for a regulator monitoring such traffic management
practices is that regulatory approaches related to net neutrality and practices is that regulatory approaches related to net neutrality and
the open Internet have been introduced in some jurisdictions. the open Internet have been introduced in some jurisdictions.
Examples of such efforts are the Internet policy as outlined by the Examples of such efforts are the Internet policy as outlined by the
Body of European Regulators for Electronic Communications Guidelines Body of European Regulators for Electronic Communications guidelines
for quality of service [BEREC Guidelines] and US FCC Preserving the for quality of service [BEREC-Guidelines] and the US FCC's
Open Internet Report and Order [FCC R&O]. Although legal challenges "Preserving the Open Internet" Report and Order [FCC-R&O]. Although
can change the status of policy, the take-away for LMAP purposes is legal challenges can change the status of policy, the take-away for
that policy-makers are looking for measurement solutions to assist LMAP purposes is that policy-makers are looking for measurement
them in discovering biased treatment of traffic flows. The exact solutions to assist them in discovering biased treatment of traffic
definitions and requirements vary from one jurisdiction to another. flows. The exact definitions and requirements vary from one
jurisdiction to another.
5 Implementation Options 5. Implementation Options
There are several ways of implementing a measurement system. The There are several ways of implementing a measurement system. The
choice may be influenced by the details of the particular use case choice may be influenced by the details of the particular use case
and what the most important criteria are for the regulator, ISP or and what the most important criteria are for the regulator, ISP, or
third party operating the measurement system. third party operating the measurement system.
One type of probe is a special hardware device that is connected One type of probe is a special hardware device that is connected
directly to the home gateway. The devices are deployed to a carefully directly to the home gateway. The devices are deployed to a
selected panel of end users and they perform measurements according carefully selected panel of end users, and they perform measurements
to a defined schedule. The schedule can run throughout the day, to according to a defined schedule. The schedule can run throughout the
allow continuous assessment of the network. Careful design ensures day, to allow continuous assessment of the network. Careful design
that measurements do not detrimentally impact the home user ensures that measurements do not detrimentally impact the home user
experience or corrupt the results by testing when the user is also experience or corrupt the results by testing when the user is also
using the broadband line. The system is therefore tightly controlled using the broadband line. The system is therefore tightly controlled
by the operator of the measurement system. One advantage of this by the operator of the measurement system. One advantage of this
approach is that it is possible to get reliable benchmarks for the approach is that it is possible to get reliable benchmarks for the
performance of a network with only a few devices. One disadvantage is performance of a network with only a few devices. One disadvantage
that it would be expensive to deploy hardware devices on a mass scale is that it would be expensive to deploy hardware devices on a mass
sufficient to understand the performance of the network at the scale sufficient to understand the performance of the network at the
granularity of a single broadband user. granularity of a single broadband user.
Another type of probe involves implementing the measurement Another type of probe involves implementing the measurement
capability as a webpage or an "app" that end users are encouraged to capability as a webpage or an "app" that end users are encouraged to
download onto their mobile phone or computing device. Measurements download onto their mobile phone or computing device. Measurements
are triggered by the end user, for example the user interface may are triggered by the end user; for example, the user interface may
have a button to "test my broadband now". One advantage of this have a button to "test my broadband now." One advantage of this
approach is that the performance is measured to the end user, rather approach is that the performance is measured to the end user, rather
than to the home gateway, and so includes the home network. Another than to the home gateway, and so includes the home network. Another
difference is that the system is much more loosely controlled, as the difference is that the system is much more loosely controlled, as the
panel of end users and the schedule of tests are determined by the panel of end users and the schedule of tests are determined by the
end users themselves rather than the measurement system. It would be end users themselves rather than the measurement system. While this
easier to get large-scale, however it is harder to get comparable approach makes it easier to make measurements on a large scale, it is
benchmarks as the measurements are affected by the home network and harder to get comparable benchmarks, as the measurements are affected
also the population is self-selecting and so potentially biased by the home network; also, the population is self-selecting and so
towards those who think they have a problem. This could be alleviated potentially biased towards those who think they have a problem. This
by stimulating widespread downloading of the app and careful post- could be alleviated by encouraging widespread downloading of the app
processing of the results to reduce biases. and careful post-processing of the results to reduce biases.
There are several other possibilities. For example, as a variant on There are several other possibilities. For example, as a variant on
the first approach, the measurement capability could be implemented the first approach, the measurement capability could be implemented
as software embedded in the home gateway, which would make it more as software embedded in the home gateway, which would make it more
viable to have the capability on every user line. As a variant on the viable to have the capability on every user line. As a variant on
second approach, the end user could initiate measurements in response the second approach, the end user could initiate measurements in
to a request from the measurement system. response to a request from the measurement system.
The operator of the measurement system should be careful to ensure The operator of the measurement system should be careful to ensure
that measurements do not detrimentally impact users. Potential issues that measurements do not detrimentally impact users. Potential
include: issues include the following:
* Measurement traffic generated on a particular user's line may * Measurement traffic generated on a particular user's line may
impact that end user's quality of experience. The danger is impact that end user's quality of experience. The danger is
greater for measurements that generate a lot of traffic over a greater for measurements that generate a lot of traffic over a
lengthy period. lengthy period.
* The measurement traffic may impact that particular user's bill * The measurement traffic may impact that particular user's bill or
or traffic cap. traffic cap.
* The measurement traffic from several end users may, in * The measurement traffic from several end users may, in
combination, congest a shared link. combination, congest a shared link.
* The traffic associated with the control and reporting of * The traffic associated with the control and reporting of
measurements may overload the network. The danger is greater where measurements may overload the network. The danger is greater
the traffic associated with many end users is synchronized. where the traffic associated with many end users is synchronized.
6 Conclusions 6. Conclusions
Large-scale measurements of broadband performance are useful for both Large-scale measurements of broadband performance are useful for both
network operators and regulators. Network operators would like to use network operators and regulators. Network operators would like to
measurements to help them better understand the quality experienced use measurements to help them better understand the quality
by their customers, identify problems in the network and design experienced by their customers, identify problems in the network, and
network improvements. Regulators would like to use measurements to design network improvements. Regulators would like to use
help promote competition between network operators, stimulate the measurements to help promote competition between network operators,
growth of broadband access and monitor 'net neutrality'. There are stimulate the growth of broadband access, and monitor 'net
other use cases that are not the focus of the initial LMAP charter neutrality'. There are other use cases that are not the focus of the
(although it is expected that the mechanisms developed would be initial LMAP charter (although it is expected that the mechanisms
readily applied), for example end users would like to use developed would be readily applied); for example, end users would
measurements to help identify problems in their home network and to like to use measurements to help identify problems in their home
monitor the performance of their broadband provider. network and to monitor the performance of their broadband provider.
From consideration of the various use cases, several common themes From consideration of the various use cases, several common themes
emerge whilst there are also some detailed differences. These emerge, while there are also some detailed differences. These
characteristics guide the development of LMAP's framework, characteristics guide the development of LMAP's framework,
information model and protocol. information model, and protocol.
A measurement capability is needed across a wide number of A measurement capability is needed across a wide number of
heterogeneous environments. Tests may be needed in the home network, heterogeneous environments. Tests may be needed in the home network,
in the ISP's network or beyond; they may be measuring a fixed or in the ISP's network, or beyond; they may be measuring a fixed or
wireless network; they may measure just the access network or across wireless network; they may measure just the access network or across
several networks; at least some of which are not operated by the several networks.
measurement provider.
There is a role for both standardized and non-standardized There is a role for both standardized and non-standardized
measurements. For example, a regulator would like to publish measurements. For example, a regulator would like to publish
standardized performance metrics for all network operators, whilst an standardized performance metrics for all network operators, while an
ISP may need their own tests to understand some feature special to ISP may need their own tests to understand some feature special to
their network. Most use cases need active measurements, which create their network. Most use cases need active measurements, which create
and measure specific test traffic, but some need passive measurements and measure specific test traffic, but some need passive measurements
of the end user's traffic. of the end user's traffic.
Regardless of the tests being operated, there needs to be a way to Regardless of the tests being operated, there needs to be a way to
demand or schedule the tests. Most use cases need a regular schedule demand or schedule the tests. Most use cases need a regular schedule
of measurements, but sometimes ad hoc testing is needed, for example of measurements, but sometimes ad hoc testing is needed -- for
for troubleshooting. It needs to be ensured that measurements do not example, for troubleshooting. It needs to be ensured that
affect the user experience and are not affected by user traffic measurements do not affect the user experience and are not affected
(unless desired). In addition there needs to be a common way to by user traffic (unless desired). In addition, there needs to be a
collect the results. Standardization of this control and reporting common way to collect the results. Standardization of this control
functionality allows the operator of a measurement system to buy the and reporting functionality allows the operator of a measurement
various components from different vendors. system to buy the various components from different vendors.
After the measurement results are collected, they need to be After the measurement results are collected, they need to be
understood and analyzed. Often it is sufficient to measure only a understood and analyzed. Often, it is sufficient to measure only a
small subset of end users, but per-line fault diagnosis requires the small subset of end users, but per-line fault diagnosis requires the
ability to test every individual line. Analysis requires accurate ability to test every individual line. Analysis requires accurate
definition and understanding of where the test points are, as well as definition and understanding of where the test points are, as well as
contextual information about the topology, line, product and the contextual information about the topology, line, product, and the
subscriber's contract. The actual analysis of results is beyond the subscriber's contract. The actual analysis of results is beyond the
scope of LMAP, as is the key challenge of how to integrate the scope of LMAP, as is the key challenge of how to integrate the
measurement system into a network operator's existing tools for measurement system into a network operator's existing tools for
diagnostics and network planning. diagnostics and network planning.
Finally the test data, along with any associated network, product or Finally, the test data, along with any associated network, product,
subscriber contract data is commercial or private information and or subscriber contract data, is commercial or private information and
needs to be protected. needs to be protected.
7 Security Considerations 7. Security Considerations
Large-scale measurements raise several potential security, privacy Large-scale measurements raise several potential security, privacy
(data protection) [RFC6973] and business sensitivity issues. (data protection) [RFC6973], and business sensitivity issues:
1. a malicious party may try to gain control of probes to launch 1. A malicious party may try to gain control of probes to launch DoS
DoS (Denial of Service) attacks at a target. A DoS attack could be (Denial of Service) attacks at a target. A DoS attack could be
targeted at a particular end user or set of end users, a certain targeted at a particular end user or set of end users, a certain
network, or a specific service provider. network, or a specific service provider.
2. a malicious party may try to gain control of probes to create a 2. A malicious party may try to gain control of probes to create a
platform for pervasive monitoring [RFC7258], or for more targeted platform for pervasive monitoring [RFC7258] or for more targeted
monitoring. [RFC7258] summarises the threats as: "an attack may monitoring. [RFC7258] summarizes the threats as follows: "An
change the content of the communication, record the content or attack may change the content of the communication, record the
external characteristics of the communication, or through content or external characteristics of the communication, or
correlation with other communication events, reveal information through correlation with other communication events, reveal
the parties did not intend to be revealed." For example, a information the parties did not intend to be revealed." For
malicious party could distribute to the probes a new measurement example, a malicious party could distribute to the probes a new
test that recorded (and later reported) information of maleficent measurement test that recorded (and later reported) information of
interest. Similar concerns also arise if the measurement results maleficent interest. Similar concerns also arise if the
are intercepted or corrupted. measurement results are intercepted or corrupted.
* from the end user's perspective, the concerns include a * From the end user's perspective, the concerns include a
malicious party monitoring the traffic they send and receive, malicious party monitoring the traffic they send and receive,
who they communicate with and the websites they visit, and who they communicate with, the websites they visit, and such
information about their behaviour such as when they are at home information about their behavior as when they are at home and
and the location of their devices. Some of the concerns may be the location of their devices. Some of the concerns may be
greater when the MA is on the end user's device rather than on greater when the probe is on the end user's device rather than
their home gateway. on their home gateway.
* from the network operator's perspective, the concerns include * From the network operator's perspective, the concerns include
the leakage of commercially-sensitive information about the the leakage of commercially sensitive information about the
design and operation of their network, their customers and design and operation of their network, their customers, and
suppliers. Some threats are indirect, for example the attacker suppliers. Some threats are indirect; for example, the
could reconnoitre potential weaknesses, such as open ports and attacker could reconnoiter potential weaknesses, such as open
paths through the network, which enabled it to launch an attack ports and paths through the network, which enabled it to launch
later. an attack later.
* from the regulator's perspective, the concerns include * From the regulator's perspective, the concerns include
distortion of the measurement tests or alteration of the distortion of the measurement tests or alteration of the
measurement results. Also, a malicious network operator could measurement results. Also, a malicious network operator could
try to identify the broadband lines that the regulator was try to identify the broadband lines that the regulator was
measuring and prioritise that traffic ("game the system"). measuring and prioritize that traffic ("game the system").
3. a measurement system that does not obtain the end user's 3. Another potential issue is a measurement system that does not
informed consent, or fails to specify a specific purpose in the obtain the end user's informed consent, fails to specify a
consent, or uses the collected information for secondary uses specific purpose in the consent, or uses the collected information
beyond those specified. for secondary uses beyond those specified.
4. a measurement system that does not indicate who is responsible 4. Another potential issue is a measurement system that does not
for the collection and processing of personal data and who is indicate who is responsible for the collection and processing of
responsible for fulfilling the rights of users. The responsible personal data and who is responsible for fulfilling the rights of
party (often termed the "data controller") should, as good users. The responsible party (often termed the "data controller")
practice, consider issues such as defining:- the purpose for which should, as good practice, consider such issues as defining:
the data is collected and used; how the data is stored, accessed,
and processed; how long it is retained for; and how the end user o the purpose for which the data is collected and used,
can view, update, and even delete their personal data. If
anonymized personal data is shared with a third party, the data o how the data is stored, accessed, and processed,
o how long the data is retained, and
o how the end user can view, update, and even delete their
personal data.
If anonymized personal data is shared with a third party, the data
controller should consider the possibility that the third party controller should consider the possibility that the third party
can de-anonymize it by combining it with other information. can de-anonymize it by combining it with other information.
These security and privacy issues will need to be considered These security and privacy issues will need to be considered
carefully by any measurement system. In the context of LMAP, the carefully by any measurement system. In the context of LMAP,
[framework] considers them further along with some potential [Framework] considers them further, along with some potential
mitigations. Other LMAP documents will specify protocol(s) that mitigations. Other LMAP documents will specify one or more protocols
enable the measurement system to instruct a probe about what that enable the measurement system to instruct a probe about what
measurements to make and that enable the probe to report the measurements to make and that enable the probe to report the
measurement results. Those documents will need to discuss solutions measurement results. Those documents will need to discuss solutions
to the security and privacy issues. However, the protocol documents to the security and privacy issues. However, the protocol documents
will not consider the actual usage of the measurement information; will not consider the actual usage of the measurement information.
many use cases can be envisaged and, earlier in this document, we Many use cases can be envisaged, and earlier in this document we
have described some likely ones for the network operator and described some likely ones for the network operator and regulator.
regulator.
8 IANA Considerations
None
Contributors
The information in this document is partially derived from text 8. Informative References
written by the following contributors:
James Miller jamesmilleresquire@gmail.com [IETF85-Plenary]
Crawford, S., "Large-Scale Active Measurement of Broadband
Networks", 'example' from slide 18, November 2012,
<http://www.ietf.org/proceedings/85/slides/
slides-85-iesg-opsandtech-7.pdf>.
Rachel Huang rachel.huang@huawei.com [Extend-TCP]
Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A.,
Handley, M., and H. Tokuda, "Is it Still Possible to
Extend TCP?", Proceedings of IETF 82, November 2011,
<http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf>.
Informative References [Framework]
Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
Aitken, P., and A. Akhter, "A framework for Large-Scale
Measurement of Broadband Performance (LMAP)", Work in
Progress, draft-ietf-lmap-framework-14, April 2015.
[IETF85-Plenary] Crawford, S., "Large-Scale Active Measurement of [RFC6973] Cooper, A., Tschofenig, H., Aboba, B., Peterson, J.,
Broadband Networks", Morris, J., Hansen, M., and R. Smith, "Privacy
http://www.ietf.org/proceedings/85/slides/slides-85-iesg- Considerations for Internet Protocols", RFC 6973,
opsandtech-7.pdf 'example' from slide 18 July 2013, <http://www.rfc-editor.org/info/rfc6973>.
[Extend TCP] Michio Honda, Yoshifumi Nishida, Costin Raiciu, Adam [RFC7258] Farrell, S. and H. Tschofenig, "Pervasive Monitoring Is an
Greenhalgh, Mark Handley and Hideyuki Tokuda. "Is it Still Attack", BCP 188, RFC 7258, May 2014,
Possible to Extend TCP?" Proc. ACM Internet Measurement <http://www.rfc-editor.org/info/rfc7258>.
Conference (IMC), November 2011, Berlin, Germany.
http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf
[framework] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., [FCC-R&O] United States Federal Communications Commission,
Aitken, P., Akhter, A. "A framework for large-scale "Preserving the Open Internet; Broadband Industries
measurement platforms (LMAP)", Practices: Report and Order", FCC 10-201, December 2010,
http://datatracker.ietf.org/doc/draft-ietf-lmap-framework/ <http://hraunfoss.fcc.gov/edocs_public/attachmatch/
FCC-10-201A1.pdf>.
[RFC6973] Cooper, A., Tschofenig, H.z., Aboba, B., Peterson, J., [BEREC-Guidelines]
Morris, J., Hansen, M., and R. Smith, "Privacy Body of European Regulators for Electronic Communications,
Considerations for Internet Protocols", RFC 6973, July "BEREC Guidelines for quality of service in the scope of
2013. net neutrality", <http://berec.europa.eu/eng/
document_register/subject_matter/berec/download/0/
1101-berec-guidelines-for-quality-of-service-_0.pdf>.
[RFC7258] Farrell, S., Tschofenig, H., "PPervasive Monitoring Is an [M-Labs_NSDI-2010]
Attack", RFC 7258, May 2014. M-Lab, "Glasnost: Enabling End Users to Detect Traffic
Differentiation", <http://www.measurementlab.net/
download/AMIfv945ljiJXzG-fgUrZSTu2hs1xRl5Oh-
rpGQMWL305BNQh-BSq5oBoYU4a7zqXOvrztpJhK9gwk5unOe-
fOzj4X-vOQz_HRrnYU-aFd0rv332RDReRfOYkJuagysstN3GZ__lQHTS8_
UHJTWkrwyqIUjffVeDxQ/>.
[FCC R&O] United States Federal Communications Commission, 10-201, [Glasnost] M-Lab tool "Glasnost", <http://mlab-live.appspot.com/
"Preserving the Open Internet, Broadband Industries tools/glasnost>.
Practices, Report and Order",
http://hraunfoss.fcc.gov/edocs_public/attachmatch/FCC-10-
201A1.pdf
[BEREC Guidelines] Body of European Regulators for Electronic [MOS] Wikipedia, "Mean Opinion Score", January 2015,
Communications, "BEREC Guidelines for quality of service <http://en.wikipedia.org/w/index.php?
in the scope of net neutrality", title=Mean_opinion_score&oldid=644494161>.
http://berec.europa.eu/eng/document_register/
subject_matter/berec/download/0/1101-berec-guidelines-for-
quality-of-service-_0.pdf
[M-Labs NSDI 2010] M-Lab, "Glasnost: Enabling End Users to Detect [DAE] Digital Agenda for Europe, COM(2010)245 final,
Traffic Differentiation", "Communication from the Commission to the European
http://www.measurementlab.net/download/AMIfv945ljiJXzG- Parliament, the Council, the European Economic and Social
fgUrZSTu2hs1xRl5Oh-rpGQMWL305BNQh- Committee and the Committee of the Regions",
BSq5oBoYU4a7zqXOvrztpJhK9gwk5unOe-fOzj4X-vOQz_HRrnYU- <http://eur-lex.europa.eu/legal-content/EN/TXT/
aFd0rv332RDReRfOYkJuagysstN3GZ__lQHTS8_UHJTWkrwyqIUjffVeDxQ/ PDF/?uri=CELEX:52010DC0245&from=EN>.
[Glasnost] M-Lab tool "Glasnost", http://mlab-live.appspot.com/tools/ Contributors
glasnost
[P.800] ITU-T, "SERIES P: TELEPHONE TRANSMISSION QUALITY Methods for The information in this document is partially derived from text
objective and subjective assessment of quality", written by the following contributors:
https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-
P.800-199608-I!!PDF-E&type=items
[MOS] Wikipedia, "Mean Opinion Score", James Miller jamesmilleresquire@gmail.com
http://en.wikipedia.org/wiki/Mean_opinion_score
[DAE] Digital Agenda for Europe, COM(2010)245 final, Communication Rachel Huang rachel.huang@huawei.com
from the Commission to the European Parliament, the
Council, the European Economic and Social Committee and
the Committee of the Regions, http://eur-
lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52010DC0245&from=EN
Authors' Addresses Authors' Addresses
Marc Linsner Marc Linsner
Cisco Systems, Inc. Cisco Systems, Inc.
Marco Island, FL Marco Island, FL
USA United States
EMail: mlinsner@cisco.com EMail: mlinsner@cisco.com
Philip Eardley Philip Eardley
BT BT
B54 Room 77, Adastral Park, Martlesham B54 Room 77, Adastral Park, Martlesham
Ipswich, IP5 3RE Ipswich, IP5 3RE
UK United Kingdom
Email: philip.eardley@bt.com EMail: philip.eardley@bt.com
Trevor Burbridge Trevor Burbridge
BT BT
B54 Room 77, Adastral Park, Martlesham B54 Room 70, Adastral Park, Martlesham
Ipswich, IP5 3RE Ipswich, IP5 3RE
UK United Kingdom
Email: trevor.burbridge@bt.com EMail: trevor.burbridge@bt.com
Frode Sorensen Frode Sorensen
Norwegian Communications Authority (Nkom) Norwegian Communications Authority (Nkom)
Lillesand Lillesand
Norway Norway
Email: frode.sorensen@nkom.no EMail: frode.sorensen@nkom.no
 End of changes. 155 change blocks. 
449 lines changed or deleted 450 lines changed or added

This html diff was produced by rfcdiff 1.42. The latest version is available from http://tools.ietf.org/tools/rfcdiff/