draft-ietf-lmap-use-cases-03.txt   draft-ietf-lmap-use-cases-04.txt 
INTERNET-DRAFT Marc Linsner INTERNET-DRAFT Marc Linsner
Intended Status: Informational Cisco Systems Intended Status: Informational Cisco Systems
Expires: October 4, 2014 Philip Eardley Expires: March 27, 2015 Philip Eardley
Trevor Burbridge Trevor Burbridge
BT BT
Frode Sorensen Frode Sorensen
NPT NPT
April 2, 2014 September 23, 2014
Large-Scale Broadband Measurement Use Cases Large-Scale Broadband Measurement Use Cases
draft-ietf-lmap-use-cases-03 draft-ietf-lmap-use-cases-04
Abstract Abstract
Measuring broadband performance on a large scale is important for Measuring broadband performance on a large scale is important for
network diagnostics by providers and users, as well as for public network diagnostics by providers and users, as well as for public
policy. Understanding the various scenarios and users of measuring policy. Understanding the various scenarios and users of measuring
broadband performance is essential to development of the framework, broadband performance is essential to development of the Large-scale
information model and protocol. This document details two use cases Measurement of Broadband Performance (LMAP) framework, information
that can assist to developing that framework. The details of the model and protocol. This document details two use cases that can
measurement metrics themselves are beyond the scope of this document. assist to developing that framework. The details of the measurement
metrics themselves are beyond the scope of this document.
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as other groups may also distribute working documents as
Internet-Drafts. Internet-Drafts.
skipping to change at page 2, line 25 skipping to change at page 2, line 25
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3 2.1 Internet Service Provider (ISP) Use Case . . . . . . . . . . 3
2.2 Regulators . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Regulator Use Case . . . . . . . . . . . . . . . . . . . . . 4
2.3 Implementation options . . . . . . . . . . . . . . . . . . . 5 3 Details of ISP Use Case . . . . . . . . . . . . . . . . . . . . 5
3 Details of ISP Use Case . . . . . . . . . . . . . . . . . . . . 6 3.1 Understanding the quality experienced by customers . . . . . 5
3.1 Understanding the quality experienced by customers . . . . . 6
3.2 Understanding the impact and operation of new devices and 3.2 Understanding the impact and operation of new devices and
technology . . . . . . . . . . . . . . . . . . . . . . . . . 7 technology . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 Design and planning . . . . . . . . . . . . . . . . . . . . 7 3.3 Design and planning . . . . . . . . . . . . . . . . . . . . 6
3.4 Monitoring Service Level Agreements . . . . . . . . . . . . 8 3.4 Monitoring Service Level Agreements . . . . . . . . . . . . 7
3.5 Identifying, isolating and fixing network problems . . . . . 8 3.5 Identifying, isolating and fixing network problems . . . . . 7
4 Details of Regulator Use Case . . . . . . . . . . . . . . . . . 9 4 Details of Regulator Use Case . . . . . . . . . . . . . . . . . 8
4.1 Promoting competition through transparency . . . . . . . . . 9 4.1 Promoting competition through transparency . . . . . . . . . 8
4.2 Promoting broadband deployment . . . . . . . . . . . . . . . 10 4.2 Promoting broadband deployment . . . . . . . . . . . . . . . 9
4.3 Monitoring "net neutrality" . . . . . . . . . . . . . . . . 11 4.3 Monitoring "net neutrality" . . . . . . . . . . . . . . . . 10
5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5 Implementation Options . . . . . . . . . . . . . . . . . . . . 11
6 Security Considerations . . . . . . . . . . . . . . . . . . . . 13 6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 12
7 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 14 7 Security Considerations . . . . . . . . . . . . . . . . . . . . 13
8 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 14
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Informative References . . . . . . . . . . . . . . . . . . . . . . 14 Informative References . . . . . . . . . . . . . . . . . . . . . . 14
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16
1 Introduction 1 Introduction
This document describes two use cases for the Large-scale Measurement This document describes two use cases for the Large-scale Measurement
of Broadband Performance (LMAP), in particular use cases for ISPs and of Broadband Performance (LMAP). Firstly, to enable network operators
regulators. Although there are many other use cases for large-scale to understand the performance of the network and the quality
measurements systems, the two described here are the consensus experienced by customers. Secondly, to enable regulators to provide
starting point for defining the system. information on the performance of the ISPs in their jurisdiction.
There are other use cases that are not the focus of the initial LMAP
work, for example end users would like to use measurements to help
identify problems in their home network and to monitor the
performance of their broadband provider; it is expected that the same
mechanisms are applicable.
2 Use Cases 2 Use Cases
From the LMAP perspective, there is no difference between fixed From the LMAP perspective, there is no difference between fixed
service and mobile (cellular) service used for Internet access. service and mobile (cellular) service used for Internet access.
Hence, like measurements will take place on both fixed and mobile Hence, like measurements will take place on both fixed and mobile
networks. Fixed services, commonly known as "Last Mile" include networks. Fixed services, commonly known as "Last Mile" include
technologies like DSL, Cable, and Carrier Ethernet. Mobile services technologies like DSL, Cable, and Carrier Ethernet. Mobile services
include all those advertised as 2G, 3G, 4G, and LTE. A metric include all those advertised as 2G, 3G, 4G, and LTE. A metric
defined to measure end-to-end services will execute similarly on all defined to measure end-to-end services will execute similarly on all
access technologies. Other metrics may be access technology specific. access technologies. Other metrics may be access technology specific.
The LMAP architecture also covers both IPv4 and IPv6 networks. The LMAP architecture also covers both IPv4 and IPv6 networks.
2.1 Internet Service Provider (ISP) Use Case 2.1 Internet Service Provider (ISP) Use Case
An ISP, or indeed another network operator, needs to understand the A network operator needs to understand the performance of their
performance of their networks, the performance of the suppliers networks, the performance of the suppliers (downstream and upstream
(downstream and upstream networks), the performance of services, and networks), the performance of services, and the impact that such
the impact that such performance has on the experience of their performance has on the experience of their customers. Largely, the
customers. Largely, the processes that ISPs operate (which are based processes that ISPs operate (which are based on network measurement)
on network measurement) include: include:
o Identifying, isolating and fixing problems in the network, o Identifying, isolating and fixing problems, which may be in the
services or with CPE and end user equipment. Such problems may be network, with the service provider, or in the end user equipment.
common to a point in the network topology (e.g. a single Such problems may be common to a point in the network topology
exchange), common to a vendor or equipment type (e.g. line card or (e.g. a single exchange), common to a vendor or equipment type
home gateway) or unique to a single user line (e.g. copper (e.g. line card or home gateway) or unique to a single user line
access). Part of this process may also be helping users understand (e.g. copper access). Part of this process may also be helping
whether the problem exists in their home network or with an over- users understand whether the problem exists in their home network
the-top service instead of with their broadband (BB) product. or with an over-the-top service instead of with their broadband
(BB) product.
o Design and planning. Through identifying the end user experience o Design and planning. Through identifying the end user experience
the ISP can design and plan their network to ensure specified the ISP can design and plan their network to ensure specified
levels of user experience. Services may be moved closer to end levels of user experience. Services may be moved closer to end
users, services upgraded, the impact of QoS assessed or more users, services upgraded, the impact of QoS assessed or more
capacity deployed at certain locations. Service Level Agreements capacity deployed at certain locations. Service Level Agreements
(SLAs) may be defined at network or product boundaries. (SLAs) may be defined at network or product boundaries.
o Understanding the quality experienced by customers. Alongside o Understanding the quality experienced by customers. Alongside
benchmarking competitors, gaining better insight into the user's benchmarking competitors, gaining better insight into the user's
skipping to change at page 4, line 4 skipping to change at page 4, line 11
o Design and planning. Through identifying the end user experience o Design and planning. Through identifying the end user experience
the ISP can design and plan their network to ensure specified the ISP can design and plan their network to ensure specified
levels of user experience. Services may be moved closer to end levels of user experience. Services may be moved closer to end
users, services upgraded, the impact of QoS assessed or more users, services upgraded, the impact of QoS assessed or more
capacity deployed at certain locations. Service Level Agreements capacity deployed at certain locations. Service Level Agreements
(SLAs) may be defined at network or product boundaries. (SLAs) may be defined at network or product boundaries.
o Understanding the quality experienced by customers. Alongside o Understanding the quality experienced by customers. Alongside
benchmarking competitors, gaining better insight into the user's benchmarking competitors, gaining better insight into the user's
service through a sample panel of the operator's own customers. service through a sample panel of the operator's own customers.
The end-to-end perspective matters, across home/enterprise The end-to-end perspective matters, across home/enterprise
networks, peering points, Content Delivery Networks (CDNs), etc. networks, peering points, Content Delivery Networks (CDNs), etc.
o Understanding the impact and operation of new devices and o Understanding the impact and operation of new devices and
technology. As a new product is deployed, or a new technology technology. As a new product is deployed, or a new technology
introduced into the network, it is essential that its operation introduced into the network, it is essential that its operation
and impact on other services is measured. This also helps to and its impact is measured. This also helps to quantify the
quantify the advantage that the new technology is bringing and advantage that the new technology is bringing and support the
support the business case for larger roll-out. business case for larger roll-out.
2.2 Regulators 2.2 Regulator Use Case
Regulators in jurisdictions around the world are responding to Regulators in jurisdictions around the world are responding to
consumers' adoption of Internet access services for traditional consumers' adoption of Internet access services for traditional
telecommunications and media services by promoting competition among telecommunications and media services by promoting competition among
providers of electronic communications, to ensure that users derive providers of electronic communications, to ensure that users derive
maximum benefit in terms of choice, price, and quality. maximum benefit in terms of choice, price, and quality.
Some jurisdictions have responded to a need for greater information Some jurisdictions have responded to a need for greater information
about Internet access service performance in the development of about Internet access service performance in the development of
regulatory policies and approaches for broadband technologies by regulatory policies and approaches for broadband technologies by
skipping to change at page 5, line 12 skipping to change at page 5, line 19
From a consumer perspective, the differentiation between fixed and From a consumer perspective, the differentiation between fixed and
mobile (cellular) Internet access services is blurring as the mobile (cellular) Internet access services is blurring as the
applications used are very similar. Hence, regulators are measuring applications used are very similar. Hence, regulators are measuring
both fixed and mobile Internet access services. both fixed and mobile Internet access services.
A regulator's role in the development and enforcement of broadband A regulator's role in the development and enforcement of broadband
Internet access service policies also requires that the measurement Internet access service policies also requires that the measurement
approaches meet a high level of verifiability, accuracy and provider- approaches meet a high level of verifiability, accuracy and provider-
independence to support valid and meaningful comparisons of Internet independence to support valid and meaningful comparisons of Internet
access service performance access service performance.
LMAP standards could answer regulators shared needs by providing LMAP standards could answer regulators shared needs by providing
scalable, cost-effective, scientifically robust solutions to the scalable, cost-effective, scientifically robust solutions to the
measurement and collection of broadband Internet access service measurement and collection of broadband Internet access service
performance information. performance information.
2.3 Implementation options
There are several ways of implementing a measurement system. The
choice may be influenced by the details of the particular use case
and what the most important criteria are for the regulator, ISP or
third party operating the measurement system.
One way involves a special hardware device that is connected directly
to the home gateway. The devices are deployed to a carefully selected
panel of end users and they perform measurements according to a
defined schedule. The schedule can run throughout the day, to allow
continuous assessment of the network. Careful design ensures that
measurements do not detrimentally impact the home user experience or
corrupt the results by testing when the user is also using the
broadband line. The system is therefore tightly controlled by the
operator of the measurement system. One advantage of this approach is
that it is possible to get reliable benchmarks for the performance of
a network with only a few devices. One disadvantage is that it would
be expensive to deploy hardware devices on a mass scale sufficient to
understand the performance of the network at the granularity of a
single broadband user.
Another approach involves implementing the measurement capability as
a webpage or an "app" that end users are encouraged to download onto
their mobile phone or computing device. Measurements are triggered by
the end user, for example the user interface may have a button to
"test my broadband now". Compared with the previous approach, the
system is much more loosely controlled, as the panel of end users and
the schedule of tests are determined by the end users themselves
rather than the measurement system. It would be easier to get large-
scale, however it is harder to get comparable benchmarks as the
measurements are affected by the home network and also the population
is self-selecting and so potentially biased towards those who think
they have a problem. This could be alleviated by stimulating
widespread downloading of the app and careful post-processing of the
results to reduce biases.
There are several other possibilities. For example, as a variant on
the first approach, the measurement capability could be implemented
as software embedded in the home gateway, which would make it more
viable to have the capability on every user line. As a variant on the
second approach, the end user could initiate measurements in response
to a request from the measurement system.
3 Details of ISP Use Case 3 Details of ISP Use Case
3.1 Understanding the quality experienced by customers 3.1 Understanding the quality experienced by customers
Operators want to understand the quality of experience (QoE) of their Operators want to understand the quality of experience (QoE) of their
broadband customers. The understanding can be gained through a broadband customers. The understanding can be gained through a
"panel", i.e., a measurement probe is deployed to a few 100 or 1000 "panel", i.e., a measurement probe is deployed to a few 100 or 1000
of its customers. The panel needs to include a representative sample of its customers. The panel needs to include a representative sample
for each of the operator's technologies (Fiber To The Premise (FTTP), for each of the operator's technologies (Fiber To The Premise (FTTP),
Fiber To The Curb (FTTC), DSL...) and broadband options (80Mb/s, Fiber To The Curb (FTTC), DSL...) and broadband options (80Mb/s,
skipping to change at page 7, line 5 skipping to change at page 6, line 16
the measurement. the measurement.
Active measurements are the most obvious approach, i.e., special Active measurements are the most obvious approach, i.e., special
measurement traffic is sent by - and to - the probe. In order not to measurement traffic is sent by - and to - the probe. In order not to
degrade the service of the customer, the measurement data should only degrade the service of the customer, the measurement data should only
be sent when the user is silent, and it shouldn't reduce the be sent when the user is silent, and it shouldn't reduce the
customer's data allowance. The other approach is passive measurements customer's data allowance. The other approach is passive measurements
on the customer's ordinary traffic; the advantage is that it measures on the customer's ordinary traffic; the advantage is that it measures
what the customer actually does, but it creates extra variability what the customer actually does, but it creates extra variability
(different traffic mixes give different results) and especially it (different traffic mixes give different results) and especially it
raises privacy concerns. raises privacy concerns. [RFC6973] discusses privacy considerations
for Internet protocols in general, whilst [framework] discusses them
specifically for large-scale measurement systems.
From an operator's viewpoint, understanding customers better enables From an operator's viewpoint, understanding customers better enables
it to offer better services. Also, simple metrics can be more easily it to offer better services. Also, simple metrics can be more easily
understood by senior managers who make investment decisions and by understood by senior managers who make investment decisions and by
sales and marketing. sales and marketing.
3.2 Understanding the impact and operation of new devices and technology 3.2 Understanding the impact and operation of new devices and technology
Another type of measurement is to test new capabilities and services Another type of measurement is to test new capabilities before they
before they are rolled out. For example, the operator may want to: are rolled out. For example, the operator may want to: check whether
check whether a customer can be upgraded to a new broadband option; a customer can be upgraded to a new broadband option; understand the
understand the impact of IPv6 before it makes it available to its impact of IPv6 before it makes it available to its customers (will v6
customers (will v6 packets get through, what will the latency be to packets get through, what will the latency be to major websites, what
major websites, what transition mechanisms will be most transition mechanisms will be most appropriate?); check whether a new
appropriate?); check whether a new capability can be signaled using capability can be signaled using TCP options (how often it will be
TCP options (how often it will be blocked by a middlebox? - along the blocked by a middlebox? - along the lines of some existing
lines of some existing experiments) [Extend TCP]; investigate a experiments) [Extend TCP]; investigate a quality of service mechanism
quality of service mechanism (e.g. checking whether Diffserv markings (e.g. checking whether Diffserv markings are respected on some path);
are respected on some path); and so on. and so on.
3.3 Design and planning 3.3 Design and planning
Operators can use large scale measurements to help with their network Operators can use large scale measurements to help with their network
planning - proactive activities to improve the network. planning - proactive activities to improve the network.
For example, by probing from several different vantage points the For example, by probing from several different vantage points the
operator can see that a particular group of customers has performance operator can see that a particular group of customers has performance
below that expected during peak hours, which should help capacity below that expected during peak hours, which should help capacity
planning. Naturally operators already have tools to help this - a planning. Naturally operators already have tools to help this - a
skipping to change at page 8, line 37 skipping to change at page 7, line 51
3.5 Identifying, isolating and fixing network problems 3.5 Identifying, isolating and fixing network problems
Operators can use large scale measurements to help identify a fault Operators can use large scale measurements to help identify a fault
more rapidly and decide how to solve it. more rapidly and decide how to solve it.
Operators already have Test and Diagnostic tools, where a network Operators already have Test and Diagnostic tools, where a network
element reports some problem or failure to a management system. element reports some problem or failure to a management system.
However, many issues are not caused by a point failure but something However, many issues are not caused by a point failure but something
wider and so will trigger too many alarms, whilst other issues will wider and so will trigger too many alarms, whilst other issues will
cause degradation rather than failure and so not trigger any alarm. cause degradation rather than failure and so not trigger any alarm.
Large scale measurements can help provide a more nuance view that Large-scale measurements can help provide a more nuanced view that
helps network management to identify and fix problems more rapidly helps network management to identify and fix problems more rapidly
and accurately. The network management tools may use simulations to and accurately. The network management tools may use simulations to
emulate the network and so help identify a fault and assess possible emulate the network and so help identify a fault and assess possible
solutions. solutions.
One example was described in [IETF85-Plenary]. The operator was An operator can obtain useful information without measuring the
running a measurement panel for reasons discussed in sub use case #1. performance on every broadband line. By measuring a subset, the
It was noticed that the performance of some lines had unexpectedly operator identify problems that affect a group of customers. For
degraded. This led to a detailed (off-line) investigation which example, the issue could be at a shared point in the network topology
discovered that a particular home gateway upgrade had caused a (such as an exchange) or common to a vendor or equipment type
(mistaken!) drop in line rate. [IETF85-Plenary] describes a case where a particular home gateway
upgrade had caused a (mistaken!) drop in line rate. A more extensive
Another example is that occasionally some internal network management deployment of the measurement capability to every broadband line
event (like re-routing) can be customer-affecting (of course this is would enable an operator to identify issues unique to a single
unusual). This affects a whole group of customers, for instance those customer. Overall, large-scale measurements can help an operator help
on the same DSLAM. Understanding this will help an operator fix the an operator fix the fault more rapidly and/or allow the affected
fault more rapidly and/or allow the affected customers to be informed customers to be informed what's happening. More accurate information
what's happening and/or request them to re-set their home hub enables the operator to reassure customers and take more rapid and
(required to cure some conditions). More accurate information enables effective action to cure the problem.
the operator to reassure customers and take more rapid and effective
action to cure the problem.
There may also be problems unique to a single user line (e.g. copper
access) that need to be identified.
Often customers experience poor broadband due to problems in the home Often customers experience poor broadband due to problems in the home
network - the ISP's network is fine. For example they may have moved network - the ISP's network is fine. For example they may have moved
too far away from their wireless access point. Perhaps 80% of too far away from their wireless access point. Perhaps 80% of
customer calls about fixed BB problems are due to in-home wireless customer calls about fixed BB problems are due to in-home wireless
issues. These issues are expensive and frustrating for an operator, issues. These issues are expensive and frustrating for an operator,
as they are extremely hard to diagnose and solve. The operator would as they are extremely hard to diagnose and solve. The operator would
like to narrow down whether the problem is in the home (with the home like to narrow down whether the problem is in the home (with the home
network or edge device or home gateway), in the operator's network, network or edge device or home gateway), in the operator's network,
or with an over-the-top service. The operator would like two or with an over-the-top service. The operator would like two
skipping to change at page 9, line 50 skipping to change at page 9, line 12
offers. In some jurisdictions regulators mandate transparent offers. In some jurisdictions regulators mandate transparent
information made available about service offers. information made available about service offers.
End users need effective transparency to be able to make informed End users need effective transparency to be able to make informed
choices throughout the different stages of their relationship with choices throughout the different stages of their relationship with
ISPs, when selecting Internet access service offers, and when ISPs, when selecting Internet access service offers, and when
considering switching service offer within an ISP or to an considering switching service offer within an ISP or to an
alternative ISP. Quality information about service offers could alternative ISP. Quality information about service offers could
include speed, delay, and jitter. Regulators can publish such include speed, delay, and jitter. Regulators can publish such
information to facilitate end users' choice of service provider and information to facilitate end users' choice of service provider and
offer. It may also help content, application, service and device offer. It may also encourage ISPs to use the same metrics in their
providers develop their Internet offerings. service level contracts, which would further help end users to choose
an ISP. Finally, transparency may help content, application, service
and device providers develop their Internet offerings.
The published information needs to be: The published information needs to be:
o Accurate - the measurement results must be correct and not o Accurate - the measurement results must be correct and not
influenced by errors or side effects. The results should be influenced by errors or side effects. The results should be
reproducible and consistent over time. reproducible and consistent over time.
o Comparable - common metrics should be used across different o Comparable - common metrics should be used across different
ISPs and service offerings so that measurement results can be ISPs and service offerings so that measurement results can be
compared. compared.
o Meaningful - the metrics used for measurements need to reflect o Meaningful - the metrics used for measurements need to reflect
what end users value about their broadband Internet access service what end users value about their broadband Internet access service
o Reliable - the number and distribution of measurement agents, o Reliable - the number and distribution of measurement agents,
and the statistical processing of the raw measurement raw data, and the statistical processing of the raw measurement data, needs
needs to be appropriate to be appropriate
A set of measurement parameters and associated measurement methods A set of measurement parameters and associated measurement methods
are used over time, e.g. speed, delay, and jitter. Then the are used over time, e.g. speed, delay, and jitter. Then the
measurement raw data are collected and go through statistical post- measurement raw data are collected and go through statistical post-
processing before the results can be published in an Internet access processing before the results can be published in an Internet access
service quality index to facilitate end users' choice of service service quality index to facilitate end users' choice of service
provider and offer. provider and offer.
The regulator can also promote competition through transparency by The regulator can also promote competition through transparency by
encouraging end users to monitor the performance of their own encouraging end users to monitor the performance of their own
skipping to change at page 11, line 27 skipping to change at page 10, line 40
4.3 Monitoring "net neutrality" 4.3 Monitoring "net neutrality"
Regulatory approaches related to net neutrality and the open Internet Regulatory approaches related to net neutrality and the open Internet
has been introduced in some jurisdictions. Examples of such efforts has been introduced in some jurisdictions. Examples of such efforts
are the Internet policy as outlined by the Body of European are the Internet policy as outlined by the Body of European
Regulators for Electronic Communications Guidelines for quality of Regulators for Electronic Communications Guidelines for quality of
service [BEREC Guidelines] and US FCC Preserving the Open Internet service [BEREC Guidelines] and US FCC Preserving the Open Internet
Report and Order [FCC R&O]. Although legal challenges can change the Report and Order [FCC R&O]. Although legal challenges can change the
status of policy such as the court action negating the FCC R&O, the status of policy such as the court action negating the FCC R&O, the
take away for LMAP purposes are that policy-makers are looking for take-away for LMAP purposes is that policy-makers are looking for
measurement solutions to assist them in discovering biased treatment measurement solutions to assist them in discovering biased treatment
of traffic flows. The exact definitions and requirements vary from of traffic flows. The exact definitions and requirements vary from
one jurisdiction to another; the comments below provide some hints one jurisdiction to another; the comments below provide some hints
about the potential role of measurements. about the potential role of measurements.
Net neutrality regulations do not necessarily require every packet to Net neutrality regulations do not necessarily require every packet to
be treated equally. Typically they allow "reasonable" traffic be treated equally. Typically they allow "reasonable" traffic
management (for example if there is exceptional congestion) and allow management (for example if there is exceptional congestion) and allow
"specialized services" in parallel to, but separate from, ordinary "specialized services" in parallel to, but separate from, ordinary
Internet access (for example for facilities-based IPTV). A regulator Internet access (for example for facilities-based IPTV). A regulator
skipping to change at page 12, line 4 skipping to change at page 11, line 17
A regulator could monitor departures from application agnosticism A regulator could monitor departures from application agnosticism
such as blocking or throttling of traffic from specific applications, such as blocking or throttling of traffic from specific applications,
and preferential treatment of specific applications. A measurement and preferential treatment of specific applications. A measurement
system could send, or passively monitor, application-specific traffic system could send, or passively monitor, application-specific traffic
and then measure in detail the transfer of the different packets. and then measure in detail the transfer of the different packets.
Whilst it is relatively easy to measure port blocking, it is a Whilst it is relatively easy to measure port blocking, it is a
research topic how to detect other types of differentiated treatment. research topic how to detect other types of differentiated treatment.
The paper, "Glasnost: Enabling End Users to Detect Traffic The paper, "Glasnost: Enabling End Users to Detect Traffic
Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost" Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost"
[Glasnost] are examples of work in this area. [Glasnost] are examples of work in this area.
A regulator could also monitor the performance of the broadband A regulator could also monitor the performance of the broadband
service over time, to try and detect if the specialized service is service over time, to try and detect if the specialized service is
provided at the expense of the Internet access service. Comparison provided at the expense of the Internet access service. Comparison
between ISPs or between different countries may also be relevant for between ISPs or between different countries may also be relevant for
this kind of evaluation. this kind of evaluation.
5 Conclusions 5 Implementation Options
There are several ways of implementing a measurement system. The
choice may be influenced by the details of the particular use case
and what the most important criteria are for the regulator, ISP or
third party operating the measurement system.
One way involves a special hardware device that is connected directly
to the home gateway. The devices are deployed to a carefully selected
panel of end users and they perform measurements according to a
defined schedule. The schedule can run throughout the day, to allow
continuous assessment of the network. Careful design ensures that
measurements do not detrimentally impact the home user experience or
corrupt the results by testing when the user is also using the
broadband line. The system is therefore tightly controlled by the
operator of the measurement system. One advantage of this approach is
that it is possible to get reliable benchmarks for the performance of
a network with only a few devices. One disadvantage is that it would
be expensive to deploy hardware devices on a mass scale sufficient to
understand the performance of the network at the granularity of a
single broadband user.
Another approach involves implementing the measurement capability as
a webpage or an "app" that end users are encouraged to download onto
their mobile phone or computing device. Measurements are triggered by
the end user, for example the user interface may have a button to
"test my broadband now". One advantage of this approach is that the
performance is measured to the end user, rather than to the home
gateway, and so includes the home network. Another difference is that
the system is much more loosely controlled, as the panel of end users
and the schedule of tests are determined by the end users themselves
rather than the measurement system. It would be easier to get large-
scale, however it is harder to get comparable benchmarks as the
measurements are affected by the home network and also the population
is self-selecting and so potentially biased towards those who think
they have a problem. This could be alleviated by stimulating
widespread downloading of the app and careful post-processing of the
results to reduce biases.
There are several other possibilities. For example, as a variant on
the first approach, the measurement capability could be implemented
as software embedded in the home gateway, which would make it more
viable to have the capability on every user line. As a variant on the
second approach, the end user could initiate measurements in response
to a request from the measurement system.
6 Conclusions
Large-scale measurements of broadband performance are useful for both Large-scale measurements of broadband performance are useful for both
network operators and regulators. Network operators would like to use network operators and regulators. Network operators would like to use
measurements to help them better understand the quality experienced measurements to help them better understand the quality experienced
by their customers, identify problems in the network and design by their customers, identify problems in the network and design
network improvements. Regulators would like to use measurements to network improvements. Regulators would like to use measurements to
help promote competition between network operators, stimulate the help promote competition between network operators, stimulate the
growth of broadband access and monitor 'net neutrality'. There are growth of broadband access and monitor 'net neutrality'. There are
other use cases that are not the focus of the initial LMAP charter other use cases that are not the focus of the initial LMAP charter
(although it is expected that the mechanisms developed would be (although it is expected that the mechanisms developed would be
skipping to change at page 13, line 24 skipping to change at page 13, line 35
contextual information about the topology, line, product and the contextual information about the topology, line, product and the
subscriber's contract. The actual analysis of results is beyond the subscriber's contract. The actual analysis of results is beyond the
scope of LMAP, as is the key challenge of how to integrate the scope of LMAP, as is the key challenge of how to integrate the
measurement system into a network operator's existing tools for measurement system into a network operator's existing tools for
diagnostics and network planning. diagnostics and network planning.
Finally the test data, along with any associated network, product or Finally the test data, along with any associated network, product or
subscriber contract data is commercial or private information and subscriber contract data is commercial or private information and
needs to be protected. needs to be protected.
6 Security Considerations 7 Security Considerations
This informational document provides an overview of the use cases for This informational document provides an overview of the use cases for
LMAP and so does not, in itself, raise any security issues. LMAP and so does not, in itself, raise any security issues.
The framework document [framework] discusses the potential security, The framework document [framework] discusses the potential security,
privacy (data protection) and business sensitivity issues that LMAP privacy (data protection) and business sensitivity issues that LMAP
raises. The main threats are: raises. The main threats are:
1. a malicious party that gains control of Measurement Agents to 1. a malicious party that gains control of Measurement Agents to
launch DoS attacks at a target, or to alter (perhaps subtly) launch DoS attacks at a target, or to alter (perhaps subtly)
Measurement Tasks in order to compromise the end user's privacy, Measurement Tasks in order to compromise the end user's privacy,
the business confidentiality of the network, or the accuracy of the business confidentiality of the network, or the accuracy of
the measurement system. the measurement system.
2. a malicious party that intercepts or corrupts the Measurement 2. a malicious party that gains control of Measurement Agents to
create a platform for pervasive monitoring [RFC7258], in order to
attack the privacy of Internet users and organisations.
3. a malicious party that intercepts or corrupts the Measurement
Results &/or other information about the Subscriber, for similar Results &/or other information about the Subscriber, for similar
nefarious purposes. nefarious purposes.
3. a malicious party that uses fingerprinting techniques to 4. a malicious party that uses fingerprinting techniques to
identify individual end users, even from anonymized data identify individual end users, even from anonymized data
4. a measurement system that does not obtain the end user's 5. a measurement system that does not obtain the end user's
informed consent, or fails to specify a specific purpose in the informed consent, or fails to specify a specific purpose in the
consent, or uses the collected information for secondary uses consent, or uses the collected information for secondary uses
beyond those specified. beyond those specified.
5. a measurement system that is vague about who is responsible for 6. a measurement system that is vague about who is responsible for
privacy (data protection); this role is often termed the "data privacy (data protection); this role is often termed the "data
controller". controller".
The [framework] also considers some potential mitigations of these The [framework] also considers some potential mitigations of these
issues. They will need to be considered by an LMAP protocol and more issues. They will need to be considered by an LMAP protocol and more
generally by any measurement system. generally by any measurement system.
7 IANA Considerations 8 IANA Considerations
None None
Contributors Contributors
The information in this document is partially derived from text The information in this document is partially derived from text
written by the following contributors: written by the following contributors:
James Miller jamesmilleresquire@gmail.com James Miller jamesmilleresquire@gmail.com
skipping to change at page 14, line 44 skipping to change at page 15, line 16
Greenhalgh, Mark Handley and Hideyuki Tokuda. "Is it Still Greenhalgh, Mark Handley and Hideyuki Tokuda. "Is it Still
Possible to Extend TCP?" Proc. ACM Internet Measurement Possible to Extend TCP?" Proc. ACM Internet Measurement
Conference (IMC), November 2011, Berlin, Germany. Conference (IMC), November 2011, Berlin, Germany.
http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf
[framework] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., [framework] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
Aitken, P., Akhter, A. "A framework for large-scale Aitken, P., Akhter, A. "A framework for large-scale
measurement platforms (LMAP)", measurement platforms (LMAP)",
http://datatracker.ietf.org/doc/draft-ietf-lmap-framework/ http://datatracker.ietf.org/doc/draft-ietf-lmap-framework/
[RFC6973] Cooper, A., Tschofenig, H.z., Aboba, B., Peterson, J.,
Morris, J., Hansen, M., and R. Smith, "Privacy
Considerations for Internet Protocols", RFC 6973, July
2013.
[RFC7258] Farrell, S., Tschofenig, H., "PPervasive Monitoring Is an
Attack", RFC 7258, May 2014.
[FCC R&O] United States Federal Communications Commission, 10-201, [FCC R&O] United States Federal Communications Commission, 10-201,
"Preserving the Open Internet, Broadband Industries "Preserving the Open Internet, Broadband Industries
Practices, Report and Order", Practices, Report and Order",
http://hraunfoss.fcc.gov/edocs_public/attachmatch/FCC-10- http://hraunfoss.fcc.gov/edocs_public/attachmatch/FCC-10-
201A1.pdf 201A1.pdf
[BEREC Guidelines] Body of European Regulators for Electronic [BEREC Guidelines] Body of European Regulators for Electronic
Communications, "BEREC Guidelines for quality of service Communications, "BEREC Guidelines for quality of service
in the scope of net neutrality", in the scope of net neutrality",
http://berec.europa.eu/eng/document_register/ http://berec.europa.eu/eng/document_register/
 End of changes. 30 change blocks. 
135 lines changed or deleted 153 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/