draft-ietf-mops-ar-use-case-00.txt   draft-ietf-mops-ar-use-case-01.txt 
MOPS R. Krishna MOPS R. Krishna
Internet-Draft InterDigital Europe Limited Internet-Draft InterDigital Europe Limited
Intended status: Informational A. Rahman Intended status: Informational A. Rahman
Expires: September 26, 2021 InterDigital Communications, LLC Expires: December 31, 2021 InterDigital Communications, LLC
March 25, 2021 June 29, 2021
Media Operations Use Case for an Augmented Reality Application on Edge Media Operations Use Case for an Augmented Reality Application on Edge
Computing Infrastructure Computing Infrastructure
draft-ietf-mops-ar-use-case-00 draft-ietf-mops-ar-use-case-01
Abstract Abstract
A use case describing transmission of an application on the Internet A use case describing transmission of an application on the Internet
that has several unique characteristics of Augmented Reality (AR) that has several unique characteristics of Augmented Reality (AR)
applications is presented for the consideration of the Media applications is presented for the consideration of the Media
Operations (MOPS) Working Group. One key requirement identified is Operations (MOPS) Working Group. One key requirement identified is
that the Adaptive-Bit-Rate (ABR) algorithms' current usage of that the Adaptive-Bit-Rate (ABR) algorithms' current usage of
policies based on heuristics and models is inadequate for AR policies based on heuristics and models is inadequate for AR
applications running on the Edge Computing infrastructure. applications running on the Edge Computing infrastructure.
skipping to change at page 1, line 38 skipping to change at page 1, line 38
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 26, 2021. This Internet-Draft will expire on December 31, 2021.
Copyright Notice Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 17 skipping to change at page 2, line 17
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions used in this document . . . . . . . . . . . . . . 3 2. Conventions used in this document . . . . . . . . . . . . . . 3
3. Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.1. Processing of Scenes . . . . . . . . . . . . . . . . . . 3 3.1. Processing of Scenes . . . . . . . . . . . . . . . . . . 3
3.2. Generation of Images . . . . . . . . . . . . . . . . . . 4 3.2. Generation of Images . . . . . . . . . . . . . . . . . . 4
4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4
5. Informative References . . . . . . . . . . . . . . . . . . . 5 5. Informative References . . . . . . . . . . . . . . . . . . . 6
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9
1. Introduction 1. Introduction
The MOPS draft, [I-D.ietf-mops-streaming-opcons], provides an The MOPS draft, [I-D.ietf-mops-streaming-opcons], provides an
overview of operational networking issues that pertain to Quality of overview of operational networking issues that pertain to Quality of
Experience (QoE) in delivery of video and other high-bitrate media Experience (QoE) in delivery of video and other high-bitrate media
over the Internet. However, as it does not cover the increasingly over the Internet. However, as it does not cover the increasingly
large number of applications with Augmented Reality (AR) large number of applications with Augmented Reality (AR)
characteristics and their requirements on ABR algorithms, the characteristics and their requirements on ABR algorithms, the
discussion in this draft compliments the overview presented in that discussion in this draft compliments the overview presented in that
skipping to change at page 4, line 38 skipping to change at page 4, line 38
A solution to the heat dissipation and battery drainge problem is to A solution to the heat dissipation and battery drainge problem is to
offload the processing and video generation tasks to the remote offload the processing and video generation tasks to the remote
cloud.However, running such tasks on the cloud is not feasible as the cloud.However, running such tasks on the cloud is not feasible as the
end-to-end delays must be within the order of a few milliseconds. end-to-end delays must be within the order of a few milliseconds.
Additionally,such applications require high bandwidth and low jitter Additionally,such applications require high bandwidth and low jitter
to provide a high QoE to the user.In order to achieve such hard to provide a high QoE to the user.In order to achieve such hard
timing constraints, computationally intensive tasks can be offloaded timing constraints, computationally intensive tasks can be offloaded
to Edge devices. to Edge devices.
Another requirement for our use case and similar applications such as
360 degree streaming is that the display on the AR/VR device should
synchronize the visual input with the way the user is moving their
head. This synchronization is necessary to avoid motion sickness
that results from a time-lag between when the user moves their head
and when the appropriate video scene is rendered. This time lag is
often called "motion-to-photon" delay. Studies have shown
[PER_SENSE], [XR], [OCCL_3] that this delay can be at most 20ms and
preferably between 7-15ms in order to avoid the motion sickness
problem. Out of these 20ms, display techniques including the refresh
rate of write displays and pixel switching take 12-13ms [OCCL_3],
[CLOUD]. This leaves 7-8ms for the processing of motion sensor
inputs, graphic rendering, and RTT between the AR/VR device and the
Edge. The use of predictive techniques to mask latencies has been
considered as a mitigating strategy to reduce motion sickness
[PREDICT]. In addition, Edge Devices that are proximate to the user
might be used to offload these computationally intensive tasks.
Towards this end, the 3GPP requires and supports an Ultra Reliable
Low Latency of 0.1ms to 1ms for communication between an Edge server
and User Equipment(UE) [URLLC].
Note that the Edge device providing the computation and storage is Note that the Edge device providing the computation and storage is
itself limited in such resources compared to the Cloud. So, for itself limited in such resources compared to the Cloud. So, for
example, a sudden surge in demand from a large group of tourists can example, a sudden surge in demand from a large group of tourists can
overwhelm that device. This will result in a degraded user overwhelm that device. This will result in a degraded user
experience as their AR device experiences delays in receiving the experience as their AR device experiences delays in receiving the
video frames. In order to deal with this problem, the client AR video frames. In order to deal with this problem, the client AR
applications will need to use Adaptive Bit Rate (ABR) algorithms that applications will need to use Adaptive Bit Rate (ABR) algorithms that
choose bit-rates policies tailored in a fine-grained manner to the choose bit-rates policies tailored in a fine-grained manner to the
resource demands and playback the videos with appropriate QoE metrics resource demands and playback the videos with appropriate QoE metrics
as the user moves around with the group of tourists. as the user moves around with the group of tourists.
skipping to change at page 6, line 22 skipping to change at page 6, line 44
Seneviratne, S., Hu, Y., Nguyen, T., Lan, G., Khalifa, S., Seneviratne, S., Hu, Y., Nguyen, T., Lan, G., Khalifa, S.,
Thilakarathna, K., Hassan, M., and A. Seneviratne, "A Thilakarathna, K., Hassan, M., and A. Seneviratne, "A
survey of wearable devices and challenges.", In IEEE survey of wearable devices and challenges.", In IEEE
Communication Surveys and Tutorials, 19(4), p.2573-2620., Communication Surveys and Tutorials, 19(4), p.2573-2620.,
2017. 2017.
[BLUR] Kan, P. and H. Kaufmann, "Physically-Based Depth of Field [BLUR] Kan, P. and H. Kaufmann, "Physically-Based Depth of Field
in Augmented Reality.", In Eurographics (Short Papers), in Augmented Reality.", In Eurographics (Short Papers),
pp. 89-92., 2012. pp. 89-92., 2012.
[CLOUD] Corneo, L., Eder, M., Mohan, N., Zavodovski, A., Bayhan,
S., Wong, W., Gunningberg, P., Kangasharju, J., and J.
Ott, "Surrounded by the Clouds: A Comprehensive Cloud
Reachability Study.", In Proceedings of the Web Conference
2021, pp. 295-304, 2021.
[DEV_HEAT_1] [DEV_HEAT_1]
LiKamWa, R., Wang, Z., Carroll, A., Lin, F., and L. Zhong, LiKamWa, R., Wang, Z., Carroll, A., Lin, F., and L. Zhong,
"Draining our Glass: An Energy and Heat characterization "Draining our Glass: An Energy and Heat characterization
of Google Glass", In Proceedings of 5th Asia-Pacific of Google Glass", In Proceedings of 5th Asia-Pacific
Workshop on Systems pp. 1-7, 2013. Workshop on Systems pp. 1-7, 2013.
[DEV_HEAT_2] [DEV_HEAT_2]
Matsuhashi, K., Kanamoto, T., and A. Kurokawa, "Thermal Matsuhashi, K., Kanamoto, T., and A. Kurokawa, "Thermal
model and countermeasures for future smart glasses.", model and countermeasures for future smart glasses.",
In Sensors, 20(5), p.1446., 2020. In Sensors, 20(5), p.1446., 2020.
skipping to change at page 7, line 39 skipping to change at page 8, line 20
[OCCL_1] Breen, D., Whitaker, R., and M. Tuceryan, "Interactive [OCCL_1] Breen, D., Whitaker, R., and M. Tuceryan, "Interactive
Occlusion and automatic object placementfor augmented Occlusion and automatic object placementfor augmented
reality", In Computer Graphics Forum, vol. 15, no. 3 , reality", In Computer Graphics Forum, vol. 15, no. 3 ,
pp. 229-238,Edinburgh, UK: Blackwell Science Ltd, 1996. pp. 229-238,Edinburgh, UK: Blackwell Science Ltd, 1996.
[OCCL_2] Zheng, F., Schmalstieg, D., and G. Welch, "Pixel-wise [OCCL_2] Zheng, F., Schmalstieg, D., and G. Welch, "Pixel-wise
closed-loop registration in video-based augmented closed-loop registration in video-based augmented
reality", In IEEE International Symposium on Mixed and reality", In IEEE International Symposium on Mixed and
Augmented Reality (ISMAR), pp. 135-143, 2014. Augmented Reality (ISMAR), pp. 135-143, 2014.
[OCCL_3] Lang, B., "Oculus Shares 5 Key Ingredients for Presence in
Virtual Reality.", https://www.roadtovr.com/oculus-
shares-5-key-ingredients-for-presence-in-virtual-reality/,
2014.
[PER_SENSE]
Mania, K., Adelstein, B., Ellis, S., and M. Hill,
"Perceptual sensitivity to head tracking latency in
virtual environments with varying degrees of scene
complexity.", In Proceedings of the 1st Symposium on
Applied perception in graphics and visualization pp.
39-47., 2004.
[PHOTO_REG] [PHOTO_REG]
Liu, Y. and X. Granier, "Online tracking of outdoor Liu, Y. and X. Granier, "Online tracking of outdoor
lighting variations for augmented reality with moving lighting variations for augmented reality with moving
cameras", In IEEE Transactions on visualization and cameras", In IEEE Transactions on visualization and
computer graphics, 18(4), pp.573-580, 2012. computer graphics, 18(4), pp.573-580, 2012.
[PREDICT] Buker, T., Vincenzi, D., and J. Deaton, "The effect of
apparent latency on simulator sickness while using a see-
through helmet-mounted display: Reducing apparent latency
with predictive compensation..", In Human factors 54.2,
pp. 235-249., 2012.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[SLAM_1] Ventura, J., Arth, C., Reitmayr, G., and D. Schmalstieg, [SLAM_1] Ventura, J., Arth, C., Reitmayr, G., and D. Schmalstieg,
"A minimal solution to the generalized pose-and-scale "A minimal solution to the generalized pose-and-scale
problem", In Proceedings of the IEEE Conference on problem", In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, pp. 422-429, Computer Vision and Pattern Recognition, pp. 422-429,
2014. 2014.
skipping to change at page 8, line 31 skipping to change at page 9, line 31
[SLAM_4] Pirchheim, C., Schmalstieg, D., and G. Reitmayr, "Handling [SLAM_4] Pirchheim, C., Schmalstieg, D., and G. Reitmayr, "Handling
pure camera rotation in keyframe-based SLAM", In 2013 pure camera rotation in keyframe-based SLAM", In 2013
IEEE international symposium on mixed and augmented IEEE international symposium on mixed and augmented
reality (ISMAR), pp. 229-238, 2013. reality (ISMAR), pp. 229-238, 2013.
[UBICOMP] Bardram, J. and A. Friday, "Ubiquitous Computing Systems", [UBICOMP] Bardram, J. and A. Friday, "Ubiquitous Computing Systems",
In Ubiquitous Computing Fundamentals pp. 37-94. CRC In Ubiquitous Computing Fundamentals pp. 37-94. CRC
Press, 2009. Press, 2009.
[URLLC] 3GPP, "3GPP TR 23.725: Study on enhancement of Ultra-
Reliable Low-Latency Communication (URLLC) support in the
5G Core network (5GC).",
https://portal.3gpp.org/desktopmodules/Specifications/
SpecificationDetails.aspx?specificationId=3453, 2019.
[VIS_INTERFERE] [VIS_INTERFERE]
Kalkofen, D., Mendez, E., and D. Schmalstieg, "Interactive Kalkofen, D., Mendez, E., and D. Schmalstieg, "Interactive
focus and context visualization for augmented reality.", focus and context visualization for augmented reality.",
In 6th IEEE and ACM International Symposium on Mixed and In 6th IEEE and ACM International Symposium on Mixed and
Augmented Reality, pp. 191-201., 2007. Augmented Reality, pp. 191-201., 2007.
Authors' Addresses [XR] 3GPP, "3GPP TR 26.928: Extended Reality (XR) in 5G.",
https://portal.3gpp.org/desktopmodules/Specifications/
SpecificationDetails.aspx?specificationId=3534, 2020.
Authors' Addresses
Renan Krishna Renan Krishna
InterDigital Europe Limited InterDigital Europe Limited
64, Great Eastern Street 64, Great Eastern Street
London EC2A 3QR London EC2A 3QR
United Kingdom United Kingdom
Email: renan.krishna@interdigital.com Email: renan.krishna@interdigital.com
Akbar Rahman Akbar Rahman
InterDigital Communications, LLC InterDigital Communications, LLC
1000 Sherbrooke Street West 1000 Sherbrooke Street West
Montreal H3A 3G4 Montreal H3A 3G4
Canada Canada
Email: Akbar.Rahman@InterDigital.com Email: Akbar.Rahman@InterDigital.com
 End of changes. 12 change blocks. 
7 lines changed or deleted 63 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/