draft-ietf-mops-ar-use-case-00.txt		draft-ietf-mops-ar-use-case-01.txt
	MOPS R. Krishna		MOPS R. Krishna
	Internet-Draft InterDigital Europe Limited		Internet-Draft InterDigital Europe Limited
	Intended status: Informational A. Rahman		Intended status: Informational A. Rahman
	Expires: September 26, 2021 InterDigital Communications, LLC		Expires: December 31, 2021 InterDigital Communications, LLC
	March 25, 2021		June 29, 2021
	Media Operations Use Case for an Augmented Reality Application on Edge		Media Operations Use Case for an Augmented Reality Application on Edge
	Computing Infrastructure		Computing Infrastructure
	draft-ietf-mops-ar-use-case-00		draft-ietf-mops-ar-use-case-01
	Abstract		Abstract
	A use case describing transmission of an application on the Internet		A use case describing transmission of an application on the Internet
	that has several unique characteristics of Augmented Reality (AR)		that has several unique characteristics of Augmented Reality (AR)
	applications is presented for the consideration of the Media		applications is presented for the consideration of the Media
	Operations (MOPS) Working Group. One key requirement identified is		Operations (MOPS) Working Group. One key requirement identified is
	that the Adaptive-Bit-Rate (ABR) algorithms' current usage of		that the Adaptive-Bit-Rate (ABR) algorithms' current usage of
	policies based on heuristics and models is inadequate for AR		policies based on heuristics and models is inadequate for AR
	applications running on the Edge Computing infrastructure.		applications running on the Edge Computing infrastructure.
	skipping to change at page 1, line 38 ¶		skipping to change at page 1, line 38 ¶
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF). Note that other groups may also distribute		Task Force (IETF). Note that other groups may also distribute
	working documents as Internet-Drafts. The list of current Internet-		working documents as Internet-Drafts. The list of current Internet-
	Drafts is at https://datatracker.ietf.org/drafts/current/.		Drafts is at https://datatracker.ietf.org/drafts/current/.
	Internet-Drafts are draft documents valid for a maximum of six months		Internet-Drafts are draft documents valid for a maximum of six months
	and may be updated, replaced, or obsoleted by other documents at any		and may be updated, replaced, or obsoleted by other documents at any
	time. It is inappropriate to use Internet-Drafts as reference		time. It is inappropriate to use Internet-Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	This Internet-Draft will expire on September 26, 2021.		This Internet-Draft will expire on December 31, 2021.
	Copyright Notice		Copyright Notice
	Copyright (c) 2021 IETF Trust and the persons identified as the		Copyright (c) 2021 IETF Trust and the persons identified as the
	document authors. All rights reserved.		document authors. All rights reserved.
	This document is subject to BCP 78 and the IETF Trust's Legal		This document is subject to BCP 78 and the IETF Trust's Legal
	Provisions Relating to IETF Documents		Provisions Relating to IETF Documents
	(https://trustee.ietf.org/license-info) in effect on the date of		(https://trustee.ietf.org/license-info) in effect on the date of
	publication of this document. Please review these documents		publication of this document. Please review these documents
	skipping to change at page 2, line 17 ¶		skipping to change at page 2, line 17 ¶
	described in the Simplified BSD License.		described in the Simplified BSD License.
	Table of Contents		Table of Contents
	1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2		1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
	2. Conventions used in this document . . . . . . . . . . . . . . 3		2. Conventions used in this document . . . . . . . . . . . . . . 3
	3. Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . 3		3. Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . 3
	3.1. Processing of Scenes . . . . . . . . . . . . . . . . . . 3		3.1. Processing of Scenes . . . . . . . . . . . . . . . . . . 3
	3.2. Generation of Images . . . . . . . . . . . . . . . . . . 4		3.2. Generation of Images . . . . . . . . . . . . . . . . . . 4
	4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4		4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4
	5. Informative References . . . . . . . . . . . . . . . . . . . 5		5. Informative References . . . . . . . . . . . . . . . . . . . 6
	Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 8		Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9
	1. Introduction		1. Introduction
	The MOPS draft, [I-D.ietf-mops-streaming-opcons], provides an		The MOPS draft, [I-D.ietf-mops-streaming-opcons], provides an
	overview of operational networking issues that pertain to Quality of		overview of operational networking issues that pertain to Quality of
	Experience (QoE) in delivery of video and other high-bitrate media		Experience (QoE) in delivery of video and other high-bitrate media
	over the Internet. However, as it does not cover the increasingly		over the Internet. However, as it does not cover the increasingly
	large number of applications with Augmented Reality (AR)		large number of applications with Augmented Reality (AR)
	characteristics and their requirements on ABR algorithms, the		characteristics and their requirements on ABR algorithms, the
	discussion in this draft compliments the overview presented in that		discussion in this draft compliments the overview presented in that
	skipping to change at page 4, line 38 ¶		skipping to change at page 4, line 38 ¶
	A solution to the heat dissipation and battery drainge problem is to		A solution to the heat dissipation and battery drainge problem is to
	offload the processing and video generation tasks to the remote		offload the processing and video generation tasks to the remote
	cloud.However, running such tasks on the cloud is not feasible as the		cloud.However, running such tasks on the cloud is not feasible as the
	end-to-end delays must be within the order of a few milliseconds.		end-to-end delays must be within the order of a few milliseconds.
	Additionally,such applications require high bandwidth and low jitter		Additionally,such applications require high bandwidth and low jitter
	to provide a high QoE to the user.In order to achieve such hard		to provide a high QoE to the user.In order to achieve such hard
	timing constraints, computationally intensive tasks can be offloaded		timing constraints, computationally intensive tasks can be offloaded
	to Edge devices.		to Edge devices.
			Another requirement for our use case and similar applications such as
			360 degree streaming is that the display on the AR/VR device should
			synchronize the visual input with the way the user is moving their
			head. This synchronization is necessary to avoid motion sickness
			that results from a time-lag between when the user moves their head
			and when the appropriate video scene is rendered. This time lag is
			often called "motion-to-photon" delay. Studies have shown
			[PER_SENSE], [XR], [OCCL_3] that this delay can be at most 20ms and
			preferably between 7-15ms in order to avoid the motion sickness
			problem. Out of these 20ms, display techniques including the refresh
			rate of write displays and pixel switching take 12-13ms [OCCL_3],
			[CLOUD]. This leaves 7-8ms for the processing of motion sensor
			inputs, graphic rendering, and RTT between the AR/VR device and the
			Edge. The use of predictive techniques to mask latencies has been
			considered as a mitigating strategy to reduce motion sickness
			[PREDICT]. In addition, Edge Devices that are proximate to the user
			might be used to offload these computationally intensive tasks.
			Towards this end, the 3GPP requires and supports an Ultra Reliable
			Low Latency of 0.1ms to 1ms for communication between an Edge server
			and User Equipment(UE) [URLLC].
	Note that the Edge device providing the computation and storage is		Note that the Edge device providing the computation and storage is
	itself limited in such resources compared to the Cloud. So, for		itself limited in such resources compared to the Cloud. So, for
	example, a sudden surge in demand from a large group of tourists can		example, a sudden surge in demand from a large group of tourists can
	overwhelm that device. This will result in a degraded user		overwhelm that device. This will result in a degraded user
	experience as their AR device experiences delays in receiving the		experience as their AR device experiences delays in receiving the
	video frames. In order to deal with this problem, the client AR		video frames. In order to deal with this problem, the client AR
	applications will need to use Adaptive Bit Rate (ABR) algorithms that		applications will need to use Adaptive Bit Rate (ABR) algorithms that
	choose bit-rates policies tailored in a fine-grained manner to the		choose bit-rates policies tailored in a fine-grained manner to the
	resource demands and playback the videos with appropriate QoE metrics		resource demands and playback the videos with appropriate QoE metrics
	as the user moves around with the group of tourists.		as the user moves around with the group of tourists.
	skipping to change at page 6, line 22 ¶		skipping to change at page 6, line 44 ¶
	Seneviratne, S., Hu, Y., Nguyen, T., Lan, G., Khalifa, S.,		Seneviratne, S., Hu, Y., Nguyen, T., Lan, G., Khalifa, S.,
	Thilakarathna, K., Hassan, M., and A. Seneviratne, "A		Thilakarathna, K., Hassan, M., and A. Seneviratne, "A
	survey of wearable devices and challenges.", In IEEE		survey of wearable devices and challenges.", In IEEE
	Communication Surveys and Tutorials, 19(4), p.2573-2620.,		Communication Surveys and Tutorials, 19(4), p.2573-2620.,
	2017.		2017.
	[BLUR] Kan, P. and H. Kaufmann, "Physically-Based Depth of Field		[BLUR] Kan, P. and H. Kaufmann, "Physically-Based Depth of Field
	in Augmented Reality.", In Eurographics (Short Papers),		in Augmented Reality.", In Eurographics (Short Papers),
	pp. 89-92., 2012.		pp. 89-92., 2012.
			[CLOUD] Corneo, L., Eder, M., Mohan, N., Zavodovski, A., Bayhan,
			S., Wong, W., Gunningberg, P., Kangasharju, J., and J.
			Ott, "Surrounded by the Clouds: A Comprehensive Cloud
			Reachability Study.", In Proceedings of the Web Conference
			2021, pp. 295-304, 2021.
	[DEV_HEAT_1]		[DEV_HEAT_1]
	LiKamWa, R., Wang, Z., Carroll, A., Lin, F., and L. Zhong,		LiKamWa, R., Wang, Z., Carroll, A., Lin, F., and L. Zhong,
	"Draining our Glass: An Energy and Heat characterization		"Draining our Glass: An Energy and Heat characterization
	of Google Glass", In Proceedings of 5th Asia-Pacific		of Google Glass", In Proceedings of 5th Asia-Pacific
	Workshop on Systems pp. 1-7, 2013.		Workshop on Systems pp. 1-7, 2013.
	[DEV_HEAT_2]		[DEV_HEAT_2]
	Matsuhashi, K., Kanamoto, T., and A. Kurokawa, "Thermal		Matsuhashi, K., Kanamoto, T., and A. Kurokawa, "Thermal
	model and countermeasures for future smart glasses.",		model and countermeasures for future smart glasses.",
	In Sensors, 20(5), p.1446., 2020.		In Sensors, 20(5), p.1446., 2020.
	skipping to change at page 7, line 39 ¶		skipping to change at page 8, line 20 ¶
	[OCCL_1] Breen, D., Whitaker, R., and M. Tuceryan, "Interactive		[OCCL_1] Breen, D., Whitaker, R., and M. Tuceryan, "Interactive
	Occlusion and automatic object placementfor augmented		Occlusion and automatic object placementfor augmented
	reality", In Computer Graphics Forum, vol. 15, no. 3 ,		reality", In Computer Graphics Forum, vol. 15, no. 3 ,
	pp. 229-238,Edinburgh, UK: Blackwell Science Ltd, 1996.		pp. 229-238,Edinburgh, UK: Blackwell Science Ltd, 1996.
	[OCCL_2] Zheng, F., Schmalstieg, D., and G. Welch, "Pixel-wise		[OCCL_2] Zheng, F., Schmalstieg, D., and G. Welch, "Pixel-wise
	closed-loop registration in video-based augmented		closed-loop registration in video-based augmented
	reality", In IEEE International Symposium on Mixed and		reality", In IEEE International Symposium on Mixed and
	Augmented Reality (ISMAR), pp. 135-143, 2014.		Augmented Reality (ISMAR), pp. 135-143, 2014.
			[OCCL_3] Lang, B., "Oculus Shares 5 Key Ingredients for Presence in
			Virtual Reality.", https://www.roadtovr.com/oculus-
			shares-5-key-ingredients-for-presence-in-virtual-reality/,
			2014.
			[PER_SENSE]
			Mania, K., Adelstein, B., Ellis, S., and M. Hill,
			"Perceptual sensitivity to head tracking latency in
			virtual environments with varying degrees of scene
			complexity.", In Proceedings of the 1st Symposium on
			Applied perception in graphics and visualization pp.
			39-47., 2004.
	[PHOTO_REG]		[PHOTO_REG]
	Liu, Y. and X. Granier, "Online tracking of outdoor		Liu, Y. and X. Granier, "Online tracking of outdoor
	lighting variations for augmented reality with moving		lighting variations for augmented reality with moving
	cameras", In IEEE Transactions on visualization and		cameras", In IEEE Transactions on visualization and
	computer graphics, 18(4), pp.573-580, 2012.		computer graphics, 18(4), pp.573-580, 2012.
			[PREDICT] Buker, T., Vincenzi, D., and J. Deaton, "The effect of
			apparent latency on simulator sickness while using a see-
			through helmet-mounted display: Reducing apparent latency
			with predictive compensation..", In Human factors 54.2,
			pp. 235-249., 2012.
	[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate		[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
	Requirement Levels", BCP 14, RFC 2119,		Requirement Levels", BCP 14, RFC 2119,
	DOI 10.17487/RFC2119, March 1997,		DOI 10.17487/RFC2119, March 1997,
	<https://www.rfc-editor.org/info/rfc2119>.		<https://www.rfc-editor.org/info/rfc2119>.
	[SLAM_1] Ventura, J., Arth, C., Reitmayr, G., and D. Schmalstieg,		[SLAM_1] Ventura, J., Arth, C., Reitmayr, G., and D. Schmalstieg,
	"A minimal solution to the generalized pose-and-scale		"A minimal solution to the generalized pose-and-scale
	problem", In Proceedings of the IEEE Conference on		problem", In Proceedings of the IEEE Conference on
	Computer Vision and Pattern Recognition, pp. 422-429,		Computer Vision and Pattern Recognition, pp. 422-429,
	2014.		2014.
	skipping to change at page 8, line 31 ¶		skipping to change at page 9, line 31 ¶
	[SLAM_4] Pirchheim, C., Schmalstieg, D., and G. Reitmayr, "Handling		[SLAM_4] Pirchheim, C., Schmalstieg, D., and G. Reitmayr, "Handling
	pure camera rotation in keyframe-based SLAM", In 2013		pure camera rotation in keyframe-based SLAM", In 2013
	IEEE international symposium on mixed and augmented		IEEE international symposium on mixed and augmented
	reality (ISMAR), pp. 229-238, 2013.		reality (ISMAR), pp. 229-238, 2013.
	[UBICOMP] Bardram, J. and A. Friday, "Ubiquitous Computing Systems",		[UBICOMP] Bardram, J. and A. Friday, "Ubiquitous Computing Systems",
	In Ubiquitous Computing Fundamentals pp. 37-94. CRC		In Ubiquitous Computing Fundamentals pp. 37-94. CRC
	Press, 2009.		Press, 2009.
			[URLLC] 3GPP, "3GPP TR 23.725: Study on enhancement of Ultra-
			Reliable Low-Latency Communication (URLLC) support in the
			5G Core network (5GC).",
			https://portal.3gpp.org/desktopmodules/Specifications/
			SpecificationDetails.aspx?specificationId=3453, 2019.
	[VIS_INTERFERE]		[VIS_INTERFERE]
	Kalkofen, D., Mendez, E., and D. Schmalstieg, "Interactive		Kalkofen, D., Mendez, E., and D. Schmalstieg, "Interactive
	focus and context visualization for augmented reality.",		focus and context visualization for augmented reality.",
	In 6th IEEE and ACM International Symposium on Mixed and		In 6th IEEE and ACM International Symposium on Mixed and
	Augmented Reality, pp. 191-201., 2007.		Augmented Reality, pp. 191-201., 2007.
	Authors' Addresses		[XR] 3GPP, "3GPP TR 26.928: Extended Reality (XR) in 5G.",
			https://portal.3gpp.org/desktopmodules/Specifications/
			SpecificationDetails.aspx?specificationId=3534, 2020.
			Authors' Addresses
	Renan Krishna		Renan Krishna
	InterDigital Europe Limited		InterDigital Europe Limited
	64, Great Eastern Street		64, Great Eastern Street
	London EC2A 3QR		London EC2A 3QR
	United Kingdom		United Kingdom
	Email: renan.krishna@interdigital.com		Email: renan.krishna@interdigital.com
	Akbar Rahman		Akbar Rahman
	InterDigital Communications, LLC		InterDigital Communications, LLC
	1000 Sherbrooke Street West		1000 Sherbrooke Street West
	Montreal H3A 3G4		Montreal H3A 3G4
	Canada		Canada
	Email: Akbar.Rahman@InterDigital.com		Email: Akbar.Rahman@InterDigital.com
End of changes. 12 change blocks.
	7 lines changed or deleted		63 lines changed or added
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/