We have an XR platform where all the AR/VR/MR contents are processed in the cloud and stream the result to the end user devices such as Mobile(Android, iOS)/HMD.
Generally, all the VR 3D 360/360 contents are large in size, streaming these contents to the HMD requires large bandwidth. Moreover, users will not be able to watch in one shot instead, he will be watching his Region of Interest data. Streaming all the ROI(Region of Interest) when users are watching an only particular region is not good design.
We are trying to address this problem by identifying the users ROI and gets particular ROI data from the cloud.
For the above solution, you will be working on the HMD module and below are the tasks we have created.
Receive Encoded data from cloud through our proprietary protocol
Identifying the user ROI
Decode the received encoded data (H264 codec)
Convert the decoded data to RGB/ARGB(Which ever format Unity accepts)
Render the frames on Screen
Integrate MQTT protocol to send user events data (ROI, HMD control commands)