Elemental image and audio synchronization and transmission technique for glasses-free 3D TV system

Abstract

Previously integral imaging based 3D technology was proposed only for still image. We propose a novel technique to synchronize elemental images and audio signal and the transmission technique for glasses-free 3D TV system based on integral imaging. The main idea behind the method is to generate 3D video based on elemental images synchronized with audio stream. The system uses the depth information and RGB data of per frame of a video through Intel RealSense 3D camera and the audio stream from microphone. The audio le is sampled according to per frame duration of the video and kept in di erent bu ers but having same index. The frames are divided into elemental images using Elemental Image Generation algorithm and the audio signal is synchronized according to the index. Then the stream of elemental images and corresponding audio data is transmitted to data server for storage. HLS streaming protocol is used to stream the TV content. A dedicated web application was made that fetches data from the server and plays video on the user end display device. By using multi-array of lenses in front of display, the video is viewed as three-dimensional with the help of integral imaging technology. The 3D TV content has an output rate of 30 frames per second and a calculated viewing angle of 18.9 . As integral imaging is an auto stereoscopic method to represent depth perception, it frees the viewer from wearing any 3D glasses.