Compression of the Concentric Mosaic - 3D Wavelet Approach

Figure 1. Framework of 3D wavelet compression of the concentric mosaic.

Since the concentric mosaic can be considered as an image sequence, a good way to compress the concentric mosaic is through 3D wavelet. The basic framework of the 3D wavelet compression is shown in Fig. 1.

We have made two key contributions for the 3D wavelet compression of the concentric mosaic:

1. To improve the compression efficiency, especially as 3D wavelet does not provide very good decorrelation accross the images.

2. To improve the speed of 3D wavelet decompression and rendering. Note that the encoding complexity is rather tolerable, since the entire data set is just compressed once. However, the concentric mosaic data is too large to be entirely decoded and then rendered from the decoded data set, it is imperative to keep the data in the compressed form, and decode and render from the compressed bitstream.

To improve the compression performance, a data rearrangement mechanism called "smart rebinning" is proposed. Smart rebinning first aligns the concentric mosaic image shots along the horizontal direction and then rebins the shots into multi-perspective panoramas. It greatly improves the cross shot correlation and enables the coder to better explore the redundancy among shots. Experimental results show that the performance of the 3D wavelet coder improves an average of 4.3dB with the use of smart rebinning. The proposed coder outperforms MPEG-2 coding of concentric mosaics by an average of 3.7dB.

To reduce the computation complexity in the decompression and rendering, a progressive inverse wavelet synthesis (PIWS) algorithm is proposed. A mixed cache is used in the PIWS, where the entropy decoded wavelet coefficient, intermediate result of lifting and fully synthesized pixel are all stored at the same memory unit because of the in-place calculation property of the lifting implementation. PIWS operates with a finite state machine, where each memory unit is attached with a state to indicate what type of content is currently stored. With PIWS, it is possible to only perform the necessary 3D inverse lifting required to reconstruct the current view of the concentric mosaic. This coupled with the selective entropy decoding, enables a 3D wavelet compressed concentric mosaic to be rendered in real time.

For details of the work, please refer to the following papers.

• L. Luo, Y. Wu, J. Li and Y. Zhang, "Compression of concentric mosaic scenery with alignment and 3D wavelet transform", SPIE: Image and Video Communication and Processing, Vol. 3974, No. 10, pp. 89-100, San Jose CA, Jan. 2000. [Basic framework, improvement on 3D wavelet packet decomposition & 3D block entropy coding]
• Y. Wu, L. Luo, J. Li and Y. Zhang, "Rendering of 3D wavelet compressed concentric mosaic scenery with progressive inverse wavelet synthesis (PIWS)", SPIE Visual Communication and Image Processing (VCIP 2000), Vol. 4067, No. 4, pp. 31-42, Perth, Australia, Jun. 2000. [The PIWS algorithm which speeds up the decompression and rendering of 3D wavelet compressed concentric mosaic, VCIP 2000 student paper award. ]
• Y. Wu, C. Zhang, J. Li and J. Xu, "Smart rebinning for compression of concentric mosaics", ACM Multimedia 2000 (oral presentation), Los Angeles, CA, Oct. 2000, pp.201-209. [HTML version, significantly improve the compression performance of 3D wavelet through smart rebinning. ]
• Y. Wu and J. Li, "Compression of concentric mosaic with rebinning of slits (ROSS)", The First IEEE Pacific-Rim Conference on Multimedia, Sydney, Australia, Dec. 13-15, 2000.

©Copyright by Jin Li, June 22, 2001