Image Blending for Virtual Environment Construction based on TIP Model
Chang Hyun Roh and Wan Bok Lee
Division of Computer Engineering
JoongbuUniversity
101 Daehak-ro, Chubu-myeon, Kumsan-gun, Chungnam, 312-702, Republic of Korea
Abstract: - Image-based rendering is an approach to generate realistic images in real-time without modeling explicit 3D geometry. Especially, TIP(Tour Into the Picture) is preferred for its simplicity in constructing 3D background scene. However, TIP has a limitation that a viewpoint cannot go far from the origin of the TIP for the lack of geometrical information. In this paper, we propose a method to interpolating the TIP images to generate smooth and realistic navigation. We construct multiple TIP models in a wide area of the virtual environment. Then we interpolate foreground objects and background object respectively to generate smooth navigation results.
Key-Words: -TIP, image blending, virtual environment, image-based rendering
1Introduction
Image-based rendering is a recently-emerging technology to rapidly represent simulation images with high reality not using 3-D modeling but using document images. As a representative, ‘Tour into the picture (TIP)’ method is being widely applied to the related industries owing to its simple operation and rapid image construction.
However, the currently-used TIP method limits the range of visual points in 3-D navigation due to insufficient information on long-distance topography. In this study, by reinforcing foreground and background objectives of TIP models constructed from multiple visual points, a wide range of virtual environment was rendered for 3-D navigation.
Chen et al. [1] proposed the Quiktime VR method which navigates a virtual environment using multiple panoramic images. McMillan et al. [2] defined the pleloptic function to found image-based rendering theory. Gortler et al. [3] and Levoy et al. [4] simplified the pleloptic function into a 4-D function and showed a possibility of realistic rendering for the objectives such as hair.
Horry et al. [5] proposed the tour into picture (TIP) method, which was simple and practical compared with the existing image-based rendering methods requiring multiple reference pictures.
The TIP method drew perspective objectives from a 2-D image, constructed a simplified model of the virtual environment, and simulated a realistic image from a new visual point. However, the TIP method could not be applied to panoramic images as well as to reference images that included multiple or indistinct vanishing points, since the method limited the reference image with one clear vanishing point.
Kang et al. [6] improved the TIP method to construct a virtual environment using vanishing line rather than vanishing point. The improved method could deal with more general images in a simpler way without limiting number of the vanishing points.
Fig.1 demonstrates a background model using vanishing points. By assuming the background plane is infinite, the images above and below the vanishing line on the camera image correspond to the background plane and the floor plane, respectively. Thus, a background model composed of the background plane and the floor plane can be constructed by applying the projective geometry to the camera image plane for a given picture.
Fig.1 A background model using vanishing points
2Estimation of an Image Acquisition Location and Search of TIP Images
To obtain a simulation image for the present location, three nearest TIP environment models were employed. Searching three nearest TIP environment models is a special case of k-nearest point problem. In this study, the nearest TIP environment models were rapidly screened by dividing space into cells using quadtree and by searching only surrounding cells.
3Correlation and Interpolation of Foreground Objectives
To construct a foreground objective at the present location, the foreground objectives in the screened surrounding TIP environment models was interpolates as follows; firstly, the user specified identical objectives from the all foreground objectives in the three TIP environment models. Unless any objective was included in all the three TIP environment models, virtual foreground objective with zero volume was created. After the identical objectives were specified, apexes of the foreground objectives were correlated. A foreground objective included in all the three TIP environment models was assumed not to have the same number and the same shape of apexes although it had a phase like circle.
In this study, the apexes were correlated using the currently-used image morphing method and foreground objectives were constructed at a new location. When any object was not included in all the three TIP environment models and hence a virtual objective was created, the virtual objective was figured out using the motion estimation method of computer vision technology.
With the created virtual objective, texture at a new location was obtained using the weighted sum for the apexes and pixel distance and image warping. Final texture image was constructed by blending the texture images obtained from the three TIP environment models at the pixel level. A weight factor that was reversely proportional to the distance between the present location and the center of the environment model was applied to the blending.
4Modification and Synthesis of Background Model
In contrast to the foreground model, a background model has a constant form independently of location of TIP environment models. Therefore, modification and synthesis of the texture images is more important than modification of the objective form.
Firstly, the pixels constructing the background model were correlated using the projective geometry. Then, a texture image at a new location was created by the image warping of each environment model. Final texture image was constructed by blending the texture images obtained from the three TIP environment models at the pixel level. A weight factor that was reversely proportional to the distance between the present location and the center of the environment model was applied to the blending.
5Allocation of Foreground Model and Synthesis of TIP Image from a New Visual Point
Finally, the foreground objectives used for the interpolation was defined using the projective geometry based on the 3-D coordination employed for image acquisition and allocated in a 3-D background model. The texture image obtained by the interpolation of the foreground objectives was used for soft blending of the background model and the foreground model as an alpha texture.
6Experimental Results
In this study, TIP model based on panorama taken by OneShot VR system (Kandan Inc.) was employed. Scanning interval was 50 m. As an image rendering system used was iPaq 3850 (HP Inc.) that has built-in 64 MB ROM and 64 MB RAM, was operated on Microsoft Windows CE, and represented 65,000 colors at a resolution of 320 X 240. Development environment was constructed on PC using Microsoft Embedded Toolkit.
Fig. 2 shows the 23 locations (red dots) where the panoramic pictures were obtained for the experiments. Fig. 3 presents the panoramic pictures samples taken at every 50 m from Daejeon Meteorological Office to KAIST. Fig. 4 exhibits the rendering image obtained on PDA with moving from Daejeon Meteorological Office to KAIST. Fig. 5a) and 5b) shows the same images obtained on PC. The acquisition direction of each panoramic image was same as the moving direction.
As results, final images of 320 X 240 size were obtained at a scan rate of 25 frames/sec on average. When the environment models used for the image rendering were modified and hence additional access to hard disc occurred, the scan rate was reduced to 15 frames/sec.
Fig.2 The 23 locations (red dots) for panoramic picture acquisition.
Fig.3a A panoramic picture taken in front of Daejeon Meteorological Office.
Fig.3b A panoramic picture taken at 50 m point from Daejeon Meteorological Office to KAIST.
Fig. 3cA panoramic picture taken at 100 m point from Daejeon Meteorological Office to KAIST.
Fig.3dA panoramic picture taken at 150 m point from Daejeon Meteorological Office to KAIST.
Fig. 3eA panoramic picture taken at 200 m point from Daejeon Meteorological Office to KAIST.
Fig 3. Panoramic picture samples.
Fig.4aRendering image of the right view from Daejeon Meteorological Office.
Fig.4bRendering image moving from Daejeon Meteorological Office to KAIST.
Fig.4cRendering image of the 45 o right view on the way from Daejeon Meteorological Office to KAIST.
Fig.4dRendering image of the side road of KAIST.
Fig.4eRendering image of the top view of the road.
Fig.4fRendering image of the right view from Daejeon KAIST.
Fig.4 Navigation images on PDA for the virtual environment.
Fig.5a Rendering image of the 30 o left view from the crossway in front of Daejeon Meteorological Office.
Fig. 5a.Rendering image of the 30 o right view from the crossway in front of Daejeon Meteorological Office
Fig.5. Navigation images on PC for the virtual environment.
7Conclusions
In this study, the currently-used TIP method was improved to widen the range of visual point in a long distance configuration. To construct a wide virtual environment, multiple TIP models were constructed using the panoramic pictures taken at many different points.
To obtain a rendering image for a given location, three nearest TIP models were interpolated. For a foreground objective, apexes and colors of the objectivewas interpolated to construct a new foreground image. For a background model, a texture image was created by image warping for the texture to obtain a new background image.
Experimental results showed that using the TIP method developed in this study, rapid and soft navigation could be displayed on PC.
References:
[1]Chen S. E, “QuickTime VR - An Image-Based Approach to Virtual Environment Navigation”, Computer Graphics (SIGGRAPH 95), 29-38, 1995.
[2][McMillan95] McMillan L. and Bishop G., “Plenoptic Modeling: An Image-Based Rendering System”, Computer Graphics (SIGGRAPH95), 39-46, 1995.
[3]Gortler S., Grzeszczuk R., Szeliski R., and Cohen M., “The Lumigraph”, Computer Graphics (SIGGRAPH 96), 43-54, 1996.
[4]Levoy M. and Hanrahan P., “Light Field Rendering”,Computer Graphics (SIGGRAPH 96), 31-42, 1996.
[5]Y. Horry, K. .Anjyo, and K. Arai. Tour into the picture: using a spidery mesh interface to make animation from single image. In Proc. SIGGRAPH, pages 225--232, 1997.
[6]Hyung Woo Kang, Soon Hyung Pyo, Ken-ichi Anjyo, Sung Yong Shin: Tour Into the Picture using a Vanishing Line and its Extension to Panoramic Images. Computer Graphics Forum 20(3): (2001).