On-Line Novel View Synthesis Capable of Handling Multiple Moving Objects

  • Indra Geys
  • Luc Van Gool
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3766)


This paper presents a new interactive teleconferencing system. It adds a ‘virtual’ camera to the scene which can move freely in between multiple real cameras. The viewpoint can automatically be selected using basic cinematographic rules, based on the position and the actions of the instructor. This produces a clearer and more engaging view for the remote audience, without the need for a human editor.

For the creation of the novel views generated by such a ‘virtual’ camera, segmentation and depth calculations are required. The system is semi-automatic, in that the user is asked to indicate a few corresponding points or edges for generating an initial rough background model. Next to the static background and moving foreground also multiple independently moving objects are catered for. The initial foreground contour is tracked over time, using a new active contour. If a second object appears, the contour prediction allows to recognize this situation and to take appropriate measures. The 3D models are continuously validated based on a Birchfield dissimilarity measure. The foreground model is updated every frame, the background is refined if necessary. The current implementation can reach approx 4 fps on a single desktop.


Active Contour Background Model Delaunay Triangulation Dissimilarity Measure Foreground Object 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arapis, C., Konstantas, D.: Design and implementation of a teleteaching environment. In: Ninth DELOS Workshop, pp. 79–83 (1999)Google Scholar
  2. 2.
    Billinghurst, M., Bowskill, J., Jessop, M., Morphett, J.: Wearable spatial conferencing space. In: ISWC 1998, pp. 76–83 (1998)Google Scholar
  3. 3.
    Billinghurst, M., Kato, H.: Real world teleconferencing. In: CHI 1999, pp. 194–195 (1999)Google Scholar
  4. 4.
    Criminisi, A., Shotton, J., Blake, A., Torr, P.: Gaze manipulation for one-to-one teleconferencing. In: ICCV 2003, pp. 191–198 (2003)Google Scholar
  5. 5.
    Criminisi, A., Blake, A.: The SPS Algorithm: Patching Figural Continuity and Transparency by Split-Patch Search. In: CVPR (1), pp. 342–349 (2004)Google Scholar
  6. 6.
    Yang, R., Welch, G., Bishop, G.: Real-time consensus-based scene reconstruction using commodity graphics hardware. In: Pacific Graphics 2002, pp. 225–235 (2002)Google Scholar
  7. 7.
    Ansar, A., Castano, A., Matthies, L.: Enhanced Real-time Stereo Using Bilateral Filtering. In: 3DPVT 2004, pp. 455–462 (2004)Google Scholar
  8. 8.
    Hirschmüller, H., Innocent, P.R., Garibaldi, J.: Real-time correlation-based stereo vision with reduced border errors. International Journal of Computer Vision 47, 229–246 (2002)zbMATHCrossRefGoogle Scholar
  9. 9.
    Triggs, B.: Autocalibration and the absolute quadric. In: CVPR 1997, p. 609 (1997)Google Scholar
  10. 10.
    Pollefeys, M., Koch, R., Van Gool, L.: Self-calibration and metric reconstruction inspite of varying and unknown intrinsic camera parameters. International Journal of Computer Vision 32, 7–25 (1999)CrossRefGoogle Scholar
  11. 11.
    Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Anal. Mach. Intell. 20, 401–406 (1998)CrossRefGoogle Scholar
  12. 12.
    Geys, I., Van Gool, L.: Hierarchical coarse to fine Depth Estimation for Realistic View Interpolation. In: 3DIM 2005, pp. 237–244 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Indra Geys
    • 1
  • Luc Van Gool
    • 1
    • 2
  1. 1.ESAT/PSI-VISICSKatholieke Universiteit LeuvenLeuvenBelgium
  2. 2.D-ITET/BIWISwiss Federal Institute of Technology (ETH)ZürichSwitzerland

Personalised recommendations