I am interested in machine learning approaches to the automatic extraction of medium level
representation of natural signals. The hope is that by structuring statistical
generative models to mimic the structure of the real world, the models will be
able to automatically adapt to audio, visual or multimodal signals during the
unsupervised model fitting (learning) stage, thus providing a medium-level
representation suitable for compression, transmission, search, editing, enhanced
viewing experience, etc. These models are object-based, where an object can
produce sounds, have a changing appearance, move and be exposed to attenuation
in audio domain, illumination in video domain, and, when other objects are
present, to occlusion or additive mixing in both domains. The main requirement
is complete adaptivity, i.e., avoiding any design parameters preset for a very specialized application. For example, the same model should be
applicable to tracking a person in front of a cluttered background, and to
tracking a flock of birds. The tracking task, as well as many other tasks
performed jointly, such as de-noising, dynamic mosaic building or object
removal as well as separating audio sources and associating them to object
appearances, are all achievable as probabilistic queries, i.e., inference of the
hidden variables associated to the world structure. All this should be doable
using the data itself, without special application-specific initialization
procedure or the separate supervised training stage.
University of
Belgrade, Belgrade, Yugoslavia B.S., Electrical Engineering,
June, 1995 B.S. thesis title: "Image compression using wavelets and
scalar coding of sub-bands", advisor: Prof. Miodrag Popovic
PUBLICATIONS
Book Chapters:
Nebojsa Jojic, Yong Rui, Yueting Zhuang, Thomas S. Huang, "A framework
for garment shopping over the Internet" to appear in May 1999 in
Handbook of Electronic Commerce, edited by Mike Shaw, Springer Verlag.
Brendan Frey and Nebojsa Jojic, "Transformation-Invariant Clustering and
Linear Component Analysis using the EM Algorithm, Part I," IEEE
Transactions on Pattern Analysis and Machine Intelligence (PAMI) accepted
for publication in 2001.
Brendan Frey and Nebojsa Jojic, "Transformation-Invariant Clustering and
Linear Component Analysis using the EM Algorithm, Part II," IEEE
Transactions on Pattern Analysis and Machine Intelligence (PAMI) accepted
for publication in 2001.
Nebojsa Jojic, Matthew Turk, Barry Brumitt and Thomas Huang "Articulated
Gaussians in Dense Disparity Maps," in preparation.
Nebojsa Jojic and
Brendan Frey, "A generative model for 2.5D vision: Estimating appearance,
transformation, illumination, transparency and occlusion," submitted,
International Journal on Computer Vision, 2002.
Ph.D. Thesis: "Generative Models for Computer Vision"
Conferences:
Matt Beal, Nebojsa Jojic and Hagai Attias, "Audio-Assisted
Visual Tracking: A Probabilistic, Self-Calibrating Algorithm," accepted, European Conference on Computer Vision, 2002.
Chris
Pal, Brendan Frey and Nebojsa Jojic, "Learning and representing objects that
change in shape using montages of transformed latent images," submitted to
European Conference on Computer Vision, 2002.
Matt Beal, Nebojsa Jojic
and Hagai Attias, "A self-calibrating algorithm for speaker tracking based on
audio-visual statistical models," accepted, IEEE International Conference
on Speech, Acoustics and Signal Processing, 2002
Brendan Frey and Nebojsa Jojic,
"Fast, Large-Scale Transformation-Invariant Clustering," in Advances in Neural
Information Processing Systems 14 (NIPS 2001)
Brendan Frey, Anitha Kannan and Nebojsa Jojic, "Product Analysis,"
in Advances in Neural Information Processing Systems 14 (NIPS 2001).
Nebojsa Jojic, Patrice Simard, Brendan Frey and David Heckerman, "Separating
Appearance from Deformation," International Conference on Computer Vision,
July 2001, Vancouver, BC, Canada.
Nebojsa Jojic, Patrice Simard, Brendan
Frey and David Heckerman, "Learning mixtures of smooth nonuniform deformation
fields for probabilistic image matching," Eight International Workshop on AI
and Statistics, January 2001, Key West, Florida.
Nebojsa Jojic, Barry Brumitt, Brian Meyers and Steve Harris, "Detecting
and estimating pointing gestures in dense disparity maps," IEEE
International Conference on Face and Gesture Recognition 2000, Grenoble,
France, March 2000.
Brendan Frey and Nebojsa Jojic, “Learning Graphical
Models of Images, Videos and Their Spatial Transformations,” Uncertainty in
Artificial Intelligence, Palo Alto, 2000.
Brendan Frey and Nebojsa Jojic,
“Transformation-Invariant Filtering Using Expectation Maximization,” Symposium
2000 on Adaptive Systems for Signal Processing, Communications and Control,
Lake Lousie, Alberta, Oct 1-4.
Bridget Carragher, Nebojsa Jojic, et al., "Automated Acquisition of Cryo
Electron Micrographs Using Leginon," Microscopy and Microanalysis '99,
pp. 376-377. Portland, Oregon, August 1-5, 1999.
Brendan Frey and Nebojsa Jojic, "Transformation Invariant Mixture Models,"
(invited paper), Machines that Learn Workshop (Snowbird), March 1999.
Nebojsa Jojic, Matthew Turk, Thomas S. Huang, "Tracking Articulated
Structures in Stereo Sequences," IEEE Information Theory Workshop on
Detection, Estimation, Classification and Imaging, Santa Fe, February
1999. pp. 77
Nebojsa Jojic, Yong Rui, Yueting Zhuang, Thomas S. Huang, "Computer
Vision and Graphics Techniques for Garment Shopping over the Internet," International Computer Symposium, Taiwan, December 1998.
Nebojsa Jojic, Thomas Huang, "On analysis of
cloth drape range data", Asian Conference on Computer Vision (ACCV)
'98, Hong Kong, January 1998, pp. 463-470.
J. Gu, N. Jojic, I. Mak, H. Shen, S. Gopalsamy and T. Huang, "Camera-Based
Human Body Reconstruction," 1998 Symposium on Image, Speech, Signal
Processing, and Robotics (ISSPR), Workshop on Computer Vision, September
'98, Hong Kong, pp. I:79-84
J. Gu, N. Jojic, H. Shen, and T. Huang, "Design of a Camera-Based 3D
Data Acquisition System," 5th International Conference on Mechatronics and
Machine Vision in Practice (MVIP'98), September '98, Nanjing, China, pp.
93-98.
D. Nandy, J. Ben Arie, N. Jojic, Z. Wang, R. K. Rao, "On the use of
the Karhunen-Loeve transform and expansion matching for generalized feature
detection", 1996 IEEE International Conference on Acoustics, Speech, and
Signal Processing Conference Proceedings (ICASSP), pp. 2223-6 vol. 4, 5
refs. 1996.
Z. Wang, R. K. Rao, D. Nandy, J. Ben Arie, N. Jojic, "A generalized
expansion matching based feature extractor" in Proceedings of the 13th
International Conference on Pattern Recognition. pp. 29-33 vol.2, 5 refs.
1996.
Z. Wang, R. K. Rao, D. Nandy, Ben Arie, N. Jojic, "Generalized
feature detection using the Karhunen-Loeve transform and expansion
matching" in Proc. SPIE - Int. Soc. Opt. Eng. (USA). vol 2727,
pt 1, pp. 416-22, 5 refs. 1996
Mehdi N. Yu X. Wu D. Jojic N. Baker B. Gosavi P. Seyd A. Bodak S. Umetani
K. He B. EDITED BY: Bajpai P K. "Body surface Laplacian mapping from
potential recordings in man" in Proceedings of the 1996 Fifteenth Southern
Biomedical Engineering Conference, pp. 359-60, 4 refs. 1996.
Acknowledgements: Most of the research was funded by
Microsoft, ARL under the cooperative agreement No. DAAL01-96-2-0003 and
NSF (grant No. IRI-9634618). The work I did in Hong Kong was also
funded by the Hong Kong Industry and Technology Development Council (grant No.
AF/122/96).
PATENTS
Microsoft Corp. has submitted the following patents on my behalf:
System and method to facilitate pattern recognition by deformable matching
Method of learning deformation models to facilitate pattern matching
A system and method for visually tracking occluded objects in real time
HONORS
Robert T. Chien Memorial Award for excellence in research, ECE Dept.
University of Illinois at Urbana Champaign, 2001.
Microsoft Graduate Fellowship for the school year 1999/2000.
Fellowship awarded by The Ministry of Science and Technology
(Yugoslavia) throughout undergraduate studies