My current big project is called ‘Raw Shaping‘, and is part of an Industrial Design research project. The object is to provide designers with an intuitive tool that allows them to simultaneously interact with real-world and virtual models of whatever they are working on. It’s a real challenge and my initial approach is going to involve a lot of computer vision technology. My biggest issue is to achieve realtime 3D reconstruction on commodity hardware… I’ll let you know when I’ve got something working properly.

A lot of approaches in computer vision are either very slow or not robust enough to actually use. At the moment of writing I’m using openCV‘s built-in dense stereo correspondence to create a disparity map from two camera images, and using this as the basis for a 3D point cloud. Although it works somewhat, I’d like to be able to use a simpler form of image rectification or make use of another, more robust algorithm that results in a better reconstruction.

Generating a 3d model from the disparity map is also a lot slower than I’ve hoped. At the moment I’m using an OpenSceneGraph heightfield primitive, but setting the heights for all points is simply too slow at even modest resolutions of the disparity map. I’ll attempt point sprites next, and if that’s also too slow I’m looking at some GPU-based implementation.