Real-time Registration of Visible and Thermal Network Cameras

Raphael Zaugg

Semester Master Project

Microengineering section, EPFL

June 2011

Abstract

It has always been a critical task to segment humans in video sequences due to partial occlusion or change of luminosity and shape. The aim of this project is to combine the images from a visual and a thermal network camera to do the person segmentation. To achieve this task, we search a transformation to map the pixels of one image to the other. In a first step the cameras are calibrated. Point matches are registered using a light bulb and the epipolar geometry is estimated. Then, a background subtraction is done for both images, so that only the shape of the person is left, before they are rectified. After this rectification the only disparity is in x-direction and for all points of the person this disparity is the same. The last step is to shift one image by the measured disparity. This method allows to get a transformation that can be used to fuse the visual and the thermal images in order to provide information to a person tracking system.

In a second part I have implemented a tool that allows to calibrate two visual images by clicking on corresponding points. The point matches are refined by a Harris keypoint detector in order to increase the precision and the quality of the calibration.