Abstract: A new method of solving the problem of camera moving, based on difference image sequence processing, was proposed, with which the complex backgrounds were converted into relatively simple ones. The data of the background moving can be obtained by template matching using obvious features of the background and transmitted to difference image sequence processing part. By this way, the edge of background is removed and only the edge of the moving target is left in difference image sequence. As proved by experiments, this method is effective.