Abstract: In visual tracking tasks, traditional particle filtering algorithms usually accumulate the error generated during model updating if targets change their appearances. To overcome this difficulty, by exploring the color information on targets differing from backgrounds, a characteristic color model was built and an improved filtering algorithm was proposed. In tracking process, targets were roughly located first by a common particle filtering, then segmented based on established color model. Experimental results show that the proposed algorithm can track targets in real time and capture the appearance changes accurately. Meanwhile, the proposed algorithm is robust to rotation, occlusion and illumination variation of the targets. This new algorithm is especially suitable for tracking objects that possess characteristic colors, such as pedestrians and automobiles.