Abstract: In order to solve natural interactive problems between human and wearable computers,a hand tracking and gesture recognition method based on wearable vision is proposed.The method is an extension of the Icondensation algorithm,which integrates the shape and depth information provided by a wearable vision system for robust hand tracking.Furthermore,gesture transition model is introduced to recognize gestures.The method can automatically switch between multiple pre-defined gestures in the tracking loop.Experimental results showed that the proposed method worked well in dynamic and complex background.The method can provide natural gesture-based interfaces for wearable computing systems.