Abstract: To solve the problem of complex scene classification, an approach based on local feature and hide conditional random fields (HCRF) is proposed. Firstly, the image is segmented into sets of super-pixels, and then the local features extracted from those regions are used as the characteristic vectors of input image observations. Secondly, the HCRF model is established to infer the scene category of the image, where every local feature has the corresponding latent variable. The parameters of the model could be estimated using random gradient ascent algorithm. On the public image dataset, the test results demonstrate that the proposed approach has better classification results than the previous methods.