Abstract: Pattern search is the essential part of all fast search motion estimation algorithms of MPEG-4 or H.264 standard. Efficient search patterns can improve the performance of the existing fast search algorithm. In this paper is analyzed the computation distribution of the block-based fast search motion estimation algorithm and the performance of different search patterns. In order to speed up the motion search procedure while maintaining the quality of the decoded sequence, we chose the double search patterns strategy, and suggested an adaptive pattern selection method. The experimental result showed that the small pattern can improve the search efficiency and the large pattern can guarantee the search accuracy. This adaptive double pattern search algorithm can achieve very similar PSNR and bits performance compared with partial distortion fast full search algorithm, while it has a speedup of 151 in real-time.