Abstract: Aiming at the problem that the numbers of operant actions are more in operant conditioning probabilistic automaton, this paper constructs a hierarchical structural operant conditioning probabilistic automaton, which is called as HS-OCPA bionic autonomous learning system. The HS-OCPA learning system which doesnt require the system model is designed mainly based on Skinner operant conditioning (Skinner OC) mechanism and probabilistic automata (PA). The HS-OCPA learning system uses OC learning mechanism to realize optimizing learning based on operant actions and system performance, and the OC learning mechanism is adjusted by the reorientation information of operant actions. Finally the optimal control strategy is searched on line. The simulation and experiment applied in two-wheeled robot poster balance control both show that the designed HS-OCPA learning system not only has quickly learning velocity but also has strong adaptive ability when numbers of operant actions are more.