This paper proposes a Particle Swarm Optimization (PSO) with hierarchical structure. In the proposed method, particles are separated into some groups, and besides, in a group particles are parted the particle of the best value from other particles. Particles of the best value in each group are applied to Gbest Model, and other particles are applied to Lbest Model. Then, the proposed method is validated through numerical simulations with several functions which are well known as optimization benchmark problems comparing to the conventional PSO methods.