As technology advances, the BIG-DATA Era is beginning. Because the data volume is increasing so fast, hardware development becomes an importance development technology. In software, there is a growing number of machine learning models using the CPU to do the parallel computing. But with limited resources, they can not catch up the data volume and demanded computing speed. Therefore, this study wants to explore the hyper parameter optimization in machine learning model training. The optimization often uses trial and error to find the best hyper-paramater combination, but it will need a lot of time. The research uses two datasets (IRIS, MNIST) and three models (neural network, nearest neighbor algorithm, extreme gradient boosting) to compare hyper-parameter optimizations with and without using the design of experiments.