In this paper, we propose a curiosity algorithm for sample efficient data collection in robotic task learning. Reinforcement learning (RL) is one of the most commonly used methods for robotic task learning. However, when the RL agent explores the environment randomly, it results in low exploration efficiency, especially in learning robotic object manipulation tasks with high-dimensional and continuous state and action space. To improve the sample efficiency, it is necessary to urge the robot to pursue unseen states where exploration is lacking. By making the robot curious, the robot would become interested in the unknown region and explore that region. In order to make the robot curious, we utilize the free energy principle, which has been attracting attention in the field of neuroscience. We propose a method that combines curiosity based on the free energy principle with Soft Actor-Critic, a state-of-the-art reinforcement learning method, and verify the effectiveness of the proposed method in simulation and real-world experiments of robotic object manipulation task learning.