Recently, great attention has been paid to the reinforcement learning (RL) algorithm in the fields of the artificial intelligence and the machine learning, as a tool to solve a class of the optimization problem. We try to construct the RL framework to find the shortest course of a ship in the following fundamental situations : (A) A ship goes on a restricted sea-area with the strong tidal current, such as the Kurushima strait. (B) Two ships go on a sea-area with no tidal current while each of them avoids the collision with the other. Q-learning algorithm, which is representative of the RL algorithm, is combined with the ship's motion equations through the quantization of their variables. Finally, the effectiveness of our framework is demonstrated with the model of the sea-area.