This paper describes the construction of a cooperative system of a dual-arm robot and an AGV using RT Middleware. This cooperative system is composed of several modules: a state machine, an AGV controller, an object pose estimation, and a pick-and-place planner. In order to realize a cooperative task, the state machine module controls the other modules using its input and output of state transition. The AGV controller module controls AGV for object transportation; the object pose estimation module estimates the poses of the objects transported by the AGV; the pick-and-place planner is used for pick-and-place of the estimated object by the dual-arm robot. We constructed the system with these modules, and confirmed the performance through an experiment.