The conventional teaching/playback scheme is widely used for robot programming due to its reliablity and general versatility. However, it is applicable only when task conditions does not vary. In this paper, we study a method of robot programming with view-based image processing: view-based teaching/playback. It can adapt to changes of task conditions such as variations in initial pose of the object, without losing the general versatility. The method is composed of two parts: teaching phase and playback phase. In the teaching phase, a human operator commands a robot to perform a manipulation task as a demonstration. Scene images and corresponding robot motions are recorded in the demonstration. A mapping from the images to the robot motions is obtained as a neural network by learning from the demonstration data. In the playback phase, a robot motion is generated by the mapping. We applied this view-based teaching/playback to pushing tasks by an industrial robot. After multiple teaching demonstrations, pushing an object to a goal position was successfully achieved from some initial poses that are not identical to those in the demonstrations. This indicates that our view-based teaching/playback can deal with a certain range of variations in initial pose of the object without object models or camera calibrations.