Current assistance systems for manual assembly reduce the efficiency of the worker by being invasive in the workflow. To restore the efficiency and at the same time to maintain the benefits of assistance systems, real time hand pose estimation can be used. However, no suitable data set is available for training such application specific detectors. In the presented work, a data set is generated that allows the use of different work gloves and prepares the overlay of realistic hand textures. We use low cost data gloves for hand pose tracking and a RGBD camera to capture the data set with 30 data points per second. This low cost approach is presented in an application for the manual assembly scenario, although transfer of the method to other scenarios is possible.