In order for robots to move in the real world, they must first be able to understand how to move their own bodies, as there are uncertainties in robot structures, camera parameters, or the tool that it holds. We propose a Dense Image Jacobian Estimator (DIJE) as a method to estimate the image Jacobian online for every pixel in the image. The estimator is based on a Kalman Filter simplified to run on every pixel, and does not need a priori knowledge of the robot kinematics. The proposed method is applied to a markerless visual servoing controller and verified on a musculoskeletal robot.