Computer vision research has seen tremendous success in recent times . This success can be attributed to recent breakthroughs in deep learning technology and such systems have been shown to achieve super human performance on several academic datasets. Driven by this success, these systems are actively being deployed in several household and industrial applications like robotics. However, current systems perform poorly when deployed in the real world, a.k.a in-the-wild, as most of the assumptions made during the modeling stage are violated. For example, consider object detectors, they require clean data for training and they are not effective in detecting or rejecting novel categories not seen in the data.In this thesis, we systematically identify problems that arise in a typical learning setup, the input, model and the output, and propose effective solutions to mitigate them.