In this thesis, we consider the forms of non-equilibrium phenomena that can arise in atoms or polar molecules trapped in a deep optical lattice and coupled by dipolar internal degrees of freedom. We specifically focus on only two internal states, which results in the systems studied behaving as spin$-1/2$ models with long-range interactions. We first study a closed system with both static and resonant near-field dipole interactions under an external drive. By studying the uniform mean-field dynamics of the system, we find the dynamics are given by Rabi oscillations, with a bifurcation in the dynamics as a function of drive strength between small scale and large scale oscillations. Analysing the stability of these oscillations to small fluctuations reveals that interactions tend to cause the oscillations to decohere. However, we find parameter regimes where coherent oscillations can persist for high enough intensity drive. We then consider the effects of an environment on the non-equilibrium dynamics of the near-field dipole model. We find that within the mean-field approximation, an environment causes the system to relax to many novel steady state spin configurations, such as spin density waves, antiferromagnetism and long-time oscillations, as well as bistabilities between these phases. To assess the validity of the mean-field approximation, we compare our mean-field results to small quantum systems. We carry out a similar analysis on a system with far-field dipole interactions, where it is necessary to introduce nonlocal dissipation, which results in several decay modes into the environment. These decay modes lead to instabilities of many of the steady state phases that occurred in the near-field dipole system, leading to the emergence of more spin density wave and oscillatory phases. Finally, we examine the dynamics on the approach to steady state in a dissipative dipolar system by studying Rydberg atoms coupled to a photonic crystal waveguide, which mediates an effective dipole-dipole interaction between the atoms. We find that if two excitations exist in the system, then bound states can form, with nonlocal dissipation resulting in a momentum dependent decay rate of the bound states and also greater freedom in engineering the bound state energy dispersion.