This cross-sectional study investigate the association between long working hours, short sleep duration, and mental health among Japanese physicians. We enrolled 232 Japanese physicians. We used the Brief Job Stress Questionnaire to assess high-stress status, and the Japanese version of the Center for Epidemiologic Studies Depression scale to assess depressive status. Daily sleep duration (DSD) and weekly working hours (WWHs) were collected using a self-administered questionnaire. Multivariable-adjusted logistic regression analysis was performed to examine the association of the combined categories of DSD and WWHs with high-stress and depressive status. Compared to physicians with WWHs <80 h and DSD ≥6 h, the multivariable-adjusted odds ratios (ORs) and 95% confidence intervals (CIs) of high-stress status for those with WWHs ≥80 and DSD ≥6, WWHs <80 and DSD <6, and WWHs ≥80 and DSD <6 were 2.76 (0.97–7.87), 3.36 (1.53–7.40), and 3.92 (1.52–10.14), respectively. The respective ORs (CIs) of depressive status were 1.82 (0.42–7.81), 4.03 (1.41–11.53), and 4.69 (1.33–16.62). The results showed that regardless of working long hours or not, physicians with DSD <6 h had significantly higher stress and depressive status, suggesting that not only regulating long working hours but also ensuring adequate sleep duration is important for preventing physicians’ mental health.