Impact Of Sleep On Sex Life! With more people frankly discussing the loss of sex drive, studies have been conducted to investigate the reasons behind this. Surprisingly, in addition to hormonal and functional causes like genital issues, one common factor that causes loss of libido in both men and women is lifestyle changes. In addition to improper food habits and loss of exercise.