Dreams.

There are two phases of sleep: active and deep. Depending on fatigue and workload, one phase is shorter, and the other is longer. In your case, the deep sleep phase predominates. Since you often don't dream, it means you're essentially waking up while in this phase, which is why you don't remember your dreams.