Humans started eating dairy very early on, perhaps early in their transition from hunter-gatherers to slightly more settled pastoralist life, but still definitely pre-agricultural. The Mongols and Kazakhs still "hunt" wild mares for their milk - they'd rope them, milk them and release them again. Some humans might be descended from stocks who didn't drink milk but that's no reason for paleo practitioners to avoid milk, in my view.

Check this video on youtube: gKjz2ae3mYU