My opinion is that since the amounts of lactose and casein in raw butter is so small, the actual problems that most people claim to experience regarding energy or other symptoms (or any quoted long term deficiencies) on raw dairy have absolutely no relation to known severe lactose or casein issues that people experience with pasteurized dairy. I agree with ys, and as I believe the other dairy critics claim the issue goes at least beyond lactose or having little to do with the lactose but with other (unknown and speculative) properties largely based in dairy not being consumed millions of years ago.
many people with LI do not in fact experience the same specific problems with raw milk as with pasteurized milk despite the fact that no one would argue that it doesn't contain lactose and casein. This does not make raw milk healthy per se or that it would not create other issues in people from selected from whole spectrum of health. Therefore it also does not mean that certain individuals are adapted to something and that this adaptation means they are healthy or unhealthy or that these things are fixed. All of this points to the issue being far more complicated then lactose, as all things being perfectly controlled, a person that had intolerance due to lactose in pasteurized milk would indeed have no problems with raw milk, or certainly raw butter from that perspective. With cultured butter or making ghee, ironically as raw-al points out, there are ways of essentially removing lactose thorough process of clarifying. Granted people can rightly claim the heat aspect in making this toxic, but the 'excuse' aspect ultimately falls on the side of those that get the same symptoms in the virtual absence of lactose.
In terms of imperfect non-controlled daily life, with fresh raw milk it is not as strait-forward because as I understand it, even folks like Aajonus (or at least his followers that I have talked to) agree that fresh milk might not be appropriate food at all times for everyone, particularly in the first few years on that diet. The issue often mentioned is indeed: (run for the hills) heavy metals.
While its always seen as an excuse that things like detox, interaction with existing past consumed dairy 'deposits', or interaction with non breast fed tissue or the like the are mentioned in regards to dairy, its by far the logical conclusion(s). We know that things like peanuts, gluten, and many industrial products that cause allergies can never result in good health but there seems to be some acceptance that raw dairy at least works for some people. Although perhaps this is a totally disingenuous compromise from people who just flat out believe raw butter is harmful.
Allergies that are really environmental/fungal issues seem to be at the root of all intolerance to healthy foods like seafood and fruits and the like and can usually be corrected through various protocols and might not go away just eating any natural diet. So its no mystery that if ones intolerance is due to some kind of internal build up of casein, there is no long term non-dairy approach that will necessarily fix this situation - that is if it is something one cares about.
Some people have terrible reactions to high potent raw foods that do have detoxifying effects like oysters, organ meats, high meats, and eggs which are considered to be part of a healthy diet. Its not always suggested that these foods be continued or the experiences are automatically positive for that moment, but rarely is it some element within the food that is causing the underlying issue. Its hard to speak about long term issues of magnesium ratios and so forth, but as I've said above, I've been through some negative experiences with raw dairy and still find now it to be a far better choice in fuel and more appropriate to my composition and heritage than all frozen fats or any tropical or sub-tropical plant fats.