In a lot of cases, moles are genetic.
I don't see it as a mutually exclusive question of diet
or genetics, but rather diet
and genetics and perhaps sun exposure, as was mentioned above, and maybe some other unknown factors.
The whole process might take years, though.
Try decades in my case. And the falling off process didn't occur at all until I went Paleo, after which multiple moles and lesions started shrinking, and others have reported the same to me. My brother eats relatively healthfully as compared to most Americans, but does not eat Paleo or raw and despite the fact that he is older than me, none of his prominent moles have disappeared. All this could be amazing coincidences, but it could also be something more.
Not everyone experiences this benefit though, so I caution folks against getting their hopes up.
Like FoxWoman said, moles tend to be benign and only rarely develop into something cancerous like malignant melanoma. If you have a lot of them or any fast-growing or unusual-looking ones, it's probably a good idea to have your physician check them every now and then, which they should be doing anyway, but often don't make the time for it unless you mention it.