If you are not facing the same problem with other milk products like curd cheese etc, the problem is in the milk not you, as you have been taking it for very long time. Hope this helps.
Next Steps
Also look for if the problem is with cow milk or buffalo milk or any milk powder
Health Tips
try milk from other sources or brands and observe your bowel habits then