I had thyroid during my pregnancy which already made my digestion very slow. Milk made my constipation worse, so I had to get my doc to adjust my diet.
My doctor explained that the high calcium in milk can sometimes slow down digestion, especially if you are on iron supplements. Try reducing the quantity or switching to warm milk.
Personally, drinking cold milk gave me bad constipation, but warm milk in the morning was totally fine. Maybe try changing the temperature and see how it suits you.
Pregnancy hormones slow down your gut, and milk can sometimes add to it. Be gentle with your body and talk to your doctor about gut-friendly alternatives.
My grandmother told me to mix a teaspoon of ghee in warm milk at night to help with motion. I checked with my doctor first, and it actually worked wonders.