Gallstones during pregnancy. This pain?

Hello Doctor, Last year i did master health checkup and got to know that i have multiple gall stones but i had no symptoms. I visited general physician and he said if there is no symptoms no need to worry. I planned for pregnancy and now i am in my 4th month of pregnancy and i feel constant pain right side below ribs which sometime radiates to left side and back as well. I consulted my gynaecologist and she said it is because of gall stones but currently nothing can be done as it needs to be managed with food only. Is this true and is there no way to get out of this pain? I have conceived after trying a lot and really want this baby and this pregnancy to go smooth. I am not able to enjoy the moments of pregnancy , kindly guide me how to overcome this pain.!