Pain in abdomen , testicl. Exercises!?

I started feeling pain in testicle and abdomen after gymming . Went for test and it showed varicocele. I Stopped doing exercises and pain went away after a month. Now i feel slight pain after long hours of standing or sitting. I started doing light exercises after 2 years of gap and pain comes as soon as i lift. Sometimes it is in left abdomen and sometimes in left testicle. Doctor says they cant operate mild varicocele. Some says it is hormonal imbalance wrongly diagnosed as varicocele. Now i want to lift and do some exercises. Cycling causes pain too. Should i stop doing exercises!? Will it become more worse and affect my daily routine after doing exercises. If it will affect me in future i wont go for lifting. I like going to gym as it makes me feel better. Please help.