Nodule under nipple. I just ignore it?

Male,20- 5 months ago i used to experience little pain whenever i used to press the area on my left chest, at the nipple. After 1 month i noticed that the area around my left nipple have swelled and it felt cushny when i touched and the pain was still there while my other nipple was flat and normal. After 1 more month the pain suddenly disappeared but the swelling was still there. I went to pgi for check up and told the surgery department physician all this, he pressed the area around my nipple and straight out told me that there's nothing. ... But whenever i look at the mirror and compare my both chest sides i can clearly see that my left side nipple area is kind of swelled up as compared to my right side. And i can also feel a uneven cup shaped small nodule. It's very small but it's there. So what to do? I know something is in there.The pain is not there and I'm not uncomfortable with it at all so should i just ignore it?