Breathing problems usually gets increased on exertion like doing gym if there s any respiratory or cardiac issue . Unlike your case you are feeling better on exercise . It could be just are having psychological stress and feeling about breathing problem in inactive time. If breathlessness associated with cough , chest pain or palpitations then consult a pulmonologist
Next Steps
watch for relevant symptoms mentioned above if breatlessness persists consult a pulmonologist