Although first time sex may cause psychological impotence due to severe anxiety but as you mentioned, you have noticed changes during rest days, it is possibly more serious.
Diabetes and obesity both are strongest risk factor for Erectile dysfunction ( ED or impotence). Please control diabetes, reduce weight, look for fasting
lipid Profile. Do good aerobic exercises, prefer vegetarian diet and fresh fruits ( will provide antioxidants), stop any tobacco product, adopt stress free life style.
Regular tadalfil 5 Mg to 10 mg will help you for time being untill above conservative measure reverse changes happened in your vasculer system.