Hello there ...well burning sensation in the upper back is bcos of various reasons....it can be a pinched nerve between your vertebra or it can be muscle strain,,or it can even be bcos of poor and faulty posture.
As you said yourself that when you dont sit for long time infront of computer or dont ride the bike u feel better...it can be a signal that maybe these sternious activities can be causing the problem.
Next Steps
You should first n foremost correct your sitting and standing posture...you can send a photo of yours in standing and sitting on chair ie how you sit and i can guide you the right way of working on your computers and doing activities.
neck and trapezius pain may or may not be related to your mid back pain...for which an xray of cervical spine and tightness of trap needs to be assessed.
Health Tips
exercises and stretching should be done under supervision as some of them might aggrevate the condition.
what all exercises are you doing right now?
resort back to me with the answers...
Regards & take care.