From the xray , it seems you have fracture of clavicle bone. With conservative treatment , it will take 4-6 weeks ro to heal. You can adjust the sling if some position is uncomfortable to you. I wouldn't advise removing the brace as it will worsen the pain.
Take analgesics in cae of pain , it will reduce in a weeks time.
Next Steps
If you can't bear the pain , better to consult orthopaedic surgeon as there are other treatment options available.