Technically speaking your hormones can be stopped now as you have crossed 12 weeks. But that is totally upto your gynaecologist/fertility consultant and you must follow their advice.
Bad taste in mouth could be due to the iron tablets that are started after 12 weeks. Just change the brand and it will go away but do not stop taking iron.
Your body busy making a whole person. Some symptoms here or there will keep happening.try to take them lightly. Usually from the end of 4th month till the end of seventh pregnancy is most enjoyable so by then you should feel much better.
Medication for your mental health can only be adjusted by your psychiatrist and since you are
pregnant there will be restrictions on how much dose they can give. Other measures you can take up meanwhile are meditation,antenatal yoga, walking, light excercises, pursuing a hobby, spending least time alone and more with family or friends, keeping yourself busy. Hope you feel better soon.