This is limited details and more precise information is needed about your skin condition which can help in planning a treatment. A thorough history, clinical examination and appropriate laboratory tests will be needed to solve the mystery. A photograph would have been helpful. Consider seeing a Dermatologist for proper diagnosis and treatment plan. I believe you might be talking of a skin condition called urticaria but it needs to be confirmed.
Your dermatologist will decide a treatment plan for you after a consultation.