Psoriasis is an autoimmune disorder, a condition in which your body's immune system works erratically and destroys its own tissues, in this case the skin. Although milk has nothing to do with it, you can avoid it because milk and milk products are allergic to you as you mentioned. Consult with an immunologist who can help you in managing your condition. The disease is characterised by occasional flare up and remissions. So periodic visits and long term management is needed. Proper diet, stress free life, enough sleep, lifestyle change all helps in managing the condition. While disease process is well monitored in modern medicine you may also try along holistic approach too.