How old is your brother? Since when you have started noticing the symptoms you have described in your question? Based on your provided history, your brother definitely seems to have some form of mental illness. This may be either obsessive compulsive disorder that is OCD or more  serious form of mental illness like schizophrenia. In both probable scenario, you need to persuade him  to visit a psychiatrist or any mental health professional like clinical psychologist who can assess him clinically and give a definitive diagnosis to his ailment. Based on his clinical profile, he will be treated with a combination of medications and psychotherapy or in simple words counselling sessions, first with patient and then with family. Hope this helps warm regards