There might be a deposit of calculous/tartar (a layer of hard yellowish deposit) over your tooth along with stains. Irrespective of how many times u brush u won't be able to get rid of the calculous & stains. These in addition to imparting yellowish look to the teeth there by compromising on aesthetics, would also lead to halitosis(bad breath), & gum bleeding because these harbour many bacterias.
Next Steps
Only solution to it is getting it removed/cleaned by a dentist. There after u could maintain it by brushing twice daily.
If u r still unhappy with the colour of your teeth after scaling(tooth cleaning) you can also get dental bleaching done for a brighter teeth.
Health Tips
Get your treatment done by a professional dentist only, and you can use Xtar tooth paste post treatment for maintaince of your teeth as it inhibits the accumulation of tartar/calculous