Yellow deposits known as plaque/tartar/calculas.it accumulate due to poor brushing mainly, other causes hormonal imbalance, etc. You need to get it cleaned from dentist, otherwise it will further damage your gums ,and eventually teeth will become loose.
Also this further lead to gaps/ teeth will move outside.
Teeth never grow in size, may be deposits giving you false illusion.
Next Steps
visit dentist for scaling and further check up.