Hi dere, this is a controversial question and I will try to my best to answer.
Physical relationship is never as it sounds. It's not just physical but psychological as well. U can not just indulge in physically contacting each other without some psychological connection happening simultaneously. There is a way to know the physical age but there is no real way to know somebody's psychological age. Physically its been agreed that 18 yrs is the right age by which one is physically competent to have sex. But then there have been instances where it happens earlier too depending on how developed a person is physically and psychologically.
There r adults who r psychologically not oriented to physical relation at all.
So in short everything bogs down to one thing and dats - MIND.
hope this helps ...but feel free to raise further queries in follow up section.