20 mm is definitely not a small size, consult a nephrologist immediately to understand the further course of action. Depending on the position of the stone, they may suggest the type of procedure/surgery required.
In case it blocks the ureteric opening, it might damage the
kidney as well due to back pressure.
In case you have anxiety related to surgery, ask your doctor to explain you the procedure with risks and benefits, or consult a psychiatrist to deal with that anxiety with short term psychotherapy or medicines.
Next Steps
Immediately consult a nephrologist