This is absolutely normal. All babies cry just before urination and while pooping. That's the only way a baby can express discomfort! An exclusively breastfed baby is not likely to get an infection. I suggest you omit the antibiotics prescribed. It is very normal that babies poop immediately when a feed is going on... That's a normal reflex.
Next Steps
Continue exclusive breastfeeding and burp baby well after each feed.
Health Tips
As long as baby is feeding well, there is no need to worry.