The only times a doctor has manipulated my penis was during an STI check.
I had some weird redness on the head of my penis a while back, so I went to a local STI open clinic. It turned out to just be caused by the harsh bodywash I was using at the time.
After taking a brief history, the doctor (40-ish mustached man) asked me to "go ahead and slip down your pants"
I stood up and complied, holding my tshirt up and out of the way, exposing myself from about my knees to my nipples. He held my penis in his hand for a long time, looking at the various trouble spots.
When he was done, I pulled up my pants, and he actually called in a second, much younger doctor in the clinic into the room for a second opinion. I thought he was just going to describe the problem, but to my surprise, the second doctor asked me to "please lower those, if you wouldn't mind"
Now I had two entirely clothed doctors, one older and one younger looking closely at my penis while the younger doctor held it and moved it around in his hand.