Should you take your passport with you off your cruise ship while on a shore excursion?

This is one of the most hotly debated questions among cruisers, as there seems to be quite a divide of what you should or shouldn't do.
I recently ran across a new thread debating the virtues of taking your passport off the ship on Reddit, and I think it's a simple decision to always take my passport with me off the ship.
If you're on the fence about what to do or haven't thought about it, I wanted to share my outlook on the debate.
While there's risk in losing it, the benefits of having it in case of an emergency outweigh any concerns.
Travelers should always be prepared

My mantra when traveling is the same as the Boy Scouts when I was a kid, "Be prepared".
You never know what might happen, and emergencies or accidents while in port are never something you see coming until it's too late.
I opt to bring my passport with me off the ship while I go on shore excursions so I have it with me, just in case.

Passports are essential, as you cannot fly home without it. So if we got stuck somewhere and missed the ship, having my passport would be critical.
Without a passport, you'd have to travel to the nearest embassy of your home country to get one. Depending on the port, the nearest embassy might be many hours away drive by car.
On any tour, I'm often far enough away from the ship that running back to it is not practical.

The counterargument to my opinion is to leave the passport in your safe on the ship, as the cruise line claims if you're a no-show to returning to the ship, they'll clear out your belongings from your stateroom (including your safe) and leave it with the port.
That's a major leap of faith that I'm not willing to take because of the multiple chain of custodies involved.
I have less concern of theft as I do it not making it to me in the end, either due to negligence or someone not knowing where the items are when they were taken.
Photocopies of a passport are as useful as photocopies of cash

Some people will say they take a photocopy of their passport and then leave the real thing in the ship safe.
A photo of your passport is useless, as you cannot travel with it. You still need the physical documents.
There's no benefit at all of having a photo of it in terms of actually being able to use it.
Three exceptions to my rule

I'll leave my passport in my stateroom safe if I'm in one of three ports of call.
First would be Perfect Day at CocoCay, which is Royal Caribbean's private island.
Because it's a cruise line operated place, it's basically impossible to be left behind. You'd have to hide in a bush to truly not make it back to the ship.

Over the years, I've encountered medical issues while on CocoCay with my family, and the first place they'll take you if it's anything remotely serious is back to the ship.
Ditto for Labadee, which is also a Royal Caribbean destination.
It just seems nearly impossible to be left behind or stuck in these spots.

The other place I'll be okay with leaving my passport behind is if I'm in a U.S. state or territory.
When my ship visits Puerto Rico, St. Thomas, or St. Croix, I'm fine with leaving my passport in the safe because I can travel home with just my driver's license.
As an American citizen, not having to rely on a passport for travel home while in a part of my country seems sufficiently safe to go off the ship with my driver's license.
It's a decision of risks

Ultimately, you have to judge which risk of taking or leaving your passport is higher and what the consequences of each risk would be.
Taking it with you means you can easily get home should you miss your cruise ship's return time for any reason, but the risk is having it lost or stolen.
But if you leave your passport on the ship, what happens if you miss the boarding time? And are you willing to risk that the ship would leave your passport and room contents at the port?

Personally, I won't go into a foreign country without my passport just to have myself covered. I think the risks of being stuck in the country without one is worse than the likelihood of losing it.