I was at Labadee just last week and loved it. I highly recommend the kayak tour - a native Haitian takes you around the bay towards the Labadee village and points out the school, medical clinic, and other structures that were built from the Royal Caribbean cruisers dollars. It was really cool.
As far as the market goes, I’m a single woman traveling alone and walked through the market area by myself to get back to the ship and see the island a little. I never felt nervous or threatened. I just kept saying, “no thank you” and kept walking. There was also Royal Caribbean security along the way who kept the vendors from coming up too close to the people walking through. It was a fantastic day and I definitely wouldn’t stay on the ship over safety concerns.