Bugged OoS gilded voyage?

Doing one of the gilded OoS bounties and I found the right landmark on the right island, but there aren't any skeletons? The ominous music that plays during the event is there but the skeletons are nowhere to be found. Additionally there were ammo pouches scattered everywhere and there are several ships around me presumably also doing these gilded voyages. Is there a cool down on each island before you can do your skeleton bounty?