Ilhabela registered high pirate activity from 1553 on, during the initial period of colonisation of Brazil. Portuguese caravels that passed by here heading toward the ports of Santos and São Vicente were a great attraction for pirates from all over. English and Dutch privateers used to take advantage of Saco Sombrio's calm waters to hide their ships and wait for commercial vessels to go past Castelhanos Bay.

The privileged position of Saco Sombrio allowed the pirates to leave their ambush and approach their targets without being noticed. Its calm, crystal-clear waters, full of fish and with an abundant supply of drinking drinkable water, made the place an ideal hideout for these thieves of the sea. Due to all of these facts, there is an abundance of legends of treasure buried by the privateers along the tracks that cross the forest on the eastern side of the Island,.

In 1591, Thomas Cavendish, in the service of the British queen Elizabeth I, left the port of Plymouth, in England, with five ships and more than 400 men, and headed towards Ilhabela. The squadron's initial objective was to attack the village of Santos, so, the fleet anchored here at the Island to restock the boats with supplies and draw up their plans. The attack had been scheduled for Christmas eve that year, when two English ships approached the small village and imprisoned its entire population, who were, at the time, attending Midnight Mass at the local church.

After taking the city, the rest of the squadron was called onto Ilhabela to carry out the looting. The pirates remained there, dominating the inhabitants and the Portuguese military garrison until February, 3rd, 1592, when they sailed off toward the Magalhães strait to try and circumnavigate the globe. By then, Cavendish had already achieved the undertaking, for which he was considered a hero by the court of Queen Elizabeth.

The squadron was almost decimated by storms at the southernmost tip of the continent. Two ships sunk, and the remaining three became separated from each other. Cavendish returned with his galleon 'Leicester' to Ilhabela, in search of a safe shelter in order to repair and replenish the ship. When they arrived they met up with another ship from the same squadron - the Roembuck, and landed. A few weeks later they had to face a Portuguese squadron commanded by Martim Corea de Sá, which came to do battle. The British ships fled northward, toward the state of Espirito Santo, and then returned to Ilhabela a few months later, with the intention of burning one of the ships, equipping the other one and following a course for the Magalhães strait again.

Frightened, part of the crew mutinied and took refuge on the Island. The captain of the Roembuk ship, Abraham Cocke, landed on Ilhabela with his men for good. It is widely believed here that the large number of blond, blue-eyed fishermen on the Island are descendants of those Englishmen. The fact is that no more was ever heard about them. Cavendish tried to return to England with the 'Leicester', but he died of illness at sea at the end of 1592, before the ship reached its destination.