Get Drifters from Events
Corsair Cove population does not simply grow naturally. Immigration encounters and sea events, including Drifter and Prisoner Ship events, are the core ways to gain more Drifters.
Release Existing Workers
Drifters fill open jobs automatically. If workers are tied up in lower-priority production, pausing that production releases labor immediately.
Move Population Between Camps
You cannot drag Drifters from one Pirate Camp to another. Instead, provide beds and jobs in the new camp to attract migration.
Use the Notoriety Path
The Notoriety-path Statue increases the number of recruits gained per event.
Single official-source detail: the Buildings Wiki states that Rising Sun unlocks at 7 Notoriety and exchanges Coins for Drifters.
Do Not Confuse Drifters with Swabbies
Drifters are island workers. Swabbies are baseline ship crew and require Stew and Booze upkeep.