History has it that, in 1631, a renegade employee of the Dutch West Indian Company called Arent Groote, acting on behalf of the English Company of Adventurers Trading to Guynney and Binney, signed an agreement with the Chief of Cormantin by which a hill site near the village was ceded to the English company. That year, the company built a lodge. Later, it was destroyed by fire and the company converted the lodge into a fort in 1638. In 1661, ownership of the fort was transferred to the Royal African Company and it became the headquarters of the English possessions on the Gold Coast.

In 1665, the Dutch Admiral De Ruyter captured Fort Cormantin in retaliation for the capture of several Dutch forts by English Admiral Holmes in 1664. The Dutch reconstructed the fort and renamed it "Fort Amsterdam". The English transferred their headquarters to Cape Coast Castle.