In 712, Tarbert was burned by King Selbach mac Ferchair of Cenél Loairn and of Dál Riata and in 731 by his son, Dúngal mac Selbaig, the latter event being recorded in the Annals of Ulster:
The burning of Tairpert Boitir by Dúngal.
— Annals of Ulster, U731.4.
King Edward II of England transferred control of the castle to the Scottish King John II de Balliol in 1292. A fortified structure was built in Tarbert during the XIII century. It was reinforced with the addition of an outer bailey and towers in the 1320s by Robert the Bruce, to protect it against the Lords of the Isles. A towerhouse was added in the XVI century, which is the most noticeable part of the remains. The castle occupies high land above Loch Fyne, providing views up East Loch Tarbert and beyond to the Firth of Clyde. This castle was captured from John MacDonald of Islay, Lord of the Isles by James IV of Scotland as part of his campaign to destroy the power of the Lords of the Isles in 1494. In 1687 the castle was involved in another skirmish when Walter Campbell of Skipness Castle seized it as a stronghold for Archibald Campbell, 9th Earl of Argyll as part of actions in support of the Monmouth Rebellion in England.