The palace was built in the first half of the sixteenth century as a place of residence of the Spanish governors of the Presidium, which had as its capital the nearby town of Orbetello. The original complex was wider than the current one, due to the damage caused during the Second World War in the rear part.
The palace of the Governors overlooks the scenic square of the village of Porto Ercole, on the opposite side of the bastion of Santa Barbara; in front of the façade there is a circular cistern well in the square, which was used to supply water in the past.
The main façade is on two levels, with on the ground floor a fine portico with three orders in late Renaissance style attributed to Baldassarre Peruzzi, with the three round arches divided by pilasters, with a capital flanked by floral decorative elements. The cross vault that covers the inside of the portico is very characteristic. At the center, there is an opening on the wall of the portico, from which the staircase leads to the upper noble floor of the building. The latter recalls the stylistic elements of the underlying portico, divided into three vertical orders, with pilasters supporting the round blind arches, which delimit the spaces in which the large quadrangular windows open. There is no shortage of floral ornamental elements and, above, there is a richly decorated summit crowning.