The church is located almost on top of the promontory of the ancient village of Porto Ercole (Monte Argentario), located further south of the village above the pier of Santa Barbara, where both the lighthouse and the Rocca aldobrandesca stands out.
The original structure dates back to the fourth century and was given to Sant’Erasmo da Formia (protector of sailors and patron of Porto Ercole), where today the choir is located [not clear]. To this, a small oratory from the 7th century was added. It was still inside the Sienese walls, to distinguish it from another Sant’Erasmo, erected outside the walls. For many centuries, in fact, it was the parish of the village and was remodeled and renovated almost entirely in the seventeenth century in the current facade. Made in a very simple Baroque Tuscan style, with a portal surmounted by a triangular pediment and a small rose that opens in the middle. The pre-existing Spanish-style bell tower (XIV century) was struck by lightning in the 18th century, seriously damaging it (some ruins remain on the roof of the sacristy). In 1915, the current one was erected.
The church is divided into two naves, one main and one smaller lateral facing the sea. The ceiling of the main nave is supported by original XVII century trusses. On the right, there is the chapel of the baptistery, with a beautiful baptismal font in polychrome marble. The chapel follows a series of niches, once bearing statues of saints now kept in the sacristy. At the back of the church there is the presbytery, raised some steps, where the high altar is in polychrome marble, of Renaissance style, with marble or inlaid marble tombstones at its feet, which cover the tombs of the nobles and governors of Porto Ercole. Behind the altar opens the choir, which is accessed by two side doors. The choir is decorated with a series of seventeenth-century chairs, plus a cross vault frescoed with the four evangelists, Sant’Erasmo di Formia and San Rocco. The right aisle consists of a series of chapels covered by cross vaults, and ends at the back entrance of the church. The first chapel is always dedicated to the saint, richly frescoed and bearing a splendid altar painted with the papier-mâché bust of the patron. Then follow other chapels, until you reach the large and luminous sacristy, with the door carved into an altar. The entire treasures and the works that remained in the church are kept in their entirety. The chapel of the sacristy, located on the sides of the presbytery, is separated from it by what remains of the ancient balustrade. The left side aisle is supported by a tunnel, about thirty meters long.
The church lost importance in the mid-twentieth century, when the parish church of San Paolo della Croce was built in the modern village, although this ancient church was opened for masses and commemorations.
Behind the little church, following an uneven open uphill staircase on the right, leaving the village in the direction of the promontory with the lighthouse south of Rocca aldobrandesca, there are now very few ruins of what was the Confraternita di Santa Croce (XV century), where there was a small sanatorium dedicated to Santa Maria Ausiliatrice. According to the official hypothesis, the well-known painter Michelangelo Merisi, called Caravaggio, was here hospitalized, on July 18, 1610, found already dying by the then portercolesi on the beach of Feniglia; always according to the official hypothesis, the cadevere was then buried in the old cemetery of San Sebastiano, where now stands a small mausoleum dedicated to him, or at the beginning of Via Nuova, in the present village. Inside, five of the nine alleged bones of the painter found.