Club San Carlos was the cultural center of the santiaguero bourgeoise before the revolution.

Club San Carlos was inaugurated with the name of Academy San Carlos de Vargas at the top of the street Santo Tomas # 1. The majority of its founders were Cuban and French people. It was established as fencing hall that continued its function uninterrupted until the cessation of Spanish rule in Cuba. After moving several times from the premises, it settled permanently to the former residence of Don Bartolomé Portuondo y Rizo, Marquis of Tempú, that is its current localization.

In 1864, it took the name of Club San Carlos by extending its cultural activities and modifying its statutes with eight articles. According to the story, article seven said: "whoever wishes to belong to the club, must be presented by two members". Likewise, the eighth warned that admission would be by secret ballot, and if it was accepted, an "admission letter and a copy of the regulation" would be sent. If no answer was received, the non-admission would have to be considered, and this unappealable decision could be never questioned.

It became popular in the commercial life and in intellectual society of the city due to its rich activities, such as receptions, dance etiquette, disguised dancing, political meetings, tribute acts, commemorative activities and banquets.

In 1899, it was the first building in the city that raised the Cuban flag.

At the end of the 19th century the owners of the club decided to build a more luxurious building, so that the old construction was demolished in 1906. New building was completed in 1913. It was designed and built by the architect Carlos Segrera. It had three floors topped by two beautiful domes that were covered with colored ceramic elements. There were wide corridors on the second and the third floors that were protecting the rooms of the west-facing building from the sun. Some of the classical architectural elements of this building were the semicircular arches, the horizontal incisions on the façades, the stone balusters, the Corinthian columns, richly ornamented entablatures and the garland motives. The second floor is an ideal place to enjoy one of the best panoramas of the city. 

In 1932, a terrible earthquake destroyed its third and also top floor completely, so that it was finally demolished, leaving the approximate image that we see today.

In the course of time this property undertook several functions. In 1953, a gallery was opened with the name of Oriente Art Gallery on the second floor to exhibit the works of art of Portocarrero, Duque de Estrada and Pedro Arranz. Since 1980s the Municipal Culture House and the Estaban Salas Concert Hall are seated on the second floor. A snack bar, Club 300 and a bank are serving on the first floor.