Built in 1756 by Bolognese Alfonso Torreggiani for the marquis of Cavriani, it is a monumental construction with severe architectural divisions and ornamentation.
Opposite the palace is a garden designed for the Cavriani in 1826 by Giovanni Battista Vergani, a well-known exponent of Milanese Neo-classicism and creator of several important projects in Mantua  and the province during the first half of the nineteenth century.

In 1835 marchese Luigi Cavriani had a marble statue of Virgil erected in the garden opposite his palazzo.Its sculptor, Stefano Gerola of Milan, was also responsible for the thirteen busts of Mantuans renowned for their contribution to letters, science and the arts situated on the columns of the railings.
The portraits include Ippolito Capilupi, Baldassarre Castiglioni, Filippo Cavriani, Marcello Donati, Teofilo Folengo, cardinal Ercole Gonzaga, marchese Francesco II Gonzaga, duca Vespasiano Gonzaga, Pietro Pomponazzo, Antonio Possevino, Battista Spagnoli, Jacopo Strada e Sordello Visconti. 
It was marchese Cavriani's clearly expressed desire to erct a modern monument to Virgil after the removal in 1821 of the one in piazza Virgiliana.