The building is a sharp triangle, with a small colonnaded portico at the tip, followed by a series of parallel rectangles, each corresponding to a vaulted, sky-lit roof. Ando’s design respects what exists; new galleries follow the arrangement of the original bays, with Ando’s trademark silken concrete surfaces providing calm contrast to the brute beauty of the centuries-old masonry walls. The architect has called concrete “the marble of the 20th century,” and no one has imbued the brutish material with as much tactility or variety.

In the space, floors are a polished concrete and resin mix, while the doorways that puncture the meter-thick walls are framed by thin, smooth concrete panels that float ever so slightly away from the brick structure. This is classic Ando: The narrow gap dramatizes the meeting of old and new, for those who are looking. The more obvious example of the same idea is the floating hollow cube that the architect placed at the center of the structure, a double-height space formed by poured-in-place concrete. A cortile of sorts (a nod to the courtyards of traditional Italian buildings), the space is the only autonomous part of the building, emphasized by the stone-block floor, a traditional pavement called masegni that is seen in virtually all of Venice’s outdoor spaces. The room, which houses a series of large-scale black-and-white photographs by Rudolf Stingel, acts as a pivot for the museum, to which all circulation paths lead.

The Punta della Dogana’s other significant architectural gesture is the replacement of grills on all the exterior doors with a woven metal screen Ando designed in homage to Carlo Scarpa’s geometric motifs in the Olivetti showroom on San Marco Square. The move was in part a response to Mayor Massimo Cacciari’s request for some outward expression of the building’s inner transformation.

Unlike the uniform spaces one often finds in museums, the Punta della Dogana’s galleries feel wonderfully varied, because the historic elements, such as the irregular, laboriously restored brick walls, are so singular, and the views of water and city beyond the arched windows are constantly changing. Interior sight lines shift too, with the upper level affording views of the loftlike double-height galleries, or into the spare central cube, for example.

Much of the art on display, such as the grouping of eight massive dark canvases by Sigmar Polke (from the series “Axial Age,” 2005–07), feels site-specific, a compliment to the curators, who rose to the challenge of choosing art that married well with the architecture while standing strong on its own. The inspired placement of a floor-to-ceiling, red-and-white-plastic-beaded curtain by Felix Gonzolez-Torres, Untitled (Blood) (1992), at the entrance to the first gallery gives visitors a pleasing, tactile introduction to the museum, leading to the quirky combination of Rachel Whiteread’s Jolly Rancher–hued casts Untitled (One Hundred Spaces) (1995) and Maurizio Cattelan’s headless stuffed horse (Untitled, 2007). Particularly clever arrangements include the pairing of Hiroshi Sugimoto’s black-and-white couture studies, Stylized Sculpture (2007), with Cattelan’s elegant, draped corpses carved from white Carrera marble, All (2008); and Jeff Koons’s self-referential, Baroque-kitsch marble Bourgois Bust-Jeff and Ilona (1991) with Cindy Sherman’s 2007–08 series of herself as an older woman in age-inappropriate party garb.

Because the building’s latest incarnation as a museum is just one chapter among many in its long history, it has the sense of calm or quiet one often feels in places where time seems to have been arrested. It’s hard to imagine a more fitting backdrop for the intensity that characterizes so much of Pinault’s collection.