Walter de Maria's all-seeing eye

The barracks where Adolf Hitler began his rise to power have been given a new lease of life – as a gallery boasting just one exhibit

Capturing the surroundings … Walter de Maria’s Large Red Sphere, at the Turkentor in Munich. Photograph: Jan Bitter

There has always been a strongly architectural element to the art of Walter de Maria: it's vast in scale, yet perfectly proportioned. The Californian captured the public imagination in 1977 with Lightning Field: 400 stainless steel rods, arranged in a grid in a New Mexico desert. As well as reflecting and refracting changing patterns of light, Lightning Field blazes magnificently in electrical storms. It has become a place of secular pilgrimage, turning its creator into a cult figure in the process.

In the 1990s, De Maria began making great stone spheres, one of which, a 25-tonne piece of highly polished red granite, has just bumped down at the Turkentor gallery in Munich. A former barracks that once provided a bunk for a young soldier called Adolf Hitler, the Turkentor was bombed in the second world war, then all but demolished in the 1970s. Only a fragment, a grand neoclassical gatehouse, remained. The Turkentor, neatly situated between two major Munich galleries, has now been reconstructed by architects Sauerbruch Hutton – and given a new life as a gallery remarkable for the fact that its purpose is to house one artwork, De Maria's sphere, and nothing else.

"I've been working on the idea of one artwork in one building for 40 years," says De Maria. The title of the polished granite sphere he gave to the Chichu Art Museum, on the Japanese island of Naoshima, does seem to be reaching towards this notion of exclusivity: Time/Timeless/No Time. That gleaming orb resides, along with some water-lilies by Monet, in a beautiful underground museum designed by Japanese architect Tadao Ando, a former truck driver and boxer.

Large Red Sphere, as the Turkentor's sole exhibit is called, measures 260cm in diameter and sits in totemic splendour at the heart of the gallery, lit solely by the sun and the moon, through a glass roof. There is nothing else to see here, although the enveloping architecture, and the way light cascades around its shapes and spaces, are striking. Yet it is hard not to be wholly absorbed by Large Red Sphere, which watches you and the world beyond like some giant unwinking eye. It has a hypnotic quality: your own eye is drawn to both its surface and into its core. You can watch it for hours – and some people do. You can even touch it. "I like people to do that," says De Maria, although most visitors are too intimidated.

De Maria made his first sphere in 1990, for the Assemblée Nationale in Paris. With their highly polished or intricately worked surfaces, his orbs all offer unexpected – and beautifully distorted – reflections of their settings and anything that happens in them. In this way, paradoxically, they seem to contain their surroundings, making them the perfect accompaniment to sensitive architecture.

Large Red Sphere was originally to have been housed in the Pinakothek der Moderne, Munich's gallery of modern art, which flanks the Turkentor. "There were disagreements," says De Maria, "so what had been a site-specific piece was left looking for a home." Having finally found one in the Turkentor, it will now, hopefully, exorcise that building's demons. It was here, when the barracks still paraded along Turkenstrasse, that Hitler stayed after serving in the first world war; it was here that the future Führer gave some of his first political speechesto fellow soldiers; and it was here, in 1919, that he was nearly killed by leftists and liberals.

The barracks was seized that year by the ultra-rightwing Freikorps, German troops spoiling for a fight with those who would turn Bavaria into a Soviet socialist republic. It's stirring to think that this place of hatred and violence has now been reborn as a haven of peace, art and contemplation.

Sauerbruch Hutton cocooned the Turkentor in a sheath of pale brick, and gave it a concealed steel-and-glass roof. The sphere, set on a black plinth designed to raise its centre to eye level, is superbly framed by weathered doric columns and worn oak beams that have been here since 1814. The result is a delicious tension between the organic and the geometric, the straight and the curved – as if the energy of the sphere is contained, or even harnessed, by the notional cube that surrounds it.

"Placing the sphere was quite an operation," says architect Matthias Sauerbruch. "We had to lower it through the roof. As this is 270cm across, you can imagine how precise the installation had to be. But as it came down, it dropped the last inch! The vibration shook the building."

De Maria had been unsure about the Turkentor. "He thought the association was too military," says Sauerbruch. "He came around. His work has a spirituality. You could liken today's Turkentor to a roadside chapel, but without the religious connotations. People drop by to walk around the sphere and are out again in a few moments. Others seem lost in contemplation."

De Maria is now planning his next sphere. "This one will be blue," he says. "I turned 75 before Christmas, so who knows? It'll probably be the last one I do."