Mrs. Astor's Mansion
5th Ave & 34th St
Caroline Schermerhorn Astor
Caroline Webster Schermerhorn was born in 1830 into one of New York's oldest and most respectable Dutch families — the Schermerhorns had been prominent in the city since the seventeenth century, and their wealth came from the deeply respectable sources of shipping and real estate. In 1853, she married William Backhouse Astor Jr., the grandson of John Jacob Astor, the fur trader and real estate baron who had been the wealthiest man in America at the time of his death in 1848. The Astor fortune was staggering, but in the status-obsessed world of old New York, money alone was never sufficient. What Caroline brought to the marriage was impeccable Knickerbocker lineage. And what she did with that combination of ancient name and enormous wealth was unprecedented. She became not merely a society hostess but the supreme arbiter of who mattered in New York — a self-appointed gatekeeper who wielded social power with the precision of a military strategist.
Her chief instrument was Ward McAllister, a Southern-born lawyer and epicure who served as her social lieutenant for over two decades. Together, they invented the modern concept of organized elite society in America. McAllister created the Patriarchs' Balls — a series of exclusive subscription dances whose guest lists were controlled by a committee of twenty-five gentlemen handpicked by McAllister and approved by Mrs. Astor. He famously declared to the New York Tribune in 1888 that there were "only about four hundred people in fashionable New York Society" — a number widely believed to correspond to the capacity of Mrs. Astor's ballroom. The "Four Hundred" became the most powerful social concept in Gilded Age America. Mrs. Astor enforced her authority through an elaborate system of rituals — the annual ball in January, the calling card protocol that determined who could visit whom, the precise seating arrangements at her dinner parties that telegraphed one's standing in the hierarchy. To receive her card was to be acknowledged. To be excluded was social death.
The Mansion at 350 Fifth Avenue
The Astor mansion was actually a double brownstone — two adjoining houses that occupied the northwest corner of Fifth Avenue and 34th Street. Mrs. Astor lived in the southern half, while her husband's brother John Jacob Astor III and his wife Charlotte Augusta Gibbes occupied the northern half. The interiors of Mrs. Astor's portion were legendarily sumptuous. The art gallery was hung with paintings by the great European masters, and the rooms were decorated with heavy gilt furniture, dark velvet draperies, and crystal chandeliers. The famous ballroom — the setting for the annual ball that defined each social season — could hold approximately four hundred guests, a physical constraint that became social law. Mrs. Astor held court there for decades, always appearing in her signature diamond stomacher and a dark wig, standing beneath a life-size portrait of herself by Carolus-Duran. By the early 1890s the neighborhood around 34th Street was growing increasingly commercial, and after her husband William's death in 1892, Mrs. Astor commissioned Richard Morris Hunt to design a new chateau-style mansion at 840 Fifth Avenue, at the corner of 65th Street, where she relocated in 1895.
Precise historical address at 350 Fifth Avenue. 97% confidence. Interiors filmed at Glenview Mansion, Hudson River Museum, Yonkers.
In HBO's The Gilded Age
In the show, Mrs. Astor is portrayed by Donna Murphy with a queenly reserve and quiet menace that captures the historical figure's extraordinary power. She appears throughout the series as the ultimate social obstacle — the woman Bertha Russell must win over to gain acceptance into polite society. The show faithfully depicts her methods of control: the guest lists, the withering silences, the calculated snubs, and the delicate negotiations conducted through intermediaries like Ward McAllister, played by Nathan Lane. The central dramatic arc of the series — Bertha's relentless campaign to breach the walls of old New York society — mirrors the real struggle that families like the Vanderbilts waged against the Astor-McAllister establishment in the 1880s and 1890s. Interior scenes set in Mrs. Astor's mansion were filmed at the Glenview Mansion in the Hudson River Museum in Yonkers, New York, a Gilded Age house museum whose ornate interiors convincingly double for the Astor residence on screen.
From Mansion to Empire State Building
The story of what happened to 350 Fifth Avenue after Mrs. Astor left is one of the great transformation tales in New York history. Her nephew William Waldorf Astor demolished his father's half of the double mansion in 1893 and built the thirteen-story Waldorf Hotel on the site, deliberately placing it right next door to his aunt's home in what many saw as an act of spite. After Mrs. Astor moved uptown in 1895, her son John Jacob Astor IV demolished her half and built the seventeen-story Astoria Hotel next to the Waldorf in 1897. The two hotels were connected by a corridor known as Peacock Alley and operated as the Waldorf-Astoria, which quickly became the most famous hotel in America. In 1929, both hotels were demolished to clear the site for the Empire State Building, completed in 1931 in just fourteen months. The Waldorf-Astoria name moved to a new building at 301 Park Avenue, where it operates to this day.
Books & Tours
Some links are affiliate links — we may earn a small commission at no cost to you.