The neighborhood was named "Back Bay" because the area was, in fact, before it was filled in, literally the back bay for Boston. To the west of the Shawmut Peninsula, on the far side from Boston Harbor, a wide bay opened between Boston and Cambridge, with the Charles River entering at the west side. As with all of the New England coast, the bay was tidal, with water rising and falling several feet over the course of the day. At low water, part of the bottom of the bay was exposed.

In 1814, the Boston and Roxbury Mill Corporation was chartered to construct a mill dam, which would also serve as a toll road connecting Boston to Watertown, bypassing Boston Neck. The dam was later buried under present day Beacon Street.

The Back Bay neighborhood was created when a parcel of land was created by filling the tidewater flats of the Charles River. This massive project was begun in 1857. The filling of present day Back Bay was completed by 1882. Filling reached Kenmore Square in 1890, and finished in the Fens in 1900. The project was the largest of a number of land reclamation projects, beginning in 1820, which, over the course of time, more than doubled the size of the original Boston peninsula. It is frequently observed that this would have been impossible under modern environmental regulations.

Back Bay's development was planned by architect Arthur Gilman with Gridley James Fox Bryant. Strict regulations produced a uniform and well-integrated architecture, consisting mostly of dignified three and four story residential brownstones.

Greatly influenced by Haussmann's renovation of Paris in the mid-to-late 19th century, the main roads of Back Bay emphasize order, with wide, parallel, tree-lined avenues and more homogenous architectural styles. Five east and west corridors run the length of the Back Bay: Beacon Street (closest to the Charles River), Marlborough Street, Commonwealth Avenue, Newbury Street, and Boylston Street. With the exception of Commonwealth Avenue, these streets are one way. These streets are named alphabetically starting at the Public Garden: Arlington, Berkeley, Clarendon, Dartmouth, Exeter, Fairfield, Gloucester, and Hereford.

The boundaries of the Back Bay, as defined by the Neighborhood Association of Back Bay, are the Charles River on the North; Arlington Street to Park Square on the East; Columbus Avenue to the 'New York, New Haven, & Hartford' right-of-way (south of Stuart Street and Copley Place), Huntington Avenue, Dalton Street, and the Massachusetts Turnpike on the South; and Charlesgate East on the West.

The Back Bay Architectural District, which is much smaller, was established by state law in 1966, and is bounded by the centerlines of Back Street on the north, Embankment Road and Arlington Street on the east, Boylston Street on the south, and Charlesgate East on the west.

The residential streets of the Back Bay are some of the best preserved examples of late 19th century urban architecture in the United States. Trinity Church, the Gibson House Museum, the Boston Public Library, and the John Hancock Tower are just some of the notable examples of architecture.

The Back Bay is best known for its historic real estate and shopping. Most stores are located on Newbury and Boylston Streets, with the ends closer to the Boston Public Garden traditionally more expensive. There is no shortage of upscale shopping, boutiques, art galleries, and fabulous restaurants. The Back Bay is also the home to many luxury hotels that include the TAJ, The Colonnade Hotel, Westin Copley Place, Fairmont Copley Plaza; including the largest hotel in the city, the Marriott Copley.