In the early 1740s, supporters of the popular minister George Whitefield determined to construct a building to house his sermons and the throngs that gathered to hear him preach. Both house and ground were to be vested in trustees, expressly for the use of any preacher of any religious persuasion who might desire to say something to the people of Philadelphia; the design in building not being to accommodate any particular sect, but the inhabitants of the City in general.

The property selected for the "New Building," as it was called, was situate on the west side of Fourth street, commencing one hundred feet south of Mulberry (now Arch) street. On November 14th, 1740, the property was deeded to a number of prominent citizens in trust. The deed recited that whereas a considerable number of persons of different denominations in religion had united their endeavors to erect a large building upon the land above described, intending that the same should be appointed to the use of a charity school for the instruction of poor children gratis in useful literature and in the Christian religion, and also that the same should be used as a house of public worship.

Whitefield preached a number of times in the building, and others followed. From these efforts, a number of Philadelphia's churches were founded, including the Second Presbyterian Church in 1743. A small burial ground was established around the New Building around the same time, perhaps originally for use by the Second Presbyterian congregation, which did not have its own house of worship or burial ground of its own until 1752.

In 1749, the building was sold to a number of prominent Philadelphia trustees to establish and a house or place of public worship, and also one free school for the instructing, teaching, and education of poor children or scholars, and that the trustees would supply "the schoolmaster or masters, usher or ushers, mistress or mistresses, to teach and instruct the said children gratis in useful literature and knowledge of the Christian religion . . . ."

Some alterations were made in the building to fit it for the uses intended. The great and lofty hall was divided into stories, with different rooms above and below for the schools. Additional ground was purchased, and thus the "New Building" changed its name to that of the "Academy," and in 1753, to "The College, Academy, and Charitable School of Philadelphia."

Following the Revolution, the Academy was united with the University of Pennsylvania which moved its quarters to Ninth Street.

In 1802, the southern portion of the great building was sold to a Methodist congregation which had split off from Saint George's Methodist Episcopal Church and took the name of the "Union Methodist Episcopal Church," and was generally known as "the Academy." The northern portion retained its original purpose for the use of religious congregations or ministers. Several religious societies which blossomed into congregations were started there, or used the room until they had means to build church buildings elsewhere. For several years there was a classical school in the old building, kept by Reverend Samuel Crawford.

The Union Methodist congregation likely used the Academy churchyard to bury their dead until they purchased their own ground at Ninth and Noble streets in 1825, before finally moving their dead to Tenth and Washington streets in 1832.

Around 1839 or 1840, the Union Methodist Church tore down their part of the old Academy structure, and erected a much larger meeting house. The northern portion of the Academy building and the large house north of it occupied by the charitable schools, and also as a dwelling by one of the teachers of the boys' school, was torn down in 1844 by the University of Pennsylvania. When they did so, workmen dug up a number of coffins of all sizes. A small portion of the "Old Academy" remained in the rear and was repurposed as an industrial manufactory.

Today, a hotel occupies the site.