Phi Mu Delta traces its roots to the National Federation of Commons Clubs. The Commons Club was founded at Wesleyan University in Middletown, Connecticut, in 1899. The Commons Club grew to an impressive 19 chapters from Washington state to Maine prior to the formation of Phi Mu Delta. At the 1918 Conclave, held at the Massachusetts Agricultural College (now UMass), Clarence Dexter Pierce and many of his supporters petitioned the assembly for the formation of a Greek letter fraternity. The petition was adopted and the original plan was in favor of all chapters of the Federation to join Phi Mu Delta. However, only four chapters did so: The Universities of Vermont, New Hampshire and Connecticut, as well as Union College.

The formation of the new fraternity met with some early resistance when the alumni of the Union chapter refused to join Phi Mu Delta. So, the Universities of Vermont, New Hampshire and Connecticut went on to become the founding chapters. Chapter designations were determined by lottery. The Connecticut chapter drew the number one and therefore became the Nu Alpha chapter, New Hampshire drew the number two ticket and became Nu Beta and Vermont became Nu Gamma (the Nu prefix was determined by the location of the chapter, New England Region).

Expansion was conservative with the first new chapter at Northwestern University (Gamma Alpha) in 1921. This was soon followed by chapters at the University of Michigan (Gamma Beta) and M.I.T. (Nu Delta) in 1922. Mu Alpha at Susquehanna University was the first expansion into the Mid-Atlantic Region and Pi Alpha at The University of California, Oakland (now Berkeley), was the first Pacific Region expansion. The end of the 1920s came with two new chapters. At Rensselaer Polytechnic Institute, the Nu Theta Chapter was formed and a merger with a local fraternity at Penn State, Omega Delta Epsilon, became the Mu Epsilon Chapter in 1930.