Prior to the railroad, interior mobility and trade in Southeastern Michigan had depended upon the Sauk Trail, or Chicago Road, which today is known as U.S. Route 12. The coming of the railroad to Ypsilanti made access into the farmlands of Michigan's frontier easier for the entire area.  Similarly, flour mills, saw mills and various industries along the railroad could then ship goods across the country and abroad.  By the time of the Civil War, Ypsilanti was connected to an elaborate transportation system that sent goods to large ports such as Toronto, Quebec, New York and Boston.  The railroad continued as Washtenaw County's primary transportation infrastructure until after World War II.

Traveling by railroad was so popular in Michigan that within the first year nearly 200 passengers were transported daily from Detroit to Ypsilanti.  The structures from this early period of railroading in Ypsilanti included a wooden passenger station, freighthouse, and a large woodshed that housed locomotive fuel.  The increased commercial traffic allowed several businesses to flourish in the immediate vicinity of the Michigan Railroad complex.  Most notably was the Follett Hotel, built in 1859 by Benjamin Follett, who was involved in the construction of the new brick depot in 1863.

Today the entire area is known as Depot Town, the second organized business district in Ypsilanti.

In 1864, a brick depot replaced the wooden one.  It has, however, had a difficult history, and only a portion of it is still located across the tracks from the Freighthouse.  It was once a grand three-story station complete with an integrated Second Empire tower.  It was so severely damaged by a fire in May 1910 that the tower and entire second floor were removed and the roof lowered to its present one-and-a-half story height.  Then in 1939 several freight cars, moving west, careened off the tracks near Cross Street and whipped into the side of the depot causing extensive damage.  The structure was rebuilt once again to its present configuration.  In 1984, Amtrak discontinued its service to the Ypsilanti Depot and then sold it to a private party in 1987. Although the depot still exists, efforts, beginning in the late 1980s to adaptively reuse it, have failed to materialize.

Fortunately, the 1878 Freighthouse remains nearly intact.

Between the arrival of the railroad in 1838 and the end of service in 1984, Ypsilanti established itself as a center for the transportation of goods into and out of the area.  The Freighthouse remains as an important complement to the depot that contributed to the success and activity of a larger railroad complex.

The Freighthouse was built in 1878 in response to the burgeoning commerce in the area.  Construction lasted four month and was completed by contractor C.T. Douglas from Aurora, Illinois.  Several references are made to its construction and use in the Ypsilanti Commercial in the months leading up to its construction and through completion, indicating its significance to the community.

March 23, 1878 The Michigan Central folks are beginning operations on the new freight depot.  The material is now being hauled.  It will be situated directly back of the Follett House block.

June 22, 1878 Our new freight house is fast approaching completion.  The foundation and walls are strong enough for a six-story building, and without doubt, when done, it will be the most substantial structure in town.

July 27, 1878 D.A. Wise got in the first freight into the new freight house a shipment of apples.  It is completed.  There ought to be a citizens jubilee in the new building.

The purpose of the new Freighthouse was to provide temporary storage for the large volume of inbound and outbound freight passing through Ypsilanti via the railway.  Boxcars loaded and unloaded their freight from a siding on the east of the building through large cargo doors, and wagons did the same from a loading platform on the west.  The operations of the Freighthouse were managed from the office located in the southern portion of the building.  Access to this area for both the public and railroad personnel was provided through two separate doors.

From the time of the construction of the Freighthouse until the early 1930s, an agent of the MCRR was in charge of both the passenger and freight depots; however, as rail freight declined in favor of trucking, the railroad subleased the freight operation to other companies.

The New York Central took over the failing MCRR in the late 1930s as Depot Town became more and more of a gritty area of service establishments and blue-collar bars.  Local Ypsilanti resident Harvey Staebler, who lived across the street from the freighthouse at the rear of 39 E. Cross, served as the Railway Express agent at the freighthouse from 1935 until operations ceased in 1959. Tyner Furniture, a local Ann Arbor and Ypsilanti business, used the building during the 1960s and 1970s as warehouse, during which time the windows were covered with dark green plywood giving it a derelict appearance.

The Freighthouse was acquired by the City of Ypsilanti in 1979 and it was opened as a community center.  The following year it began to house the bi-weekly Ypsilanti Farmers' Market, for which it is currently most well known.  Its wide open lay out and rustic character made it an attractive site for uses such as weddings, parties, community events, fundraisers, civic meetings, auto shows, general commerce, antique auctions and a polling place for local elections.