The current Madison Avenue Bridge was not the first to span the Harlem River to connect 138th Street in The Bronx with Manhattan. In 1874, New York City annexed three mainland Westchester County towns west of the Bronx River; making them into the 23rd and 24th wards. On October 21, 1874, 327 people in this Annexed District petitioned for a wooden pile bridge to be built at that site.
After lengthy examination and planning by the Army Corps of Engineers and New York City agencies, construction began in 1879. This bridge was opened to traffic in November 1884 but was not finished until a year later. The total cost of erecting this span was $509,106.22.
Various improvements were made to the new bridge until 1898. Although still in good condition, street traffic had expanded so much that the span was outmoded. Plans for new approaches and a new bridge were made and considered from 1898 to 1903. The final plans were approved and the city's Board of 'Estimate provided the sums to begin work on July 29, 1903.
Since the new bridge would be on practically the same site as the original one, a temporary bridge had to be built to carry traffic during construction. Plans for both the temporary and permanent spans were approved by the Secretary of War on August 28, 1905.
On both sides of the river, temporary wood trestles formed the approach to two steel fixed spans and the repositioned draw from the original bridge. At midnight, May 4, 1908, traffic over the old bridge ceased, the draw repositioned, and the temporary bridge opened on May 11, 1908.
Construction for the new permanent bridge began October 8, 1907 by John C. Rodgers. The current Madison Avenue Bridge opened to the public July 18, 1910. The temporary structure was removed by Edward S. Reise starting on August 24, 1910.
The foundations for the Madison Avenue Bridge are on piles. The river piers are of concrete faced with granite. The approaches are of concrete steel. Unlike the other Harlem River Bridges, the approaches to the Madison Avenue Bridge are not filled with earth between granite retaining walls. They are new-built with reinforced concrete, with the roadway supported in columns. This provides space for storage facilities and work rooms.