Mostly self-taught, Williams began playing drums in his teens. He played with pianist Judy Roberts for several years beginning around 1959. He moved to New York in the mid-'60s and played with tenor saxophonist Ervin in 1967. The next year he played with Rollins, tenor saxophonist Archie Shepp (at the Newport Jazz Festival), and tenor saxophonist Jordan. In 1969, Williams began a long-standing association with Barry Harris, recording the first of several albums with the pianist. He played with tenor saxophonist Mobley and bassist Wilbur Ware in 1970; he also got the job with Monk that year (probably through Ware, who was also in the band). Williams toured with Monk that summer; he left just prior to an October tour of Japan, replaced by Lenny McBrowne. During the '70s, he freelanced with such leaders as tenor saxophonist Yusef Lateef, pianist Ray Bryant, and saxophonist Charles McPherson; also Getz, pianist Andrew Hill, saxophonists Sonny Stitt, Junior Cook, Al Cohn, Buddy Tate, and Bob Wilber, among many others. In the '80s, he combined with bassist Art Davis to form a rhythm section for two different pianists, Barry Harris and Tommy Flanagan. He also recorded with trombonist Steve Turre and trumpeter Bill Hardman. In the early '90s, he recorded with alto saxophonist Lee Konitz, and also Anthony Braxton on one of the avant-gardist's "standards" projects.

By the mid-'90s, most of the legends Williams had played with were gone, but he remained active, playing with a new generation of bop-oriented players including saxophonist Ralph Lalama and pianist Pete Malinverni. At the turn of the decade, Williams was a member of the cooperative band el Mollenium, which also included guitarist Roni Ben-hur, pianist Bertha Hope, and bassist Walter Booker. The band is devoted largely to interpreting the music of Hope's late husband, the pianist Elmo Hope.