Raye was an accomplished dancer as a boy, and won the Virginia State Dancing Championship. From the mid-20s he worked as a singer and dancer in Vaudeville and, later, toured theatres and nightclubs in France and England, while also writing songs for himself and other performers. In 1935 he collaborated with Sammy Cahn, Saul Chaplin and band leader Jimmie Lunceford on Rhythm In My Nursery Rhymes and, in the late '30s, worked for a New York music publishing house. After moving to Hollywood in 1940, Raye was commissioned to write the songs for Argentine Nights, in which the Andrews Sisters made their screen debut. Together with Hughie Prince and the Sisters' arranger Vic Schoen, Raye wrote Hit The Road and Oh! How He Loves Me. With Prince, he produced Rhumboogie, the first of a series of 'boogie woogie' numbers, several of which became hits for the Andrews Sisters, pianist Freddie Slack, and Will Bradley And His Orchestra. Raye and Prince's next assignment was Buck Privates, which also featured the Andrews Sisters, and rocketed the comedy duo, Abbott And Costello to movie stardom. His long partnership with Gene De Paul followed, in which they collaborated on many songs for films in the 1940s.

De Paul and Raye's last film work together was in 1949, for another Disney project, the highly acclaimed cartoon, The Adventures of Ichabod and Mr Toad. After the mid-50s, Raye only wrote occasionally, although his Well, All Right, written with Frances Faye and Dan Howell, and a hit in 1959 for the Andrews Sisters, was interpolated into the 1978 bio-pic, The Buddy Holly Story.