Winding session arranger Garry Sherman contacted friend and colleague Ragovoy after Kai had expressed an interest in going in a more commercial and rhythmic direction (in gratitude for the gig Ragovoy granted Sherman 50% of the publishing rights, which he kept until an expensive divorce in 1983). But Ragovoy had thought of no lyrics for the song other than "time is on my side". Produced by Creed Taylor and engineered by Phil Ramone, and including background vocals by Cissy Houston, Dionne Warwick and Dee Dee Warwick, the recording was released on the Verve Records label in October 1963.