Brown attended Lincoln Park High School, in Lincoln Park, Michigan, before enrolling at the University of Michigan. As a sophomore in 1966, he was a back-up to quarterback Dick Vidmer. His first appearance for Michigan was as a substitute for Vidmer in a 41–0 victory over Oregon State on September 17, 1966. He completed both of his passes, including a touchdown pass to Jim Berline. Brown also saw brief action in a loss to an undefeated Michigan State team (0–1 passing) and against Minnesota (1–1 passing).