Grimes was born in New York City. His acting career began at age seven, during the depression, when he helped earn money for his family. He appeared as Jackie Grimes in the Broadway play The Old Maid, which won a Pulitzer Prize and ran for 10 months in New York. It then went on tour for another 11 months.

Grimes worked on radio, beginning with the CBS program Let's Pretend. He was also a regular on The Fred Allen Show, The Philip Morris Playhouse, Second Husband, CBS Radio Myster Theater, and Death Valley Days. By age 12, he was appearing on 35 to 40 radio shows a week.