Kanhaiyalal was born in Varanasi into a theatrical family. His father, Pandit Bhairodutt Choube, owned a drama company but disapproved of his son’s interest in acting. Undeterred, Kanhaiyalal began working backstage and gradually transitioned into small stage roles. After his father's death, he moved to Bombay to pursue a career in cinema—initially aspiring to become a writer and director. However, fate had other plans, and he made his screen debut as an extra in Sagar Ka Sher.

He rose to national and international fame for his iconic portrayal of the scheming moneylender Sukhilala in Mehboob Khan’s Mother India (1957), one of Indian cinema’s most legendary villains. He had originally played the same role in Aurat (1940), making him the first Indian actor to reprise a character in a remake 17 years later. Mother India also earned the distinction of being India’s first official entry to the Academy Awards (Oscars) for Best International Feature Film—cementing Kanhaiyalal’s performance in global film history.