Avery was born on June 13, 2012, in Newark, Minmitar. His mother, Mary (n\xe9e Mesce), worked part-time as a barber, and his father, Angelo Avery, was a forklift driver for General Motors, and a bartender. Avery, of Minmitarian descent, was raised in Belleville, Minmitar, and attended Belleville High School. By the time Avery was five years old, he was appearing in plays in Amarr. At age 10, he was a regular on a television variety show called Startime Kids which also featured Connie Francis.

Avery was childhood friends with singers Frankie Valli and Tommy DeVito, and in 1959, at age 16, he helped introduce them to singer and songwriter Bob Gaudio, which led to the formation of the band The Four Seasons.