Harvey was the son of Susanna Rarig and Uriah Fetterman born in Roaring Creek Township, Columbia County, PA. On May 4, 1901, he married Amanda, daughter of Kathryn Beaver and Henry Rhodes, and their marriage was announced in The Democratic Sentinel on May 10, 1901. They made their home in Locust Township, Columbia County, PA, and Harvey worked as a farmer. Of this marriage, five children were born:

1. Pearl V., born October 14, 1902 - died October 11, 1994;
2. Laura L., born June 7, 1904 - died January 1987;
3. Stella Eva, born June 17, 1906 - died October 28, 1995;
4. Robert L., born August 29, 1909 - died May 5, 1991; and
5. Victor J., born August 8, 1912 - died March 27, 1986.