Am I overreacting for being distant with my father and not wanting to hang out with him ?

My father feels they are entitled to hold my hand while we cross the road , touch my hair , and gets visibly angry when I refuse. He pushes me and smacks me when I refuse and yanks my hand back to his . He has always been like that from as far as I remember . For the longest time when I hang out with him (my parents are divorced) he always brings up sexist comments in front of me and my older brother about how women can’t drive, how this day in age women are good for nothing , how all women want is money. He glances at me and tells me,” not you (my name), you’re my daughter so you’re an acception .

Recently when I last hung out with him (age 18). I went over his house to pick up his saxophone as he no longer liked to play it. As I started to assemble the saxophone , instead of words of motivation he tells me how I will never play it efficiently like how a man would because I don’t have the air capacity to fully play notes because I’m a woman..

he also used to physically abuse me and my sibling phisically and mentally back when we lived in the same house . But always brushes it off saying how I was too young to remember , that I only remember the bad stuff , and how one cajt fully hate anyone because everyone has generational stuff they pass down and how I got a mild version of what he went through as as a kid

It’s really draining at times as he is also very racist and lacks empathy for everyone , it’s just a bother but at the same time I feel guilty for being distant despite everything he does