What makes a video game horrifying?

We all have different interpretations of subjective things, and I've been thinking about the horror genre in video games a lot lately. I'm developing a game that I consider spooky, but not necessarily "horror". Yet I've played similar titles that users tag as horror, even though I wouldn't rate the scary-factor in those games much higher than the average creepy pasta. I've also had playtesters for my own game that got pretty spooked at certain points, even though I'm not trying to design anything more frightening than "somewhat creepy". (One of the more, er... "fear-inclined" among the testers actually shrieked at one point, which I was not expecting).

I know similar questions about genre definitions have been asked here before, but most of the answers are focused on film/TV. I get the sense that there's a trickier, blurrier line to find when if comes to video games. Somewhere along the way between Luigi's Mansion and Silent Hill 2, a video game acquires the horror genre. Where's that line for you? How does you having active agency in the medium of games change the line compared to flim/TV/written works?

To make matters more befuddling, my lens for horror games is a little scuffed as a developer. The Haunting of Hill House spooked me right out for days, but every time I play the latest, scariest games, I spend time trying to figure out how the enemy AI works or admiring the lighting rather than being scared -__-