Camera is jittery

Whenever I press the mouse scroller in and rotate the camera, it sputters and it's super annoying. I do have Smooth Camera rotation enabled. What can I do about this? Any help is appreciated!