On the Apple TV 4K Remote on iPad, you can use buttons instead of swipe gestures. With Directional Buttons on, the Apple TV Remote on iPad works with buttons, so you don’t need to swipe. It’s one of the accessibility features for mobility on iPad.
Go to Settings > Accessibility > Apple TV Remote, then turn on Directional Buttons.