I've had no problems using remote controllers with my VR headsets (on my Galaxy S6 and Pixel phones). However, a couple of the cheapest ones just didn't work. I didn't have to install any special apps or root my phone.
The one I wound up using the most is the iPega: http://www.hypergridbusiness.com/2016/02/review-of-ipega-nice-android-vr-controller/
I got it specifically for the Gear VR, since the trackpad is lousy for playing games with. But it works for Cardboard apps, too.
A cheaper option is a tiny little controller like this one:
There are millions of them out there, and many come free with headsets. Just pair them the way you do any Bluetooth device.
Where they won't work at all is with iPhones. Apparently, Apple doesn't believe in remote controllers anymore.
Here are some workarounds:
http://www.hypergridbusiness.com/2016/12/vr-games-that-work-with-iphone-controllers/