The trouble is that something slightly DIY is so much cheaper for what you get.
Gonna echo that the Anbernic 353 handhelds are going to be your easiest all in one solution by a longshot. TV output works fine and they come with everything you need within your budget.
BUT depending on what you're trying to accomplish, if you're not interested in a handheld and are willing to tinker, for your consideration. This is the kind of stuff you want to be looking at. General-purpose SBCs. Anything marketes for gaming will be more expensive than it should be.
Issue is you need an SD card to store ROMs on and those will eat into your budget hard.
What systems are you looking to play?
Here's what I'd do personally:
NanoPi R6C This thing's a monster, you could buy something significantly cheaper of you're not looking to play the PS2 / GC era. 4gb memory is fine, we'll get the SD card elsewhere. $100 with a case. Really anything with this RockChip RC3588 in it plus some sort of cooling.
2 wired 8BitDo Pro 2 controllers (Or just one, take your pick)
The Daijisho Android frontend to easily launch games.
For a slightly more than your budget you're getting PS2 emulation.
Alternatively easier, manage to find a Raspberry Pi 4 (good luck) and you'd have access to things like Batocera and RecallBox which would feel a little more "plug & play"-esque.