Amazon kits are a great start, there’s plenty of options for different budgets - just punch “needlefelting kit” in the search! For a more minimal kit for just testing if you like the medium before investing in it, expect to spend $8-15. For a pretty complete kit with lots of color options and tools, expect a price tag of $20-25.
Note that these kits do not normally come with instructions. If you’re looking for a project pattern to follow to get some practice/test the waters before diving into your own original needlefelts, consider a project kit like this Woolpets kit or these Daiso mini-packs for your first purchase.
If you decide you like the medium enough to venture out and make bigger stuff without guidance, and need large amounts of single specific colors or fiber types, Etsy is a great go-to.