Here's a decent beginner's setup. You should also pick up a carving glove, a strop and stropping compound.
This is about the cheapest you're going to get away with for a setup with good tools that are versatile and of a quality to last a good while.
As for wood to start, Basswood is the best wood for beginners and is also used by professional carvers. Here's a good starter pack that will get them going for a while.
Get yourself some flex cut knives and some bassword. I started with this and its worked well for the past 6 months https://www.amazon.com/gp/aw/d/B005EG033Y/ref=mp_s_a_1_3?qid=1437522906&sr=8-3&pi=AC_SY200_QL40&keywords=flexcut&dpPl=1&dpID=51mjev6rXKL&ref=plSrch
I just got into whittling and I started with beavercrafts spoon carving kit. The knives are pretty great and cheap but I bought a flex cut hook knife that just blew beavercraft out of the water. Looking back I think I would’ve been better off spending more up front to get a higher quality knife.