I got started with a DE for under $60 total. There's no reason to go fancy. Start with a soap stick (instead of a bowl that you lather up the soap in, you just put it directly on your face and then brush up the lather) and get a synthetic brush rather than badger hair (I really like my synthetic brush, actually – linked below) and you'll be fine. I'd recomend getting a sample pack of blades, so that you can try different ones.

Razor - $30

Blades – $15

Brush – $8

Soap Stick – $6

You might also want to get a styptic pencil to stop nicks from bleeding. You can find them at the drugstore for not much.