With a $500 to $700 budget I might suggest the following. Steer clear of the "packaged podcast stuff". And others will have their own opinions and thoughts but here are mine.
Recorder: Zoom H4nPro $230. Portable, flexible, will accept up to 4 inputs. Records on a SD Card.
Software: Audacity. It is free and allows you to manipulate and put together a cohesive product.
Mics: 2 of the ATR 2100 USB $150. Again, flexible and a decent beginner mic.
Accessories needed. 2 wind screen/pop filters for mics ($20), 2 mic stands ($20), 2 XLR cables for mics ($25)
Headphones: Sony MDR7560 $80 2 might be nice, but 1 required.
That is about $550 to $600 at this point.
If you wanted a studio mic, a decent starter mic is the MXL 990 at $100/ea. But beware that this is a condenser mic and it will pick up a ton of ambient sound especially in a non-treated room.
You may want to add a mixer into the mix at some point. I have a Behringer Xenyx 1204USB $139, but wish I had known more when I bought it and I would have bought the Behringer UFX1204 with the difference being that the one I have sends out a single stereo track to the recorder, but the latter has the ability to send separate tracks. Woudl be very handy if your guest is VERY soft spoken, there is a lot of talk over one another, etc.
Audio-Technica ATR2100-USB Cardioid Dynamic USB/XLR Microphone https://www.amazon.com/dp/B004QJOZS4/ref=cm_sw_r_cp_apa_8xwqzbM9CGGYW
This microphone is highly recommended. It has a USB port to directly connect this to your computer, and headphone ports for playback.
I'd suggest picking up a boom arm and a nice little windscreen and that's all.
Here is a nice review about it: https://youtu.be/1GPiaLGgnz4
Audio-Technica ATR2100-USB Cardioid Dynamic USB/XLR Microphone https://smile.amazon.com/dp/B004QJOZS4/ref=cm_sw_r_cp_apa_cnU9BbA8EK7GR
get this and a cheap scissor stand. you won't regret it
Thanks! I'm currently torn between two (after hours of research lol)
This is the one I'm leaning toward: (Samson C01U Pro USB Studio Condenser Microphone)
https://www.amazon.com/gp/product/B00HXE4BYW/ref=ox_sc_sfl_title_6?ie=UTF8&psc=1&smid=ATVPDKIKX0DER
And this is the other possibility (Audio-Technica ATR2100-USB Cardioid Dynamic USB/XLR Microphone):
https://www.amazon.com/gp/product/B004QJOZS4/ref=ox_sc_sfl_title_11?ie=UTF8&psc=1&smid=ATVPDKIKX0DER
My primary concern is obviously audio quality, but I'm also very concerned with background noise. I've got 4 kids, and keeping them quiet is...challenging. I think the majority of my recording will be done after they're in bed, but on weekends and stuff they tend to stay up late...so, a mix of me telling them to chill out and a mic that can filter out background noise is probably my best bet. I've already started selling my wife on me turning the guest bedroom into my office lol
edit: I'd like to officially open this up to anybody who has mic advice based on what I just said!
This mic is certainly a great upgrade over something like a headset mic. They are fantastic for a acoustically well-controlled recording space (like studios :). That being said, do think twice when purchasing a condenser mic for a home-studio environment: condensers are very sensitive, so they pick up great detail in your voice, which is great... They also pick up all the other sounds around (furnace kicking on, doors closing, water pipes, toilet flushes). You know your budget better than I do, but if you think extraneous noise may be an issue for where you're recording, consider a dynamic mic instead. A slightly more expensive "starter" dynamic mic is the ATR 2100, for example.
/u/AlanDavison also has a great list of dynamic setups at many different price points.
Audio technica atr2100 - sorry I have no idea how to properly link. These things have been on sale for as low as $35 in the past, but are still a good deal at $60. http://www.amazon.com/Audio-Technica-ATR2100-USB-Cardioid-Dynamic-Microphone/dp/B004QJOZS4/