If you want a looser fit, I like Levis 550s for jeans and Izod big and tall pleated for khakis.
The Izod khakis fit in either as casual or business casual and they have an expanding waist to make putting them on easier.
If you want something a little more form fitting but still comfortable, I recommend the Levi's 541 with the stretch material for jeans and their 541 twill variations for slightly dressier. I do recommend going up a size on the twill though, as they have less give.
6'4", 320lbs