Yeah, it's a technique popularized by the /r/Ultralight crowd.
You can get 5 for $6 on Amazon. https://www.amazon.com/Hefty-Trash-Compactor-Bags-GAL/dp/B002BXRGQA
Pack covers do protect the fabric of your pack. However most modern backpacks are made of nylon materials that shed water and don't really retain water weight. If you've got a bunch of stuff strapped to the outside of your pack - the cover may still be necessary (although that's one reason we typically recommend not strapping sensitive gear to the outside of your bag).
Pack covers tend to flap in the wind. They make it a pain to get into your gear without getting everything wet.
A liner also helps eliminate extra dry bags. I used to have a dry bag for clothes, jackets, sleeping bag, etc. Now I just jam everything into my single pack liner and roll it up tight. Even more weight/space/cost savings there.
This helps with the 'pebble' problem. When you have many dry bags and other hard items in your pack, they take up extra space. Think of pebbles in a glass (pebbles = gear, glass = pack). There is air / unused space between the rocks. However if you take the gear out of those smaller / multiple dry sacks - and jam them all together - you use the space inside your pack more effectively.
Big 4 for $502
The rest of the gear is relatively cheap, like:
Random stuff from your house / DIY:
Clothes could be a whole post of their own. Start with what you already own and upgrade to wool/synthetics over time.
Food from the grocery store is cheap. Skip the fancy freeze dried stuff. Shout out /r/trailmeals.
Skipping unnecessary items is the best way to save money. Amazon will make you think that a huge knife, lantern, chair, axesaw, and other 'camping' gear is important. Don't let them trick you into wasting money / weight on garbage.
EDIT: added a few items