but I don't know if they sell one with 4 ports.

You can get up to 6 keystone jacks in a 1-gang box

I thought that I would buy a new rack-mountable switch and patch panel. This is what I've been considering:

Like /u/toaster_knight said, get a vertical or deeper one. I would strongly recommend an actual enclosure with a locking door for an office environment.

I also see no reason to put anything smaller than a 24-port in a rack. It's just a waste of space and money.

I would also recommend at least a SMB-level switch. Yes, they are more expensive, but their uptime and management capabilities are worth it IMO. The Ubiquiti 24-port switch is under $200 IIRC.

I'm assuming that I can rest the router on top of the switch.

That will probably be fine, but you can get rackmount shelves and drawers as well. The shelf is nice because you can remove the switch without messing about with the router.

I'd also like to buy some type of cable organizers for inside the ceiling. I normally use those small cable clips, but they seem to be intended for one cable. I've seen velcro used, but I'm not sure how to actually secure the cables.

You can get joist hangers, but inside a ceiling, the cables are usually just bundled and left loose or in a tray. If you have a drop ceiling, there are a wide variety of products designed for your support system.

I would also recommend a UPS and a power distribution panel. UPS for obvious reasons and the power panel will let you easily power cycle each component without having to dig into the back--something that gets much harder after rack-mounting.