You'll want a USB Switch (not a hub).
Something like this - https://www.amazon.com/dp/B08MBXMZLV
Go for all HDMI ,easier to use and more common port.
Also, some displays may not support DP (e.g. you want to pipe one output to your TV or Projector, these usually don't come with DP ports....
Male DP to Male HDMI https://www.amazon.com/DisplayPort-Gold-Plated-Avacon-Display-Adapter/dp/B01FX3K7T2/ref=sr_1_3?crid=1COJZOP737CAG&keywords=dp+to+hdmi&qid=1688331148&sprefix=dp+to+hdmi%2Caps%2C262&sr=8-3
Also, I dont think you need a KVM switch, see your monitors have both DP and HDMI inputs. Now my monitors have 2 short cut buttons that I have programmed input 1 (DP) and Input 2 (HDMI). All input 1 have work computer, Input 2 are personal pc. So when I need to switch, I just press the relevant short cut button on the monitors. If your monitors have such a feature, then you are good.
For Mouse and KB, get a USB only KVM. https://www.amazon.com/ABLEWE-Selector-Computers-Switcher-Compatible/dp/B08MBXMZLV/ref=sr_1_5?crid=1SZZRZ97F4EE1&keywords=usb+kvm&qid=1688331544&sprefix=usb+kvm%2Caps%2C111&sr=8-5
In this method, you have to press 4 buttons when you want to change PC, but you get flexibility to display which ever PC you want on which ever screen.
USB hub is used to connected into the USB port on the laptop, or it won't wok. As you said , USB KVM switch is a good choice for you.
Following are links of USB Hub and USB KVM Switch, you can see their difference clearly.