Back when I had to maintain a separate laptop for work and my personal computer, I used some displayport switch boxes to switch three displays between two computers. I used a separate mouse and keyboard for each system, as I wanted to be able to use both systems at the same time and just switch the displays between the two systems.
I had (3) ASUS 32" displays, each connected via DisplayPort. Essentially, each of the monitors had a switch box that connected to both computers and this allowed me to toggle which displays each computer used. I mounted the switch boxes under my desk and reached under to flip the switch and change the displays between systems. For audio, I used a set of external speakers that connected to both systems via a y cable, although I mainly used a set of bluetooth headphones.
I too went through several different kvm devices and finally decided the main part I wanted to switch was the monitors, so I finally found and settled on the Lindy 2-Port 4K DisplayPort switch (https://www.amazon.com/LINDY-Port-4K-DisplayPort-Switch/dp/B00ZK2S71W/ref=sr_1_10?keywords=LINDY-USA&qid=1688137721&sr=8-10). It doesn't look like these are available on Amazon anymore, but I paid around $40 each for them, there are a couple others out there for similar pricing. With these switch boxes, I could have 1, 2 or 3 monitors assigned to each system. Often I had 1 monitor per system and toggled the 3rd monitor to which ever computer I needed the 2nd display on at that point in my workflow. But often afterhours I would have all three monitors switched to my personal computer.
You could probably mix and match HDMI and DisplayPort switches to accommodate the monitors you have and the available ports on the computers. But I would try and standardize on either HDMI or DisplayPort. There are converters out there to convert between HDMI and DisplayPort, but the more pieces you cobble together, the more issues you would probably face. You could also potentially add more DisplayPort or HDMI ports to each pc using USB or thunderbolt adapters, docks, etc.
There are really good quality multi-monitor KVM switches out but you'll end up paying $800 - $1200 depending on your needs and these are mostly only HDMI or DisplayPort, not both.
The last time I had this setup was maybe 3 years ago now? Just before the covid lockdown, we started using Amazon Workspaces at work as an option. So today, I just run the amazon workspace client on my personal Mac and do all of my work in a window. I no longer have a work laptop, I returned it as I wasn't using it.
As you also mentioned using a Remote Desktop app could potentially work. Microsoft Remote Desktop features are built into windows and are usually enabled on work pc's, so you could potentially run a RDP client on your personal pc and connect to the work pc via RDP without installing any software on the work pc. Parsec is just a fancy low-latency RDP client/server software. If both pc's are on the same home network, it's overkill anyway and the basic/free RDP client should suffice.
I have a similar issue. I found this on Amazon, but only 2 reviews.