Basically. If you knew the correct pins to connect.

USB-C has insufficient bandwidth to drive 2x 4K@60Hz displays, so you are probably getting 30 Hz when you plug in a second display (thunderbolt can do 2 @ 60 Hz, but not USB-C). You can go to chrome://flags and enable list-all-display-modes, which should tell you exactly which mode you're getting in the display settings.

So yes, you'll have to use both ports on the Pixelbook to get 60 Hz on both monitors. The second port doesn't require a full dock -- just a relatively cheap USB-C to DisplayPort dongle/cable that's rated for 4K60 such as this one should do it.