You bought the worst possible rack for "milk crate" usage.

I have the Ibera Touring Rack+ and that is the rack you need. Notice how the pannier bars are lower than the flat top, and offset to make the panniers easy to install and remove while a basket is attached to the flat top.

https://www.amazon.com/Ibera-Bike-Rack-Frame-Mounted-Adjustable/dp/B09VMB3ZXF