Usually, the biggest difference between topsoil and potting soil is that potting soil has been amended and fertilized with the Colonel's Secret 11 herbs and spices (sometimes quite literally with chicken manure)
Topsoil typically does not have a bunch of fertilizer or manure premixed into it. It is graded and blended with a couple of different types of soils to allow for filling and compacting, dressing or starting a new lawn.
I wouldn't shock the plant further by diffing them up and replanting. Buy a cheap NPK kit off Amazon and test the soil, then add a fertilizer that will bring everything up to the correct levels.
They run anywhere from $10-way too much money.
Or look into a local conservatory group/school/county/state run free soil testing center. Most states have some form of free soil testing available where you take a sample of your soil and mail it off, then you get the results back with recommendations for fertilizers. Google Free Soil Testing...and your State to find options.
Sometimes they will ask what you are planting (lawn, veggies, flowers, etc) and customize a year-round fertilizer plan. I bought a test kit from RX Soil and paid $40 iirc, and they gave me the first full year of amendments by seasons for my specific planting. I did it to find out what was wrong with my lawn and felt the $$ was worth it, but the less expensive DIY kits are just as accurate (plus it can be an interesting evening project for the family)