Connect solar panel to a solar charger, connect the solar charger to the battery, connect the battery to an inverter, connect the invert to an automatic transfer switch (ATS), connect the grid power to the ATS, connect the ATS to the outlet/lights.

The outlet/lights will draw power from the battery/panels/inverter when possible, and if the voltage drops it will instantly draw power from the grid instead. If you have a computer plugged in, I would also use a UPS. Even though some ATS's are nearly instant, it might not make your computer happy.

Here's an example of an ATS that might work for you: https://www.amazon.com/MOES-Controller-Automatic-Transfer-Switch/dp/B07F12RDZ2

this product seems to do that https://www.amazon.com/MOES-Controller-Automatic-Transfer-Switch/dp/B07F12RDZ2

Thanks for responses. I was able to find a automatic transfer switch that does what I was looking to do. https://www.youtube.com/watch?v=ni0A_ZdH7aU MOES Dual Power Controller this allows you to run off your solar/battery, but cuts over to the mains when it runs low. Ebay lists them for about $100-150 amazon has one for $119 https://www.amazon.com/MOES-Controller-Automatic-Transfer-Switch/dp/B07F12RDZ2 This device could be used to offset constant power items (outdoor refrigerator/freezer, tiny servers) during the day with solar, and kick over to mains at night. This makes sense for high electric rate areas (Like our rate, Hawaii, at 0.44/kwh)