Thank you for the info! I figured that "it depends" might be the answer. I agree that I'd rather overdo it than try and min/max my way into just barely having enough.
As for weather, we have probably less than a dozen nights below freezing a year and maybe 15 days over 100, pretty moderate. It sounds like with the controller being a crucial part, the one I linked originally would not be ideal. I found this 30w panel +mppt controller. Would that + a 30AH deep cycle battery seem sufficient? The 50w version+mppt is only about $20 more so I'm definitely considering just upsizing if that's not absurd.