For sure, Japanese grammar rules can seem arbitrary and frustrating to study, especially if you spend a big part of your time on that. But I wouldn't give up completely on it, at least leave yourself open to the idea that your mind will eventually adjust to certain grammar patterns. It may be that you are actually better at picking up grammar patterns from actual speech/text as opposed to learning from a textbook, the former being more important anyway.

But I think you are on the right track in terms of spending your limited time on something you enjoy. I would suggest reading children's stories, since generally they user simpler grammar and can be easier to finish in one sitting.

There are a bunch of books and sites out there, but I happen to have created my own set of books of classic Japanese fairy tales with bilingual Japanese/English text so you can check your understanding. Here is the link if interested: https://www.amazon.com/dp/B087JSVCC6

Podcasts are also a good option if you are time limited, since you listen on the train/bus/car, etc. Here is one I have enjoyed: https://manga-sensei.com/podcast/