Did you add "replacement" to your search? Because it turned up a bunch of results for me;

https://www.amazon.com/BRHE-Replacement-Nintendo-Original-Joystick/dp/B08D6L94R6

https://www.amazon.com/Skull-Co-Replacement-Joystick-Nintendo/dp/B07W6VSJ8P

The little teeth on the underside of the thumb caps aren't meant to be removed in the first place, so decorative caps are usually the more popular choice. As cleanly fitting a replacement cap on can be quite difficult.