I too hate condoms for this reason. Latex condoms have a texture that seems to grab or stick to me, like how a balloon does when you run your fingers across it. Lube helps but not enough. The first thing is to use a high quality lube. The water based ones can get sticky fast sometimes. I’ve been using this one for years and it doesn’t get sticky on me.

Second thing is you may not have a latex allergy but that doesn’t mean you can’t try latex free condoms. For me they are much more comfortable and as someone else said you can use silicone lube with them. I don’t have a latex allergy but I only use non latex condoms.