The tension bolts on the rear axle?

https://www.amazon.com/gp/aw/d/B0711TSFP5?psc=1&ref=ppx_pop_mob_b_asin_title

You need to replace that 'little threaded rod'. That's the tension bolt, and the main way to adjust your chain..... they're cheap and break all the time, I usually buy them by the dozen from amazon.

After you replace the tension bolt(and might as well do the other one too) tighten the axle just enough so that it still moves in it's slot. And then tighten/loosen the nut on the tension bolt to get the right slack on the chain, then tighten the axle up.

Keep in mind that an old chain, specially a stock one will stretch out over time. So if you find that you've tighten the tensions bolts all the way and there's still some slack you might have to take a link or two out of the chain.

You want to have the top run of the chain to be as tight as possible, and the bottom run should have about 1/2-3/4 inch of movement when you press down or push up on it.

Aside from all of that make sure the chain is aligned properly, and the spacers and washers are in the right order on the rear axle. And if the chain is old there's no kinks, bends, or warped links that would make it pop off.