2x cells that equal 6.4V would sound more like they are 3.2v nominal cells. They might be LiFePO4 batteries and not standard Lithium cells.

Do you know what the full charge voltage is on them, or what the charger outputs?

If they are LiFePO4 cells, then they should be 3.6v per cell when fully charged, so the charger would likely output 7.2v total, as opposed to standard Lithium cells that would be 4.2v per cell when gully charged and use an 8.4v charger.

If it is in a kids toy that would make sense to LiFePO4 cells since the electrolyte in them is non-flammable, so there is no risk of fire or exploding if the cells are shorted, overcharged or even punctured. the trade off for the safety is the slightly lower maximum voltage, and they tend to be a little more expensive.

Here is a holder: https://www.amazon.com/Aobao-Single-Battery-Holder-Leads/dp/B09MYCMBBF

You will need to check if the LiFePO4 will fit. Otherwise find another. This one will fit an AA holder: https://www.amazon.com/Shockli-LiFePo4-Phosphate-Rechargeable-Batteries/dp/B07RGQX6W7/

I would swap out batteries (and not permanently connect and have to recharge with the circuit).