If you want some vintage feel to it, buy a 3-position rotary switch and wire your power leads to whichever connection is made in the middle position.

To turn it on you'd just turn it all the way in one direction - that makes contact momentarily in the center position, acting as a momentary switch. Turn it off the same way, turning it all the way the other direction.

I used that method in this build