So I found this transformer, and an upward facing lamp base that can take 250 volts. Would this work?
It will work for all things that would work if you plugged them in by themselves, ie anything that's 120V/240V compatible. I don't think there's any reason it wouldn't be safe as long as you don't exceed the rating of the extension cord.
If you plug something in that's 120V-only, though, there will be, at best, a blown fuse, at worst a fire. But that's no different than if you plug it on directly.
For anything that's 120V-only, you need a converter. (Beware: while I'm thinking of getting one for the same purpose, I have not researched any models in particular. The reviews seem to indicate that some of them seem sketchy, quality wise.)