That RAM is intended for servers/workstations, and won't work with an Optiplex 390; you need unbuffered non-ECC memory. Something like this would work, though it's much more expensive as buffered ECC RAM isn't exactly in high demand. For what it's worth Dell's manuals are usually outdated, and underestimate the max amount of RAM you can use. That i3 supports 8GB sticks so your max is really 16GB, although it doesn't make much sense to use that much RAM.

The 750 Ti is still a good choice, but it's getting long in the tooth. A 1050 Ti is the most powerful 75W card out there, although with prices starting at $145 and only going up it might be a bit out of reach. 1050's are still more powerful than a 750 Ti but those start at $115ish new. The Radeon RX 460/560 are worth considering too, and are priced similarly to the Nvidia options. Really it comes down to what deals are available to you. I'd go for the cheapest of the 4 cards I mentioned if you can, and a 750 Ti otherwise.