The games that I've tried so far in this or a similar genre:
SOTS
SINS
GC2+
X-3:TC
Armada 2526
MOO1,2,3
Planetarion (browser game)
Star Ruler
The ones I have not tried that I've heard various interesting things about is:
Star Empires IV, V
Distant Worlds
Now, for the first list, I'll bulk them according to features.
Explore
How fun is the game world? Is a planet 'one,normal' or 'a teeming green world orbited by a precursor defence system'. Can you get surprises?
Staple games of ths are GC2, MOO's, SOTS. However, X-3:TC should be mentioned here. As it is a flight sim-rpg kind of game (with the option to build a space emprie) it does this bit really well.
Expand
Colonization. Explore is about how much life there is in the galaxy. How much life is there in your empire? Interesting races? Unique features? Racial traits? Advisors? Research and technology?
GC2, SOTS are good at this. SINS with all the expansions are decent as well. MOO series was good at this too.
Exploit
Diplomacy and strategy. Is diplomacy limited to 'shoot first, talk to the hand' or can you have genuine allies and enemies? Strategic advantages? That sort of thing.
SOTS does the strategy well, with the radical travel differences between the races. Diplomacy is less good, though. SINS, GC2 are good at diplomacy. But here is the reason I mentioned Planetarion - it's massive MO, with hundreds or thousands of players, resulting in coalition building and diplomacy more complex than in any 'normal' game - same with EVE online, incidentally, but it has no empire building so it can't really be counted - but once you're in a corp, it qualifies well enough. This is also where X-3:TC falls through.
Exterminate:
Or, 'combat! What's it about and is it fun?. For my part, I like combat to include ship design and tactical combat, with an option to autoresolve inconsequential battles. SOTS, IMO, is the best of the listed in that, with MOO2, X-3:TC good contenders. GC2+, planetarion would probably be the worst. Stack fights is in my opinion the worst possible option. SINS, while the combat system is good and exciting, lacks the unqiue components 4X games should include, IMO.
Special mention: Armada 2526 and Star Ruler. They were heavily recommended indy games, but for me, they failed to deliver enjoyment. Star Ruler remains a tech demo, Armada 2526 suffers from crashes and lack of being fun (tm).
Another special mention:
Pirates of the Burning Seas. Yes, it's a pirate MMO. It does, however, feature ship combat, exploration, building up a trade empire (if that strikes your fancy), etc. Worth a look now that it (soon) enters Free to Play. And the ship combat? It's brilliant. Probably the best ship combat I've run into, period.