Ultimately two things really conspired against Demigod.
1. The launch was brutal. It's been covered in a few places just what went wrong here, but the short version is that connectivity was a mess. For a predominatly single player game (like say Civ) that's annoying but not the end of the world, but Demigod is a multplayer focused game.
First of all Gamestop broke the street date, so a pirate version was out even faster then usual. That pirate version was connecting to the servers (in numbers far higher then paid for versions) looking for updates, which brought down some servers. They got that fixed, but then the real ugliness reared its head: player connectivity also didn't work very well. I for example couldn't play at all the first week it was out, and could only play on Hamachi (emulating LAN) for another week after that.
Technical details are also elsewhere, but the peer to peer code along with the NAT traversal stuff didn't really work right at launch. Getting into a game where everybody could talk to everybody was all but impossible for large games, and not easy for small games. Many people just couldn't play at all (like me). Those who could wound up using things like Hamachi and Gameranger (recommended by Stardock at one point) to get around the problems.
This was eventually cleaned up due to a herculean effort on Stardock's part, but by then the damage was done. The community was split into several smaller communities, and many people were just fed up.
2. The game was too pretty. That might sound strange, but it's true. It was a ton of work to add content to the game. The levels in particular didn't use a conventional map editor, they used some crazy 3d software that nobody has. So, user created maps were non-existant. GPG isn't exactly known for awesome post-release support, and they didn't put out much new content. I'm not sure if any new maps came out. They did put out a couple new Demigods eventually, but it took forever.
DotA itself was reincarnated on at least one occasion, and got updates and things changed. Demigod was really static.
IMO a game without the super pretty 3d maps that allowed people to actually create custom maps would have done better. Ditto with GPG adding new Demigods faster.
At the end of the day, the game was pretty fun... but the beta test and internal QA somehow missed MASSIVE connectivity problems (to be fair, it's pretty hard for internal QA to duplicate exactly what happened in a test lab). After that GPG's support was lacking. On the whole, I've got a very negative opinion of GPG and pretty much won't buy anything they put out until its on the discount rack and been patched to the point of working.