a great solution for ragequititng is (1) giving players incentive to win (like a well-developed ladder) in order to achieve a legitimate record, (2) have a matchmaking system that matches up based on equal skill levels (as best as possible), (3) letting players get organized as best as possible within your game
you (SD & GPG) want a competitive game, so you should take hints from the successes and failures of those before you (guild wars, wc3, dow2, even wow arena). in WC3's glory days of ladder play, people didn't get upset as much when the opposing player or player team left, because people often left in the spirit of concession. professional WC3 players will say "gg" and leave before the very final units and structures are destroyed, because it's wasting time to stay. it was actually bad manners in some cases to stay while clearly losing. why did this happen in WC3? players were organized, sometimes familiar with each other, and generally trying to win to improve their rank and record. players were given incentives (records, rank, etc) to use their good judgment when deciding whether to leave or stay, rather than letting their emotions get the better of them and exiting the game.
the random teaming and custom gaming of WC3, on the other hand, could be just as bad as DotA's or what demigod's is shaping up to be. ragequitting became a symptom of playing in PUBs constantly. a clear lack of organization + a lack of a preexisting relationship = frustration when trying to win.
give good incentives to TRY and WIN games consistently. let players get organized for teams fights against other organized teams. then put in penalizations if needed.