There's a difference between underexploited genres, and underexploited settings that's not really being discussed. As far as genres, go, we're basically looking at fantasy, sci-fi, and historical, with all their cross-overs like time travellers/alternative histories, space fantasy like Star Wars, superheroes in the modern day, etc. It's only once you really start to narrow it down that you can talk usefully about the setting.
What's strange about this debate is that even weird stuff like Fallen London was pretty popular - and it wasn't for the mechanics, it really was all down to the weird setting. Thief and Dishonoured are hardly traditional fantasy games, and neither is stuff like the Legacy of Kain/Soul Reaver games.
Going by sales on something like this, it's pretty hard to argue that 'setting' really has anything to do with sales, even in cRPG terms. Elderscrolls and Borderlands are nearly tied on sales, at least according to wiki, and the highest grossing RPG series is of course Final Fantasy, which is hardly trad high fantasy. If I was going to make a AAA rpg, purely for the money, I'd go either go 'elite military/mercenary/criminal team/person cross the globe, visit exotic locations, and rob them/fix their problems/kill everyone' or 'Mass Effect in Modern Warfare/Modern Crime/Football'. (They would of course be flashy, real-time action rpgs to maximise $$$)
The 'problem' is not setting - in fact, different settings could lead to more sales (look how much more successful Mass Effect has been for BioWare than Dragon Age) but one of accessibility. It's not rocket science to figure out that the easier, flashier, and less complex a game is, while still offering choices in build, etc, the bigger your potential pool of buyers is. To pick on Age of Decadence a bit, its problem is not that it's set in a Roman-themed post-apoc fantasy world (dude, just say it's fantasy and let them figure the rest out for themselves) it's that the pool of buyers for 'games just like Tim Cain used to make, except not rushed out the door, and with brutally hard combat' is pretty small. The things that make it so appealing to the Codexer down the street will never appeal to the more casual audience. Skyrim didn't sell on being able to play as a gender-fluid Khajit skooma addicted thief, or on the fact that it has orcs, elves and furries, or on choices having consequences - it sold on VIKING! EXPLORING! BIG MONSTER! SHOUTING THINGS TO DEATH! That's stuff even a toddler can relate to.
You never know - the Wire by Obsidian could have been their breakout, Rockstar-style hit - but we all know it woulda been a trainwreck.
I think that's the real reason fantasy rpgs are the dominant genre - fantasy fans have a high tolerance of shit content, and they're not really being served anywhere else in the games market. It doesn't matter if the game's broken, the plot's phoned in, the gameplay is dull and repetitive - it's still fantasy, and that's all that counts. For people coming from tabletop play, standards are even lower. If you put up with Gary playing a seductive elf-maid, you can smile along with Minsc and Boo. There's also the whole correlation between traditional 'hero saves kingdom' fantasy, nostalgia and romanticism, and religious authoritarianism. Many fantasy fans are the type of people for whom any deviation is absolutely haram. If you're dealing with a sizable chunk of audience whose last book since school was R.A. Salvatore, and who genuinely like nice, neat, good vs evil stories that make them feel safe and comfortable, there's not really much incentive to change settings from fantasy, or do anything that interesting with it. China Mieville fantasy is as likely as Banksian sci-fi.
As for mechanics, you can always create shit or good mechanics for any setting. Banner Saga consists almost entirely of dudes hitting other dudes, but the turn-by-turn decisions per character have far more tactical depth than shitty kitchen-sink systems like Kotor, Shadowrun, or Arcanum.