It depends quite a bit on what the appeal of the specific game is. BG3 is a cinematic, licensed DnD experience, so including dice rolls in conversation checks, and having a huge quantity of content in terms of dialogue, voice-overs and sequencer cinematics is basically required, regardless of whether it is the ideal speech implementation. More open-ended sandboxy type RPGs don't need, and pretty much could not produce the mass of content for, that kind of speech implementation (BG3 had like 400+ employees working on it full-time at peak, took many years to make, and included many years in early access, and even for that is pretty constrained/on-rails compared to a sandbox fully open world, and is also still quite buggy).
I lean more towards favoring the semi-sandboxy open world game you get from Morrowind or Daggerfall, where the story is what you make it, instead of the BG3 cinematic approach (and Morrowind was made by ~30 people 20 years ago). As I read over this thread, I kept thinking that the Morrowind way was the least bad of all those mentioned so far, especially if we're talking about a standard combat RPG, and not some kind of slice of life roleplay.
Morrowind already had individual disposition, a personality ability score, a speech skill, racial modifiers, faction modifiers, and reputation.
AFAIK Morrowind didn't utilize its well-realized disposition system and its various modifiers to its best potential. Hence, I wish that another game could've learned from MW and taken its already great base and expanded it.
It's a logical and intuitive system, Someone's willingness to give up information almost always depends on how much they like/dislike you and that's what the disposition system is meant to represent. if you have a high personality stat, People are naturally going to like you more due to the innate traits that make you more likable as a person - as much as I can excuse Fallout and similar games' use of the speech mechanic, I don't like how sometimes a character who might utterly despise you will still give up valuable info based purely on a single line of dialogue...
The above makes the most sense to me. Say individual NPCs have an out of 100 disposition towards you that defaults to 40 for the sake of the argument. You then have numeric alterations like:
-Base Charisma attribute bonus/detriment to all NPC dispositions (or whatever system the game uses, maybe charisma/speech is a trait that can be none/major/minor)
-Same Race bonus / Rival Race detriment
-Same Faction bonus / Rival Faction detriment
-Same Religion bonus / Rival Religion detriment
-bespoke bonuses for specific quests or favors done in game
-dialogue options to increase bonuses based off information you learn in-game (or decrease disposition if you are rude/aggressive/taunting)
-bribery to smooth things over (but it's a risk as it decreases the disposition of NPCs who can't be bribed, or who wanted more money than you offered)
-maybe charm spells (just not broken like in Morrowind)
-perks that balance bonuses/detriments to disposition based on class, profession, wealth, whatever of the NPC in question (like bonus to women but detriment to men or vice versa, or bonus to disposition from the poor but detriment to the wealthy, etc...).
And perhaps a few more things, and all this disposition boost really does is open up more dialogue options as it goes up. Maybe below 10 they won't talk to you. From 10-24 they give curt/rude answers. From 25-49 they provide basic personal and world information like directions and shop locations. From 50-74 they share rumors and secrets, and will participate in most quest specific dialogue. From 75-89 maybe they offer specific bits of help, or are required to be in that range for more advanced quest dialogue. And >= 90 is for the rare times you need someone to straight auto-win the quest for you, or agree to appoint you head of a guild, or follow you as a companion, or share info about hidden treasures, secret doors and such, or some other kind of very high disposition thing. It would also affect merchant prices. I think it would probably be a nice simplification if options always fell in these discrete ranges, and the player knows with decent certainty what kind of disposition they need from who for what (ie 0-10 is just "hates you" and 90-100 is "loves you" and everything in-between labelled, and you just know they need to "like" you for whatever).
If it was just a better balanced, more sensible, slightly evolved form of Morrowind, and used well by quest writers, it would be more than good enough for most combat/exploration/questing based RPGs. I think it fits well with the wiki-type dialogue as well, so you don't have to click through the same dialogue tree repeatedly as often. You come to an NPC to get info or advance a quest, and it's nice to just immediately select what you want, or see what's new, instead of going through tedious dialogue sequences over and over, especially many hours into the game.
As for going much further than that for speech options in an RPG, you could go all the way down the path of different types of speech, and many non-combat options with minigames and other complex mechanics for non-combat builds, but that then strikes me as bordering on a different genre of game. If you can be a dedicated diplomat, or run a store, or be a travelling merchant, or just a pillar of the community around town, then you're talking about the mechanics of The Sims, The Guild, Potion Craft, Recettear, a bunch of social/romance games, etc...basically a different genre to an RPG (although with "RPG elements" oftentimes). A given game needs to pick a focus, even an open world game, and including a full suite of both combat and non-combat play/lifestyles in an RPG is gonna be like making 2+ games at once. Who is the audience for that game? And who would make two games and sell it for the price of one?