I thought all Witcher 3 romance encounters were good, although I think only Yennefer quest features tough choice to make and real emotion. Concidering I am instinctively turned away from women of her type that's saying something.
Other good romances were mostly done by Obsidian and people who hate romances, like MoTB & KOTOR parts and PST.
JRPGs featured romances since forever, sometimes sweet, often fanservice'y, although it some respect their "light" and fanservice'y approach actually might not be a bad thing - games are there to entertain after all, and if that's the point you don't feel awkward since you know it's not serious writing.
Bioware, for all their later fault, did some nice stuff even in relatively new games. DA:O featured a romance for females that could get you into struggle if you were playing a non-noble in a world of nobles marrying nobles; and Morrigan, I think, was a good romance for males in terms of C&C and how "difficult" she was. Guessing what was the bitches problem was entertaining and her fucking you over for her own magical goals was nice.
BG2, I think, was good in respect that romances were prolongued, even into expansion; complicated due other NPCs present in party and not afraid to carry over into realms of future and family. In comparison, later games from Bioware just went for fuck'em and go for next one or something, where the goal for everything was a bad cutscene.
Modders are generally terrible, but some did stand out. Dynaheir romance in NPC project for BG1 was as lawful-good as it gets; and my personal favorite is Edwin romance for BG2 which is just so fun.
Some adventure games had cool romances, since those have a lot of time for characters to grow. The Last Express comes to mind with it's dramatic setting of meeting and breaking away because of WW1.
There is also a black pit of various obscure Eroges that can suddenly manifest psychologically-heavy instances of romantic writing, but that is due to their character-focused nature.
In the end what makes or breaks romance is writer's ability. Generally speaking, the one flaw of later Bioware is that it tries to be dramatic yet still falls into a realm of fanservice.
But the most difficult part is probably incorporating a romance into RPG where character is blank slate and people just can't roleplay, since that means everything is turned to the player. When you play game like Witcher, you begin sometimes to think what would Geralt do, and what is right and relevant for his character in the story. When you're in a sandbox, NPCs become your virtual waifus/husbands. Even JRPG eroges often work around that and just don't give you choice and make protagonist act on his own. KOTOR, PST and MoTB also did that - your character has a backstory that you, the player, had no control over during making him.
As a tl;dr, my opinion is:
- Light romances and fanservice shouldn't be ashamed of as long as they run along with overall style of the game (JRPGs, VTM: Bloodlines, and stuff written by Larian like Dragon Commander amazing waifus and D:OS pairing);
- Romances with pre-defined characters work better than with blank slates as proved by top storyfag RPGs;
- Incorporating setting nuances into romance makes them more "realistic" (arranged marriage in medieval game; being a force leecher in KOTOR who controls everyone without player realising it);
- Being good writer and understanding human nature can still trump over everything (Witcher 3). Might be easier than one thinks, concidering how low the bar is due to Bioware fucking up all the time.
- Personal Observation (of a game master who by the nature of games had to deal with everything including playing waifus himself because-well-shit): The cardinal rule of ye old storytelling of yore aka make player think and imagine things that are not explicitly stated in the story might be an elegant way out; since basically no_love in the setting completely is unrealistic, allowing player somehow aknowledge something, and NPC aknowledging that in return but without distasteful Bioware cutscenes can work pretty well. Old example: option to gift flowers to some NPCs in Ys 2 jrpg and one of those NPCs staying for a while to give you goodbye at the end of the game. Cute, simple, elegant, non-cringy.