Series books: Without backstory, do I care?

Having enjoyed Cordelia’s Honor by Lois mcMaster Bujold, I’ve been checking the library for the next book in the series (by internal chronology). Having had no luck, I figured I’d just pick a later one at random, and selected A Civil Campaign. This was a mistake.
Not because the book is bad, or because I had trouble following it without the backstory in earlier volumes. But without the backstory, I care as much as I’d have liked.
Series protagonist Miles Vorkosigan fell in love in the previous book. His love interest, Ekaterin, is newly widowed, obviously ready for a new relationship, and very desirable to other men of their social class. To stake out his turf until Ekaterin is ready for new love, Miles hires her to design gardens for he estate, figuring this will let him monopolize her time and get inside her defenses (this does not work well).
In a B plot, Miles clone and adopted brother has his own romantic issues, and comedic problems involving his new business venture. In the C plot, a woman tries to outwit Barrayar inheritance law by getting a sex-change operation.
Bujold does an impressive job filling in backstory without info dumping and leaving some backstory unexplained without confusing me. But while I could follow the plot, I had no emotional connection with a lot of it.
I care about Miles’ romance even though I haven’t read the previous book, because he’s the central character. No problem there. But I care a lot less about Mark the clone and his struggles. His plot line is funny at times,but his backstory is complex enough that without having read it, I wasn’t responding to his struggles at any level beyond light comedy.
The sex change plot I didn’t care about at all. Other than Cordelia’s Honor I know nothing of Barrayaran politics or of those who play power games. Trying to care about the outcome was like trying to care about the next election in Poland or Latvia. I don’t.
Having the book run well over 500 paperback pages didn’t help, as it meant the two lesser plots got lots and lots of space they might not have received in a shorter novel.
Lesson learned? Just because I can follow a late-series book doesn’t mean I’ll enjoy it. Next Bujold book, I’ll pick something in sequence.