How much is enough? Second writing group post

(First post here).
So at the writing group Tuesday, the same response kept coming up regarding two of the other readings. One was a kid on a frontier planet trying to run away from home, the other was a flashback to the lead character’s childhood. Neither one, someone said, really had much SF in it. And one of my colleagues invoked the old observation that if you can take the SF out without changing the plot, it’s not really SF.
I wouldn’t hold that as a universal standard but I would agree I feel cheated if I pick up an SF/fantasy novel and it turns out not to be. I remember a fantasy I read about 20 years ago that involved elves in a rock band; if the author had taken out the elves, it would have been mainstream without another word changing (no magic, nothing, just elves acting like human rockers).
Definitely nobody should put down an SF/fantasy novel feeling that way (I haven’t looked at the author’s books since), but what constitutes “not really specfic” is a little subjective. I thought the runaway story perfectly good SF——the fact the protagonist is trying to hop a space freighter and that there’s a serious risk of getting stranded on a mining-planet hellhole if he does gave it a very different feel from a boy running away on Earth. The fact that abusive families are present in the real world notwithstanding.
I find it hard to see how Katherine Kurtz’s Deryni books are not fantasy, but Ursula LeGuin dismissed them as Not Really Fantasy on the grounds the scenes of intrigue in the Deryni Council could just as easily be politicians in the US (and sounded like it).
And of course, Asimov’s Foundation Trilogy, as he freely admitted, borrowed heavily from Imperial Roman history, but I don’t know anyone who’d consider it not SF.
What makes something “not really” speculative? Feel and tone is part of it, I think. LeGuin found Kurtz’s prose style too flat for fantasy, and didn’t think a fantasy novel should waste time on grubby politics. I thought King’s The Stand was more a disaster movie and a very bad one than a horror novel (it’s one of the most deadly dull post-apocalypse stories I’ve ever read). That’s a subjective standard it’s hard to set terms too
Sometimes magic or high-tech is just the McGuffin, like the spy stories where an SF superweapon is the bone of contention. In the movies, I’ve frequently seen low-budget horror/SF that my friend Ross describes as “just enough”——a tacked-on SF element to what’s otherwise a straight thriller.
Sometimes it’s marketing. Dan Simmons’ Song of Kali is a good drama about one man’s nightmarish experiences in Calcutta, but it isn’t supernatural horror——however, I can’t blame Simmons that this is how the book was promoted.
Sometimes “not really SF” works. Fritz Leiber’s “Foxholes of Mars” started as a contemporary political story, but didn’t sell until he set it in the future. I think that makes the political aspects (embittered veteran starts whipping up war fever for political gain) more universal and less specific. Spider Robinson’s “Time Machine” has no SF at all——it’s premise is that a character who went to prison overseas in the early sixties and emerged a decade letter suffers the same culture shock as if he’d time-traveled those years.
So I’m not quite sure what my point is——”Don’t write SF stories that don’t have clear SF elements, unless you can make it work.”? But I felt it was worth touching on anyway.