Why should a male person's self-perception override a woman's perception of reality? As though she has no right to feel her space has been invaded if a male person genuinely believes it hasn't.
I think the focus on "good vs ill intent" (virtually impossible to prove anyhow) draws attention away from the fact that "good intent" does not necessarily make a traumatised woman more comfortable with a single-sex environment no longer being so.
Is she supposed to feel so grateful for good intent and sincerity that she sets aside her need for a space of her own? So grateful that no one actually wants to attack her that the social, emotional and psychological benefits of a female-only space should be sacrificed?
Maybe her desire for a female-only space is "genuine" (and not, as she will be told, "a dogwhistle for hate" etc.). But who cares about how genuine her feelings are? We've no problem with ascribing ill intent to an inconvenient woman.
The real "moral panic" is over women setting boundaries. There, you can impute as much ill intent as you like, yell in parliament, point your finger, imply they're only out to harm others, vilify to your heart's content on the basis that women are never "genuine".