I think the question is, should we treat abstract ideas and entities just as real as physical entities? Is "masculine" just as real as "male"? I would say yes.
If biological sex determined everything, then there's no point in talking about psychology. We might as well as biologically determine everything. But an abstract computer program can affect the hardware in the physical world, and potentially, it can start to change its own hardware or it can affect things in the physical world.
The neo-Darwinian, "selfish gene" theory imply that the body is nothing but a vessel for the gene to carry and replicate itself. The mammal is mostly a visual being, and so it must rely on visual or some other sensual markers in order for it to mate. It does not know what "masculine" or "feminine" means. But we humans know what "masculine" or "feminine" mean without relying on any sensual markers. We can abstractly know and understand that just from reading texts, for example. We become attracted to people chatting online or reading fictional books.
The Meme evolution theory imply that abstract memes are influenced by evolution, which makes the evolution of abstract ideas just as real as the evolution of physical objects. I would say that "male" and "female" are simply physical evolutions, and "masculine" and "feminine" are abstract, memetic evolutions that occurred within a complex social society, and it co-evolved with the biological sex.