This seems like one of those Russian folklore things that you generally shouldn't put too much stock in.
I had an art teacher who I think was SLI. It worked out fine, she liked me a lot. I've also been in a similar position with IEIs, I didn't see it as being a problem either. Maybe the other way is more "natural", though.