When I was in high school, I would actually wear things that I knew looked unattractive on me. It took a little while to realize that this took its own form of effort, and now I focus on comfort and low-profile. I am lucky enough to be an average sort of size that I can get ALL my outerwear from Goodwill and other thrift stores, and I actually find enough choices that I can put back items that either look too bad or too good. I don't mind something looking well-suited to me, but I don't want my clothing to call attention to me. I also only allow myself to own a limited collection at any given time; if I buy a new shirt, I make myself get rid of an old one, etc.
I *can't* wear makeup, or jewelry or perfume or most synthetic fabrics. I can actually feel that stuff on me, and it drives me up a wall. I can barely even stand to be in the same room as someone else wearing perfume/cologne. Even strong-smelling shampoos often wrinkle my nose.
So, I'd say my physical hypersensitivity plus my unwillingness to call attention to myself are the two biggest factors shaping the way I present myself to others. I do have a wee bit of a style in my office attire: (muted color) Hawaiian shirts and khakis. Thank the flying spaghetti monster for casual business dress.
My LSE SO likes dressing understatedly dressy. He'll wear a nice button-down shirt to go drinking with the guys, but it doesn't come off as pretentious or attention-seeking.