It's not that the math is 'too hard.' What I've concluded is that the math is almost all by rote (a description of something 'MBTI sensing' and then the math just 'tacked on'), so I'm either going to have to deal with that or get out of it...
I also think INTp and INTj are the exact opposite at physics: for INTps, they are studying math simply as a tool to learn the physics. I see physics almost as applied mathematics, and so I'm studying physics simply as an interesting way to learn math - I cannot possibly look at math as a tool...