Results 1 to 5 of 5

Thread: Distortions in Self-Report Data

  1. #1
    Creepy-

    Default Distortions in Self-Report Data

    We all know this is a big problem with testing, so I thought maybe we should collaborate and put our insights and ideas into one place for easy access if anyone wants to develop a new test or improve on an old one, etc.

    Here's a list of some common distortions (lists and definitions are incredibly useful things).

    Social-desirability bias
    - a tendency to give "socially approved" answers
    - subjects try to create a favourable impression

    Misunderstanding
    - respondents misunderstand questions
    - the way questions are worded can shape participants' responses

    Memory errors
    (pretty much self-explanatory)

    Tendency to agree
    - respondent tends to agree with nearly all items

    Tendency to disagree
    - respondent tends to disagree with nearly all items

    There's probably not a great deal one can do about the last three distortions (although a large scale like rmcnew's might help somewhat for the last two).

    Social-desirability bias is likely responsible for the "ENTp bias" that SG illustrated here. Maybe test-developers need to make sure that they don't make any functions sound better than others. (I don't have any answer's, I'm just speculating for the moment).

    The distortion I'm particularly interested in is the misunderstanding. How does a test developer minimise misunderstanding? Maybe rather than have 1 statement per item, there should be 2 statements which illustrate the same concept in different wordings? Or maybe pairs of positive and negative statements in each item?

    Anyway, ideas are wonderful! This is an area that deserves our focus.

  2. #2
    Creepy-

    Default

    Bump! I really think this is important.

  3. #3
    Cone's Avatar
    Join Date
    Mar 2005
    Location
    Pennsylvania
    Posts
    2,717
    Mentioned
    4 Post(s)
    Tagged
    0 Thread(s)

    Default

    Haha, I was going to make the exact same reply yesterday, but I thought this topic was still too recent to bump.

    I also think it is important and something to consider for future test makers.
    Binary or dichotomous systems, although regulated by a principle, are among the most artificial arrangements that have ever been invented. -- William Swainson, A Treatise on the Geography and Classification of Animals (1835)

  4. #4
    Creepy-

    Default

    Quote Originally Posted by Cone
    Haha, I was going to make the exact same reply yesterday, but I thought this topic was still too recent to bump.

    I also think it is important and something to consider for future test makers.
    I thought it might be too recent as well but I wasn't sure anyone was going to read it in the next few days anyway. I am also going to post (in another thread, for clarity) about Experimenter bias (also important).

  5. #5
    Kim's Avatar
    Join Date
    Jun 2005
    TIM
    IEE e7 783 sx so
    Posts
    6,857
    Mentioned
    380 Post(s)
    Tagged
    2 Thread(s)

    Default

    I think the only way to work around that is to test for the accuracy of your type in several ways:

    1) Take tests AND have people who know you well take the test with you in mind. Compare results. It has worked for me.

    2) Read type descriptions and see if they fit

    3) Check how the functions work for you

    As for tests alone, you can never work around bias in my humble opinion. But one good way really is to be typed by people who know you well.

    Maybe test-developers need to make sure that they don't make any functions sound better than others. (I don't have any answer's, I'm just speculating for the moment).
    As long as people themselves have a perception of desirable and not-so desirable functions in their head, the way they are presented on the test is not going to make a difference. Especially when they read around here ::

    The distortion I'm particularly interested in is the misunderstanding. How does a test developer minimise misunderstanding? Maybe rather than have 1 statement per item, there should be 2 statements which illustrate the same concept in different wordings? Or maybe pairs of positive and negative statements in each item?
    Yes, that sounds interesting. But I would have to think about it more
    “Let us forget with generosity those who cannot love us”
    ― Pablo Neruda

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •