Ch. 14 - Evaluating Measurement & Data Quality (QUANTITATIVE)
Terms
undefined, object
copy deck
- What is MEASUREMENT?
-
- assigning NUMERIC VALUES to ATTRIBUTES (qualities/characteristics) of objects so they can be quantified
- Attributes vary (ex. from day to day, person to person, etc.)
- VARIABILITY can be captured by NUMBERS that signify "how much"
- Measurement allows comparison among people on particular characteristics
- requires that rules be applied in a SYSTEMIC fashion
- rules for physical or physiological measurements are well known (ex. BP, temperature, weight)
- rules for many variables must be designed by the researcher (ex. depression, pain, satisfaction, happiness, etc.)
- rules enhance OBJECTIVITY
- provides for PRECISE information about an ATTRIBUTE - What is MEASUREMENT ERROR?
-
- Scores from even the best designed instruments have error
- the difference between true and obtained scores
- OBTAINED SCORE: True Score +/- Error
- TRUE SCORE: the score that would be obtained if it were possible to have an infallible measure (no human or instrument error); hypothetical
- Measure Error: the error that occurs as a result of EXTRANEOUS factors - 5 FACTORS that contribute to ERROR are:
-
1. SITUATIONAL (participant awareness, environment)
2. RESPONSE SET BIAS
3. PERSONAL FACTORS (lack of interest, fatigue)
4. ADMINISTRATION VARIATION
5. ITEM SAMPLING - 2 Criteria used to assess the rigor of instruments are:
-
1. Reliability
2. Validity - RELIABILITY
-
- CONSISTENCY of findings from the instrument; the LESS VARIATION in measurement, the more RELIABLE the instrument
- maximizes the true score and minimizes the error - 3 Types of MEASUREMENT RELIABILITY:
-
1. Stability
2. Internal Consistency
3. Equivalence - STABILITY
-
- extent to which the SAME SCORE is obtained when the instrument is used on the SAME PEOPLE on DIFFERENT OCCASIONS
- appropriate for characteristics that are enduring
- determined through test-retest procedure
- reliability coefficient calculated, r (.00 -1.00)
- the HIGHER the coefficient, the MORE STABLE the instrument
- r > 0.70 is DESIRED!
- every test should show the RELIABILITY COEFFICIENT in the research article; reliability tends to be GREATER if the pre and post test occur over a short time span
*r = the symbol for a bivariate correlation coefficient, summarizing the MAGNITUDE and DIRECTION of the RELATIONSHIP between 2 VARIABLES - Internal Consistency
-
- extent to which the items on a scale all measure the SAME ATTRIBUTE
- determined through:
a) SPLIT HALF PROCEDURE: reliability coefficient calculated on the 2 HALVES (odd vs. even)
- first 50 items will be one test and the 2nd 50 items will be another test; the results should be the same
b) CRONBACH ALPHA (coefficient alpha) - split half correlations for ALL possible halves
- what are the possible COMBINATIONS of halves (ex. 51-1; 53-2)
- many versions of 2 halves and many ways to combine them so 2 samples are different
- test EVERY possible to half to make sure they are equivalent - Equivalence
-
- consistency between raters who score the instrument
- assessed through interrater reliability
- used primarily with observational data; can be used with other QUALITATIVE data
- 2 or more people make simultaneous assessments
- index of equivalence determined - reliability coefficient - What is the importance of RELIABILITY COEFFICIENTS?
-
- Low reliability prevents adequate testing of hypotheses - failure to confirm
- Important for INTERPRETING research results - VALIDITY
-
- degree to which an instrument measures what it is supposed to - degree to which the scores reflect the variable being examined, and not something else
- an instrument cannot be valid if it is not reliable; can be reliable but not valid - 4 Types of Measurement Validity:
-
1. FACE Validity
2. CONTENT Validity
3. CRITERION Validity
4. CONSTRUCT Validity - FACE validity
- - do the items look like they are measuring the concept
- CONTENT validity
-
- extent to which the concept is adequately covered
- SUBJECTIVE assessment of experts
- can be determined by a CONTENT VALIDITY INDEX: extent of EXPERT agreement - CRITERION-RELATED VALIDITY:
-
- extent of the relationship between the scores on an instrument and an external criterion
- determined by a validity coefficient (r): strong correlation is desired, >.70 - CONSTRUCT VALIDITY:
-
- extent to which expected theoretical relationships are confirmed
- determined through correlation coefficient
- factor analysis can also be employed - How to Critique Instrument Quality:
-
- assess the report for information about the RELIABILITY and VALIDITY of the instruments used
- are CONCEPTUAL and OPERATIONAL DEFINITIONS provided?
- what validity and reliability assessments were used?
- do the validity and reliability assessments give you confidence in the findings?