Bond and Fox, Chapters 3 and 4
Major Characteristics of items and scales from a Rasch perspective
Unidimensionality – very important in Rasch modeling as implemented in B&F Steps.
Responses to each item should be determined by differences on only one dimension.
This means that the bifactor models that I’ve been working on cannot be analyzed as such iusing B&F Steps.
More complex IRT models CAN handle the multiple determination associated with bifactor models.)
Reliability –
Person reliability index – the correlation of person ordering we could expect if this sample of persons were given another parallel set of items measuring the same construct (BF p 40). (The parallel set of items could be the same items.) Loosely – how much we could expect a person’s score to remain the same on two equivalent tests. The reliability here is the reliability of a person’s score across different administrations of the “same” test.
Same people – different items -> Same person ability values.
Is each person going to have the same score across different administrations of the test.
Item reliability index – the expected correlation of item placements along the pathway if these same items were given to another equivalent sample of persons of the same size that behaved in the same way (BF p 41). (Or the same people after a memory erasing drug.) Loosely – how much we could expect an item’s difficulty to remain the same across equivalent samples of people. The reliability here is the reliability of an item’s score across different samples.
Same items – different people -> Same item difficulty values.
Is each item going to have the same difficulty across different samples of persons.
Inconsistency - Whether the pattern of right and wrong (or agree and disagree) responses makes sense.
There should be a monotonic increase in probability of correct / agree from the lowest probability for low ability people responding to high difficulty items to the highest probability for high ability people responding to low difficulty items. Reversals indicate inconsistency.
Consistency – Whether the pattern or right and wrong (agree and disagree) responses makes too much sense. It is assumed that there is some randomness associated with each response. So the response patterns are not expected to be absolutely perfect. If they are, that could indicate a problem.
The pathway
A graphical representation of certain measures from the testing situation. Figure 3.2 from B3
Information presented in the pathway
Person ability
Item difficulty
Precision of estimation of item and person characteristics – By symbol size
Item and person over consistency / inconsistency – by location on the horizontal dimension
As we’ll see – having all the information presented in the pathway will give us a more easily appreciated picture of the measurement process than, for example, just having the scalogram.
Bond and Fox Chapter 4 – Developing a test
Data are: Bond & Fox BLOT data: Chapter 4
The Bond Logical Operations Test (BLOT): A test of childhood cognitive development suitable for administration to a whole class of students.
Items were taken one by one from Chapter 17 of Inhelder and Piaget (1958), The Growth of Logical Thinking.
The 35 items on the BLOT are identified as follows
01 Negation (to negate identity) ; Item labels courtesy of Trevor Bond
02 Reciprocal (to negate identity)
03 Implication
04 Incompatibility
05 Multiplicative compensation
06 Correlations
07 Correlations
08 Correlations
09 Conjunction
10 Disjunction
11 Conjunctive negation
12 Affirmation of p
13 Reciprocal exclusion
14 Probability
15 Reciprocal implication
16 Reciprocal (to negate identity)
17 Identity (to negate reciprocal)
18 Negation (to negate correlative)
19 Reciprocal (to cause disequilibrium)
20 Negation (to cause disequilibrium)
21 Correlative + negation > equilibrium
22 Reciprocal + negation > disequilibrium
23 Correlative + identity > disequilibrium
24 Coordination of two systems of reference
25 Complete negation
26 Complete affirmation
27 Negation of p
28 Non-implication
29 Affirmation of q
30 Equivalence
31 Negation of q
32 Negation of reciprocal implication
33 Probability
34 Coordination of two systems of reference
35 Coordination of two systems of reference
The test was administered to a group of 158 children. The results of the administration are below.
PSY 5950 L13 - 1
001 11111111110110101101011111111011111
002 11111111111111111111111111101111111
003 11010111111111011111011111101011111
004 11111111111111111111101111111111111
005 11111111111101111111011111111111111
006 11111111111110111101011111111111111
007 11111111111101111111011111111111111
008 11111111111111111111111111101011111
009 11111111111111111111111101111111111
010 11111111111111111111111111111001111
011 11111110111111111111111111111111111
012 11011111011111011111011111000110111
013 11111110111111111111011011111101111
014 11111110111111111111111111101001111
015 11111111111111011111010111101111111
016 11111111111101111101111111111111111
017 11111111111101111101111111111111111
018 11111111111101111111011111101110111
019 11111110111111111111111111111111111
020 11111111111111111110011111111110111
021 11101110111111111111111111101110111
022 11001111011101010111011111111111111
023 11111111111111111111111111111111111
024 11111111111111111111111111111101111
025 11111111111111111111011111111111111
026 11111111011111011111111110110010111
027 11111111111111111111111111111111111
028 11101111111111111111001101111011111
029 11111111111101111111110110111010111
030 11111111111111111101010110101111111
031 11111111111101111111011111111111111
032 00101110111111110111011111101101111
033 11111111011111011111011011111110111
034 11111111111111111111111111101111111
035 11111111111111111111111111111101111
036 01111111111101010101011010111001101
037 11011111111101111111011111111011111
038 11111111111110111111011111111011111
039 10011111101111011011011111111111111
040 11111111110111111111111111101011011
041 11111111110111001111011111101001111
042 11011111111111101111111111111101100
043 11111111111111111011111101101110111
044 11011111000110000111101011101100111
045 00111111111111111111010100111010111
046 11111111111111111111111111111111111
047 11111111011111110111110101111111111
048 11111110011101011111111111101100011
049 01110110110101111111011110110111111
050 11111111111101111111011101111111111
051 10010110110101101111110111111110011
052 11001101101101011111010101111011111
054 11111111110111011111011111111110111
055 11111111111101011111111111111111100
056 11111111110111111101011111101111111
057 11011110111101111110111111001011111
058 11001110111111011111011111111011111
059 11111111111111111111011111111111111
060 11111111111111011101011101110010111
061 11001110010111110111011111101110111
062 11110110111101111011110110101001111
063 11011110110110111111011111111110111
064 11111111111111011111011111111001111
065 11111111111111110011010111111111111
066 11111111111111111111011011111011011
067 11011111101110011111011011101011100
068 11111111111111011110011001111010100
070 11101101111101001001010101101111100
072 11011111010101111111011110111011111
073 11011111100101101111011101101111111
074 11101111111111111111011111101110111
075 10111111111111010001111100111011000
076 11011111001100111110010111111011111
077 11111111111101101111010111111011111
078 11000100111111011111011100101001111
079 00111111110111011111011100101111011
080 11111110101111010101110011111111100
081 11111111011111100111111111111111111
082 11011111111111111111111101111111111
083 11111111111111111111111111101111111
084 11111111111111111111011111111111111
086 00011111011101011110011110100011111
087 11011100111111011111111000101110100
088 11111111010110111111111111101111111
090 11111110111101100101011110101010111
091 11111101100101111111001100101000111
092 11111111111111111111011111111111111
093 11111111101101010101011111100011111
094 11111111111111111101111110101110111
095 01111111011111010011010101110011100
096 11011110111111111001011110001100010
097 11111111010101011101011100101110111
098 01101010000100011110010000100100011
099 11011111111111000101010110100110011
100 11100111111111001111011001011011111
101 11111111111111111111111011111111111
102 11111111111101111111111111111111111
103 11101111111101101001000101001000111
104 11101101111101111001011111101001011
105 11011100110110100101011110101101110
106 11011110011101110011010110110110111
107 11010110101101100001010111111011111
108 11010111011101000011011001010100111
109 11101010011111111111011111111110111
110 11111100111111111101110000101110011
111 11111110100101111101011001101000000
112 11011100110101101010001100010110111
113 00011111111101010010011111111011011
114 11000110001111110011111110101111111
115 11101110111111111111100000100111111
116 01011111111101001100011110000010111
117 00001110100101010111011011101111000
118 10001100010010010000010000010000111
119 00000100000001010001000010000000000
120 11111111110111111101011011111111111
121 10100111010111000001011001101110011
122 11001100100111011111011010101100100
123 10101010011111000001010010101011111
124 11111111101110111111010100101110111
125 11111101111101011101011100101100111
126 10011110001111111000010100111110100
127 11111111101100110101011110111011011
128 11111110111111001111010110101011111
129 11111110010111010001110111101011010
130 00010110110011110101111000001000111
132 10111111010101111111110111111000011
133 11111101101111111111011000101100110
131 00111100011110011001110011100010101
134 11001100000111010000110000101100101
135 11010110001111010100000100100100011
136 11001111111100011011011000101010100
137 00011110011100110011110010101010011
138 11101110100001000000001010000010100
139 10111010101001010000001000100100100
140 11001110110101110111011000000010000
141 11101110001100010111110000110111110
143 01010110110001000000000010010010110
144 01000110000000010011010001001000100
145 11010100000101010101011001111001101
146 11001110111100000000010111101010111
147 01011100100110000000100000001100110
149 11111111111111101111110100100000101
150 11111111111111011111011111101111111
151 10000100000001000100100110000000010
152 00011101111101011011011010101010100
153 11111110010111111111010111001110111
154 11001000001001101111001100101010011
155 11011111111111111111111111101110111
156 01101110011100110101011001101100101
158 11001101111101110111100110101111111
PSY 5950 L13 - 1
Think for a moment about the complex process that begins with this 158x35 collection of 1s and 0s (5530 in all) and ends with the output, tables, graphs, and interpretations that will be created from them.
A Regular (dinosaur?) analysis of the data in SPSS.
data list fixed /id 1-3 q1 to q35 5-39.
begin data.
001 11111111110110101101011111111011111
002 11111111111111111111111111101111111
.
.
.
156 01101110011100110101011001101100101
158 11001101111101110111100110101111111
end data.
compute totalscore = sum (q1 to q35).<---- This is the usual person estimate.
frevar=totalscore /format=notable /histogram.
This is a classic “easy” test, with scores “piled up” near the top of the range of possible scores.
Note that distribution of scores such as this makes it easier for us to distinguish between different Low levels of person ability.
But it is not easy to distinguish between different high levels of the ability. Consider the top 3 persons. Who is best? We can’t tell – all of the same score.
Classic Reliability Estimation – person reliability
reliability variables = q1 to q35 /summary=total.
Reliability StatisticsCronbach's Alpha / N of Items
.876 / 35
Item-Total Statistics
Scale Mean if Item Deleted / Scale Variance if Item Deleted / Corrected Item-Total Correlation / Cronbach's Alpha if Item Deleted
q1 / 25.47 / 38.277 / .380 / .873
q2 / 25.47 / 38.345 / .355 / .873
q3 / 25.68 / 37.226 / .436 / .871
q4 / 25.56 / 37.765 / .398 / .872
q5 / 25.45 / 38.517 / .349 / .873
q6 / 25.37 / 39.496 / .207 / .875
q7 / 25.48 / 38.117 / .400 / .872
q8 / 25.70 / 36.775 / .509 / .870
q9 / 25.59 / 37.935 / .348 / .873
q10 / 25.53 / 37.539 / .467 / .871
q11 / 25.59 / 37.734 / .386 / .873
q12 / 25.39 / 38.294 / .558 / .871
q13 / 25.73 / 37.958 / .298 / .875
q14 / 25.47 / 38.949 / .213 / .876
q15 / 25.73 / 37.032 / .457 / .871
q16 / 25.52 / 38.520 / .273 / .875
q17 / 25.62 / 36.868 / .530 / .869
q18 / 25.55 / 37.349 / .487 / .870
q19 / 25.63 / 37.496 / .406 / .872
q20 / 25.46 / 38.116 / .429 / .872
q21 / 25.97 / 38.711 / .176 / .878
q22 / 25.44 / 38.423 / .385 / .873
q23 / 25.61 / 37.782 / .363 / .873
q24 / 25.59 / 37.129 / .498 / .870
q25 / 25.64 / 37.843 / .341 / .874
q26 / 25.69 / 36.847 / .501 / .870
q27 / 25.45 / 38.008 / .468 / .871
q28 / 25.85 / 37.661 / .338 / .874
q29 / 25.50 / 37.876 / .430 / .872
q30 / 25.74 / 38.113 / .269 / .876
q31 / 25.59 / 37.976 / .340 / .874
q32 / 25.75 / 36.898 / .474 / .871
q33 / 25.49 / 38.534 / .291 / .874
q34 / 25.51 / 38.037 / .387 / .873
q35 / 25.52 / 37.674 / .452 / .871
This what we typically get from SPSS when we analyze a test.
The test would be judged as being highly reliable. This means that if we gave the test twice to the same people, somehow obliterating their memory of how they responded on the first test, we would expect a .85 to .90 correlation between the two sets of test scores.
Note that classic reliability is what Bond and Fox call Person Reliability in Chapter 3 – Same people given two equivalent tests (or the same test twice).
Each person’s score on the 2nd administration would be pretty close to his/her score on the 1st administration.
Item information from SPSS Reliability output
Scale Mean if Item Deleted / Scale Variance if Item Deleted / Corrected Item-Total Correlation / Cronbach's Alpha if Item Deletedq1 / 25.47 / 38.277 / .380 / .873
q2 / 25.47 / 38.345 / .355 / .873
q3 / 25.68 / 37.226 / .436 / .871
q4 / 25.56 / 37.765 / .398 / .872
q5 / 25.45 / 38.517 / .349 / .873
q6 / 25.37 / 39.496 / .207 / .875
q7 / 25.48 / 38.117 / .400 / .872
q8 / 25.70 / 36.775 / .509 / .870
q9 / 25.59 / 37.935 / .348 / .873
q10 / 25.53 / 37.539 / .467 / .871
q11 / 25.59 / 37.734 / .386 / .873
q12 / 25.39 / 38.294 / .558 / .871
q13 / 25.73 / 37.958 / .298 / .875
q14 / 25.47 / 38.949 / .213 / .876
q15 / 25.73 / 37.032 / .457 / .871
q16 / 25.52 / 38.520 / .273 / .875
q17 / 25.62 / 36.868 / .530 / .869
q18 / 25.55 / 37.349 / .487 / .870
q19 / 25.63 / 37.496 / .406 / .872
q20 / 25.46 / 38.116 / .429 / .872
q21 / 25.97 / 38.711 / .176 / .878
q22 / 25.44 / 38.423 / .385 / .873
q23 / 25.61 / 37.782 / .363 / .873
q24 / 25.59 / 37.129 / .498 / .870
q25 / 25.64 / 37.843 / .341 / .874
q26 / 25.69 / 36.847 / .501 / .870
q27 / 25.45 / 38.008 / .468 / .871
q28 / 25.85 / 37.661 / .338 / .874
q29 / 25.50 / 37.876 / .430 / .872
q30 / 25.74 / 38.113 / .269 / .876
q31 / 25.59 / 37.976 / .340 / .874
q32 / 25.75 / 36.898 / .474 / .871
q33 / 25.49 / 38.534 / .291 / .874
q34 / 25.51 / 38.037 / .387 / .873
q35 / 25.52 / 37.674 / .452 / .871
The “Corrected Item-Total Correlation” gives us some information about the individual items.
The term correct is used to indicate that the item being considered is not included in the “Total”. That is, for each item, the total of all the other items is computed and that “corrected” total is correlated with the item’s score.
A larger correlation means that responses to that item correlate quite highly with the total score on the rest of the items. This means that the item is a “good” item from a psychometric point of view – it discriminates good performers from poor performers.
Note that there is no easy-to-use item difficulty information in this display.
There is also no information on person ability in this display, and no information on the relationship of person ability values to item difficulty values, and no information on consistency of person responses or on consistency of responses to items, or on precision of item or person estimates. A pretty useless display.
The Rasch analysis of the BLOT data
The Rasch control file from Bond & Fox 2nd Edition
&INST ; initial line (can be omitted)
TITLE = "Bond & Fox BLOT data: Chapter 4"
PERSON = Person ; persons are ...
ITEM = Item ; items are ...
ITEM1 = 5 ; column of response to first item in data record
NI = 35 ; number of items
NAME1 = 1 ; column of first character of person label
NAMELEN = 3 ; length of person identifying label
XWIDE = 1 ; number of columns per item response
CODES = 10 ; valid codes in data file
UIMEAN = 0 ; item mean for local origin
USCALE = 1 ; user scaling for logits (so they’re Z scores – M=0; S=1)
UDECIM = 2 ; reported decimal places for user scaling
TOTAL = Yes ; show total raw scores
CHART = Yes ; produce across-pathway picture
MNSQ = No ; use Standardized fit statistics
CONVERGE= L ; Convergence decided by logit change
LCONVERGE=.00001 ; Set logit convergence tight because of anchoring
IAFILE = * ; Item anchor file to preset the difficulty of an item
4 0 ;Item 4 exactly at 0 logit point.(freezing temp of H2O)
* ; End of anchor list
&END
01 Negation (to negate identity) ; Item labels courtesy of Trevor Bond
02 Reciprocal (to negate identity)
.
.
.
35 Coordination of two systems of reference
END NAMES
001 11111111110110101101011111111011111
002 11111111111111111111111111101111111
003 11010111111111011111011111101011111
.
.
.
.
.
The Chapter 4 control file from Bond and Fox 3rdEdition.
&INST
TITLE = "Bond & Fox BLOT data: Chapter 4"
PERSON = Person ; persons are ...
ITEM = Item ; items are ...
ITEM1 = 5 ; column of response to first item in data record
NI = 35 ; number of items
NAME1 = 1 ; column of first character of person label
NAMELEN = 3 ; length of person identifying label
XWIDE = 1 ; number of columns per item response
CODES = 10 ; valid codes in data file
UIMEAN = 0 ; item mean for local origin
USCALE = 1 ; user scaling for logits
UDECIM = 2 ; reported decimal places for user scaling
TOTAL = Yes ; show total raw scores
CHART = Yes ; produce across-pathway picture
MNSQ = No ; use Standardized fit statistics
&END
01 Negation
02 Reciproc
.
.
.
28 Non-impl
29 Aff of q
30 Equiv
31 Neg of q
32 NegRecipImpl
33 Probab
34 CoordinA
35 CoordinB
END LABELS
001 11111111110110101101011111111011111
002 11111111111111111111111111101111111
003 11010111111111011111011111101011111
004 11111111111111111111101111111111111
005 11111111111101111111011111111111111
006 11111111111110111101011111111111111
007 11111111111101111111011111111111111
008 11111111111111111111111111101011111
009 11111111111111111111111101111111111
010 11111111111111111111111111111001111
011 11111110111111111111111111111111111
012 11011111011111011111011111000110111
013 11111110111111111111011011111101111
014 11111110111111111111111111101001111
The Rasch Person Map – Person names/labels on the right; item difficulties on the left.
TABLE 16.3 Bond & Fox BLOT data: Chapter 4 ZOU210WS.TXT Feb 27 23:28 2014
INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0
------
Items MAP OF Persons
<rare>|<more>
4 + 002 004 009 011 019 023 024 025 027 034 035 046
059 083 084 092 101 102
|
|
|
|
|
| 005 007 008 010 016 017 031 082
|
3 +
S|
| 006 020 038 050 081 150 155
|
|
X | 013 014 015 018 021 037 040 043 047 054 055 056
064 065 066 074 088 094 120
|
| 028 029 030 033 042 077
2 +
|T 001 003 026 039 058 063 109
|
X M| 022 041 057 060 072
|
| 012 032 045 048 049 073 124 128 153
| 061 062 068 076 079 080 093 100 115 127 132 158
XX | 051 052 067 090 097 104 110 114 125 133 149
1 XX +S
X | 036 075 087 106 113
XX | 078 086 091 095 096 099 107 129
| 044 070 105
XX | 103 116 141
XX | 108 111 112 122 126 136 152 156
X S| 117 121 123 131 137 146
XXX | 130 145
0 XX +M
| 140 154
XXX | 135
X | 134
XX | 098
X | 139
XXX | 138 143 147
XX | 118
-1 X T+S
X | 144
|
| 151
|
|
|
X |T
-2 +
| 119
|
|
X |
|
|
|
-3 +
<frequ>|<less>
The Rasch Item Map – Item names/labels on the right
TABLE 12.2 Bond & Fox BLOT data: Chapter 4 ZOU032WS.TXT Mar 24 18:13 2012
INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0
------
Persons MAP OF Items
<more>|<rare>
4 ######### +
|
|
|
|
|
#### |
|
3 +
S|
.### |
|
|
.######### | 21 Correlative + negation > equilibrium
|
### |
2 +
.### |T
|
.## M| 28 Non-implication
|
.#### |
###### |
.##### | 30 Equivalence
32 Negation of reciprocal implication
1 +S 13 Reciprocal exclusion
15 Reciprocal implication
.## | 08 Correlations
#### | 03 Implication
26 Complete affirmation
.# |
.# | 19 Reciprocal (to cause disequilibrium)
25 Complete negation
#### | 17 Identity (to negate reciprocal)
23 Correlative + identity > disequilibrium
### S| 24 Coordination of two systems of reference
# | 09 Conjunction
11 Conjunctive negation
31 Negation of q
0 +M 04 Incompatibility
18 Negation (to negate correlative)
# |
. | 10 Disjunction
16 Reciprocal (to negate identitiy)
35 Coordination of two systems of reference
. | 34 Coordination of two systems of reference
. | 29 Affirmation of q
33 Probability
. | 07 Correlations
.# | 01 Negation (to negate identity)
02 Reciprocal (to negate identity)
14 Probability
. | 20 Negation (to cause disequilibrium)
27 Negation of p
-1 T+S 05 Multiplicative compensation
. | 22 Reciprocal + negation > disequilibrium
|
. |
|
|
|
|T 12 Affirmation of p
-2 +
. |
|
|
| 06Correlations
|
|
|
-3 +
<less>|<frequ
EACH '#' IS 2.
Item characteristics . . . Item STATISTICS ordered by Measure
TABLE 13.1 Bond & Fox BLOT data: Chapter 4 ZOU527WS.TXT Mar 25 11:43 2012
INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0
------
Person: REAL SEP.: 2.04 REL.: .81 ... Item: REAL SEP.: 3.79 REL.: .93
Item STATISTICS: MEASURE ORDER
+------+
|ENTRY TOTAL MODEL| INFIT | OUTFIT |PTMEA|EXACT MATCH| |
|NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR.| OBS% EXP%| Item |
|------+------+------+-----+------+------|
| 21 54 150 2.40 .20|1.27 2.6|1.75 3.7| .32| 69.4 74.8| 21 Correlative |
| 28 73 150 1.68 .19|1.12 1.4|1.23 1.7| .43| 70.1 71.9| 28 Non-implicat|
| 32 87 150 1.17 .19| .96 -.5| .85 -1.1| .53| 72.8 71.2| 32 Negation of |
| 30 89 150 1.10 .19|1.19 2.3|1.15 1.0| .38| 63.3 71.4| 30 Equivalence |
| 13 91 150 1.03 .19|1.16 2.0|1.32 2.0| .37| 66.7 71.7| 13 Reciprocal e|
| 15 91 150 1.03 .19| .97 -.4| .84 -1.1| .52| 73.5 71.7| 15 Reciprocal i|
| 8 95 150 .88 .19| .91 -1.1|1.00 .1| .52| 75.5 72.5| 08 Correlations|
| 26 97 150 .80 .20| .90 -1.3| .75 -1.6| .55| 72.8 72.9| 26 Complete aff|
| 3 98 150 .76 .20| .98 -.2| .90 -.5| .49| 73.5 73.1| 03 Implication |
| 25 104 150 .52 .20|1.07 .8|1.26 1.3| .40| 74.1 74.9| 25 Complete neg|
| 19 105 150 .48 .20|1.01 .2|1.05 .3| .44| 74.8 75.2| 19 Reciprocal (|
| 17 107 150 .40 .20| .87 -1.4| .75 -1.3| .54| 78.2 75.8| 17 Identity (to|
| 23 108 150 .36 .21|1.06 .7| .92 -.3| .42| 70.7 76.2| 23 Correlative |
| 24 111 150 .23 .21| .89 -1.1|1.03 .2| .49| 82.3 77.4| 24 Coordination|
| 9 112 150 .18 .21|1.07 .7| .97 .0| .40| 76.2 77.8| 09 Conjunction |
| 11 112 150 .18 .21|1.02 .3| .96 -.1| .42| 80.3 77.8| 11 Conjunctive |
| 31 112 150 .18 .21|1.07 .7|1.55 2.2| .36| 78.9 77.8| 31 Negation of |
| 4 116 150 .00A .22|1.00 .0| .88 -.4| .43| 80.3 79.8| 04 Incompatibil|
| 18 117 150 -.05 .22| .90 -.9| .74 -1.0| .49| 79.6 80.3| 18 Negation (to|
| 10 120 150 -.20 .23| .92 -.7| .68 -1.2| .47| 84.4 81.8| 10 Disjunction |
| 16 122 150 -.31 .23|1.13 1.0|1.03 .2| .33| 79.6 82.9| 16 Reciprocal (|
| 35 122 150 -.31 .23| .93 -.5| .73 -.9| .45| 83.7 82.9| 35 Coordination|
| 34 124 150 -.42 .24|1.00 .1| .79 -.6| .41| 81.6 83.9| 34 Coordination|
| 29 125 150 -.48 .24| .94 -.4| .71 -.9| .43| 86.4 84.5| 29 Affirmation |
| 33 126 150 -.53 .25|1.10 .7| .93 -.1| .33| 81.0 85.0| 33 Probability |
| 7 128 150 -.66 .25| .97 -.1| .65 -1.0| .41| 85.7 86.0| 07 Correlations|
| 2 129 150 -.72 .26|1.01 .1| .75 -.6| .37| 85.0 86.6| 02 Reciprocal (|
| 14 129 150 -.72 .26|1.15 1.0|1.32 .9| .25| 85.0 86.6| 14 Probability |
| 1 130 150 -.79 .26| .98 .0| .69 -.8| .39| 86.4 87.1| 01 Negation (to|
| 20 131 150 -.86 .27| .91 -.5| .81 -.4| .40| 87.1 87.7| 20 Negation (to|
| 27 132 150 -.94 .27| .85 -.8| .62 -.9| .43| 89.8 88.3| 27 Negation of |
| 5 133 150 -1.01 .28| .98 -.1| .76 -.5| .35| 90.5 88.9| 05 Multiplicati|
| 22 134 150 -1.09 .29| .91 -.4|1.69 1.4| .35| 90.5 89.5| 22 Reciprocal +|
| 12 141 150 -1.81 .36| .69 -1.1| .24 -1.5| .46| 94.6 94.0| 12 Affirmation |
| 6 145 150 -2.49 .47|1.06 .3| .83 .0| .20| 96.6 96.6| 06 Correlations|
|------+------+------+-----+------+------|
| MEAN 109.9 147.0 .00 .24|1.00 .1| .95 -.1| | 80.0 80.5| |
| S.D. 19.5 .0 .97 .05| .11 1.0| .31 1.2| | 7.8 6.9| |
+------+
The key quantities in the table are
1. Measure – the difficulty of the item.
2. S.E. – the standard error of the estimate of the item’s difficulty. Note that the SEs of the more difficult items are smaller than those of the easy items. This is because there were many persons with ability values close to (and above) the difficulty of the difficult items at the top of the axis but few persons at the bottom of the axis. It’s a sample size problem – smaller local (around an item) sample sizes lead to larger standard errors – less precise estimates of the actual item difficulty.
3. Infit – A measure of the extent to which the item was consistently gotten correct by high ability persons and incorrect by low ability persons. According to text, Rasch analysts give this more weight, p. 57, p. 239-240. Z = 0 is best; > 0 is too inconsistent; <0 is suspicious because it’s too good.
4. Outfit – the mean of squared residuals with all residuals weighted equally.
5. Note that Item 4 was anchored at difficulty = 0 for the 2nd Edition. That was dropped for the 3rd Ed.
Item Characteristics . . . “Items FITgraph” ordered by Measure
This table presents the same information as above, but graphically.
Output Tables -> 13. Item: measure -> Scroll down to Table 13.2.
TABLE 13.2 Bond & Fox BLOT data: Chapter 4 ZOU527WS.TXT Mar 25 11:43 2012
INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0
------
Items FIT GRAPH: MEASURE ORDER
+------+
| ENTRY | MEASURE | INFIT STANDARDIZED | OUTFIT STANDARDIZED | |
| NUMBER| - + |-3 -2 -1 0 1 2 3 |-3 -2 -1 0 1 2 3 | Items |
|------+------+------+------+------|
| 21| *| : . :* | : . : *| 21 Correlative + negation > equilibrium |
| 28| * | : . * : | : . *: | 28 Non-implication |
| 32| * | : *. : | : * . : | 32 Negation of reciprocal implication |
| 30| * | : . * | : . * : | 30 Equivalence |
| 13| * | : . *: | : . * | 13 Reciprocal exclusion |
| 15| * | : *. : | : * . : | 15 Reciprocal implication |
| 8| * | : * . : | : * : | 08 Correlations |
| 26| * | : * . : | : * . : | 26 Complete affirmation |
| 3| * | : * : | : *. : | 03 Implication |
| 25| * | : . * : | : . * : | 25 Complete negation |
| 19| * | : * : | : * : | 19 Reciprocal (to cause disequilibrium) |
| 17| * | : * . : | : * . : | 17 Identity (to negate reciprocal) |
| 23| * | : .* : | : * : | 23 Correlative + identity > disequilibrium |
| 24| * | : * . : | : * : | 24 Coordination of two systems of reference|
| 9| * | : .* : | : * : | 09 Conjunction |
| 11| * | : * : | : * : | 11 Conjunctive negation |
| 31| * | : . * : | : . * | 31 Negation of q |
| 4| A | : * : | : *. : | 04 Incompatibility |
| 18| * | : * . : | : * . : | 18 Negation (to negate correlative) |
| 10| * | : *. : | : * . : | 10 Disjunction |
| 16| * | : . * : | : * : | 16 Reciprocal (to negate identitiy) |
| 35| * | : *. : | : * . : | 35 Coordination of two systems of reference|
| 34| * | : * : | : *. : | 34 Coordination of two systems of reference|
| 29| * | : *. : | : * . : | 29 Affirmation of q |
| 33| * | : . * : | : * : | 33 Probability |
| 7| * | : * : | : * . : | 07 Correlations |
| 2| * | : * : | : *. : | 02 Reciprocal (to negate identity) |
| 14| * | : . * : | : . * : | 14 Probability |
| 1| * | : * : | : * . : | 01 Negation (to negate identity) |
| 20| * | : *. : | : *. : | 20 Negation (to cause disequilibrium) |
| 27| * | : * . : | : * . : | 27 Negation of p |
| 5| * | : * : | : *. : | 05 Multiplicative compensation |
| 22| * | : *. : | : . * : | 22 Reciprocal + negation > disequilibrium |
| 12| * | : * . : | : * . : | 12 Affirmation of p |
| 6|* | : * : | : * : | 06 Correlations |
^ ^| |
Least Most
Difficult Difficult
(I red’d the items whose infit values were in the gray area.)
Infit and Outfit are represented graphically by horizontal position of the asterisks representing the item. Test makers prefer the asterisks to be between -2 and +2.
Note that Item 4 is “anchored” at Measure = 0. Since the measures are interval scaled, any zero point is appropriate.(Compare with temperature.) The program automatically picks the item whose difficulty is closest to the median and assigns it the difficulty value, 0. This is a reminder that the measures are interval scaled, so the zero point is arbitrary.
Item Characteristics – the bubble plot . . .
This plot show the same information as above, but it’s more colorful.
Plots -> Bubble Chart -> Items (columns in data) -> Entry number
For this plot, I chose Items only and used Entry Numbers to identify items.
Note that 3 items have been identified that are inconsistently responded to. Alas, they also happen to be the most difficult items. Perhaps guessing played a role here.
Fit is measured by using the equation shown in last week’s lecture to compute the probability of each person getting each item correct.
e(Bn-Di)
P(x=1|B,D) = ------.
1 + e(Bn-Di)
That probability is subtracted from 1 if the person got it correct or from 0 if the person missed it. The resulting difference is a residual, analogous to Y – Y-hat.
If the model is fitting the data generally OK, then the probabilities from the above equation with person and item values substituted for Bn and Dishould be close to 1 for those person/item combinations in which the response was correct and should be close to 0 for those person/item combinations in which the response was incorrect.
Thus the sum of the squared residuals for all persons responding to a given item should be small. If the sum of squared residuals for an item is large, then that means that the item is probably screwy in some way. If the sum for a person is large, then that person may be too inconsistent.
A comparison of 3 ways of scoring the BLOT – Summated score, log odds, and Rasch.
To make the comparison, I had to put the Rasch Person measures into an SPSS file.
Within the Steps program analyzing the BLOT data: Output Tables -> 18. Person: entry.
TABLE 18.1 Bond & Fox BLOT data: Chapter 4 ZOU518WS.TXT Mar 9 10:45 2016
INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0
------
Person: REAL SEP.: 2.04 REL.: .81 ... Item: REAL SEP.: 3.79 REL.: .93
Person STATISTICS: ENTRY ORDER
+------+
|ENTRY TOTAL MODEL| INFIT | OUTFIT |PTMEA|EXACT MATCH| |
|NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR.| OBS% EXP%| Person|
|------+------+------+-----+------+------|
| 1 29 35 1.84 .48|1.06 .3|1.13 .4| .25| 85.7 83.6| 001 |
| 2 34 35 3.95 1.03| .94 .2| .30 -.2| .30| 97.1 97.1| 002 |
| 3 29 35 1.84 .48| .81 -.6| .90 .0| .47| 85.7 83.6| 003 |
| 4 34 35 3.95 1.03|1.12 .4|4.48 1.9| -.19| 97.1 97.1| 004 |
| 5 33 35 3.19 .75| .79 -.2| .36 -.5| .44| 94.3 94.3| 005 |
| 6 32 35 2.72 .63| .95 .0|1.29 .6| .23| 91.4 91.4| 006 |
| 7 33 35 3.19 .75| .79 -.2| .36 -.5| .44| 94.3 94.3| 007 |
| 8 33 35 3.19 .75| .93 .1| .42 -.4| .36| 94.3 94.3| 008 |
| 9 34 35 3.95 1.03|1.07 .4| .91 .4| .09| 97.1 97.1| 009 |
| 10 33 35 3.19 .75|1.08 .3| .87 .2| .16| 94.3 94.3| 010 |
| 11 34 35 3.95 1.03|1.04 .4| .65 .2| .16| 97.1 97.1| 011 |
| 12 27 35 1.43 .43| .91 -.3| .97 .0| .41| 82.9 78.8| 012 |
| 13 31 35 2.37 .56| .94 .0| .76 -.2| .34| 91.4 88.6| 013 |
| 14 31 35 2.37 .56| .98 .1| .66 -.4| .36| 85.7 88.6| 014 |
I copied thetable and pasted it into Word.
In Word I then Alt+Selected the measure column and pasted it into a column in SPSS.
In SPSS I created a “rough” Rasch score for each person, logoddsscore, using the following syntax. (Recall it’s ln(score/(total possible - score)).
compute logoddsscore = ln(totalscore/(35-totalscore)).
Note that if (totalscore/(35-totalscore) = 0 or ∞, the cell that was to receive the result will remain blank. Here’s an excerpt from the SPSS file . . .
Here are the correlations of totalscore, logoddsscore, and raschscore
Note that the sample size in all correlations involving logoddsscoreis 147. This is because there were 3 persons who got either all items correct or all items incorrect. Logoddsscore was not available for those persons.
Here are scatterplots of the relationships ..(They’ll be presented again on a following page.)
Note that the logoddsscore and the raschscore are virtually identical.
Note also that the relationships of logoddsscore and Rasch to total score are linear in the middle, but the Rasch measures “stretch” in the tails, especially the upper tail.
Of course, this tells us that for some datasets we don’t need the program to compute person measures. For those datasets we can simply compute the ln(score/(total possible-score)) and use it.
The disadvantage of this is that we don’t get the other cool stuff that the BF program gives us.
One advantage of using the program is that it will give us an estimate of the Rasch value for persons whose total score is perfect or 0, something the log method cannot do.
Dot plots of the three measures . . .