Estimation methods: JMLE, PROX

Winsteps implements three methods of estimating Rasch parameters from ordered qualitative observations: JMLE and PROX. Estimates of the Rasch measures are obtained by iterating through the data. Initially all unanchored parameter estimates (measures) are set to zero. Then the PROX method is employed to obtain rough estimates. Each iteration through the data improves the PROX estimates until they are usefully good. Then those PROX estimates are the initial estimates for JMLE which fine-tunes them, again by iterating through the data, in order to obtain the final JMLE estimates. The iterative process ceases when the convergence criteria are met. These are set by MJMLE=, CONVERGE=, LCONV= and RCONV=. Depending on the data design, this process can take hundreds of iterations (Convergence: Statistics or Substance?). When only rough estimates are needed, force convergence by pressing Ctrl+F or by selecting "Finish iterating" on the File pull-down menu.

Extreme scores: (perfect, maximum possible scores, and zero, minimum possible scores) are dropped from the main estimation procedure. Their measures are estimated separately using EXTRSC=.

Missing data: most Rasch estimation methods do not require that missing data be imputed, or that there be case-wise or list-wise omission of data records with missing data. For datasets that accord with the Rasch model, missing data lower the precision of the measures and lessen the sensitivity of the fit statistics, but do not bias the measure estimates.

Likelihood: Using the current parameter estimates (Rasch measures), the probability of observing each data point is computed, assuming the data fit the model. The probabilities of all the data points are multiplied together to obtain the likelihood of the entire data set. The parameter estimates are then improved (in accordance with the estimation method) and a new likelihood for the data is obtained. The values of the parameters for which the likelihood of the data has its maximum are the "maximum likelihood estimates" (Ronald A. Fisher, 1922).

JMLE "Joint Maximum Likelihood Estimation" is also called UCON, "Unconditional maximum likelihood estimation". It was devised by Wright & Panchapakesan, www.rasch.org/memo46.htm. In this formulation, the estimate of the Rasch parameter (for which the observed data are most likely, assuming those data fit the Rasch model) occurs when the observed raw score for the parameter matches the expected raw score. "Joint" means that the estimates for the persons (rows) and items (columns) and rating scale structures (if any) of the data matrix are obtained simultaneously. The iterative estimation process is described at Iteration.

Advantages - these are implementation dependent, and are implemented in Winsteps:

(1) independence from specific person and item distributional forms.

(2) flexibility with missing data

(3) the ability to analyze test lengths and sample sizes of any size

(4) symmetrical analysis of person and item parameters so that transposing rows and columns does not change the estimates

(5) flexibility with person, item and rating scale structure anchor values

(6) flexibility to include different variants of the Rasch model in the same analysis (dichotomous, rating scale, partial credit, etc.)

(7) unobserved intermediate categories of rating scales can be maintained in the estimation with exact probabilities.

(8) all non-extreme score estimable (after elimination of extreme scores and rarely-observed Guttman subsets)

(9) all persons with the same total raw scores on the same items have the same measures; all items with the same raw scores across the same persons have the same measures.

Disadvantages:

(11) measures for extreme (zero, perfect) scores for persons or items require post-hoc estimation.

(12) estimates are statistically inconsistent

(13) estimation bias, particularly with small samples or short tests, inflates the logit distance between estimates.

(14) chi-squares reported for fit tests (particularly global fit tests) may be somewhat inflated, exaggerating misfit to the Rasch model.

Comment on (8): An on-going debate is whether measures should be adjusted up or down based on the misfit in response patterns. With conventional test scoring and Rasch JMLE, a lucky guess counts as a correct answer exactly like any other correct answer. Unexpected responses can be identified by fit statistics. With the three-parameter-logistic item-response-theory (3-PL IRT) model, the score value of an unexpected correct answer is diminished whether it is a lucky guess or due to special knowledge. In Winsteps, responses to off-target items (the locations of lucky guesses and careless mistakes) can be trimmed with CUTLO= and CUTHI=, or be diminished using TARGET=Yes.

Comment on (13): JMLE exhibits some estimation bias in small data sets, but this rarely exceeds the precision (model standard error of measurement, SEM) of the measures. Estimation bias is only of concern when exact probabilistic inferences are to be made from short tests or small samples. It can be exactly corrected for paired-comparison data with PAIRED=Yes. For other data, It can be approximately corrected with STBIAS=Yes, but, in practice, this is not necessary (and sometimes not advisable).

PROX is the Normal Approximation Algorithm devised of Cohen (1979). This algorithm capitalizes on the similar shapes of the logistic and normal ogives. It models both the persons and the items to be normally distributed. The variant of PROX implemented in Winsteps allows missing data. The form of the estimation equations is:

Ability of person = Mean difficulty of items encountered +

log ( (observed score - minimum possible score on items encountered) /

(maximum possible score on items encountered - observed score) )

* square-root ( 1 + (variance of difficulty of items encountered) / 2.9 )

In Winsteps, PROX iterations cease when the variance of the items encountered does not increase substantially from one iteration to the next.

Advantages - these are implementation dependent, and are implemented in Winsteps:

(2)-(9) of JMLE

Computationally the fastest estimation method.

Disadvantages

(1) Person and item measures assumed to be normally distributed.

(11)-(14) of JMLE

Other estimation methods in common use (but not implemented in Winsteps):

Gaussian least-squares finds the Rasch parameter values which minimize the overall difference between the observations and their expectations, Sum((Xni - Eni)²) where the sum is overall all observations, Xni is the observation when person encounters item i, and Eni is the expected value of the observation according to the current Rasch parameter estimates. For Effectively, off-target observations are down-weighted, similar to TARGET=Yes in Winsteps.

Minimum chi-square finds the Rasch parameter values which minimize the overall statistical misfit of the data to the model, Sum((Xni - Eni)² / Vni) where Vni is the modeled binomial or multinomial variance of the observation around its expectation. Effectively off-target observations are up-weighted to make them less improbable.

Gaussian least-squares and Minimum chi-square:

Advantages - these are implementation dependent:

(1)-(8) All those of JMLE.

Disadvantages:

(9) persons with the same total raw scores on the same items generally have different measures; items with the same raw scores across the same persons generally have different measures.

(11)-(13) of JMLE

(14) global fit tests uncertain.

CMLE. Conditional maximum likelihood estimation. Item difficulties are structural parameters. Person abilities are incidental parameters, conditioned out for item difficulty estimation by means of their raw scores. The item difficulty estimates are those that maximize the likelihood of the data given the person raw scores and assuming the data fit the model. The item difficulties are then used for person ability estimation using a JMLE approach.

Advantages - these are implementation dependent:

(1), (6)-(9) of JMLE

(3) the ability to analyze person sample sizes of any size

(5) flexibility with item and rating scale structure anchor values

(12) statistically-consistent item estimates

(13) minimally estimation-biased item estimates

(14) exact global fit statistics

Disadvantages:

(2) limited flexibility with missing data

(3) test length severely limited by mathematical precision of the computer

(4) asymmetric analysis of person and item parameters so that transposing rows and columns changes the estimates

(5) no person anchor values

(11) of JMLE

(13) estimation-biased of person estimates small but uncertain

EAP. Expected A Posteriori estimation derives from Bayesian statistical principles. This requires assumptions about the expected parameter distribution. An assumption is usually normality, so EAP estimates are usually more normally distributed than Winsteps estimates (which are as parameter-distribution-free as possible). EAP is not implemented in Winsteps.

MMLE. Marginal maximum likelihood estimation. Item difficulties are structural parameters. Person abilities are incidental parameters, integrated out for item difficulty estimation by imputing a person measure distribution. The item difficulties are then used for person ability estimation using a JMLE approach.

Advantages - these are implementation dependent:

(3), (6)-(9) of JMLE

(1) independence from specific item distributional forms.

(2) flexibility with missing data extends to minimal length person response strings

(5) flexibility with item and rating scale structure anchor values

(11) extreme (zero, perfect) scores for persons are used for item estimation.

(12) statistically-consistent item estimates

(13) minimally estimation-biased item estimates

(14) exact global fit statistics

Disadvantages:

(1) specific person distribution required

(4) asymmetric analysis of person and item parameters so that transposing rows and columns changes the estimates

(5) no person anchor values

(11) measures for extreme (zero, perfect) scores for specific persons or items require post-hoc estimation.

(13) estimation-biased of person estimates small but uncertain

PMLE. Pairwise maximum likelihood estimation. Person abilities are incidental parameters, conditioned out for item difficulty estimation by means of pairing equivalent person observations. The item difficulties are then used for person ability estimation using a JMLE approach.

Advantages - these are implementation dependent:

(1), (3), (6), (7) of JMLE

(5) flexibility with item and rating scale structure anchor values

(8) all persons with the same total raw scores on the same items have the same measure

(12) statistically-consistent item estimates

Disadvantages:

(11) of JMLE

(2) reduced flexibility with missing data

(4) asymmetric analysis of person and item parameters so that transposing rows and columns changes the estimates

(5) no person anchor values

(8) items with the same total raw scores across the same persons generally have different measures.

(13) estimation-biased or item and person estimates small but uncertain

(14) global fit tests uncertain.

(15) uneven use of data in estimation renders standard errors and estimates less secure

WMLE. WMLE (or WLE) estimates are usually slightly more central than Winsteps estimates.Warm's (1989) Weighted Maximum Likelihood Estimation. Standard MLE estimates are the maximum values of the likelihood function and so statistical modes. Warm shows that the likelihood function is skewed, leading to an additional source of estimation bias. The mean likelihood estimate is less biased. Warm suggests an unbiasing correction that can be applied, in principle, to any MLE method, but there are computational constraints. Even when feasible, this fine tuning appears to be less than the relevant standard errors and have no practical benefit. It is not currently implemented in Winsteps.

Cohen Leslie. (1979) Approximate Expressions for Parameter Estimates in the Rasch Model, The British Journal of Mathematical and Statistical Psychology, 32, 113-120

Fisher R.A. On the mathematical foundations of theoretical statistics. Proc. Roy. Soc. 1922 Vol. CCXXII p. 309-368

Warm T.A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450