The basic assumption of a multinomial model like the one shown in Figure 1 below is that the observed response patterns can be seen as the final product of a number of different cognitive processes, each of which occurs with a certain probability. In tasks modelled by multinomial models such as the "Who said what?" task (Klauer & Wegener, 1998) participants have a number of separate response options. The combination of cognitive processes determines how often each response option is chosen: in mathematical terms, the relative frequency of a response option is modelled by the product of the probabilities of the involved cognitive processes. The unknown probabilities of the different cognitive processes can therefore be estimated on the basis of these response frequencies as explained below. The present introduction to multinomial modeling is based on the pioneering work of Riefer and Batchelder (1988; Batchelder & Riefer, 1990). A comprehensive review of technical issues and applications is given by Batchelder and Riefer (in press).
The model in Figure 1 is based on Taylor, Fiske, Etcoff, and
``Who said what?'' paradigm. In that paradigm,
perceivers observe the discussion of members of two
categories, for example of men and women.
In a recognition test, they are again shown
the discussion statements and asked to assign each statement
to its speaker. Participants' assignment errors can be
classified into two kinds. Within-category errors occur
if a statement is wrongly assigned to a person who is
a member of the same category as the speaker. Between-categories
errors arise if a statement is assigned to a person from
the wrong category. If there are more within-category errors
than between-categories errors (after a suitable chance
correction), this is often interpreted as evidence for
category memory, and the size of the difference of the two
kinds of error as a measure of category salience in perceiving
and interpreting the group discussion.
Klauer and Wegener (1998) have proposed a small modification of the paradigm that allows one to disentangle these different processes by means of a mathematical model. The modification involves only the assignment phase of the paradigm. Apart from the statements that occur in the discussion, henceforth called targets or old statements, new statements or distractors are presented in the assignment phase. For each statement, the participant is first asked whether or not the statement occurred in the discussion. If the participant judges the statement old, he or she is required to assign the statement to a speaker in a second step. If the statement is judged new, an assignment to a speaker is not required.
Figure 1 shows the multinomial model proposed by Klauer and Wegener (1998) for the modified ``Who said what?'' paradigm. Multinomial models can be characterized as discrete-state models. They are discrete models in the sense that they postulate only a finite number of processing states, represented as the nodes of the processing trees depicted in Figure 1. A comprehensive review of the theory and applications of multinomial modeling is given by Batchelder and Riefer (in press). The present model describes participants' responses by means of the processes of item discrimination, person discrimination, and category discrimination as well as three guessing processes.
The model assumptions are represented by means of three processing trees, one for each kind of statement, that is, for statements that were made by a speaker from Category A, for those by a speaker from Category B, and for new statements. In Figure 1, the item categories are shown on the left side and the response categories in rectangles on the right side. In between, the mediating latent processes are depicted within ellipses. As can be seen, some of the response categories appear repeatedly because different processes can result in the same observable response.
Consider first the tree for A-items. The first branching of that tree models the process of item discrimination. With the probability the participant detects a presented A-item as old, whereas the item is not detected with complementary probability . If the item is detected, the process of person discrimination is considered next in the processing-tree representation. With probability the retrieved information about statement and speaker is sufficient to identify the speaker. In this case, the participant responds with the correct assignment. With probability , however, the information does not suffice to identify the speaker. If the speaker's social category has been encoded along with the statement, the participant may then still remember that category. The probability for successful category discrimination is modelled by the parameter . If the category is recalled, but not the individual speaker, then the model assumes that one of the persons of category A is guessed to be the speaker. With a fixed probability , guessing results in a correct assignment, whereas with complementary probability , a within-category error is made.
With probability , category discrimination does not succeed. In this case, the participant can still guess the speaker's category. The probability of guessing Category A rather than B is given by parameter a. Note that this bias parameter ensures that response biases favoring one of the categories over the other can be accomodated by the model. If Category A is guessed, a second guess picks the speaker with probability and a wrong person within that category with probability as above. If instead Category B is guessed, the participant necessarily responds with a between-categories error.
Consider now the lower half of the tree for A-items. It describes the case that item discrimination fails (probability ). The model assumes that the participant then guesses whether the item is old or new. The probability of guessing old is given by the bias parameter b. Again, b need not equal .5, so that the model takes into account that participants may be biased toward one of the response alternatives "old" or "new".
If the participant guesses that the statement is old, the same processes as described above for the case of failing person discrimination and category discrimination are assumed to occur. That is, both category and a person therein are guessed. If the participant guesses on the other hand that the statement is new, he or she falsely responds ``new'' with probability 1-b.
The processing tree for statements made by a speaker from Category B is analogous. Different discrimination parameters, indexed by the category, are assumed, however, so that the strenghts of the processes may differ as a function of category.
The tree for distractors is simpler. With probability the item is correctly discriminated as new. Such discrimination can be based, for example, on so-called autonoetic processes (Strack & Bless, 1994). Autonoetic processes are inferences based on peculiarities of distractors that allow the participant to reason that he or she would surely have recognized this particular statement if it had been presented at all. In this case, the participant responds "new". Otherwise, with probability , the same subtree of processes follows as in the other trees after failed item discrimination with the only difference that there can be no correct assignment for distractors.
On the basis of the observed assignment frequencies, the parameters can be estimated by means of the maximum-likelihood method; the model fit can be assessed by comparing observed and expected frequencies, and hypotheses about parameter values can be tested as explained in Klauer and Wegener (1998).
The following, in part conditional processes and corresponding parameters, are considered:
The different combinations of processes resulting in the responses are visualized in a multinomial processing-tree representation like the one in Figure 1. On the left side there are three starting points in this model representing the three situations a participant may be confronted with in each trial of the assignment task: the presented statement may be a statement made by a member of Category A, a statement made by a member of Category B, or a new statement. The combination of cognitive processes involved may be different for each of the three situations. Hence the model comprises three different trees.
On the right side all possible response options are depicted in Figure 1. Note that some of the response options occur several times in each tree, and thus there can be different combinations of cognitive processes resulting in the same response. In contrast to the observable starting points and the observable responses, the mediating cognitive processes are not observable. This fact is visualized by depicting the observable events in rectangles and the latent processes in ellipses.
Every cognitive process takes place with a certain probability, and these probabilities are represented by the parameters attached to the connecting lines between the ellipses. In the present model there are always two alternatives at each branching: either a cognitive process takes place with a certain probability, or it does not take place with the complementary probability. Thus the present model is called a binary model, but other models may sometimes have more than two options branching out from a given node.
Because the branches of each partial tree represent a combination of cognitive processes and since each cognitive process takes place with a certain probability, the probability for each branch is the product of the probabilities of all processes that constitute the branch. Consequently, the response option a branch ends in will be chosen with the joint probability of all involved cognitive processes. Because some branches end in the same response option, that option will ultimately be chosen with probability equalling the sum of several joint probabilities.
Note that not only response options can appear repeatedly in the same tree, but also cognitive processes. Consider, for example, the process of guessing the category that is associated with parameter a. As can be seen, the process occurs twice in the first tree in Figure 1.
Sometimes it is assumed that the same cognitive process even occurs in different trees of a model, as is the case for the guessing process associated with parameter a, among others, in the present model. Thus we assume that the process of guessing that a statement was made by a speaker from Category A is the same regardless of whether the statement was actually made by a speaker from Category A, was made by a speaker from Category B, or was not presented in the discussion phase at all. Parameter a therefore appears in all partial trees of the present model.
For the statistical analysis of a multinomial model the data are aggregated over all participants of a given experimental condition. For every partial tree and response option, the frequency with which the response option was chosen by the participants is obtained.
As has been said, the
probability with which a response option is chosen can be
expressed as the product of the probabilities
of the involved cognitive processes or as the sum
of joint probabilities of this kind
if more than one branch leads to the same response option.
Hence the whole multinomial model can
be described as a system of equations: for every
response option of each partial tree one equation is obtained.
In the present multinomial model, for example, the
Because the frequencies of the response options can be observed and the same parameters appear in several independent equations, the resulting system of equations can be solved for the unknown parameters under certain conditions, and the probabilities of the involved cognitive processes can thereby be estimated. The job of estimating the probabilities is performed by a computer program by means of the maximum likelihood method. In this iteration process, several combinations of probabilities are processed until the best solution is found, i.e. until the difference between the observed frequencies and those that would be expected on the basis of the estimated parameters is minimized, where the difference is measured in terms of the ratio of the likelihoods of the observed and the expected frequencies.
In applying a model, a global goodness-of-fit test of the model should be conducted. To answer the question of whether the observed response pattern can be explained by the model, a goodness-of-fit test evaluates the differences between the observed and the estimated response frequencies. To test the goodness of fit, the likelihood ratio statistic G2 is computed, which is asymptotically distributed. If this test does not yield a significant result, given a satisfactory statistical test power, it can be concluded that there are no substantial deviations from the model. In this case, the model fits the data. Note that for the goodness-of-fit test the degrees of freedom have to be computed. As a rule of thumb, they are given by the number of independent response options minus the number of parameters to be estimated. See the body of the paper for examples.
Special hypotheses can be tested by restricting the model. Usually, a hypothesis assumes that a certain treatment affects a certain cognitive process and hence will cause the probability of this process to change. Fitting the model to the experimental condition should thus yield an estimated probability of this process which differs from the one obtained by fitting the model to the control condition. To test this hypothesis statistically, the model is doubled and treated as one joint model for both experimental and control condition. The parameters of the resulting model are indexed by condition, thereby allowing for different probabilities in the two situations, especially allowing for different probabilities of the cognitive process that is supposed to have been manipulated. Then the change in these latter probabilities can be tested statistically by using a goodness-of-fit test again: If there is indeed a change, it should be impossible to restrict the model to use only one parameter for both probabilities, thereby forcing them to be equal. Thus, if the restriction leads to a significant loss of goodness of fit, the treatment can be concluded to have affected the cognitive process in question. The loss of goodness of fit is the difference of the goodness of fit of the restricted model minus that of the unrestricted model. The degrees of freedom for testing the significance of this distributed loss value are given by the difference in the degrees of freedom of the two models.
Before a new model can be used to examine the cognitive processes involved in a given task, its substantive interpretation should be validated properly. The global goodness-of-fit test examines whether a certain probability structure can explain the frequencies with which the different response patterns were observed. Since many different models with different substantive implications can usually be fitted to the same set of data, the goodness-of-fit test does not imply that the parameters really measure the intended cognitive processes. Hence, a series of experiments should aim at validating the different process parameters.
Computer Software for estimating parameters and testing the models is available via Internet (http://xhuoffice.psyc.memphis.edu/gpt/index.htm) from Xiangen Hu. There are two programs available: an older DOS version called MBT (Hu, 1991) and a Windows95/NT version called GPT (Hu, 1997).
For working with MBT, an ASCII document containing the system of equations of the model and an ASCII document containing the aggregated data are needed. After reading the two files, the program asks for restrictions of the model. For example, parameters can be constrained to be equal, or they can be fixed at a user-determined value. Then the parameters are estimated and an ASCII document is created containing parameter estimates, their 90%-confidence intervals, the value of goodness of fit, the estimated and the empirical frequency distribution as well as the so-called Fisher information matrix from which the confidence intervals are computed (Rao, 1973). In addition, the program checks the identifiability of the model, i.e. whether the system of equations can be solved uniquely. For this purpose, several parameter estimations are conducted from different starting values of the parameters, and it has to be checked whether the final estimates for each parameter are always the same.
GPT can work with ASCII documents containing equation systems and aggregated data or, alternatively, the processing-tree representation of the multinomial model can be drawn and the data has to be entered via the keyboard. Additional options like analysis of power and model simulations are integrated. Whereas MBT can handle models with only up to 61 branches, GPT is able to deal with very large models.
Figure 1. The two-high-threshold multinomial model of social categorization in the modified ``Who said what?'' paradigm. DA=probability of detecting a statement made by a speaker from Category A, DB=probability of detecting a statement made by a speaker from Category B, DN=probability of detecting that a distractor is new; dA=probability of correctly discriminating the category of a statement made by a speaker from Category A, dB=probability of correctly discriminating the category of a statement made by a speaker from Category B; a=probability of guessing that a statement is made by a speaker from Category A; b=probability of guessing that a statement is old.