The basic assumption of a multinomial model like the one shown in Figure 1 below is that the observed response patterns can be seen as the final product of a number of different cognitive processes, each of which occurs with a certain probability. In tasks modelled by multinomial models such as the "Who said what?" task (Klauer & Wegener, 1998) participants have a number of separate response options. The combination of cognitive processes determines how often each response option is chosen: in mathematical terms, the relative frequency of a response option is modelled by the product of the probabilities of the involved cognitive processes. The unknown probabilities of the different cognitive processes can therefore be estimated on the basis of these response frequencies as explained below. The present introduction to multinomial modeling is based on the pioneering work of Riefer and Batchelder (1988; Batchelder & Riefer, 1990). A comprehensive review of technical issues and applications is given by Batchelder and Riefer (in press).
The model in Figure 1 is based on Taylor, Fiske, Etcoff, and
Ruderman's (1978)
``Who said what?'' paradigm. In that paradigm,
perceivers observe the discussion of members of two
categories, for example of men and women.
In a recognition test, they are again shown
the discussion statements and asked to assign each statement
to its speaker. Participants' assignment errors can be
classified into two kinds. Within-category errors occur
if a statement is wrongly assigned to a person who is
a member of the same category as the speaker. Between-categories
errors arise if a statement is assigned to a person from
the wrong category. If there are more within-category errors
than between-categories errors (after a suitable chance
correction), this is often interpreted as evidence for
category memory, and the size of the difference of the two
kinds of error as a measure of category salience in perceiving
and interpreting the group discussion.
Klauer and Wegener (1998) have proposed a small modification of the paradigm that allows one to disentangle these different processes by means of a mathematical model. The modification involves only the assignment phase of the paradigm. Apart from the statements that occur in the discussion, henceforth called targets or old statements, new statements or distractors are presented in the assignment phase. For each statement, the participant is first asked whether or not the statement occurred in the discussion. If the participant judges the statement old, he or she is required to assign the statement to a speaker in a second step. If the statement is judged new, an assignment to a speaker is not required.
Figure 1 shows the multinomial model proposed by Klauer and Wegener (1998) for the modified ``Who said what?'' paradigm. Multinomial models can be characterized as discrete-state models. They are discrete models in the sense that they postulate only a finite number of processing states, represented as the nodes of the processing trees depicted in Figure 1. A comprehensive review of the theory and applications of multinomial modeling is given by Batchelder and Riefer (in press). The present model describes participants' responses by means of the processes of item discrimination, person discrimination, and category discrimination as well as three guessing processes.
The model assumptions are represented by means of three processing trees, one for each kind of statement, that is, for statements that were made by a speaker from Category A, for those by a speaker from Category B, and for new statements. In Figure 1, the item categories are shown on the left side and the response categories in rectangles on the right side. In between, the mediating latent processes are depicted within ellipses. As can be seen, some of the response categories appear repeatedly because different processes can result in the same observable response.
Consider first
the tree for A-items. The first branching of that tree models the
process of item discrimination. With the probability
the participant detects a presented A-item as old, whereas
the item is not detected with complementary probability
. If the item is detected, the process of
person discrimination is considered next in the processing-tree
representation. With probability
the retrieved
information about statement and speaker is sufficient
to identify the speaker. In this case, the
participant responds with the correct assignment. With
probability
, however, the information does not
suffice to identify the speaker.
If the speaker's social category has been encoded along with the statement,
the participant may then still remember that category.
The probability for successful category discrimination
is modelled by the parameter
. If the category is recalled,
but not the individual speaker, then the model assumes that one
of the
persons of category A is guessed to be the speaker.
With a fixed probability
, guessing results in a
correct assignment, whereas with complementary probability
,
a within-category error is made.
With probability
, category discrimination does not succeed.
In this case, the participant can still guess
the speaker's category. The probability of guessing Category A
rather than B is given by parameter a.
Note that this bias parameter ensures
that response biases favoring one of the categories over the other
can be accomodated by the model.
If Category A is guessed, a second guess picks the speaker
with probability
and a wrong person within that
category with probability
as above. If instead
Category B is guessed, the participant necessarily responds with a
between-categories error.
Consider now the lower half of the tree for A-items. It describes
the case that item discrimination fails (probability
).
The model assumes that the participant then guesses whether the
item is old or new. The probability of guessing old is given
by the bias parameter b. Again, b need not equal .5,
so that the model takes into account that participants may be biased
toward one of the response alternatives "old" or "new".
If the participant guesses that the statement is old, the same processes as described above for the case of failing person discrimination and category discrimination are assumed to occur. That is, both category and a person therein are guessed. If the participant guesses on the other hand that the statement is new, he or she falsely responds ``new'' with probability 1-b.
The processing tree for statements made by a speaker from Category B is analogous. Different discrimination parameters, indexed by the category, are assumed, however, so that the strenghts of the processes may differ as a function of category.
The tree for distractors is simpler. With probability
the item is correctly discriminated as new. Such discrimination
can be based, for example, on so-called autonoetic processes (Strack
& Bless, 1994).
Autonoetic processes are inferences based on
peculiarities of distractors that allow the participant to reason
that he or she would surely have recognized this particular statement if it
had been presented at all. In this case, the participant responds "new".
Otherwise, with probability
, the same subtree of processes
follows as in the other trees after failed item discrimination with
the only difference that there can be no correct assignment for distractors.
On the basis of the observed assignment frequencies, the parameters can be estimated by means of the maximum-likelihood method; the model fit can be assessed by comparing observed and expected frequencies, and hypotheses about parameter values can be tested as explained in Klauer and Wegener (1998).
The following, in part conditional processes and corresponding parameters, are considered:
,
,
and
according
to category of statement.
,
according
to the speaker's category.
,
according to
the speaker's category.
The different combinations of processes resulting in the responses are visualized in a multinomial processing-tree representation like the one in Figure 1. On the left side there are three starting points in this model representing the three situations a participant may be confronted with in each trial of the assignment task: the presented statement may be a statement made by a member of Category A, a statement made by a member of Category B, or a new statement. The combination of cognitive processes involved may be different for each of the three situations. Hence the model comprises three different trees.
On the right side all possible response options are depicted in Figure 1. Note that some of the response options occur several times in each tree, and thus there can be different combinations of cognitive processes resulting in the same response. In contrast to the observable starting points and the observable responses, the mediating cognitive processes are not observable. This fact is visualized by depicting the observable events in rectangles and the latent processes in ellipses.
Every cognitive process takes place with a certain probability, and these probabilities are represented by the parameters attached to the connecting lines between the ellipses. In the present model there are always two alternatives at each branching: either a cognitive process takes place with a certain probability, or it does not take place with the complementary probability. Thus the present model is called a binary model, but other models may sometimes have more than two options branching out from a given node.
Because the branches of each partial tree represent a combination of cognitive processes and since each cognitive process takes place with a certain probability, the probability for each branch is the product of the probabilities of all processes that constitute the branch. Consequently, the response option a branch ends in will be chosen with the joint probability of all involved cognitive processes. Because some branches end in the same response option, that option will ultimately be chosen with probability equalling the sum of several joint probabilities.
Note that not only response options can appear repeatedly in the same tree, but also cognitive processes. Consider, for example, the process of guessing the category that is associated with parameter a. As can be seen, the process occurs twice in the first tree in Figure 1.
Sometimes it is assumed that the same cognitive process even occurs in different trees of a model, as is the case for the guessing process associated with parameter a, among others, in the present model. Thus we assume that the process of guessing that a statement was made by a speaker from Category A is the same regardless of whether the statement was actually made by a speaker from Category A, was made by a speaker from Category B, or was not presented in the discussion phase at all. Parameter a therefore appears in all partial trees of the present model.
For the statistical analysis of a multinomial model the data are aggregated over all participants of a given experimental condition. For every partial tree and response option, the frequency with which the response option was chosen by the participants is obtained.
As has been said, the
probability with which a response option is chosen can be
expressed as the product of the probabilities
of the involved cognitive processes or as the sum
of joint probabilities of this kind
if more than one branch leads to the same response option.
Hence the whole multinomial model can
be described as a system of equations: for every
response option of each partial tree one equation is obtained.
In the present multinomial model, for example, the
term,
![\begin{displaymath}[ D_A (1-c_A) (1-d_A) (1-a) ]\quad + \quad [ (1-D_A) b (1-a) ],
\end{displaymath}](img16.gif){kind=small}
Because the frequencies of the response options can be observed and the same parameters appear in several independent equations, the resulting system of equations can be solved for the unknown parameters under certain conditions, and the probabilities of the involved cognitive processes can thereby be estimated. The job of estimating the probabilities is performed by a computer program by means of the maximum likelihood method. In this iteration process, several combinations of probabilities are processed until the best solution is found, i.e. until the difference between the observed frequencies and those that would be expected on the basis of the estimated parameters is minimized, where the difference is measured in terms of the ratio of the likelihoods of the observed and the expected frequencies.
In applying a model, a global
goodness-of-fit test of the model should be conducted. To answer
the question of whether the observed
response pattern can be explained by the model, a goodness-of-fit
test evaluates the differences between the
observed and the estimated response
frequencies. To test the goodness of fit,
the likelihood ratio statistic G2
is computed, which is
asymptotically 
distributed. If this test does
not yield a significant result, given a satisfactory
statistical test power, it can be
concluded that there are no substantial deviations from the
model. In this case, the model fits the data. Note that for the
goodness-of-fit test the degrees
of freedom have to be computed. As a rule of thumb, they are
given by the number of independent
response options minus the number of parameters to be
estimated. See the body of the paper for examples.
Special hypotheses can be tested by restricting the model.
Usually, a hypothesis assumes
that a certain treatment affects a
certain cognitive process and hence will cause the probability
of this process to change.
Fitting the model to the experimental condition should thus yield an
estimated probability of this process which differs from the one obtained by
fitting the model to the control condition. To test this
hypothesis statistically, the model
is doubled and treated as one joint model for both
experimental and control condition.
The parameters of the resulting model are indexed by
condition, thereby allowing for different
probabilities in the two situations, especially allowing
for different probabilities of the
cognitive process that is supposed to have been
manipulated. Then the
change in these latter probabilities can be tested statistically by
using a goodness-of-fit test again: If there is indeed a change,
it should be
impossible to restrict the model to use
only one parameter for both probabilities, thereby forcing
them to be equal. Thus, if the restriction leads to a
significant loss of goodness of fit, the treatment
can be concluded to have affected the cognitive
process in question.
The loss of goodness of fit is
the difference of the goodness of fit of the restricted model
minus that of the
unrestricted model. The degrees of freedom for testing
the significance of this distributed loss value are given by
the difference
in the degrees of freedom of the two models.
Before a new model can be used to examine the cognitive processes involved in a given task, its substantive interpretation should be validated properly. The global goodness-of-fit test examines whether a certain probability structure can explain the frequencies with which the different response patterns were observed. Since many different models with different substantive implications can usually be fitted to the same set of data, the goodness-of-fit test does not imply that the parameters really measure the intended cognitive processes. Hence, a series of experiments should aim at validating the different process parameters.
Computer Software for estimating parameters and testing the models is available via Internet (http://xhuoffice.psyc.memphis.edu/gpt/index.htm) from Xiangen Hu. There are two programs available: an older DOS version called MBT (Hu, 1991) and a Windows95/NT version called GPT (Hu, 1997).
For working with MBT, an ASCII document
containing the system of equations of the
model and an ASCII document containing the aggregated data are needed. After reading the two
files, the program asks for restrictions of the model.
For example, parameters can be constrained to be equal,
or they can be fixed at a user-determined value. Then the parameters
are estimated and an ASCII
document is created containing parameter estimates,
their 90%-confidence intervals,
the value
of goodness of fit, the estimated and the empirical frequency
distribution as well as the so-called Fisher information matrix from which the confidence intervals
are computed (Rao, 1973).
In addition, the program checks the
identifiability of the model, i.e. whether the
system of equations can be solved uniquely.
For this purpose, several parameter estimations are
conducted from different starting values of the parameters,
and it has to be checked whether the
final estimates for each parameter are always the same.
GPT can work with ASCII documents containing equation systems and aggregated data or, alternatively, the processing-tree representation of the multinomial model can be drawn and the data has to be entered via the keyboard. Additional options like analysis of power and model simulations are integrated. Whereas MBT can handle models with only up to 61 branches, GPT is able to deal with very large models.
Figure 1. The two-high-threshold multinomial model of social categorization in the modified ``Who said what?'' paradigm. DA=probability of detecting a statement made by a speaker from Category A, DB=probability of detecting a statement made by a speaker from Category B, DN=probability of detecting that a distractor is new; dA=probability of correctly discriminating the category of a statement made by a speaker from Category A, dB=probability of correctly discriminating the category of a statement made by a speaker from Category B; a=probability of guessing that a statement is made by a speaker from Category A; b=probability of guessing that a statement is old.