5.8 General protocol for \(X^2\) tests

In any contingency table situation, there is a general protocol to completing an analysis.

1. Identify the data collection method and whether the proper analysis is based on the Independence or Homogeneity hypotheses (Section 5.1).

2. Make contingency table and get a general sense of response patterns. Pay attention to “small” counts, especially cells with 0 counts.

   1. If there are many small count cells, consider combining categories on one or both variables to make a new variable with fewer categories that has larger counts per cell to have more robust inferences (see Section 5.10 for a related example).

3. Make the appropriate graphical display of results and generally describe the pattern of responses.

   1. For Homogeneity, make a stacked bar-chart.

   2. For Independence, make a mosaic plot.

   3. Consider a more general exploration using a tableplot if other variables were measured to check for confounding and other interesting multi-variable relationships. Also check for missing data if you have not done this before.

4. Conduct the 6+ steps of the appropriate type of hypothesis test.

   1. Use permutations if any expected cell counts are below 5.

   2. If all expected cell counts greater than 5, either permutation or parametric approaches are acceptable.

5. Explore the standardized residuals for the “source” of any evidence against the null – this can be the start of your “size” discussion.

   1. Tie the interpretation of the “large” standardized residuals and their direction (above or below expected under the null) back into the original data display (this really gets to “size”). Work to find a story for the pattern of responses. If little evidence is found against the null, there is not much to do here.