Exact Methods

EXACT METHODS

Among the class of nonparametric techniques is a group of methods called permutation, or randomization, methods. The methods have the advantage that conditioned on some aspect of the data at hand, they have a significance level that is exactly the specified level. The conditioning we refer to is conditioning on the marginal totals in a 2 × 2 table. In a two-sample problem, we condition on observing the combined observations without regard to which population they came from.

For the parametric techniques that we have studied in this course, achieving the correct significance level is simply a matter of finding the correct critical value(s) in a table of the sampling distribution under the null hypothesis. For more complicated testing situations in which nonparametric methods are used or approximate distributions are applied, the test may not be exact. For example, many bootstrap testing procedures provide useful nonparametric tests but they are not exact over the entire range of distributions that we consider under the null hypothesis. For such hypothesis tests that have a large set of possible distributions for the population being sampled, this exactness property is not obtainable.

We saw that Fisher’s exact test, an alternative to the chi-square test for a 2 × 2 contingency table, is one example of an exact permutation test. Cytel Corp. is one of the few companies that produce software specializing in exact methods. Cyrus Mehta, Cytel’s president, began to develop the corporation’s main products, StatXact and LogXact, in 1987. Cytel provides the most extensive and best algorithms for performing exact probability calculations. The software programs em-ploy fast algorithms based on network optimization algorithms that were originally developed for operations research problems. Cytel’s current products are described on their website at www.cytel.com. The latest version of StatXact includes sample size and power calculations.