The permutation test is a more sensitive analytic strategy than simple corrections for multiple comparisons, because it takes into account deviations from the normal distribution and spatial correlation in the data. This advantage was evident in the results from our test cases, in which the permutation test always activated a superset of the regions activated by the simpler method, tended to enlarge clusters rather than adding isolated voxels, almost always produced tail probabilities greater than or equal to those of the simpler test, and activated voxels whose signals were weaker than those picked up by the simpler test. Although the complete step-down version of the permutation test may at first seem computationally infeasible [Holmes & al. 1996], some finesse with data structures can make the time bound of this algorithm nearly linear, and complete analysis of a data set takes only nine minutes of computer time using current technology (a 500 MHz Alpha 21164 processor). We have implemented this optimised permutation test as part of AFNI [Cox 1996], a freely available, widely used package of routines for analysis of functional brain images.
In an attempt to deal with one problem at a time, and in keeping with past work on resampling methods in functional imaging [Holmes & al. 1996; Arndt & al. 1996; Brammer & al. 1997; Heckel & al. 1998], we have not attempted to account for autocorrelation in the observed time series. In cases in which the ideal time series represents a blocked design, the presence of autocorrelation within the observed time series tends to exaggerate estimates of significance slightly, since the empirical distribution is based on data whose autocorrelation has been removed by shuffling. One way to reduce this slight biasing effect would be to shuffle the observed time series in chunks, so that the order of chunks with respect to each other is randomised but the original order of the samples within each chunk is preserved. A better method, of course, would be to model the autocorrelation and remove it. Locascio et al. [1997] present an elegant model of autocorrelation in fMRI time series, based on autoregressive and moving-average techniques originally developed for the analysis of economic data. As Locascio et al. observe, an implementation of such a model as part of a software package tailored specifically for the analysis of fMRI data (e.g. [Cox 1996]) would be a useful tool.
We view the permutation test software presented here as one among several potential improvements and optimisations to correlation-based strategies for fMRI data analysis. We invite the addition of other optimisation steps, and regard the system that we have described as a first step towards an even more sophisticated, freely available fMRI analysis package that takes advantage of currently available levels of computational speed.