Please use this identifier to cite or link to this item:
Title: Permutations of functional magnetic resonance imaging classification may not be normally distributed
Author: Al-Rawi, M. S.
Freitas, A.
Duarte, J. V.
Cunha, J. P.
Castelo-Branco, M.
Keywords: Anderson–Darling test
Central limit theorem
Classification analysis
Permutation testing
Issue Date: Dec-2017
Publisher: Sage
Abstract: A fundamental question that often occurs in statistical tests is the normality of distributions. Countless distributions exist in science and life, but one distribution that is obtained via permutations, usually referred to as permutation distribution, is interesting. Although a permutation distribution should behave in accord with the central limit theorem, if both the independence condition and the identical distribution condition are fulfilled, no studies have corroborated this concurrence in functional magnetic resonance imaging data. In this work, we used Anderson–Darling test to evaluate the accordance level of permutation distributions of classification accuracies to normality expected under central limit theorem. A simulation study has been carried out using functional magnetic resonance imaging data collected, while human subjects responded to visual stimulation paradigms. Two scrambling schemes are evaluated: the first based on permuting both the training and the testing sets and the second on permuting only the testing set. The results showed that, while a normal distribution does not adequately fit to permutation distributions most of the times, it tends to be quite well acceptable when mean classification accuracies averaged over a set of different classifiers is considered. The results also showed that permutation distributions can be probabilistically affected by performing motion correction to functional magnetic resonance imaging data, and thus may weaken the approximation of permutation distributions to a normal law. Such findings, however, have no relation to univariate/univoxel analysis of functional magnetic resonance imaging data. Overall, the results revealed a strong dependence across the folds of cross-validation and across functional magnetic resonance imaging runs and that may hinder the reliability of using cross-validation. The obtained p-values and the drawn confidence level intervals exhibited beyond doubt that different permutation schemes may beget different permutation distributions as well as different levels of accord with central limit theorem. We also found that different permutation schemes can lead to different permutation distributions and that may lead to different assessment of the statistical significance of classification accuracy. © 2015, © The Author(s) 2015.
Peer review: yes
DOI: 10.1177/0962280215601707
ISSN: 0962-2802
Appears in Collections:CIDMA - Comunicações
IEETA - Comunicações
PSG - Comunicações

Files in This Item:
File Description SizeFormat 
Permutation distributions paper in RIA December 2017.pdfDocumento principal versão preprint749.75 kBAdobe PDFView/Open

Formato BibTex MendeleyEndnote Degois 

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.