Possible warnings for machine learning studies? "Chance" classification depends on the number of samples - if samples are not infinite, it may be higher than expected.
UPDATE: The general consensus seems to be that permutation testing, as commonly done, is valid and avoids the potential problems laid out by the authors.
