(PDF) Using multivariate decoding to go

PERSPECTIVE ARTICLE published: 30 October 2014 doi: 10.3389/fpsyg.2014.01250 Using multivariate decoding to go beyond contrastive analyses in consciousness research Kristian Sandberg1,2 *, Lau M. Andersen 3 and Morten Overgaard 1,4 1 Cognitive Neuroscience Research Unit, Hammel Neurorehabilitation Centre and University Research Clinic, Aarhus University, Aarhus, Denmark 2 Institute of Cognitive Neuroscience, University College London, London, UK 3 Cognitive Neuroscience Research Unit, Aarhus University, Aarhus, Denmark 4 Center for Cognitive Neuroscience, Department of Communication and Psychology, Aalborg University, Aalborg, Denmark Edited by: Contrasting conditions with and without awareness has been the preferred method for Jaan Aru, University of Tartu, Estonia investigating the neural correlates of consciousness (NCC) for decades, yet recently it Reviewed by: has been suggested that further insights can be made by moving beyond this method, Martin Nikolai Hebart, University Medical Center Hamburg-Eppendorf, specifically by meticulously controlling that potential precursors and consequences of Germany the NCC are not mistaken for an NCC. Here, we briefly review the advantages and Nathan Faivre, California Institute of potential pitfalls of existing paradigms going beyond the contrastive method, and we Technology, USA propose multivariate decoding of neural activity patterns as a supplement to other methods. *Correspondence: Specifically, we emphasize the ability of multivariate decoding to detect which patterns Kristian Sandberg, Danish Neuroscience Center, Aarhus of neural activity are consistently predictive of conscious experiences at the single trial University Hospital, Nørrebrogade 44, level.This is relevant as the “NCC proper” is expected to be consistently predictive whereas Building 10G, 8000 Aarhus, Denmark processes that are consequences of consciousness may not occur on every trial (making e-mail:

[email protected]

them less predictive) and prerequisites without conscious Keywords: consciousness, THE EVOLUTION OF CONTRASTIVE ANALYSIS In early outlines of contrastive analyses in consciousness research, emphasis was placed on comparing pairs of psychological phe- nomena of which one was conscious and the other was not (e.g., Baars, 1994). Behavioral characteristics and neural activity could thus be compared between the conscious and unconscious cases. In the case of vision, for instance, neural activity related to masked and unmasked stimulus presentations (Dehaene et al., 2001) or to stimuli presented at various durations (Kjaer et al., 2001) has been investigated. Over the last two decades, meth- ods have evolved so rapidly that it is now difficult to determine what is a natural extension of the contrastive analysis method and what is an alternative method. In this article, we discuss some of the recent developments, and we consider how mul- tivariate decoding, as an extension of or in combination with contrastive analysis, can contribute to identifying neural correlates of consciousness (NCC). Many recent paradigms were developed in order to avoid con- founds present in the original proposals and experiments. For instance, if stimulus duration is varied, the two conditions no longer differ exclusively in terms of the subjective experience of the participant, but also in terms of an important stimulus char- acteristic, which could be expected to have an impact on conscious as well as unconscious processing (Overgaard, 2004). For this reason, some scientists have preferred paradigms where the phys- ical parameters remain stable, but only the conscious experience varies. This has been done, for instance, using masked stimuli by contrasting trials based on reports of awareness (e.g., Babiloni et al., 2010). Furthermore, in some relatively early studies par- ticipants primarily performed objective tasks, and to the extent www.frontiersin.org Sandberg et al. Lamme, 2006). Bistable perception provides another method for ensuring both conscious and unconscious perception under equal stimulation conditions. Many earlier studies using ambigu- ous perception examined differences in neural activity related to ambiguity/non-ambiguity (Lumer et al., 1998) or reversals of perception (Kornmeier and Bach, 2004), but some have also compared neural activity related to one perceptual state versus another (Andrews et al., 2002; Sterzer and Rees, 2008; Sandberg et al., 2013). RECENT DEVELOPMENTS Recently, it has been argued that it is possible that studies using contrastive analyses cannot distinguish between a NCC and its pre- requisites (NCC-pr) and consequences (NCC-co; Aru et al., 2012). An NCC-pr is neural activity associated with task specific initial processing (which predicts later conscious experiences) whereas an NCC-co is neural activity related to a process that occurs for conscious stimuli only, for instance encoding in working memory. Aru et al. (2012) have argued that by manipulating stimulus pro- cessing in various ways, NCC-pr and NCC-co should change, but the NCC should remain stable. In one experiment, Melloni et al. (2011) manipulated the stimulus expectation across conditions and found that an early EEG component (around 100 ms) only reflected differences between seen and unseen stimuli when there was no expectation of the stimulus, and similarly a later compo- nent (the P300) only correlated with awareness when stimuli had to be encoded in working memory, but not when a representation was already present. In contrast, a component between the two, at around 200–300 ms, correlated with conscious perception inde- pendently of condition. This indicated that the first component was an NCC-pr, the middle component at 200–300 ms a likely NCC candidate, and the P300 an NCC-co. Although this method for moving beyond contrastive analysis is certainly novel and useful, it assumes one can evoke the same experience by means of multiple, very different manipulations. However, there is no guarantee that the experience is identical even if the same proportion of awareness responses is obtained across conditions. Ratings of awareness can be viewed as a deci- sion process in which evidence is gathered for a particular response (e.g., Lau, 2008), for example “seen,” but when different manipu- lations are made, the decision axis is no longer shared, and thus it is unknown if the NCC can be expected to remain unchanged (Jannati and Di Lollo, 2012). A potential solution to this could be the use of more detailed awareness ratings, but it may also be possi- ble to improve the paradigm in general using decoding approaches as we will return to later. Accordingly, we still have no paradigm to investigate NCCs without potential systematic confounds. Newer paradigms, to some degree, have solved problems in previous paradigms, yet have introduced new ones. For this reason, we argue that converging evidence across multiple paradigms is essential in the search for the “NCC proper” (Overgaard, 2011). MULTIVARIATE DECODING Here, we use the term multivariate decoding [also sometimes referred to as multivariate/multi-voxel pattern analysis, pattern classification, “brain reading,” or simply decoding (Haynes and Frontiers in Psychology | Consciousness Research Sandberg et al. FIGURE 1 | Illustration of a hypothetical classification analysis and the steps involved. (A) The classifier is to distinguish “Aware” (red) and “Unaware” (blue). Trials are sets in three different ways to ensure cross-validation. The plots show fMRI data. It is precisely the heavy spatial and temporal correla- tions of neuroimaging data that make them fit for multivariate analyses. In contrast to univariate tests, multivariate tests can facilitate the information contained in the temporal and spatial dependencies between data points in both sensor and source space (MEEG) and in voxel space (fMRI) in a single test. www.frontiersin.org Sandberg et al. that a core NCC (or “NCC proper”) should in principle to predict a conscious state perfectly. From this it follows higher decoding accuracy is generally a sign of greater tional accuracy although it must be emphasized that care should be taken when comparing decoding accuracies across different brain areas, and there are several aspects to consider. Kamitani and Tong (2006) found that perceived motion direc- tion was only decoded as well from MT+ as from earlier visual areas V1–V4 when the same number of voxels was used. Indeed, a later article by Smith et al. (2011) mention that when compar- ing fMRI decoding accuracies across conditions, participants, or brain regions, it is important that several factors are for including the number of voxels and stimulus repetitions (and we might add that not only the number of spatial, but also the number of temporal, features should be controlled for). Addi- tionally, they specifically emphasize the importance of for or taking into account the mean amplitude of the compo- nent of interest as they show that decoding accuracy increases as a function of mean amplitude even if specificity is not increased. The function with which classifier accuracy increases as a func- tion of response amplitude (measured as percent signal change for fMRI) can nevertheless be estimated and compared across areas for a more valid comparison of decoding accuracy. A simpler, but not always feasible solution is to compare components of equal amplitude. A note of caution is necessary, however: even when mean ampli- tude is controlled for, the obtainable signal from two components may differ in their signal-to-noise ratios (for instance, if the angle of the neurons prevents a good signal in MEEG). This necessi- tates that one is cautious when interpreting differences in accuracy between MEEG components unless one has a good way to esti- mate differences in noise ceilings. Such estimations are with encoding models (Kay et al., 2008) or with representational similarity analysis (Nili et al., 2014), but it is presently solved issue for decoding models and further work in this field is important for ensuring the validity of comparisons of decod- ing accuracies. It should be emphasized that the issue is not likely to be dramatic and presently a rough estimate of noise ceiling may be achieved by prior knowledge of decoding accuracies across different tasks for various brain regions/components. Univariate tests are of course sensitive to differences that are present on all epochs, but crucially they can, in addition, be sensi- tive to differences that appear only on some epochs, but show some average difference between conditions (e.g., aware/unaware). This has important implications for the attempt to separate NCC-pr, NCC, and NCC-co. In Figure 2, we show simulated data with three components for which there are average differences between tri- als reported as “aware” and “unaware” by a participant. We would expect the actual NCC to vary consistently with the conscious experience – whenever the participant has an experience of the stimulus, the relevant component should reflect this. The NCC-pr, however, might be present without the NCC on some trials (i.e., one particular prerequisite of conscious experience was present on a trial, but perhaps some others were not, and the participant thus had no experience) in which case the component becomes an unreliable predictor and should not be assigned high weights by the classifier when all data are taken into account, and it should Frontiers in Psychology | Consciousness Research Sandberg et al. holds more information about the label of the state, say “aware” or “unaware.” In cases where the confounding processes occur on every sin- gle trial with an awareness response, multivariate decoding on its own will not be able to distinguish between NCC and NCC- pr/NCC-co as all responses could be equally predictive. For this reason, we believe that the optimal paradigm is a combination of decoding and the methods suggested by Melloni et al. (2011) and Aru et al. (2012). One way to combine methods would be to use cross-task decoding – i.e., using several tasks resulting in similar conscious experiences and training/testing on different tasks using a leave-one-out procedure. In this case, decoding performance should be best for components that generalize across experimental contexts. Using multivariate decoding on MEG data, a study by our group have found that conscious experience during binocular rivalry was predicted relatively accurately by activity around 130–320 ms after stimulus onset and that an earlier and a later component was not consistently predictive (Sandberg et al., 2013). In an addi- tional (ongoing) MEG study, multivariate decoding furthermore showed that activity around this time was the most predictive of small, graded differences in the clarity of conscious experience on the single trial level (Andersen et al., in preparation). Simi- larly, decoding can be used on different brain areas in turn in order to compare how consistently predictive these are separately (and/or combined; Norman et al., 2006). For binocular rivalry, this was done for V1–V3 by Haynes and Rees (2005) and across the cortex by Sandberg et al. (2013). Lastly, it should be acknowl- edged that when doing multivariate analyses, “decoding” is not strictly necessary. There are ways of doing “encoding” as well, where one can extract parameters from the model, as in classi- cal univariate models. Encoding applications are at the moment, however, less available than decoding applications, both theoreti- cally and practically, but see Allefeld and Haynes (2014) for a novel approach. OTHER POSSIBILITIES USING MULTIVARIATE DECODING The use of multivariate decoding opens up for potential research, which would otherwise be difficult or even impossible to conduct. For MEG, conscious experience can be decoded using only a few milliseconds of data gathered within the first 200 ms after stimulus presentations (Sandberg et al., 2013, 2014). Particularly, if near- perfect, near real-time decoding can be achieved, it may be possible to exploit such speed in the control of brain-computer interfaces. At present, one study was able to achieve above 85% decoding accuracy for three of eight participants (and around 95% for one; Sandberg et al., 2013). In comparison, univariate decoding (i.e., using the single best sensor at the single best time point) resulted in lower accuracies (around 10% lower), and would furthermore require both time point and sensor to be specified in advance. Additionally, other studies have shown cases in which multivariate decoding is above chance in the absence of an average activity difference (Sterzer et al., 2008). Because decoding can be accomplished prior to report, the possibility that an MEG based brain-computer interface could be used to generate changes in the environment even before they are produced by the motor behavior of the individual, which could www.frontiersin.org Sandberg et al. Andrews, T. J., Schluppeck, D., Homfray, D., Matthews, Activity in the fusiform gyrus predicts conscious perception illusion. Neuroimage 17, 890–901. doi: 10.1006/nimg.2002.1243 Aru, J., Bachmann, T., Singer, W., and Melloni, L. (2012). ral correlates of consciousness. Neurosci. Biobehav. 10.1016/j.neubiorev.2011.12.003 Baars, B. (1994). A Thoroughly Empirical Approach To Consciousness. 1–25. Babiloni, C., Vecchio, F., Buffo, P., Buttiglione, M., M. (2010). Cortical responses to consciousness of schematic expressions: a high-resolution EEG study. Hum. Brain 10.1002/hbm.20958 Christensen, M. S., Ramsøy, T. Z., Lund, T. E., Madsen, An fMRI study of the neural correlates of graded visual 31, 1711–1725. doi: 10.1016/j.neuroimage.2006.02.023 Dehaene, S., Naccache, L., Cohen, L., Bihan, D. L., Mangin, et al. (2001). Cerebral mechanisms of word masking and priming. Nat. Neurosci. 4, 752–758. doi: 10.1038/89551 Haxby, J. V., Guntupalli, J. S., Connolly, A. C., Halchenko, Gobbini, M. I., et al. (2011). A common, high-dimensional sentational space in human ventral temporal cortex. Neuron 10.1016/j.neuron.2011.08.026 Haynes, J.-D. (2009). Decoding visual consciousness from Trends Cogn. Sci. 13, 194–202. doi: 10.1016/j.tics.2009.02.004 Haynes, J.-D., and Rees, G. (2005). Predicting the stream of consciousness from activity in human visual cortex. Curr. Biol. 15, 1301–1307. doi: 10.1016/j.cub.2005.06.026 Haynes, J.-D., and Rees, G. (2006). Decoding mental states humans. Nat. Rev. Neurosci. 7, 523–534. doi: 10.1038/nrn1931 Hesselmann, G., Hebart, M., and Malach, R. (2011). Differential associated with subjective and objective reports during mal observers. J. Neurosci. 31, 12936–12944. doi: 10.1523/JNEUROSCI.1556- 11.2011 Jannati, A., and Di Lollo, V. (2012). Relative blindsight confound in metacontrast masking: implications for theories Conscious. Cogn. 21, 307–314. doi: 10.1016/j.concog.2011.10.003 Kamitani, Y., and Tong, F. (2006). Decoding seen and attended motion direc- tions from activity in the human visual cortex. Curr. 10.1016/j.cub.2006.04.003 Kay, K. N., Naselaris, T., Prenger, R. J., and Gallant, ing natural images from human brain activity. Nature 452, 352–355. doi: 10.1038/nature06713 Kjaer, T. W., Nowak, M., Kjaer, K. W., Lou, A. R., and prefrontal activity during awareness of visual verbal 356–365. doi: 10.1006/ccog.2001.0509 Koivisto, M., Mäntylä, T., and Silvanto, J. (2010). The (V1/V2) in conscious and unconscious visual perception. 834. doi: 10.1016/j.neuroimage.2010.02.042 Kornmeier, J., and Bach, M. (2004). Early neural activity evidence for low-level processing of a gestalt phenomenon. 1–8. doi: 10.1046/j.1469-8986.2003.00126.x Lamme, V. A. F. (2006). Towards a true neural stance on Sci. 10, 494–501. doi: 10.1016/j.tics.2006.09.001 Lau, H. C. (2008). A higher order Bayesian decision theory Brain Res. 168, 35–48. doi: 10.1016/S0079-6123(07)68004-2 Lau, H. C., and Passingham, R. E. (2006). Relative blindsight and the neural correlate of visual consciousness. Proc. 18763–18768. doi: 10.1073/pnas.0607716103 Luck, S. J. (2014). An Introduction to the Event-Related Cambridge, MA: The MIT Press. Lumer, E. D., Friston, K. J., and Rees, G. (1998). Neural tual rivalry in the human brain. Science 280, 1930–1934. ence.280.5371.1930 Frontiers in Psychology | Consciousness Research of consciousness may be present on some trials experience (making them less predictive). multivariate decoding, multivariate pattern analysis, contrastive analyses, MEG, fMRI that awareness reports were used, they were used to confirm that conditions could be treated as subliminal/supraliminal (Dehaene et al., 2001; Kjaer et al., 2001; Silvanto et al., 2005). In contrast, in some later studies, scientists have more often preferred to base analyses on trial-by-trial reports of awareness (or confidence) even when multiple physical stimulus conditions are used (Christensen et al., 2006; Koivisto, Mäntylä et al., 2010). The use of awareness reports can be seen as a necessary consequence of the wish to control for physical parameters. Methodologically speaking, these reports separate conditions when trials no longer differ in terms of objective characteristics. But their use is also partly a conse- quence of theoretical arguments in favor of the crucial role of awareness ratings as a key measure of validity in consciousness research (Overgaard, 2006, 2010). Some scientists even prefer to keep accuracy stable so that only the level of awareness varies between conditions (Lau and Passingham, 2006; Lau, 2008) or to examine the correlates of accuracy and awareness separately while ensuring that mask and stimulus have very different neural signatures (Hesselmann et al., 2011). Common to most recent studies is that the need to con- trol for potential confounds has resulted in a shift from the examination of complete unawareness versus complete awareness to the examination of smaller differences in graded awareness ratings or changes in the probability of obtaining reports of awareness. As the change between conscious and unconscious perception occurs more suddenly across stimulus intensity for the attentional blink (than for masking), this paradigm has sometimes been preferred (e.g., Sergent et al., 2005) although others are reluctant to use the paradigm as they suspect it reflects failure to attend (possibly conscious) perception (e.g., October 2014 | Volume 5 | Article 1250 | 1 Multivariate decoding and contrastive analysis Rees, 2006; Norman et al., 2006; Haynes, 2009)] as an umbrella term for a group of analysis techniques for which the goal, in this context, is to decode the conscious experience of a partic- ipant based on large amounts of brain data. We will exemplify the general logic behind multivariate decoding by example of a within-subject decoding. Take an MEG dataset (Figure 1), for instance, of a subject with x epochs of class A (e.g., “aware”) and x epochs of class B (e.g., “no awareness”): each data point of each epoch is called a feature. For a given dataset with n sensors/sources and t time points, one will thus have n X t features for each epoch. The dataset is then divided into two parts, a training set (often 90% of the data) and a test set (the remaining 10%; Figure 1A). A model is fitted to the training set and each feature is assigned a weight. Dependent on the sign of a given weight, it raises the posterior probability of a given epoch to belong to class A or B, respectively. The fitted training set, with its feature weights, is then used to predict the class of each epoch for the test set (Figure 1B). The predicted class label for a given epoch is the class label that has the highest posterior probability assigned to it when the feature weights for that epoch are summed together. One can then obtain a classification score, which is the percentage of correctly classified epochs. Figure 1C shows an example of this. To test the generality of the classification score, one can cross-validate the score by dividing the data set into training and test sets in different ways. We believe that multivariate decoding has a role in neuroscien- tific consciousness research for several reasons and in the following we will go through these. We will, however, first emphasize that decoding results should be interpreted with care: although a given mental state can be decoded above chance from particular neu- ral activity, this does not in itself imply a causal relationship. In this sense, multivariate decoding shares some of the limitations of correlation studies. Multivariate decoding, nevertheless, opens up new possibilities that have not previously been available. INCREASED SENSITIVITY OF MULTIVARIATE DECODING One main advantage of multivariate decoding is the greater sensi- tivity than that of traditional mass-univariate approaches typically used in contrastive analyses (i.e., the testing of single variables one at a time; Haynes and Rees, 2006; Norman et al., 2006). Mul- tivariate decoding is more sensitive that univariate testing due to pooling of information and the informativeness of the co-variance of the features (Haynes and Rees, 2006). Furthermore, univariate tests typically test for linear relationships whereas the nature of the relationship does not need to be specified to achieve successful decoding (Haynes, 2009). The advantage of multivariate decoding in consciousness research has been shown for fMRI where Haynes and Rees (2005) showed that decoding based on V1–V3 voxels combined was more predictive of perception during binocular rivalry than decoding based on the combined mean of the same voxels. Similarly, using MEG Sandberg et al. (2013) showed that perception during binocular rivalry can be decoded at an accuracy just a few percent below peak decoding accuracy (around 75%) using just 10 occipital sensors, which were individually at chance (below 51.5%). At its core, all univariate testing regards data points as indepen- dent of one another, which is evidently false for both MEEG and October 2014 | Volume 5 | Article 1250 | 2 Multivariate decoding and contrastive analysis hypothetical activity developing over time courses for ten trials of aware and between two categories: unaware respectively. (B) The decisions reached based on the model fit in the separated into training and test training set. (C) Classification is performed for 100 trials (50 aware and 50 unaware) with a non-linear decision boundary. FINDING CONSISTENT CORRELATES USING MULTIVARIATE DECODING Multivariate tests are more sensitive to differences between con- ditions that are present during all epochs, and that they are less sensitive to differences between conditions that are only present during some of the epochs. Indeed, Haynes (2009) emphasized October 2014 | Volume 5 | Article 1250 | 3 Multivariate decoding and contrastive analysis be able that representa- For instance, controlled controlling FIGURE 2 | Consistency of the neural correlates of consciousness (NCC). Three simulated, hypothetical signals of differing consistency and strength are plotted. All could be candidate NCC, thus reflecting differences between trials classified as “aware” and “unaware” by a participant. For the first component, there is a small average difference, but the component is not consistently larger for “aware” trials, making it unlikely that the component reflects awareness. The component could reflect a prerequisite for consciousness (NCC-pr) as it has to be present for awareness, but it does not guarantee awareness. For the second component, there is a medium average difference, and the component is consistently larger for “aware” trials. On the single trial level, the component thus reflects awareness and it may thus be an actual NCC. Finally, for the third component, there is a large average difference, but the component is only found on a subset of “aware” trials, and it does thus not consistently reflect awareness. The component could thus reflect processes that are consequences of awareness (NCC-co), which occur exclusively for “aware” trials, but may not occur on every single aware trial. Note that traditional univariate statistics based averaged participant-specific possible averages would erroneously find more evidence for the last component being the NCC proper in this example. an unre- produce suboptimal decoding accuracy when used to train/test the classifier alone. This corresponds to the first component in Figure 2. The NCC-co, on the other hand, might not occur after each single NCC component (even if it occurs after some NCC components), and it should never occur without an NCC compo- nent. It is thus expected to be similarly suboptimal for decoding even if it produces very large responses on some trials and a large average difference. This corresponds to the third component in Figure 2. The actual NCC is thus expected to be consistently the most predictive at the single trial level even if it does not produce the largest average difference. This corresponds to the second component in Figure 2. As mentioned above, multivariate decoding approaches are able to identify the most consistent cor- relates, but traditional univariate analyses typically base statistics on participant-specific means and would in our example find sig- nificant evidence in favor of the third component even though it only occurs on some trials. Importantly, if the aim is to compare components, as in our example (Figure 2), univariate tests are not readily interpretable. There is no straightforward interpretation of what a difference in amplitude between components means (Luck, 2014). In comparison, the interpretation of differences in decod- ing accuracy is straightforward – it simply means that the pattern October 2014 | Volume 5 | Article 1250 | 4 Multivariate decoding and contrastive analysis be of key importance in the study of overt behavior and sense of agency. Furthermore, neural correlates can be analyzed before and after the preparation to report in the attempt to filter out cor- relates of introspection, metacognition, and motor preparation. And finally, fast and accurate decoding allows for manipulations of stimuli or brain activity (using TMS, for instance) around the time where an event is experienced, but before it is reported, and it may allow for the study of awareness without report. Haynes and Rees (2006) emphasized the importance of the then unresolved issue of how well activity generalizes over time, across situations (paradigms) and even across participants. This can be examined by conventional methods using correlations, but decod- ing provides a method of examining whether minor changes are critical or whether the overall patterns are generally maintained. Haynes and Rees (2005) used fMRI to examine drops in decoding accuracy across days, but the first long-term study was conducted by Sandberg et al. (2014), who found that the decrease in decoding accuracy within participants across 2.5 years was only around 1%, which was comparable to the drop across a few days. This study also found that the drop when attempting to generalize across participants (even at the source level) was much greater (around 10%). Further studies examining whether minor details in patterns of activity predict related changes in perceptual experience can be used to address theoretical questions about multiple realization in the brain. It has also been established that it is possible to decode the conscious experience of one individual using a classifier trained on a different individual although the accuracy is lower than for within-individual decoding (Poldrack et al., 2009; Haxby et al., 2011; Sandberg et al., 2013, 2014). This opens up possibilities that so far have been outside the reach of cognitive neuroscience meth- ods. One might apply multivariate decoding to investigate whether neural correlates generated in experiments using one paradigm can be used to train a classifier to decode the experience in other paradigms as we discuss above. Furthermore, between-participant decoding opens possibilities of decoding across groups for which it is uncertain whether one has conscious experiences, such as vegetative or minimally conscious patients. When consciousness has been examined in non-human animals, methods such as flash suppression have been used to ensure the validity of report as the stimuli are bistable but conscious perception can be manipulated by the experimental setup (Sheinberg and Logothetis, 1997). Such or similar methods could in principle also be used with patients, and it could be possible to decode both within individuals but also to examine how well classifiers generalize from healthy individuals to reduced consciousness patients. Here again, the improved accu- racy of multivariate decoding provides an advantage compared to univariate approaches. ACKNOWLEDGMENTS This work was supported by the European Research Council (Kristian Sandberg and Morten Overgaard). it raises REFERENCES Allefeld, C., and Haynes, J.-D. (2014). Searchlight-based multi-voxel pattern analysis of fMRI by cross-validated MANOVA. Neuroimage 89, 345–357. doi: 10.1016/j.neuroimage.2013.11.043 October 2014 | Volume 5 | Article 1250 | 5 Multivariate decoding and contrastive analysis P., and Blakemore, C. (2002). Melloni, L., Schwiedrzik, C. M., Müller, N., Rodriguez, E., and Singer, W. of Rubin’s vase-face (2011). Expectations change the signatures and timing of electrophysiolog- ical correlates of perceptual awareness. J. Neurosci. 31, 1386–1396. doi: Distilling the neu- 10.1523/JNEUROSCI.4570-10.2011 Rev. 36, 737–746. doi: Nili, H., Wingfield, C., Walther, A., Su, L., Marslen-Wilson, W., and Kriegeskorte, N. (2014). A toolbox for representational similarity analysis. PLoS Comput. Biol. Psyche 1, 10:e1003553. doi: 10.1371/journal.pcbi.1003553 Norman, K. A., Polyn, S. M., Detre, G. J., and Haxby, J. V. (2006). Beyond mind- Cibelli, G., and Rossini, P. reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430. emotional facial doi: 10.1016/j.tics.2006.07.005 Mapp. 31, 1556–1569. doi: Overgaard, M. (2004). Confounding factors in contrastive analysis. Synthese 141, 217–231. doi: 10.1023/B:SYNT.0000043019.64052.e0 K. H., and Rowe, J. B. (2006). Overgaard, M. (2006). Introspection in science. Conscious. Cogn. 15, 629–633. doi: perception. Neuroimage 10.1016/j.concog.2006.10.004 Overgaard, M. (2010). How consciousness will change our view on neuroscience. J. F., Poline, J. B., Cogn. Neurosci. 1, 224–225. doi: 10.1080/17588928.2010.497585 unconscious repetition Overgaard, M. (2011). Visual experience and blindsight: a methodological review. Exp. Brain Res. 209, 473–479. doi: 10.1007/s00221-011-2578-2 Y. O., Conroy, B. R., Poldrack, R. A., Halchenko, Y. O., and Hanson, S. J. (2009). Decoding the large-scale model of the repre- structure of brain function by classifying mental States across individuals. Psychol. 72, 404–416. doi: Sci. 20, 1364–1372. doi: 10.1111/j.1467-9280.2009.02460.x Sandberg, K., Bahrami, B., Kanai, R., Barnes, G. R., Overgaard, M., and Rees, human brain signals. G. (2013). Early visual responses predict conscious face perception within and between subjects during binocular rivalry. J. Cogn. Neurosci. 25, 969–985. doi: 10.1162/jocn_a_00353 Sandberg, K., Barnes, G. R., Rees, G., and Overgaard, M. (2014). Magnetoencephalo- graphic activity related to conscious perception is stable within individuals from brain activity in across years but not between individuals. J. Cogn. Neurosci. 26, 840–853. doi: 10.1162/jocn_a_00525 BOLD activity Sergent, C., Baillet, S., and Dehaene, S. (2005). Timing of the brain events underlying “blindsight” in nor- access to consciousness during the attentional blink. Nat. Neurosci. 8, 1391–1400. doi: 10.1038/nn1549 Sheinberg, D. L., and Logothetis, N. K. (1997). The role of temporal cortical areas arises from a criterion in perceptual organization. Proc. Natl. Acad. Sci. U.S.A. 94, 3408–3413. doi: of consciousness. 10.1073/pnas.94.7.3408 Silvanto, J., Lavie, N., and Walsh, V. (2005). Double dissociation of V1 and V5/MT activity in visual awareness. Cereb. Cortex 15, 1736–1741. doi: Biol. 16, 1096–1102. doi: 10.1093/cercor/bhi050 Smith, A. T., Kosillo, P., and Williams, A. L. (2011). The confounding effect of J. L. (2008). Identify- response amplitude on MVPA performance measures. Neuroimage 56, 525–530. doi: 10.1016/j.neuroimage.2010.05.079 Sterzer, P., Haynes, J.-D., and Rees, G. (2008). Fine-scale activity patterns in high- Lou, H. C. (2001). Precuneus- level visual areas encode the category of invisible objects. J. Vis. 8, 10.1–12. doi: stimuli. Conscious. Cogn. 10, 10.1167/8.15.10 Sterzer, P., and Rees, G. (2008). A neural basis for percept stabilization in binocular role of early visual cortex rivalry. J. Cogn. Neurosci. 20, 389–399. doi: 10.1162/jocn.2008.20039 Neuroimage 51, 828– Conflict of Interest Statement: The authors declare that the research was conducted in Necker-cube reversal: in the absence of any commercial or financial relationships that could be construed Psychophysiology 41, as a potential conflict of interest. consciousness. Trends Cogn. Received: 28 May 2014; accepted: 14 October 2014; published online: 30 October 2014. Citation: Sandberg K, Andersen LM and Overgaard M (2014) Using multivariate of consciousness. Prog. decoding to go beyond contrastive analyses in consciousness research. Front. Psychol. 5:1250. doi: 10.3389/fpsyg.2014.01250 in normal observers This article was submitted to Consciousness Research, a section of the journal Frontiers Natl. Acad. Sci. U.S.A. 103, in Psychology. Copyright © 2014 Sandberg, Andersen and Overgaard. This is an open-access article Potential Technique, 2nd Edn. distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original correlates of percep- author(s) or licensor are credited and that the original publication in this journal is cited, doi: 10.1126/sci- in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. October 2014 | Volume 5 | Article 1250 | 6

(PDF) Using multivariate decoding to go beyond contrastive analyses in consciousness research