For over 40 years, researchers have explored whether cognitive processes can occur without conscious awareness. Still, to this day, it remains a relatively open question, with no consensus about the extent and scope of unconscious processing. Past research has produced many conflicting findings, possibly due to the great variability in the methods employed. These methodological variations have been suggested to influence the results, but their impact has not yet been systematically examined or mapped. To address this gap, we introduce the UnconTrust database (https://osf.io/2jgsx; see also http://uncontrustdb.tau.ac.il), classifying the methodological aspects of studies that focus on unconscious processing, specifically asking if such processing reaches the semantic level, and if it can capture attention. 

The database provides a comprehensive overview on the field, offering insights on how unconscious processing has been studied empirically, and allows researchers to explore the relationship between methods and the obtained results. Thus, it represents a novel meta-scientific tool that enables researchers to explore methodological trends in the field.

The dataset contains detailed information about the methods, including factors such as how conscious awareness has been manipulated and measured, the type of unconscious processes examined, information regarding the sample, tasks, and analysis (for example, whether researchers employed post-hoc participants or trials selection). 

The dataset is presented as an interactive website (see link above). Users of the website can perform custom queries, explore methodological trends in unconscious processing research, examine co-occurrence of specific methodological decisions, and assess how results may be influenced by different experimental approaches. They can generate plots, download them as well as the data, and upload their own papers to the database.

The database is continuously updated, so the number of studies included in the website is already larger. Though it is currently focused on behavioral studies involving healthy adult participants and mainly on the semantic and attentional processing domains, we plan to gradually expand the database to include additional areas of unconscious processing, such as emotional and perceptual processes, as well as data from additional populations (e.g., children, animals) and neuroscientific findings. Future updates will involve systematically searching for studies of unconscious affective processing and of learning without awareness, and also by extracting the data from newly published studies, which will be regularly added to the interactive website. Yet even at this stage, the UnconTrust database is the largest and most comprehensive collection of metadata and classifications of studies of unconscious processes, following the FAIR principles (Findable, Accessible, Interoperable and Reusable). Below we briefly describe the process of compiling and validating the data.

Currently, the included experiments come from two unpublished meta-analyses focused on semantic processing and attentional allocation without awareness. For them, we first conducted database searches in PsycInfo (for both meta-analyses) and also in Medline (for the semantic processing meta-analysis), Embase, Scopus and PubMed (for the attentional meta-analysis), yielding 6155 records for the attention meta-analysis, and 5315 for the semantic meta-analysis. Then we detected 64 additional papers based on previously published meta-analyses and review papers, as well as the papers that cited them. Then we screened these papers in two stages - first, by title and abstract, and then based on the full text. Papers that were found eligible after the second screening (n=793) were subsequently classified and coded. From each study, we collected key details, including the experimental paradigm used; the suppression method applied to render stimuli unconscious; the measure used to assess awareness; stimulus characteristics, such as type, size, and duration; the processing domain investigated; the outcome measures and tasks participants were asked to perform. 

We then integrated the two previously separate datasets (collected for each meta-analysis) into a unified structure with consistent coding conventions. The first step was to define a standardized set of variables and their possible values for each experiment, and then, where necessary, missing values were supplemented.

As validation to the dataset, we took a few measures. First, the second screening as well as the data extraction carried out by two independent coders, with any discrepancies resolved in consultation with Liad Mudrik, who supervised the project. Second, the inclusion criteria and the scope were approved by a steering committee that includes, in addition to the authors of this paper, leading researchers in the field - Ned Block, Axel Cleeremans, Stephen Fleming, Dominique Lamy, Lucia Melloni, Megan Peters, and Anil Seth. In addition, any new data that will be added to the database by future users will also undergo a validation process, being reviewed by at least one member of the steering committee. Additionally, if the entry is not submitted by the publication's authors, we will send the classification to them for validation and approval.

We hope the field will benefit from the dataset as well as the website, which allows an easy exploration of methodological choices, guiding future research.