Razib Khan One-stop-shopping for all of my content

April 13, 2017

The coming reign of the Baby Boomer gerontocracy

Filed under: Gerontocracy,Psychology,Psychometrics — Razib Khan @ 1:22 pm

From Dawn to Decadence: 1500 to the Present: 500 Years of Western Cultural Life is one of my favorite books. It’s one of those works whose breadth and depth is such that I would recommend it to anyone. Jacques Barzun began writing this work when he was 84, and it was published in his 93rd year. Born in 1907 Barzun saw the full efflorescence of 20th century Western culture across much of its span firsthand. When people say that when you age you gain wisdom, surely in the domain of scholarship Barzun’s production in the last few decades of his life would qualify.

But not everyone is Jacques Barzun. If you read Intelligence: All That Matters or peruse some of Eliott Tucker-Drob’s work you will know that cognitive function declines with age beyond your twenties. Different subcomponents may decline at different rates. And, they decline differently in different people (e.g., some people may develop dementia, so their faculties will decline far faster at an earlier age). But, by and large any gains in experience or wisdom are going to be balanced against declines in raw analytic ability, as well as the slow entropic loss of information.

This is not an inconsequential matter. Our governing class is quite old. The average age in Congress may be 55 to 60, but it is almost certainly true that more senior members with more power and authority are older. The president of the United States is 70 years old. If you look at the plots in these figures by 70 there has been a notable drop in intelligence by this age, though again, it may vary from person to person.

But most important in light of these figures is that the Supreme Court is a lifetime appointment, and many of its members are quite old, an anticipate serving until they are quite old if they are younger. In the mid-1970s justice William O. Douglas had a stroke and was basically not mentally competent to serve. Because of this fact, and Douglas’ reluctance to retire his fellow justices basically did not take his vote into account. Three of the justices today are over the age of 70, with Clarence Thomas nearing that age, and two are over the age of 80.

When it comes to Congress, or even the President, there seems to be some sort of institutional support as well as the larger collective vote in the case of Congress, which might buffer the cognitive impact of a gerontocracy. But aside from law clerks Supreme Court justices have to rely on their own individual mental capacities.

The Mormon Church has a gerontocracy among its we openleadership. Even my most devout friends in the church sometimes found it amusing how old their leadership was, and how quickly they died in succession due to the seniority principle. But The Supreme Court is not the leadership of a relatively small church. It impacts our whole nation. This sort of gerontocracy is no laughing matter.

Will we openly speak of the age issue? I doubt it. Today the Baby Boomers are between the ages of 53 an 71. They are coming into their own as a cohort into the highest reaches of the gerontocracy. If there is any generation with the grace and humility to step aside for the greater good, it will not be this generation.

October 30, 2012

Which results from cognitive psychology are robust & real?

Filed under: Cognitive Psychology,Psychology — Razib Khan @ 8:11 pm

A paper on the psychology of religious belief, Paranormal and Religious Believers Are More Prone to Illusory Face Perception than Skeptics and Non-believers, came onto my radar recently. I used to talk a lot about the theory of religious cognitive psychology years ago, but the interest kind of faded when it seemed that empirical results were relatively thin in relation to the system building (Ara Norenzayan’s work being an exception to this generality). The theory is rather straightforward: religious belief is a naturally evoked consequence of the general architecture of our minds. For example, gods are simply extensions of persons, and make natural sense in light of our tendency to anthromorphize the world around us (this may have had evolutionary benefit, in that false positives for detection of other agents was far less costly than false negatives; think an ambush by a rival clan).*

 

But enough theory. Are religious people cognitively different from those who are atheists? I suspect so. I speak as someone who never ever really believed in God, despite being inculcated in religious ideas from childhood. By the time I was seven years of age I realized that I was an atheist, and that my prior “beliefs” ...

September 7, 2012

Who believes in I.Q.?

Filed under: Psychology — Razib Khan @ 12:49 am

There are many things that a given individual believes which are ‘heterodox’ in their social circle. For example, I have long thought that intelligence tests are predictive of life outcomes, and somewhat heritable in a genetic sense (these are both true, the objection of skeptics usually rests on the fact that they are skeptical of the construct itself). As I have explained here before I did not always hold to these views. Rather, when I was in seventh grade a teacher who mentored me somewhat took me aside after class, and suggested that perhaps some of my slower classmates were not quite as lazy as I obviously presumed (I tended to get impatient during mandatory group projects). When I was 5 years old and starting kindergarten my command of English was rather weak, and my mother explained to me that Americans were a very smart people. By the end of the year I was excelling. Throughout my elementary school years I frankly had a smugness about me, because I accepted what my parents told me, that academic outcome is a function of the virtue of effort. And I had quite a bit of virtue if the results were any gauge.

But ...

July 19, 2012

Cousin marriage can reduce I.Q. a lot

Filed under: I.Q.,Psychology — Razib Khan @ 11:33 pm

In light of the previous post I was curious about the literature on inbreeding depression of IQ. A literature search led me to conclude two things:

- This is not a sexy field. A lot of the results are old.

- The range in depression for first cousin marriages seems to be on the order of 2.5 to 10 IQ points. In other words ~0.15 to ~0.65 standard deviation units of decline in intelligence.

The most extreme case was this paper from 1993, Inbreeding depression and intelligence quotient among north Indian children. The authors compared the children of first cousin marriages, and non-bred in individuals, from a sample of Muslims in Uttar Pradesh of comparable socioeconomic status (though the authors note that inbreeding has a positive correlation with socioeconomic status in this community). A table with results speaks for itself:


May 29, 2012

Reason: the God that fails, but we keep socially promoting….

Filed under: Anthroplogy,Cognitive Science,Psychology — Razib Khan @ 12:03 pm

One point which I’ve made on this weblog several times is that on a whole range of issues and behaviors people simply follow the consensus of their self-identified group. This group conformity probably has deep evolutionary origins. It is often much cognitively “cheaper” to simply utilize a heuristic “do what my peers do” than reason from first principles. The “wisdom of the crowds” and “irrational herds” both arise from this dynamic, positive and negative manifestations. The interesting point is that from a proximate (game-theoretic rational actor) and ultimate (evolutionary fitness) perspective ditching reason is often quite reasonable (in fact, it may be the only feasible option if you want to “understand,” for example, celestial mechanics).


If you’re faced with a complex environment or set of issues “re-inventing the wheel” is often both laborious and impossible. Laborious because our individual general intelligence is simply not that sharp. Impossible because most of us are too stupid to do something like invent calculus. Many people can learn the rules for obtaining derivatives and integrals, but far fewer can come up with the fundamental theorem of calculus. Similarly, in the 18th century engineers who utilized Newtonian mechanics for practical purposes were not capable ...

Reason: the God that fails, but we keep socially promoting….

Filed under: Anthroplogy,Cognitive Science,Psychology — Razib Khan @ 12:03 pm

One point which I’ve made on this weblog several times is that on a whole range of issues and behaviors people simply follow the consensus of their self-identified group. This group conformity probably has deep evolutionary origins. It is often much cognitively “cheaper” to simply utilize a heuristic “do what my peers do” than reason from first principles. The “wisdom of the crowds” and “irrational herds” both arise from this dynamic, positive and negative manifestations. The interesting point is that from a proximate (game-theoretic rational actor) and ultimate (evolutionary fitness) perspective ditching reason is often quite reasonable (in fact, it may be the only feasible option if you want to “understand,” for example, celestial mechanics).


If you’re faced with a complex environment or set of issues “re-inventing the wheel” is often both laborious and impossible. Laborious because our individual general intelligence is simply not that sharp. Impossible because most of us are too stupid to do something like invent calculus. Many people can learn the rules for obtaining derivatives and integrals, but far fewer can come up with the fundamental theorem of calculus. Similarly, in the 18th century engineers who utilized Newtonian mechanics for practical purposes were not capable ...

May 16, 2012

Bias in psychology

Filed under: Psychology,science — Razib Khan @ 9:46 pm

Ed Yong has a piece in Nature on the problems of confirmation bias and replication in psychology. Yong notes that “It has become common practice, for example, to tweak experimental designs in ways that practically guarantee positive results.” The way this has been explained to me is that you perform an experiment, get a p-value of > 0.05 (significance). You know that your hunch is warranted, so just modulate the experiment, and hope that the p-value comes in at < 0.05, and you have publishable results! Obviously this is not just a problem in psychology; John Ioannidis has famously focused on medicine. But here’s a chart which shows that positive results are particular prevalent in psychology:

There are many angles to this story, but one which Ed did not touch upon is the political homogeneity of of psychology as a discipline. The vast majority of psychologists are political liberals. This issue of false positive results being ubiquitous is pretty well known within psychology, so I’m sure that that’s one reason Jonathan Haidt has emphasized the ideological blinders of scholars so much. Let’s assume that the ...

January 5, 2012

The “sex difference factor”?

Filed under: Psychology,Sex Differences — Razib Khan @ 3:10 pm

There’s a new paper in PLoS ONE, The Distance Between Mars and Venus: Measuring Global Sex Differences in Personality*, which suggests that by measuring variation of single observed personality traits researchers are missing larger underlying patterns of difference. The Distance Between Mars and Venus: Measuring Global Sex Differences in Personality:

In conclusion, we believe we made it clear that the true extent of sex differences in human personality has been consistently underestimated. While our current estimate represents a substantial improvement on the existing literature, we urge researchers to replicate this type of analysis with other datasets and different personality measures. An especially critical task will be to compare self-reported personality with observer ratings and other, more objective evaluation methods. Of course, the methodological guidelines presented in this paper can and should be applied to domains of individual differences other than personality, including vocational interests, cognitive abilities, creativity, and so forth. Moreover, the pattern of global sex differences in these domains may help elucidate the meaning and generality of the broad dimension of individual differences known as “masculinity-femininity”…In this way, it will be possible to build a solid foundation for the scientific study of psychological sex differences and their biological and cultural origins.


I’m curious about the reaction of people in psychology to this result. The reason is that I am generally confused or skeptical about measurements of personality difference. I’m not confused or skeptical of differences in personality between individuals or groups. I agree that these exist. I just don’t have a good sense of the informativeness of the measures of difference. People may criticize psychometrics intelligence testing all they want, but at least their methods are relatively clear.

From what I can gather the authors discovered that the differences between sexes on personality were much clearer once you looked for the correlation across numerous single measured traits. This strikes me as similar to what you see in population genetics when you move from variation in one gene across populations to many. While a single gene is not very informative in terms of population differences (e.g., the standard assertion that ~15 percent of variation is between races), synthesizing the variation of many genes allows one to easily distinguish populations, because there is such strong discordance in the correlation of differences. An analogy with traits makes understanding this easy. If you were told that population X tended toward black hair, that would not be very informative. Nor if you were told that population X tended toward straight hair. And what if you were told that population X tended toward light skin? All these traits are common across many different populations. But if you told that population X tended toward straight black hair and light skin, the set of populations which intersect at those three traits together in this direction is far smaller than evaluating on a trait-by-trait basis.

But in regards to the evolution of sex differences there is something that I feel that I can say here. Humans seem to lay between other ape lineages in terms of physical dimorphism. For example, in size the difference between males and females is not as extreme as gorillas, but not as equitable as among gibbons. These differences are traditionally correlated with social structure. Groillas are highly polygynous, and there is a great deal of male-male competition, therefore driving sexual selection. In contrast, gibbons tend toward monogamy (at least in the ideal, as with “monogamous birds” the reality seems to differ from the ideal).

But there is also an evolutionary genetic aspect to sexual dimorphism we must consider: in Genetics and Analysis of Quantitative Traits the authors note that evolution of sex specific traits is not going to occur fast. The reason is simple: aside from the peculiarities on the sex chromosomes males and females are genetically the same. This implies that sex differences on the genetic level may emerge via modulation of gene expression across networks of genes tuned by some “master controllers” associated with differential sex development. All of this added complexity takes time to evolve, with the rough result that sexual differences in trait value take about an order of magnitude longer than other traits to come to the fore. The intuition here is simple: if there is selection for large males, there will be selection for large daughters indirectly. Modifiers which dampen this effect need to emerge, so that sex-specific selection doesn’t have the side effect of dragging the other sex along in terms of trait value (this is a concern when you have traits, such as high testosterone, which might increase fitness in males, but reduce it their daughters). Therefore, if there are sex differences in behavioral tendencies which are biologically rooted (I believe there), they will tend to be universal across human societies and have a very deep evolutionary history.

So that would be the strategy to understand differences in personality across the sexes. Go beyond W.E.I.R.D. populations, as they did in this study. And look for traits where males and females seem to exhibit consistent differences across these range of social environments. I suspect environment does effect the magnitude of differences, but I would be willing to bet money that some differences are going to persist (e.g., inter-personal violence is an area where males will differ due to size and personality).

* I’m really sick of the use of the Mars vs. Venus dichotomy in the scholarship.

January 1, 2012

Too smart to be a good cop

Filed under: IQ,Psychology — Razib Khan @ 3:35 pm

Several readers have pointed me to this amusing story, Court OKs Barring High IQs for Cops:

A man whose bid to become a police officer was rejected after he scored too high on an intelligence test has lost an appeal in his federal lawsuit against the city.

“This kind of puts an official face on discrimination in America against people of a certain class,” Jordan said today from his Waterford home. “I maintain you have no more control over your basic intelligence than your eye color or your gender or anything else.”

Jordan, a 49-year-old college graduate, took the exam in 1996 and scored 33 points, the equivalent of an IQ of 125. But New London police interviewed only candidates who scored 20 to 27, on the theory that those who scored too high could get bored with police work and leave soon after undergoing costly training.

The average score nationally for police officers is 21 to 22, the equivalent of an IQ of 104, or just a little above average.

But the U.S. District Court found that New London had “shown a rational basis for the policy.” In a ruling dated Aug. 23, the 2nd Circuit agreed. The court said the policy might be unwise but was a rational way to reduce job turnover.


First, is the theory empirically justified? If so, I can see where civil authorities are coming from. That being said, it’s obvious that there are some areas where “rational discrimination” is socially acceptable, and others where it is not. The same arguments used to be applied to women, in terms of the actuarial probabilities that they would get pregnant and so have to leave the workforce. And disparate impact always looms large in the utilization of these sorts of tests.

Second, can’t you just fake a lower score on an intelligence test? Do police departments hire statisticians to smoke out evidence of conscious selection of incorrect scores? I doubt it. Jordan may be smart, but perhaps he lacks common sense if the upper bound for IQ was well known.

My initial thought was that an IQ of 104 seemed too low for a median police officer, but poking around it does seem plausible as a descriptive statistic. Honestly I don’t have much acquaintance with the police, so I’ll trust the scholars no this. That being said, is it in our social interest for police officers to be so average? I don’t know. Though is it in the social interest that someone with an IQ as high as Robert Jordan’s ends up a prison guard?

December 27, 2011

Richard Feynman’s intelligence

Filed under: Psychology,Richard Feynman — Razib Khan @ 1:32 am

Interesting interview of Steve Hsu. I’ll reproduce the part about Feynman:

3. Is it true Feynman’s IQ score was only 125?

Feynman was universally regarded as one of the fastest thinking and most creative theorists in his generation. Yet it has been reported-including by Feynman himself-that he only obtained a score of 125 on a school IQ test. I suspect that this test emphasized verbal, as opposed to mathematical, ability. Feynman received the highest score in the country by a large margin on the notoriously difficult Putnam mathematics competition exam, although he joined the MIT team on short notice and did not prepare for the test. He also reportedly had the highest scores on record on the math/physics graduate admission exams at Princeton. It seems quite possible to me that Feynman’s cognitive abilities might have been a bit lopsided-his vocabulary and verbal ability were well above average, but perhaps not as great as his mathematical abilities. I recall looking at excerpts from a notebook Feynman kept while an undergraduate. While the notes covered very advanced topics for an undergraduate-including general relativity and the Dirac equation-it also contained a number of misspellings and grammatical errors. I doubt Feynman cared very much about such things.


One thing I have always wondered about is the fact that Richard Feynman had substantive accomplishments which marked him as definitively brilliant by the time he was talking about his 125 I.Q. score (which is smart, but not exceedingly smart). Intelligence scores are supposed to be predictors of accomplishments, but Feynman already had those accomplishments. Bright people take many psychometric tests, so there will be a range of score about a mean. My personal experience is that there’s a bias in reporting the highest scores. But it may be that Feynman gloried in reporting his lowest scores because that made his accomplishments even more impressive. Unlike most he had nothing to prove to anyone.

December 26, 2011

The aliens among us

Filed under: Asperger syndrome,Psychology — Razib Khan @ 3:48 pm

Amy Harmon has a very long piece in The New York Times, Navigating Love and Autism. It’s about a couple who both have been diagnosed with Asperger syndrome. Like cancer I suspect that this term brackets a lot of different issues into one catchall label, not to mention the acknowledgment that it’s a spectrum. When I spent time with the Bay Area Less Wrong community I would observe the range in tendencies and neurological diversity of people who clearly would be classified as “high functioning autistic” (to be clear, these were individuals strongly selected for high general intelligence, with a minimum threshold of around two standard deviations above the norm). The lack of comprehension of religiosity and bias toward libertarianism were two salient characteristics of this sect (though people who have met me don’t classify me as having Asperger syndrome, I have these two cognitive biases myself)

 

In any case, the bigger issue which Amy Harmon’s piece brought out to me is that people with high-functioning autism develope their own micro-norms, meaning that they are often not very compatible with each other despite their deviation from “neurotypicals.” There’s no guarantee that you’ll deviate away from the norm in the same dimension when the norm is highly multidimensional!

People with Asperger are often non-conformists. This is not a bad thing necessarily, at least for society as a whole. But as explained in Not by Genes Alone: How Culture Transformed Human Evolution a very strong tendency toward within-group conformity is a major hallmark in human behavior. It’s probably biological encoded. So, for example, speaking with your parents’ accents, as opposed to that of your hypothetical peer group, is a trait of many people with high functioning autism (or, a tendency toward hyper-formalism of speech). This is a tell for lack of group conformity. The problems autistic people have with conventional “manners,” and not just basic universal human niceties, is an outgrowth of this tendency I suspect. Manners can differ greatly across societies, and require cultural conditioning. But the human tendency to want some set of regular norms does apply to those with Asperger. The diversity among this set is what results in the difficulties of negotiating conflicts (and may explain a bit why libertarians and the hyper-atheistic tend to fracture along what seem trivial deviations from the outside!). You can imagine that in some ways people with Asperger syndrome explore the full parameter space of cultural possibilities, unencumbered by the positive feedback loops of group conformity which is the human norm.

December 11, 2011

Most Reported Genetic Associations with General Intelligence Are Probably False Positives

Filed under: Behavior Genetics,Genomics,Human Genomics,I.Q.,Psychology — Razib Khan @ 4:49 pm

The title says it all, and I yanked it from a paper that is now online (and free). It’s of interest because of its relevance to the future genetic understanding of complex cognitive and behavioral traits. Here’s the abstract:

General intelligence (g) and virtually all other behavioral traits are heritable. Associations between g and specific single-nucleotide polymorphisms (SNPs) in several candidate genes involved in brain function have been reported. We sought to replicate published associations between 12 specific genetic variants and g using three independent, well-characterized, longitudinal datasets of 5571, 1759, and 2441 individuals. Of 32 independent tests across all three datasets, only one was nominally significant at the p ~ .05 level. By contrast, power analyses showed that we should have expected 10–15 significant associations, given reasonable assumptions for genotype effect sizes. As positive controls, we confirmed accepted genetic associations for Alzheimer disease and body mass index, and we used SNP-based relatedness calculations to replicate estimates that about half of the variance in g is accounted for by common genetic variation among individuals. We conclude that different approaches than candidate genes are needed in the molecular genetics of psychology and social science.


My hunch is that these results will be unsatisfying to many people. The authors confirm and reassert the heritability of general intelligence, both by reiterating classical results, and utilizing novel genomic techniques. But, they also suggest that the candidate gene literature is nearly worthless because of the lack of power of most of the earlier studies. The latter is probably due to the genetic architecture of the trait. Intelligence may be determined by numerous genes of very small effect (e.g., 0.01% of the variance effected by one particular SNP), or, “rare, perhaps structural, genetic variants with modest to large effect sizes.” The former case is pretty obvious, but what about the latter? I’m mildly skeptical of this because I’m curious why modest-to-large effect variants didn’t show up in family-based studies (presumably within the family the same variants would localize to sections of the genetic map)? But I’m not fluent enough in the literature to know if there was a lot of work in this area with families previously.

Related: Here’s the first author’s article in Commentary from the late 1990s, IQ Since “The Bell Curve”.

November 28, 2011

The New York Times on violence and Pinker

The New York Times has a short piece on Steven Pinker up. Nothing too new to long time followers of the man and his work. I would like to point readers to the fact that Steven Pinker has a F.A.Q. up for The Better Angels of Our Nature: Why Violence Has Declined. He links to my post, Relative angels and absolute demons, as supporting his dismissal of Elizabeth Kolbert’s review in The New Yorker. I have to admit that I find much, though not all, of the coverage of science in The New Yorker to exhibit some of the more annoying stereotypical caricatures of humanists when confronting the specter of natural philosophy.

I should also mention I started reading The Better Angels of Our Nature over Thanksgiving. I’m only ~20% through it, and probably won’t finish until Christmas season gets into high gear, but so far it’s a huge mess. In both a good way, and a bad way. The good way is that it’s incredibly rich in its bibliography, with fascinating facts strewn about the path of the narrative. The bad way is that so far it lacks the tightness of  The Blank Slate or The Language Instinct in terms of argument. This may change. Finally, I think I should mention that Pinker has already addressed some of the criticisms of his methodologies brought up in the comments sections of my posts. Those who have specific critiques probably should read the book, because he seems to try sincerely to address those. Or at least they should address those critiques to people who have bothered to read the book.

November 10, 2011

The problem of false positives

Filed under: Psychology — Razib Khan @ 7:01 pm

False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant:

In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.

Since the paper is behind a paywall, I’ve cut & pasted the solutions belows:

We propose the following six requirements for authors.

  1. Authors must decide the rule for terminating data collection before data collection begins and report this rule in the article. Following this requirement may mean reporting the outcome of power calculations or disclosing arbitrary rules, such as “we decided to collect 100 observations” or “we decided to collect as many observations as we could before the end of the semester.” The rule itself is secondary, but it must be determined ex ante and be reported.

  2. Authors must collect at least 20 observations per cell or else provide a compelling cost-of-data-collection justification. This requirement offers extra protection for the first requirement. Samples smaller than 20 per cell are simply not powerful enough to detect most effects, and so there is usually no good reason to decide in advance to collect such a small number of observations. Smaller samples, it follows, are much more likely to reflect interim data analysis and a flexible termination rule. In addition, as Figure 1shows, larger minimum sample sizes can lessen the impact of violating Requirement 1.

  3. Authors must list all variables collected in a study. This requirement prevents researchers from reporting only a convenient subset of the many measures that were collected, allowing readers and reviewers to easily identify possible researcher degrees of freedom. Because authors are required to just list those variables rather than describe them in detail, this requirement increases the length of an article by only a few words per otherwise shrouded variable. We encourage authors to begin the list with “only,” to assure readers that the list is exhaustive (e.g., “participants reported only their age and gender”).

  4. Authors must report all experimental conditions, including failed manipulations. This requirement prevents authors from selectively choosing only to report the condition comparisons that yield results that are consistent with their hypothesis. As with the previous requirement, we encourage authors to include the word “only” (e.g., “participants were randomly assigned to one of only three conditions”).

  5. If observations are eliminated, authors must also report what the statistical results are if those observations are included. This requirement makes transparent the extent to which a finding is reliant on the exclusion of observations, puts appropriate pressure on authors to justify the elimination of data, and encourages reviewers to explicitly consider whether such exclusions are warranted. Correctly interpreting a finding may require some data exclusions; this requirement is merely designed to draw attention to those results that hinge on ex post decisions about which data to exclude.

  6. If an analysis includes a covariate, authors must report the statistical results of the analysis without the covariate. Reporting covariate-free results makes transparent the extent to which a finding is reliant on the presence of a covariate, puts appropriate pressure on authors to justify the use of the covariate, and encourages reviewers to consider whether including it is warranted. Some findings may be persuasive even if covariates are required for their detection, but one should place greater scrutiny on results that do hinge on covariates despite random assignment.

Guidelines for reviewers

We propose the following four guidelines for reviewers.

  1. Reviewers should ensure that authors follow the requirements. Review teams are the gatekeepers of the scientific community, and they should encourage authors not only to rule out alternative explanations, but also to more convincingly demonstrate that their findings are not due to chance alone. This means prioritizing transparency over tidiness; if a wonderful study is partially marred by a peculiar exclusion or an inconsistent condition, those imperfections should be retained. If reviewers require authors to follow these requirements, they will.

  2. Reviewers should be more tolerant of imperfections in results. One reason researchers exploit researcher degrees of freedom is the unreasonable expectation we often impose as reviewers for every data pattern to be (significantly) as predicted. Underpowered studies with perfect results are the ones that should invite extra scrutiny.

  3. Reviewers should require authors to demonstrate that their results do not hinge on arbitrary analytic decisions. Even if authors follow all of our guidelines, they will necessarily still face arbitrary decisions. For example, should they subtract the baseline measure of the dependent variable from the final result or should they use the baseline measure as a covariate? When there is no obviously correct way to answer questions like this, the reviewer should ask for alternatives. For example, reviewer reports might include questions such as, “Do the results also hold if the baseline measure is instead used as a covariate?” Similarly, reviewers should ensure that arbitrary decisions are used consistently across studies (e.g., “Do the results hold for Study 3 if gender is entered as a covariate, as was done in Study 2?”).5 If a result holds only for one arbitrary specification, then everyone involved has learned a great deal about the robustness (or lack thereof) of the effect.

  4. If justifications of data collection or analysis are not compelling, reviewers should require the authors to conduct an exact replication. If a reviewer is not persuaded by the justifications for a given researcher degree of freedom or the results from a robustness check, the reviewer should ask the author to conduct an exact replication of the study and its analysis. We realize that this is a costly solution, and it should be used selectively; however, “never” is too selective.

To preempt angry and offended psychology professors: this problem is not limited to their discipline. It is probably a bigger problem in medicine because it costs us a lot of money and likely kills people.

October 31, 2011

Social psychology: like medical science (except it doesn’t kill you)

Filed under: Diederik Stapel,Psychology — Razib Khan @ 11:13 pm

I was warned off social psychology years ago by a friend (who was a research psychologist) because of the field’s propensity for ‘sexy’ results which get a lot of media play. As a lay person he doubted I could tell the fake from the reliable, the one off from the replicable. Later someone else told me about how dispiriting it was engaging in data dredging to find something publishable in a lab where they were working, even though many of the principals involved obviously assumed that the results weren’t robust. Keep that in mind when you read about the mind-boggling scientific fraud perpetrated by Diederik Stapel. It looks like he might be the Bernie Madoff of psychology. The Lehman Brothers to the Long Term Capital Management of Marc Hauser. Here’s a taste:

Stapel’s work encompassed a broad range of attention-catching topics, including the influence of power on moral thinking and the reaction of psychologists to a plagiarism scandal. The committee, which interviewed dozens of Stapel’s former students, postdoctoral researchers, co-authors, and colleagues, found that Stapel alone was responsible for the fraud. The panel reported that he would discuss in detail experimental designs, including drafting questionnaires, and would then claim to conduct the experiments at high schools and universities with which he had special arrangements. The experiments, however, never took place, the universities concluded. Stapel made up the data sets, which he then gave the student or collaborator for analysis, investigators allege. In other instances, the report says, he told colleagues that he had an old data set lying around that he hadn’t yet had a chance to analyze. When Stapel did conduct actual experiments, the committee found evidence that he manipulated the results.

Here’s the controversial thing I’m going to say: instances of flamboyant fraud are probably far less of an issue in a discipline like social psychology than more subtle biases and systematic incentives. Similarly, someone like Bernie Madoff was notable for the havoc he caused on an individual level, but he paled in the aggregate against the death by a thousand cuts that occurred as the housing market collapsed under the pall of modest “liar loans” and the like. And this isn’t just a problem with science. ‘Sexy’ social psychological research is routinely blasted by the press, and devoured by the public. Incentives matter, and Diederik Stapel is just the reductio ad absurdum of scientists who respond to the repeated 15 minutes of fame which the press provides. Most of the time it’s not so blatant, nor frankly so malicious in consequence. I doubt that there are many Diederik Stapel’s in the field, else he wouldn’t have gotten so far with bluster and bluff.

September 25, 2011

Like modestly attracts like

Filed under: I.Q.,marriage,Psychology,Sociology — Razib Khan @ 11:04 pm

I saw this link posted on twitter, IQ and Human Intelligence:

An interesting finding from genetic research, which Mackintosh mentions, only in passing, as posing a problem in the estimation of the heritability of g, is that there is greater assortative mating for g than for any other behavioral trait; that is, spouse correlations are only ∼.1 for personality and only ∼.2 for height or weight, but the correlation for assortative mating for g is ∼.4. In addition to indicating that people are able to make judgments about g in real life, this finding suggests that assortative mating may contribute to the substantial additive genetic variance for g, because positive assortative mating for a character can increase its additive genetic variance.

I’ve seen these sort of results before. The review is from 1999. In general I always wonder if quantitative values for personality are not to be trusted because of issues with the measurement of personality types. But this is clearly not an issue with height or weight. And in the case of height the overwhelming causal explanation for variation in the West is genetic variation. Overall I’m rather surprised by the rather low correlations for some of these traits, such as height and intelligence. I wonder if beauty, perhaps measured by an index of facial symmetry, might exhibit higher correlation values?

September 20, 2011

God is intuitive

Filed under: Anthroplogy,atheism,Cognitive Science,Psychology,Religion — Razib Khan @ 10:15 pm

Update: An ungated version of the paper.

I used to spend a lot more time talking about cognitive science of religion on this weblog. It was an interest of mine, but I’ve come to a general resolution of what I think on this topic, and so I don’t spend much time discussing it. But in the comments below there was a lot of fast & furious accusation, often out of ignorance. I personally find that a little strange. I’ve been involved in freethought organizations in the past, and so have some acquaintance with “professional atheists.” Additionally, I’ve also been a participant and observer of the internet freethought websites since the mid-1990s (yes, I remember when alt.atheism was relevant!). In other words, I know of whom I speak (and I am not completely unsympathetic to their role in the broader ecology of ideas).

But the bigger issue is a cognitive model of how religiosity emerges. Luckily for me a paper came out which speaks to many of the points which I alluded to, Divine intuition: Cognitive style influences belief in God:

Some have argued that belief in God is intuitive, a natural (by-)product of the human mind given its cognitive structure and social context. If this is true, the extent to which one believes in God may be influenced by one’s more general tendency to rely on intuition versus reflection. Three studies support this hypothesis, linking intuitive cognitive style to belief in God. Study 1 showed that individual differences in cognitive style predict belief in God. Participants completed the Cognitive Reflection Test (CRT; Frederick, 2005), which employs math problems that, although easily solvable, have intuitively compelling incorrect answers. Participants who gave more intuitive answers on the CRT reported stronger belief in God. This effect was not mediated by education level, income, political orientation, or other demographic variables. Study 2 showed that the correlation between CRT scores and belief in God also holds when cognitive ability (IQ) and aspects of personality were controlled. Moreover, both studies demonstrated that intuitive CRT responses predicted the degree to which individuals reported having strengthened their belief in God since childhood, but not their familial religiosity during childhood, suggesting a causal relationship between cognitive style and change in belief over time. Study 3 revealed such a causal relationship over the short term: Experimentally inducing a mindset that favors intuition over reflection increases self-reported belief in God.

Recall that in many social domains where neurotypicals rely on innate, intuitive, and “fast” cognition, high functioning autistic individuals must reflect and reason. I don’t have access to the original paper, but there’s a nice piece in Harvard Gazette on the research. Here’s the last sentence: ““How people think about tricky math problems is reflected in their thinking — and ultimately their convictions — about the metaphysical order of the universe,” Shenhav said.”

August 18, 2011

There would have been a Plato without Plato

Filed under: Idealism,Platonism,Psychology — Razib Khan @ 3:20 pm

In the comments of my post “Platonism is useful only when it’s useful” several people made a few references to Plato, as well as Platonism. That is fair and makes sense. And there’s a deep strain of anti-Plato sentiment amongst respectable people (e.g., Karl Popper). I assume of the two ancient Greek philosophers of renown, Plato and Aristotle, most readers would be more sympathetic to the latter. He may have gotten a lot wrong, but Aristotle’s more empirical bent is probably more congenial to many moderns than Plato’s greater reliance on abstract theory.

But I don’t think that we can put Platonism at the feet of Plato. There are deep human intuitions about the nature of reality, which Plato and his followers systematized at an early date. But this systematization would have happened at some point in history, and we would have termed it by some name, which would be reviled and lauded by intellectual partisans of a later age.

Rather than inveigh against Plato, we need to be cognizant and aware of our own biases. The human cognitive toolkit seems to be a faulty statistical machine, and that’s because it leans strongly on ideals. Archetypes, prototypes, whatever ...

August 10, 2011

Half the variation in I.Q. due to variation in genes

A new paper in Molecular Psychiatry has been reported on extensively in the media, and readers have mentioned it several times in the comments. I read it. It’s titled Genome-wide association studies establish that human intelligence is highly heritable and polygenic. But the fact is that I read this paper last year. Back then it was titled Common SNPs explain a large proportion of the heritability for human height. I kid, but you get the picture. The new paper establishes for intelligence what we already suspected: most of the genetic variation in this heritable trait is accounted for by numerous genes of small effect. You inherit variants of these numerous genes from your two parents, and your own trait value is to a large extent a combination of the parental values. The issue is not if intelligence is heritable, but the extent of that heritability.


The standard way to estimate human heritability was to track similarities across individuals with varying degrees of relatedness. For example, compare identical twin correlations on a trait with fraternal twin correlations. The main objection to these methods is that one could argue ...

July 11, 2011

Blank slate when you want it that way

Filed under: Behavior Genetics,Genetics,Heritability,homosexuality,Psychology — Razib Khan @ 10:44 am

Tim Pawlenty debates Lady Gaga’s ‘Born This Way’ idea:

Gregory pressed, asking “Is being gay a choice?”

Pawlenty ultimately said, “I defer to the scientists in that regard.”

Again, Gregory pressed: “So you, you think it’s not a choice. … That you are, as Lady Gaga says, you’re born that way.”

Said Pawlenty: “There’s no scientific conclusion that it’s genetic. We don’t know that. So we don’t know to what extent, you know, it’s behavioral, and that’s something that’s been debated by scientists for a long time. But as I understand the science, there’s no current conclusion that it’s genetic.”


This is one issue where the American Left has a tendency to be on the side of the hereditarians. In contrast, the American Right emphasizes the plasticity of human behavior, and its amenability to cultural pressures and individual will and contingency. Transpose the structure of the arguments to male-female sex differences, and many of the basic elements would be preserved, but those espousing them would invert politically.

One issue which needs to be clarified is the distinction between something which is explainable by genetics, and something which is not explainable by genetics but may still have a biological basis. It does seem that

Older Posts »

Powered by WordPress