Human Varieties

Category: Psychometrics (Page 1 of 4)

Controversy over the predictive validity of IQ on job performance

June 1, 2024 / / 4 Comments

Sackett et al. (2022) recently questioned prior meta-analytic conclusions about the high IQ validity since the studies by Schmidt & Hunter decades ago. The crux of the issue is complex, but while the debate regarding range restriction correction is not (at least not completely) resolved yet, one thing is certain. The validity depends on the measure of job performance that is used in the meta-analysis. Maybe the importance of IQ has declined in recent years but the debate is not settled yet.

(updated: August 2nd 2024)

CONTENT
1. The groundbreaking study
2. The controversy
3. The choice of criterion (dependent) variable matters

Continue reading

How IQ became less important than personality: A critical examination of Borghans et al. (2016)

May 7, 2024 / / 0 Comments

Borghans et al. (2016) analyze 4 datasets with diverse measures of IQ and, shockingly, concluded that the impact of IQ on social outcomes is weak compared to personality measures, despite what the earlier reviews and meta-analyses showed (Gottfredson, 1997; Poropat, 2009; Schmidt & Hunter, 2004). Indeed, as reviewed prior, most studies found that personality measures generally have weak relationship with outcomes once IQ is accounted for. Yet their work has not been subjected to critical examination, just various uninteresting comments (Ganzach & Zisman, 2022; Golsteyn et al., 2022; Stankov, 2023) and replication failures (Zisman & Ganzach, 2022).

Continue reading

The Structure of Well Designed Online IQ Tests

February 23, 2024 / / 5 Comments

There are convenient ways researchers can collect IQ scores and correlate the observed scores with measures of self-reported health, socio-economic attainment, personality or political views. In platforms such as Prolific or MTurk, participants make money in their spare time by completing tasks. Designing a test that displays both a high loading on the general factor of intelligence, while avoiding measurement bias and bad quality data from online participants, is quite a challenging task.

(update: July 19th 2024)

CONTENT

  1. Introduction page content
  2. Item’s pass rate and g-loading
  3. Lazy and dishonest test takers
  4. Short versus long test
  5. Scrolling dilemma
  6. Item type “write-in”
  7. Instruction and rules
  8. Cultural content and cultural bias
  9. Computerized Ability Test

The issues related to online testing are illustrated based on the numerous IQ tests Jurij Fedorov devised, with my assistance, using Alchemer’s professional software.

Continue reading

Spearman’s g Explains Black-White but not Sex Differences in Cognitive Abilities in the Project Talent

September 1, 2023 / / 3 Comments

In this analysis of the Project Talent data, the g factor model as represented by the Spearman’s Hypothesis (SH) was confirmed for the black-white cognitive difference but not for the sex difference. Results from MGCFA and MCV corroborate each other. MGCFA detected small-modest bias with respect to race but strong bias with respect to sex cognitive difference. Full result is available at OSF.

Continue reading

DIF Review and Analysis of Racial Bias in Wordsum Test using IRT and LCA

February 10, 2023 / / 0 Comments

As reviewed in my previous article, the majority of studies on measurement bias, either on the item- or subtest-level, reached an agreement about the fairness of IQ test. Unfortunately, even among studies which use acceptable Differential Item Functioning (DIF) methods, the procedure was often sub-optimal. This probably leads to more spurious DIFs being detected.

The advantages (and shortcomings) of each DIF method are presented. The GSS data is used to compare the performance of the two best DIF methods, namely IRT and LCA, at detecting bias in the wordsum vocabulary test between whites and blacks.
Continue reading

Fair and Square: A Conclusion on IQ Test Bias

January 2, 2023 / / 0 Comments

This is a 2-part article. In this first part, the most important studies on internal test bias with respect to racial groups in the item-level, subtest-level and construct-level are reviewed. The proposed causes will be discussed. Generally, the most commonly used IQ tests aren’t biased or only minimally biased as to be of no practical value.

The best methodologies with an application using the Wordsum GSS for the Black-White group will be discussed in the second part of the article : DIF Review and Analysis of Racial Bias in Wordsum Test using IRT and LCA.
Continue reading

Schooling enhances IQ, not intelligence

December 22, 2022 / / 2 Comments

The idea that schooling raises intelligence still prevails. The influential study review of Ceci (1991) concluded that schooling has a strong impact on IQ scores despite his final warning that observed score does not equate real intelligence. After, many more studies were published, including latent factor modeling and quasi-experimental designs. It is unclear whether education truly improves general intelligence modeled as latent factor or whether long-lasting IQ gain involves far transfer effect. More likely, the answer to all of these questions is negative.

Continue reading

Re-analysis of Willerman’s Study: Race of Mother’s Hypothesis

September 11, 2022 / / 0 Comments

It’s been almost 50 years now that the famous study of Willerman et al. (1974) has been published. This study is regularly cited as one of the most convincing evidence against the hereditarian hypothesis, despite strong emphasis by hereditarians on the failure of experimental efforts to raise IQ (more specifically, g) and population differences magnifying during adolescence or adulthood due to increasing heritability with age (Jensen, 1998, pp. 333-344, 359, 474; See Malloy [2013] for a case of a stability model with respect to the Black-White gap). Caution about this study is now vindicated. The data used by Willerman also revealed a pattern: the IQ deficits related to having a Black mother seem to vanish over time (Hu, 2022). Continue reading

Measurement Error, Regression to the Mean, and Group Differences

July 1, 2017 / / 5 Comments

Regression to the mean, RTM for short, is a statistical phenomenon which occurs when a variable that is in some sense unreliable or unstable is measured on two different occasions. Another way to put it is that RTM is to be expected whenever there is a less than perfect correlation between two measurements of the same thing. The most conspicuous consequence of RTM is that individuals who are far from the mean value of the distribution on first measurement tend to be noticeably closer to the mean on second measurement. As most variables aren’t perfectly stable over time, RTM is a more or less universal phenomenon.

In this post, I will attempt to explain why regression to the mean happens. I will also try to clarify certain common misconceptions about it, such as why RTM does not make people more average over time. Much of the post is devoted to demonstrating how RTM complicates group comparisons, and what can be done about it. My approach is didactic and I will repeat myself a lot, but I think that’s warranted given how often people are misled by this phenomenon.
Continue reading

« Older posts

© 2024 Human Varieties

Theme by Anders Noren — Up ↑