Category: Psychometrics (Page 1 of 3)

The Structure of Well Designed Online IQ Tests

There are convenient ways researchers can collect IQ scores and correlate the observed scores with measures of self-reported health, socio-economic attainment, personality or political views. In platforms such as Prolific or MTurk, participants make money in their spare time by completing tasks. Designing a test that displays both a high loading on the general factor of intelligence, while avoiding measurement bias and bad quality data from online participants, is quite a challenging task.

CONTENT

  1. Introduction page content
  2. Item’s pass rate and g-loading
  3. Lazy and dishonest test takers
  4. Short versus long test
  5. Scrolling dilemma
  6. Item type “write-in”
  7. Instruction and rules
  8. Cultural content and cultural bias
  9. Computerized Ability Test

The issues related to online testing are illustrated based on the numerous IQ tests Jurij Fedorov devised, with my assistance, using Alchemer’s professional software.

Continue reading

Spearman’s g Explains Black-White but not Sex Differences in Cognitive Abilities in the Project Talent

In this analysis of the Project Talent data, the g factor model as represented by the Spearman’s Hypothesis (SH) was confirmed for the black-white cognitive difference but not for the sex difference. Results from MGCFA and MCV corroborate each other. MGCFA detected small-modest bias with respect to race but strong bias with respect to sex cognitive difference. Full result is available at OSF.

Continue reading

DIF Review and Analysis of Racial Bias in Wordsum Test using IRT and LCA

As reviewed in my previous article, the majority of studies on measurement bias, either on the item- or subtest-level, reached an agreement about the fairness of IQ test. Unfortunately, even among studies which use acceptable Differential Item Functioning (DIF) methods, the procedure was often sub-optimal. This probably leads to more spurious DIFs being detected.

The advantages (and shortcomings) of each DIF method are presented. The GSS data is used to compare the performance of the two best DIF methods, namely IRT and LCA, at detecting bias in the wordsum vocabulary test between whites and blacks.
Continue reading

Fair and Square: A Conclusion on IQ Test Bias

This is a 2-part article. In this first part, the most important studies on internal test bias with respect to racial groups in the item-level, subtest-level and construct-level are reviewed. The proposed causes will be discussed. Generally, the most commonly used IQ tests aren’t biased or only minimally biased as to be of no practical value.

The best methodologies with an application using the Wordsum GSS for the Black-White group will be discussed in the second part of the article : DIF Review and Analysis of Racial Bias in Wordsum Test using IRT and LCA.
Continue reading

Schooling enhances IQ, not intelligence

The idea that schooling raises intelligence still prevails. The influential study review of Ceci (1991) concluded that schooling has a strong impact on IQ scores despite his final warning that observed score does not equate real intelligence. After, many more studies were published, including latent factor modeling and quasi-experimental designs. It is unclear whether education truly improves general intelligence modeled as latent factor or whether long-lasting IQ gain involves far transfer effect. More likely, the answer to all of these questions is negative.

Continue reading

Re-analysis of Willerman’s Study: Race of Mother’s Hypothesis

It’s been almost 50 years now that the famous study of Willerman et al. (1974) has been published. This study is regularly cited as one of the most convincing evidence against the hereditarian hypothesis, despite strong emphasis by hereditarians on the failure of experimental efforts to raise IQ (more specifically, g) and population differences magnifying during adolescence or adulthood due to increasing heritability with age (Jensen, 1998, pp. 333-344, 359, 474; See Malloy [2013] for a case of a stability model with respect to the Black-White gap). Caution about this study is now vindicated. The data used by Willerman also revealed a pattern: the IQ deficits related to having a Black mother seem to vanish over time (Hu, 2022). Continue reading

Measurement Error, Regression to the Mean, and Group Differences

Regression to the mean, RTM for short, is a statistical phenomenon which occurs when a variable that is in some sense unreliable or unstable is measured on two different occasions. Another way to put it is that RTM is to be expected whenever there is a less than perfect correlation between two measurements of the same thing. The most conspicuous consequence of RTM is that individuals who are far from the mean value of the distribution on first measurement tend to be noticeably closer to the mean on second measurement. As most variables aren’t perfectly stable over time, RTM is a more or less universal phenomenon.

In this post, I will attempt to explain why regression to the mean happens. I will also try to clarify certain common misconceptions about it, such as why RTM does not make people more average over time. Much of the post is devoted to demonstrating how RTM complicates group comparisons, and what can be done about it. My approach is didactic and I will repeat myself a lot, but I think that’s warranted given how often people are misled by this phenomenon.
Continue reading

Into the IQ shredder

Wang, M., Fuerst, J., Ren, J. (2016). Evidence of dysgenic fertility in China. Intelligence, 57, 15-24.

From the discussion: “We’ve seen, in Table 4, that urban populations in China exhibited a relatively high dysgenic fertility trend in the 1951–1970 birth cohort. For this same cohort, the trend was much smaller in the rural populations. It suggests that dysgenic selection is related to urbanity. This supports Pan’s (1923) observation that “modern urbanization has had so many dysgenic effects upon the race.”

IQ and Personality: What James Heckman Got Wrong

A few years ago James Heckman, together with some other economists, published a study arguing that “achievement tests” and “IQ tests” are different beasts: the former, they claim, are better predictors of criterion outcomes (such as grade point averages) and are more strongly influenced by personality differences than the latter. Like most of Heckman’s forays into psychometrics — he has been obsessed with trying to shoot down Bell Curve -type arguments ever since the book was released — the study leaves much to be desired. David Salkever has published a nifty reanalysis of Heckman and colleagues’ study, showing that their results stem from faulty imputation and a failure to take into account age effects. Continue reading

« Older posts

© 2024 Human Varieties

Theme by Anders NorenUp ↑