Nature paper flags 504 label flaws across 449 psychology scales

Across 449 personality scales, a semantic-embedding audit flagged 504 label errors: 340 'jingle' fallacies, where different measures share a construct name, and 164 'jangle' fallacies, where one measure runs under different names. The Nature Human Behaviour paper by Dirk U. Wulff and Rui Mata, published in March 2025, fed 4,452 items from the International Personality Item Pool into a fine-tuned MPNet embedding model trained on 200,000 personality item pairs.

The predictions held up. Across 449 scales the method predicted Cronbach's α at r = 0.75 in-sample and r = 0.61 out-of-sample. Across 443 scales it fully recovered factor structure for 81% in-sample and 79% out-of-sample.

Collapsing 254 original construct labels into 68 cut jingle-jangle overlaps by 58%.

The authors frame the problem as 'difficulty comparing scientific theories due to different concept uses and operationalizations', the taxonomic incommensurability of the title. Their embeddings, they write, 'quantify similarity between construct labels and associated measures.' Validation also ran against the Open-Source Psychometrics Project (16PF, BIG5, FFM, HEXACO), a NEO inventory assessment of 212,625 US respondents, and the Eugene-Springfield Community Sample.

The paper surfaced on our radar via two tracked researchers who shared it.