pilot test: Semantic Similarity and Item Correlation

First, the outrageous figure: $r = .620, p < .001$

Although this figure doesn't explain much, it's shocking enough. Because the absolute value was taken directly, rather than negated based on opposite semantics.

Algorithm

Git

Use Embedding to calculate cosine similarity, call numpy to calculate Pearson $r$ of variables.

For each pair of items, output a point (similarity, correlation) for regression analysis.

If no special processing is done, it is easy to find that when similarity is high, the absolute value of correlation is high. Consider manually flipping reverse items.

This is the normal figure: $r = .305, p<.001$

Next, plan to finish this dataset and then take a look. Then merge other datasets.

Data

The data was found online: Answers to Cattell's 16 Personality Factors Test with items from the IPIP. https://openpsychometrics.org/_rawdata/

According to the codebook, subscales A, B, and C were selected for computation. Not many subscales were selected, but with about fifty thousand data points, the computational load is not small either.