Since the New York Times bought the popular new word game, Wordle, some players have accused the Times of making it too difficult. This is not likely. The entire answer list—before and after the change in ownership—is publicly available and the only modifications the Times has made so far is to remove a handful of words.
But all this speculation raises a question: What is it that makes a Wordle answer hard to guess? Uncommon words? Words with uncommon letters? Or perhaps other factors related to the how the game is played?
A Twitter bot created by Kevin O'Connor offers some clues. The bot has collected and tabulated data on Wordle scores based on Twitter posts since early January.
Because very few people fail at a given Wordle (typically around 3%), I use the percent of people who use all six guesses, whether they win (6/6) or fail (X/6) as a proxy for a word's difficulty. This percentage varies quite a bit, suggesting that some answers are definitely harder than others. The hardest answers so far include SWILL (40% using six guesses), PROXY (32%), SHAKE (30%) and ELDER (29%), while the easiest include THORN (4%), POINT (5%), PANIC (6%) and DRINK (6%).
Some Wordle answers are harder to guess than others
Percent of players using six guesses varies from 4% to 40%
Data source: @WordleStats
Uncommon words
Some have suggested that uncommon words are harder to guess. But the correlation between difficulty and how common the words are only accounts for about 10% of the variance in difficulty.
Weak relationship between word difficulty and word usage
Zipf numbers represent how commonly words are used
Data sources: @WordleStats, word frequencies from Robyn Speer
Other factors
Many attempts to identify the best first guess focus on letter frequencies. If people prioritize more common letters in their guesses it might make words with uncommon letters harder to guess. I calculate Scrabble scores as a summary measure of letter frequencies in each word.
Another possibility is that words with a letter appearing more than once are tricky for guessers since they require foregoing the opportunity to learn about a new letter in order to win.
Finally, another challenge players can face is knowing four letters, but having many options for the remaining letter. This can be the case with a pattern like S?ILL, which represents many potential answers: SHILL, SKILL, SPILL, STILL, SWILL.
I estimate a linear model of these factors on the percent of people who use all six guesses, and they all appear to matter somewhat, altogether accounting for 49% of the variance. A word having two of the same letter results in a 10 percentage point increase in the percent of people using all six guesses. A word having three other common words spelled differently by only one letter implies a 7 percentage point increase. A seven-point increase in Scrabble score (the difference between an X and an E tile) increases the percent of six-guess players by 6 points.
Duplicate letters, similarity to other words and uncommon letters make harder answers
The obscurity of the word appears less important
"Similar to other common words" is an indicator of whether the word shares four letters with a group of 3 other common words (Zipf score greater than 3). Estimates shown for Scrabble score and word obscurity (reversed Zipf score) are standardized. Bars represent 95% confidence intervals.
Data sources: @WordleStats, word frequencies from Robyn Speer
The obscurity of the word appears less important. A word that is 10 times less common than another word (a Zipf score decrease of 1) is only associated with a 1 percentage point increase in people using six guesses.
Biased reporting
One potential problem with the data being used here is that it is self-reported. People are probably more likely to tweet their scores when they do well, and less likely to do so when they do poorly. That means there is probably more variation in people scores than is apparent in the Twitter data.
I test this by decomposing the trend in the total number of players into its seasonal component (day of week effects), its overall trend and the remaining variation. The remaining variation appears to be negatively correlated with the difficulty of the word, suggesting that people do in fact choose to report more often when the words are easier to guess.
An even stronger relationship is apparent between reporting and doing well (winning in 3 guesses or fewer).
This probably shouldn't impact the overall conclusions of what makes words difficult to guess. If anything, it makes it slightly harder to explain why some words are harder than others, since it reduces the variation between easy and hard words in the reported data.
Data Availability
Data used for this post is available for download here.
