Does the information from Givewell and Glassdoor reviews tell the same story?
By Rachel Ker, Luis Eduardo San Martin and Carla Solis
Short answer: They don't seem to.
We scraped data from the websites Glassdoor and Givewell, and analyzed if the sentiments of the pooled Glassdoor reviews and the Givewell review were correlated. The graphs and the table below summarizes our results. Each axis indicates the proportion of words with a certain sentiment -- positive or negative, according to the graph. For the proportion of positive words graph, the y axis represents the rate of words with a positive sentiment in the pooled Glassdoor review and the x axis represents the rate of words with a positive sentiment in the Givewell review. The proportion of negative words graph has a similar interpretation, but for the rates of words with negative sentiment.
Rate of words with positive sentiment
Rate of words with negative sentiments
We also constructed a sentiment compound score which can be interpreted as a measurement of the general sentiments in a given text. It ranges from -1 to +1, where a value below zero indicates the text has a stronger negative than positive sentiment.
Compound score
The graphs don't show a clear correlation between the sentiments of the reviews from both sources. More interestingly, after getting the correlations we can see they are not statiscally significant. Even though our very small sample (29 organizations) might be affecting significance here, the correlation between the percentage of negative words is very close to zero.
Correlations
Rate of words with positive sentiment: 0.17
Rate of words with negative sentiment: 0.01
Compound score: 0.24
None of them is statistically different than zero.
Which organizations have the biggest review discrepancies?
We also analyzed the discrepancies between the pooled Glassdoor reviews and the Givewell review. Then, we got the difference between those three indicators from the pooled Glassdoor reviews and from the Givewell review, for each organization.
The graphs below summarizes such results by organization. A positive score indicates that the pooled Glassdoor reviews produce a higher result than the Givewell review.
Compound score
Positive words
Negative words
