We analyze a lot of reddit AMA’s individually to look for factors that caused the AMA to do well or not so well. While this subjective approach has been great for identifying factors that make for good and bad AMA’s, we started to wonder if there was a more objective approach that would allow us to measure what makes for a great AMA.
To get started, we decided to explore a simple question:
Is there a correlation between AMA title/body length and score?
Top AMA’s vs Random AMA’s
The first dataset we looked at is the 917 top scoring reddit AMA’s (posted to r/IAmA) from the past 12 months (as of November 19th, 2014). We figured it would be interesting to see if the most popular AMA’s could tell us something about optimal title and body length.
The second dataset was the 709 most recent reddit AMA’s (posted to r/IAmA). This is a more random set as it includes low and high scoring AMA’s.
Here are the numbers for each dataset side-by-side:
|Top AMA’s||Random AMA’s|
|Avg. Title Length||95.15||94.92|
|Avg. Title Length (middle 80%)||85.44||86.35|
|Avg. Body Length||1192.78||906.95|
|Avg. Body Length (middle 80%)||909.37||723.81|
All length values are in characters
Things to note:
- As expected, the Top AMA’s have a much higher average score than the Random AMA’s.
- The average title length of the Top AMA’s and the Random AMA’s is about the same (95 characters).
- The average body length of the Top AMA’s is longer than the Random AMA’s by about 283 characters. If you look at the middle 80%, which cuts out the 10% longest/shortest bodies, the Top AMA’s average body length is about 183 characters longer.
The only thing interesting here is that the average body length of the Top AMA’s is longer than the Random AMA’s. One possible conclusion is that on average, more planning went into the Top AMA’s than the Random AMA’s and therefore have a longer average body length.
Score vs. Body Length
We decided to take a closer look at the correlation between body length and score. To do so, we plotted the body length and score for the Top AMA’s and the Random AMA’s. Below is what the graph looked like for the Top AMA’s.
As you can see, it is a random cluster (the Random AMA’s dataset produced a similar looking graph). We tried putting a trend line to the data, but the correlation coefficient was near-zero. In other words, the correlation between score and body length is really small and possibly statistically insignificant.
The length of the title or body of an AMA could have a measurable effect on the success of an AMA, but that effect is masked by other more important factors (e.g. the popular demand for the AMA). A celebrity, professional athlete, or relevant expert can all have sucessful AMA’s regardless of the length of their title or body. So to get to the bottom of our original question, we would need to equalize the demand factor, which is not the easiest thing to measure. Even if we were able to do that, we would still only discover a correlation that has very little to do with the success of an AMA.
So finding a correlation between title/body length and score is something we will likely put to bed. In the future, we will look at other ways to analyze reddit AMA’s to see if there is any secret sauce hiding in the numbers.