What’s the Word – Adding Context to Your Rank?

Posted by on November 26, 2012

Browsing Tableau’s Viz of the Day I stumbled upon Ben Jones post on DataRemixed, October 3rd Debate Word Counter. I enjoyed it immediately and appreciated that the pick list defaulted to “teachers”.  I was surprised to learn that Romney said teachers 3.5 times as frequently as Obamas did (7 to 2).

As I scrolled through the words in the viz and discovered that I had questions I couldn’t answer simply using the pick list: What were the Top 10 words used by each candidate and how did they compare?  How did their top ranking words change over the course of the debates?

How do I get from here To here?

Tableau Presidential Debate 2012 Word Count Vis

Ben Jones’s Presidential Debate Word Count Vis

Leigh's Debate Word Count Vis

Leigh’s Presidential Debate Word Count Vis

This is where Tableau Public gets fun.  I downloaded Ben’s workbook and opened it in my local copy of Tableau Desktop and started exploring the data fields, the available data and building my own views of how I might answer these new questions. Thank you, Ben, for inspiriting me in exploration.

Initially I tried answering the Top 10 Words by Candidate using the October 3rd Debate Word Counter workbook but was unable to sort the ranking by candidate because the candidates were not Dimensions.  What to do but go to the source: I downloaded the debate transcripts from the New York Times (links below), parsed and edited the transcripts and copied the data to an Excel file with three columns: Debate, Candidate and Word.  (I did some additional text editing including grouping words (a, but, and, etc.) and excluded nulls and non-verbal words from the transcripts like applause and chuckle.)

In a new Tableau workbook I imported Excel file and went to my toolbox and pulled out a Parameter and a couple of Calculated Fields to create a Top Words selector and to limit the viz display.

Let’s start with the Calculated Fields:

  1. First, an indexing function
    Name: Rank
    Formula: Index()
  2. Second, a Boolean field for our Filters mark
    Name: Words to Display
  3. Formula: [Rank] <= [Select Top Words]
  4. Finally, a Parameter with a range for selecting the ranking
Tableau Top N Parameter Example

Tableau Parameter Example

Once I put this on the viz I ran into trouble with my Rank Table Calculation — how could I get the Top 10 for both Obama and Romney if they didn’t share the same Top 10 set?

The Tableau Forums saved me: I came across a similar question answered by and Richard Leeke, two gems on the Forum. The question posed was basically How to create Top N Parameter with a filter option. The poster was asking about applying a date filters with a ranking parameter and I wanted to do basically the same thing only with a candidate filter. The trick I picked up here was the value of adding a Context Filter using Candidate. The Context Filter creates a temporary table of the data using the context filter criteria. This causes the data to be filtered by the context criteria before any other filters are applied. By adding Candidate to the Filters Mark and then selecting Add to Context, the Candidate Filter was applied before the Top 10 was calculated and now both candidates could have independent Top 10s and we could compare them side by side. I added an additional parameter for Debate # and here’s what we get:


Powered by Tableau

 

We can see now that Obama led Romney in talk of Markets and the Marketplace in the first debate and while many have commented that Obama was more engaged in the final debate, Romney still got in 10% more words than Obama.

I noticed some really interesting things while looking at the data. For instance, in the first two debates Obama referenced Governor Romney by title significantly more times than Romney called him President Obama. But in the last debate, the President drops his references to Governor Romney. And if we look back at the This is What I’m Sayin’ viz we see that President Obama is talking to the audience more than he’s addressing his opponent.


Powered by Tableau

 

Finally, I decided to explore some word groupings.

Firstly, How often did the candidates use I or We words or Us and Them words. It’s clear how important We become in the final debate. Notice any other trends?


Powered by Tableau

 

Lastly, How do the candidates talk about foreign policy issues?


Powered by Tableau

 

I also noticed that phrases would be interesting to explore–You’re wrong and I’m not wrong were biggies–but decided to leave that for another post.

Who’s work have you been inspired by? Share a link to your explorations in language or viz you’ve added a little something to. And thanks again to Ben Jones for his inspiration.

The 2012 Presidential Debate transcripts are available from the New York Times at:

Debate 1: http://nyti.ms/V2KTQD
Debate 2: http://nyti.ms/TqNZ3z
Debate 3: http://nyti.ms/XXhleD

Posted in: commentary
Tags: , ,

Comments

Be the first to comment.

Leave a Comment

You must be logged in to post a comment.