Category Archives: text analysis

Social Networks of Shakespearean Plays – part 2

As a follow on to my previous work about social networks in Shakespeare, I wanted to see how the social network changes throughout the course of the play.  Using the same techniques as last time we can look at the social network structure by act. The density of the social network underlies the style of […]

Social Networks in Shakespeare

Privacy settings are very important when getting them wrong results in a duel to the death. When I think of social networks, I immediately start to think of Facebook, LinkedIn and the rest of their ilk.  These tend to dominate the landscape of our thoughts on social networks simply because they’re the biggest.  But social […]

A/S/L/(Neil Gaiman fan?) – Fusion tables and OKCupid

I’ve been thinking about data visualization tools lately.  In particular, I got some advice to checkout Google’s Fusion Tables.  I needed some data to start playing around with, but, luckily, I happened to have 29,035 OKCupid profiles laying around in a database (learn how I got them here). Age: First question, what does the age […]

Data mining OKCupid

I’ve been thinking quite a bit about natural language processing lately.  This started with my series on text message analysis and looking at gender specific twitter usage.  Lately I’ve been pointed at the Natural Language Toolkit (NLTK), a library in python, to make this analysis more robust.  I want to apply this toolkit on a […]

Text Message Analysis, Chapter 3

So far in our exploration text messaging, we’ve analyzed the time of day each message arrives and the structure of each message.  We’ve been able to pick out typical behaviors like timing of sleep cycles and life events like a trip to Switzerland.  We’ve found a relationship between the structure of messages and the kind […]

Victor Hugo, the original texter

By way of today’s episode of Ze Frank’s “a show” Victor Hugo wanted to know how his book, Les Miserables, was doing.  So he wrote his publisher.  All he included in his letter was  a single character, a “?”  His publisher wrote back with a single character, a “!”  So little information, so much exformation, […]

Text message analysis, chapter 2

Let’s continue with our analysis of text message behavior by shifting over into content.  One of the first things to understand is the form of each message.  What’s the length of each message? As a function of length, how does the content of the message change?  To begin with…

Text message analysis, chapter 1

A couple of months ago I read Stephen Wolfram’s article about analyzing his emails and saved files.  If you’ve not seen it yet, it’s definitely worth a read.  I was intrigued and wanted to apply a few of these analysis ideas to my own life.  I don’t have 30 years of saved email (yet), but […]

Downloading iPhone text messages to your computer

I’m working on a program to analyze text messages for sentiment, as well as other features of how communication occurs. In the course of this, I’ve been playing around with a way to get text messages off the iPhone and into a data format that’s more directly usable.