Semantic Void

I started this project as a weekend thingy a day back but hit a hurdle due to the Twitter rate limits. Since I couldn’t complete this project, I thought it would be rather nice to write a post explaining what I had in mind.

Twitterers like bloggers are driven/motivated by their *follower* counts (who doesn’t like attention). But unlike blogs there is no mechanism in Twitter to determine the visibility of ones tweets or gain insights into how one can maximize the visibility of their tweets. By visibility I mean the chance of my tweet being read by my followers. I’ll try to explain my approach to estimate this *visibility* with the following example:

The image above depicts the user (imagine yourself) at the center along with his followers and the users they are following (users which your followers are following - this can get crazy). The idea is just this dead simple - the visibility of your tweet depends on the twittering habits of your followers followings. Let me slow down - imagine a queue of size 20 assigned to each of your followers. This is the list of recent tweets displayed on Twitter for every user. Your tweet’s lifetime is as long it doesn’t get pushed out of this queue - which depends solely on the twittering frequency of your followers followings. The faster they tweet, the faster your tweet vanishes off this list and vice-versa. This lifetime/visibility of your tweet averaged over all your followers can be figured out algorithmically as follows:

• Fetch all your followers and their followings (lets call this FF).
• For each FF - calculate their average twittering time (do this by analyzing their last 50-100 tweets).
• Using the times calculated above, calculate the time for your tweet to get pushed off the queue for each of your followers. This can be done by simulating tweets from each of the followers following (FF) at the frequency calculated above and pushing them into the queue.
• Average this value across all your followers.

The above not only gives you insight on how long would your tweet be visible but also which of your followers could potentially read your tweet. For example, you could answer queries like - ‘if I tweet now, which of my followers could still read this tweet after 5 hours’ or ‘if I tweet now, which of my followers will have already read/missed this tweet after 3 hours’.

For more accurate results, one could also analyze the twittering habits of the FF’s (followers following) and generate a probabilistic model of their twittering frequency relative to the time of the day. This could help generate dynamic statistics depending on the time you intend to twitter.

Another insight that could be provided with the above statistics could be the duration one should wait before putting another tweet. I, for example, twitter sometimes in bursts (with just seconds delay between consecutive tweets). Followers usually give the most attention to your most recent tweet. The more consecutive tweets you have, the higher the chances of the older tweets getting less attention. If one can figure out the average time it would take for ones older tweet to get pushed off the ‘recent 20′ list of his follower/s, it would give one a better idea of how long should one wait before twittering again.

I’m sure that there are many more cool statistics/insights one can figure out from ones network in Twitter. I would be glad to hear out your take on this as well as any improvements you might have to suggest. I would love to make this an app once I can get Twitter to raise the hourly request limit for me - Twitter are you listening?

I assume that most of you have watched the movie ‘21‘. Well if you haven’t, I recommend that you watch it. Around 20 minutes into the movie there is a scene where the professor poses a problem in front of the protagonist:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice? [via Whitaker, Craig F. (1990). [Letter]. “Ask Marilyn” column, Parade Magazine p. 16]

The protagonist chooses to switch. It is not very intuitive as to why one should deviate from one’s original choice. It seems as if the probability of winning the car just bumped up to 1/2 from 1/3 after the host opens a door with a goat behind it. But it can be empirically proven that switching the choice is the rational strategy for the player. This ubiquitous game show dilemma is the Monty Hall problem.

Let me try to prove why is it so.