Month: January 2013

by on

A HBR article titled Smartphones, Silly Users perfectly describes why I have moved my personal information management system away from apps that sync across desktop and mobile:

  1. “We don’t remember anything anymore.”
    • “We’re increasingly outsourcing our personal memory banks to Google and other search engines, effectively wiping our own brains of easily accessible information.” a.k.a. the Google effect
  2. “We waste time preserving optionality.”
    • “We’re refusing to finalize our plans until critical moments. The ability to make reservations, check opening hours, look up driving directions, and review ratings on our mobile devices means that we’re increasingly iterating our schedules and keeping our options open until the very last moment before that meeting, lunch, or coffee catchup is set to begin.”
  3. “We get stuck in the infinite notification loop.”
    • “As we endlessly loop between Facebook, Twitter, LinkedIn, and other app notifications, our attention fragments, and it becomes difficult to focus on larger, more important tasks.”

Till this month, I was obsessed with syncing everything across my desktop and mobile. The problem was that I became obsessed with the mobile phone unnecessarily and once you’re using the phone, Point no. 3 kicks in – the infinite notification loop swallows a lot of time and attention.

Once I shifted my system to laptop-only, I don’t have all my tasks and calendar at hand, I’m forced to remember things (see point 1 above), and strangely, I’m more likely to remember things to pick up from the grocery store now than I was likely to remember to check my mobile phone app for things to buy when I was near a grocery store!

The most important thing is that notes and todos are in the same place, for example, if I’m on a call, I can take notes and then I can keep referring back to those notes while creating todos and working on tasks. The tasks come out of notes, they’re not separate! It really helps to have one system that can handle and encourage the normal flow instead of being forced to use separate notes and tasks apps.

Today, I’m all OrgMode. Again.

(more…)

by on

Recently, I finished reading the latest “early access” version of the Big Data Book by Nathan Marz.

What is Big Data

Let’s look up Wikipedia:

In information technology, big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, analysis, and visualization.

So, Big Data is relevant for any technical and business person whose company deals with lots of information and wants to make use of it. For example, Gmail search, etc.

Why this book is awesome

The book has been a fascinating and engaging learning for me because of two reasons:

First, it has a strong and simple “first principles” approach to an architecture and scalability problem, as opposed to the confusing (to me) and mushrooming complexity and treating Hadoop as a panacea in the Big Data world.

Second, Nathan Marz was one of the only 3 engineers who made the BackType search engine (the company was acq-hired by Twitter):

BackType captures online conversations, everything from tweets to blog comments to checkins and Facebook interactions. Its business is aimed at helping marketers and others understand those conversations by measuring them in a lot of ways, which means processing a massive amount of data.

To give you an idea of the scale of its task, it has about 25 terabytes of compressed binary data on its servers, holding over 100 billion individual records. Its API serves 400 requests per second on average, and it has 60 EC2 servers around at all times, scaling up to 150 for peak loads.

It has pulled this off with only seed funding and just three employees: Christopher Golda, Michael Montano and Nathan Marz. They’re all engineers, so there’s not even any sysadmins to take some of the load.

Note: BackType’s (now open sourced) real time data processing engine Storm powers Twitter’s analytics product and real-time trends among other things.

(more…)