Database as a REST API

Advantages would be:

• The sharing of database ORM models across multiple programming languages will be seamless.
• Making the API available via Thrift (like the Evernote API) implies native API wrapper libraries can be automatically generated for almost any programming language.
• Using Thrift also means the storage of logs, etc. in JSON / Thrift format will be straightforward and migration to a Lambda Architecture in future would be far more plausible as well.
• Performance enhancements such as converting long keys to actually short keys in backend (for MongoDB, HBase) and using good hash keys (for DynamoDB) can be handled in this layer without every language-specific wrapper having to implement it.
• Switching some data from Postgres to MongoDB or sharding in MongoDB, etc. could be hidden as internals of this layer.
• Rate-limiting, monitoring and scaling would have been easy – and Django memory leaks, etc. would not affect other parts of the system…
• This would have easily enabled having a lot of small apps instead of a big monolithic Django app, etc.

Did we just sort-of reinvent Datomic?

Of course, this is not a new idea at all, take restSQL as an example – my question is why is this not talked about more often?

Do most frameworks support this? If not, why not? If so, why don’t most frameworks don’t talk about such a use case in their documentation? If I use Django, I’ll start writing the models and use South to create migrations, and that’s that. If I have to reuse those model, from say, Java, then you’re on your own. The point is that, by default, Django (or Rails) doesn’t encourage you to do such a thing. If you go for a lighter framework such as Flask, then this becomes easier because the ORM is anyway not part of the framework.

Is this concept felt needed only in a polyglot case (multiple database systems, multiple programming languages)?

P.S. Also read Stevey’s Google Platforms rant.

Update on [2013-04-28 Sun]: Also see the very useful tech talk Designing a Beautiful REST+JSON API.

Mystery Spot was interesting and the strange phenomenon there could only be explained by some kind of magnetic force, we would literally stand at an angle in the centre of the mystery spot, and the host was very comic and entertaining, so the visit felt worth it.

Ocean View Boulevard is one of the most beautiful places to drive around and we actually saw a couple of weddings in the open lawns on that road which gives an idea of how scenic that place was.

17-Mile Drive was a nice drive along the coastline. The fun part for me was when Karthick decided to teach me how to drive in the USA and I was so nervous of all the rules (solid line vs. dotted line, whoa!), but he taught well and I got the hang of it, although, maintaining an average of 60 miles per hour on the highway is daunting as I’m not used to those speeds.

A R Karthick

Leaning in Mystery Spot

Entertaining host at Mystery Spot

Passing by green fields

17-mile drive

17-mile drive

17-mile drive

17-mile drive

17-mile drive

Thanks Karthick for spending time with us and taking us out :)

The first thing that I’ve noticed is that people generally smile here and are cordial which is quite striking and I would probably attribute that to something about parenting here, because I’ve never seen American kids cry, and I mean never. In India, if you smile at a stranger, it’s because you know them, not because you’re walking by.

The second thing is that even if they are cordial, they have some kind of “force field” around them, it’s a “Don’t enter my personal space” thing. No wonder Americans thought of force fields in cartoons and movies. And I’m not the first person to talk about this, a good friend of mine Anu who has moved from USA to India to run a for-profit social enterprise (which I had earlier freelanced for) talks about What being Indian means to her:

It’s about love.  In all senses of the word.  It’s about having this big love for your family, your friends, the world, everything.  It’s exemplified in Bollywood- the enormity of emotion.  In the West, we have a very limited view on love, mainly referring to the romantic kind.  Maybe that’s because we are a much younger civilization and haven’t quite figured out how to express the nuances of this amazing force. But in India, that sort of love is directed towards everything and everyone.  I was hanging out with some new friends I made in Mumbai yesterday, and it was amazing- they treated me as if they’ve known me forever and brought me into their friend circle, making me feel so welcome. And it was genuine- absolutely and completely genuine.  This isn’t new, as the same thing happened when I first came to India in college, where new friends took me in and treated me as their own, again, no questions asked, no strings attached, just..because.  I haven’t been able to be in touch as much as I want, but it’s one of those feelings that you know they’re not judging you, and that bond you have is still strong.  Family in India is the same way.  They take you in with open arms (friends of yours included) and treat you like their own son or daughter.  It’s incredible… So I think to me, being Indian means being capable of exuding that love, and reflecting it back on the world unconditionally.

Interestingly, we met some friends of friends (including a typical white American male) who intend to eventually move from America to Asia because of the individualistic culture in America.

On the other hand, the city of San Francisco is a wonderful place to visit:

View from Potrero Hill, San Francisco

Yerba Buena garden, San Francisco

Pillow Fight at Embarcadero, San Francisco

Long cars / limos is a thing in San Francisco

Some random anecdotes:

Once I saw a person riding a Segway on the road! The person riding it was too fast for me to take a photo but I was fascinated to see this.

Internet speeds are fast but cellphone connections (at least T-Mobile) are pathetic. I can download a file at 2.5 Mbps actual speed on a home connection but I can’t call anyone because the cell tower connectivity is zero when indoors, so I have to call them via Skype, heh.

I walked into a “Chatz coffee” place and the person at the counter told me she’s studying her masters at San Francisco State University and had come to Himalayas last year for studying “land tectonic shifts”. Wow, people actually study those things? And I wonder how it works that she’s working behind the counter at a coffee table and doing Masters at the same time – perhaps someone who has done masters in USA can enlighten me on the economics and time availability aspects.

I saw a grandpa walking with his grandchild and asking her “Does your mommy have a boyfriend? Does your mommy have a husband? No?” I was not sure if he was simply trying to have a conversation with the grandchild or trying to find out the truth via the grandchild…

Booking a taxi through Uber iPhone app was surreal because it actually shows you the number of taxis near your location so that you have an idea of how quickly a taxi can reach you, and once you click on a button, the taxi reaches your location usually in a span of 4-5 minutes and you can actually see the taxi moving towards you as if it’s a car racing game! And at the end of the trip, it automatically charges your credit card, so no money exchange has to happen at the trip, this is great especially when you’re in a hurry to reach your destination.

Similarly, the experience of going around San Francisco using the superb public transportation via one standard Clipper Card and using Google Maps app to figure out which Muni bus / Muni train / cable car / BART / Caltrain (and combination therein) to take is simply great.

The stores here are a treat to visit if you’re into arts and crafts:

Imagiknit store

Imagiknit store

Some more photos:

Bubbles in Dolores Park

Hula hoops in Dolores Park

Palace of Fine Arts, San Francisco

Palace of Fine Arts, San Francisco

Lombard Street, the famous twisted road

Funny sign at a children’s playground

Civic Center, some kind of Vidhan Soudha equivalent

View from Ferry Building, Embarcadero

Sausalito city

Cycling is a common tourist activity in Sausalito

Hiking route from 61 bus drop-off point to Muir Woods

Hiking route from 61 bus drop-off point to Muir Woods

The Lonely Planet San Francisco edition tells us that there is still lots to see and they say that it’s the top city for travelling in 2013, so I hope I get time to explore more!

The startup that I’ve been part of for more than a year has launched this week : Automatic – Your Smart Driving Assistant. It’s a hardware device + mobile app + cloud combination that helps you save money by helping you drive in a more fuel-efficient way, monitor your car engine’s health, automatic call to local authorities/911 in case of a car crash, and will automatically remember where you parked your car (note that the product is currently USA-only).

Watch the video:

Now, go and pre-order it now at www.automatic.com.

So how did I get involved? A couple of years back when I was considering freelancing full-time, Thejo Kote got in touch with me and I got started with NextDrop for which he was one of the co-founders. After some time, I was looking for something more long-term, so I pinged Thejo and he welcomed me to join his new startup.

It was a YCombinator company and was well-funded and I actually liked the idea, although at the time I was skeptical if this is something people will buy, that question has been comprehensively answered by the audience response and media coverage this week. Finally, I’m part of a hardware startup that had a working product before preorder launch :)

It was funny that as we were working on this in stealth, I would see many articles such as:

• Forget the Internet of Things: Here Comes the ‘Internet of Cars’
• Cars Are The Next Playground For App Developers
• The startup hardware M&A market will be vibrant in 2013
• Your Car: The Next Big App Platform!
• U.S. govt. report that household expenditures for gasoline account for nearly 4% of pretax income
• Oak Ridge National Laboratory researchers quantify the effect of increasing highway speed on fuel economy
• Here’s How Much Speeding Really Costs Drivers At The Pump
• How much is fast driving costing you?

And I would think to myself that we are working on the same area.

The past few weeks have been all about the company showing demos to journalists and interested people and the result has been there for all to see, snippets below.

Press coverage:

Fast Company says:

A visionary gadget that makes any drive more fuel-efficient.

CNBC says:

Automatic could do for driving what the iPod did for music.

The Verge says:

Cars have been equipped with onboard computers for decades now, but as they get smarter, they remain stubbornly closed off. You might see an miles-per-gallon reading on your dashboard, but there’s no way to check it on your phone, and as soon as you finish a trip on an on-board GPS system, it’s gone forever. As Automatic CEO Thejo Kote puts it, “your car is the most expensive computer you own.” And it’s usually a pretty bad one. Naturally, Kote has a plan to fix that.

The most prominent feature is a fuel-efficiency score, designed to nudge you into gas-saving driving habits. Automatic is playing off a Department of Energy report which found that avoiding hard stops, fast starts, and speeding can increase gas mileage by up to 33 percent.

It could be the most practical take so far on what a cloud-connected car should look like — an area that auto-averse startup scene has mostly overlooked. Automatic is working towards a time when users and developers can access a car’s data whenever they want, and use that data to build new features. From there, it’s easy to imagine more applications, from parental car-tracking to next-generation traffic apps, but the first step is getting the data off the dashboard. With the app-connector combination, Automotive has set up a remarkably painless way to do it.

MIT Technology Review says:

The device combines two burgeoning trends—the “Internet of things,” where traditionally offline gadgets are connected to the Internet to amplify their usefulness (see “50 Disruptive Companies 2013: Nest’s Smarter Home”), and the mining of data that’s collected by our devices for meaning (see “Every Step You Take, Tracked Automatically”). By putting these two together, the company thinks it can get users to conserve gas and spend less—and make a profit itself while doing so.

TechCrunch says:

People spend a ton of money on their cars every year, from car payments to insurance to gas to maintenance. But for such expensive assets, most people normally don’t know a whole lot about what’s happening under the hood, or how they can drive or maintain their cars better over time. The folks at Automatic want to change all that, with a smart combination of hardware and mobile apps to keep people better informed of how their cars are doing.

Automatic investors include Andreessen Horowitz, Founders Fund and Y Combinator, as well as a group of angels.

Gigaom says:

In addition to helping people spend less money on gas, the app also shows you why the check-engine light comes on (and permits users to turn it off), directs you to your parking spot and automatically calls 911 if you get into a collision. It also tracks the cost of every mile you drive, a calculation based on a continually updated stream of data of gas prices from stations across the country. All these functions came about after the founders realized what a device that could monitor fuel use can do, said Jariyasunant, a co-founder.

Gizmodo says:

We saw Automatic in action on a little drive around the streets of New York City last week. Along with an iPhone app, the little dongle will track your gas mileage and remember where you parked. It’ll also even give you a little notification if you’re braking too hard or accelerating too fast. Why? These are apparently a couple of habits that make you use more gas than you need to. How are you driving? It’ll give you personalized feedback every week in score from 1-100. And it tries to lookout for your car too. Engine light flips on? You can tap a notification to see what the issue is. If it’s no big deal, you can turn the light off all from within the app. Or you can look up a nearby mechanic to get things sorted out. And god forbid you get in a wreck, Automatic will automatically of course call emergency responders and send texts to three numbers you pre-determined to let them know you’re ok.

Wired says:

We spent a little time with Automatic and were impressed.

One of the biggest highs for me was seeing it on TechMeme:

Tweets:

Automatic seems like such a good idea. If it delivers, it's going to be a huge product: automatic.com—
Jason Fried (@jasonfried) March 12, 2013

@collision @stripe @automatic Pre-order placed. Great checkout process.—
Jason Fried (@jasonfried) March 12, 2013

I'm checking it out too! MT @bre Excited! I just pre-ordered @automatic to upgrade my car with my smartphone. bit.ly/ZHQ0cz.—
Tim O'Reilly (@timoreilly) March 13, 2013

The latest installment of the hardware renaissance: Automatic. automatic.com—
Paul Graham (@paulg) March 12, 2013

Pre-order @automatic to upgrade your car with your phone. Check it out at automatic.com.

Seriously. This is awesome. #thefuture—
Josh Brewer (@jbrewer) March 12, 2013

Shut up and take my money: Automatic gets telemetry from your car into your iPhone, does smart things with it. automatic.com—
Rafe Needleman (@Rafe) March 12, 2013

Game changing product, check. YC backed, check. Ridiculous domain name and twitter handle, check. This is @automatic automatic.com—
Arush (@arush) March 13, 2013

I have never gone from product discovery to purchase faster than this. Get the data to help you drive smarter. automatic.com—
Drew Michael Blake (@drewble) March 12, 2013

Your old car + phone + automatic.com = The smart car we’ve been waiting for. Too cool.—
Bryan Birsic (@birsic) March 13, 2013

We love to see design/tech that improves daily life. Meet @automatic, which monitors your driving and car's health. pco.lt/14XOaGb—
  (@Jawbone) March 13, 2013

$70 is trying to fly out of my pocket. Sweet car gadget by @automatic zagg.com/community/blog… (via @zarinf)—
Morgan Williams (@mrgnw) March 13, 2013

You guys took the mobile world to another level!! @Automatic—
Jinesh (@civilizedgauti) March 14, 2013

Too good to resist a preorder – check out @automatic to upgrade your car with a smartphone. automatic.com.—
andreas lieber (@lieber) March 14, 2013

jesus automatic.com is so amazing it makes me want to buy a car and learn how to drive—
Dan Nolan (@dannolan) March 12, 2013

Automatic seems too good to be true. I think I’ll get one. automatic.com—
Marc Edwards (@marcedwards) March 13, 2013

I would buy this just to remember where I parked my car. '@aronsolomon: Agreed. automatic.com seems to be amazing.—
Jonas Grankvist (@jgrankvist) March 13, 2013

The code reader alone will pay for itself. definitely the killer app for #mobile now. @automatic
lnkd.in/ys6JjR—
Ryan Osilla (@ryanosilla) March 13, 2013

@automatic WOOHOO! I just preordered. This is super cool!!! Can't wait to get it.—
Charlie Sheets (@charliesheets) March 13, 2013

Just pre-ordered an @automatic

Looks so effing cool!
automatic.com—
AJ Fleming (@lolheyaj) March 13, 2013

@automatic how the?! That's it! I'm moving to the States just so I can have this in my life!—
Ether Creative (@ethercreative) March 13, 2013

@stevederico I agree, but I also said that about Gowalla. @Automatic has a solid team and more funding though. I'm betting on that horse.—
Joshua Sortino (@JoshuaSortino) March 13, 2013

First I'll buy this: automatic.com. Then I'll buy a car. via @ronnieliew—
Onur Özer (@onurozer) March 14, 2013

Automatic ( automatic.com) and Leap Motion (www.leap motion.com) have made my must have list this year. Cool products.—
Brentt Baltimore (@brenttbalt) March 14, 2013

@waze @automatic If you guys can work together and get car-to-car comms happening I'll forego my jetpack. http://t.co/mtpIyaPIbb—
Dean K (@deanlk) March 14, 2013

My mother needs this. automatic.com—
Parker (@parkr) March 14, 2013

Check out @Automatic for your car. Welcome to truly integrated technology. If this thing works, it's way too cheap. automatic.com—
Shawn Lein (@SMLein) March 14, 2013

Informative post about @automatic, an awesome app for driving safer and saving gas by @Brit + Co. brit.co/automatic/—
Randi Zuckerberg (@randizuckerberg) March 21, 2013

A simple twitter search for @automatic will reveal a lot more feedback :)

Friends

I hope now my dear friends will forgive me for not revealing a lot what I’ve been working on:

@swaroopch is up-to something & its huge. All he does is show me journalists' tweets that say "biggest idea since the iPhone" #automatic—
Abishek Nair (@abisheknair) March 09, 2013

Couple of young entrepreneurs just showed me the biggest idea I've seen since the iPhone. #automatic automatic.com—
Jon Fortt (@jonfortt) March 06, 2013

with @swaroopch & @ntkeep doing crazy entrepreneurial 'monkey business' it makes me feel stale & gives me crazy career ideas—
Abishek Nair (@abisheknair) March 09, 2013

Team

The past one year and three months has been an interesting time for me, and we have a great team in-place:

If you are interesting in the kind of things we are doing, we have open positions for engineers, so join us – we’re looking for a Server engineer, iOS engineer, Android engineer and Firmware engineer – please get in touch with us at jobs@automatic.com or directly contact me if you’re interested.

Forward

In the end, we have our eyes on only one thing: delivering on the promise the company’s been making and I’m confident that Thejo will lead us there successfully. So after the launch frenzy, its heads down and back to work.

This struck a chord with me on one of the most under-focused roles of managers, because, of course, it’s hard to make it happen, but at the same time, this is what employees appreciate the most, besides salary raises (and sometimes in lieu of salary as well).

P.S. I also liked the “A leader’s job is to absorb anxiety and instill confidence among employees” part in the above linked article.

Recently the Obama re-election campaign employed similar population-wide behavioral analytics to micro-target voters to ensure his re-election. There is no reason why we in India must not look to technology to devise ingenious methods for near real time data collection and population-wide analytics of social performance. This will not only help micro-target and localise welfare Interventions by local Governments (as opposed to centralised schemes) but it will also shift the focus away from agenda-driven politicking based on lagging indicators and towards a debate on actionable interventions that can make a difference here and now.

I love this idea in this article in Niti Central on how we can leverage technology for social welfare.

The bit about Obama’s re-election campaign is best read about in this article called The nerds go marching in, although I still haven’t been able to find specifics about the tools that they built.

P.S. Also see Wikipedia article on Microtargeting.

1. “We don’t remember anything anymore.”
   • “We’re increasingly outsourcing our personal memory banks to Google and other search engines, effectively wiping our own brains of easily accessible information.” a.k.a. the Google effect
2. “We waste time preserving optionality.”
   • “We’re refusing to finalize our plans until critical moments. The ability to make reservations, check opening hours, look up driving directions, and review ratings on our mobile devices means that we’re increasingly iterating our schedules and keeping our options open until the very last moment before that meeting, lunch, or coffee catchup is set to begin.”
3. “We get stuck in the infinite notification loop.”
   • “As we endlessly loop between Facebook, Twitter, LinkedIn, and other app notifications, our attention fragments, and it becomes difficult to focus on larger, more important tasks.”

Till this month, I was obsessed with syncing everything across my desktop and mobile. The problem was that I became obsessed with the mobile phone unnecessarily and once you’re using the phone, Point no. 3 kicks in – the infinite notification loop swallows a lot of time and attention.

Once I shifted my system to laptop-only, I don’t have all my tasks and calendar at hand, I’m forced to remember things (see point 1 above), and strangely, I’m more likely to remember things to pick up from the grocery store now than I was likely to remember to check my mobile phone app for things to buy when I was near a grocery store!

The most important thing is that notes and todos are in the same place, for example, if I’m on a call, I can take notes and then I can keep referring back to those notes while creating todos and working on tasks. The tasks come out of notes, they’re not separate! It really helps to have one system that can handle and encourage the normal flow instead of being forced to use separate notes and tasks apps.

Today, I’m all OrgMode. Again.

OrgMode is a notes and tasks organization system all inside Emacs. I won’t go into details about OrgMode itself, the simple way to get started is to use BG’s dotemacs and read the OrgMode Manual.¹

Here’s my transformation:

• Tasks were moved from Things.app to a file called life.org
• Notes were moved from Evernote to a reference.org file for “I may need someday” kinda info and background.org for things that “I want to keep in mind”.
  • I still use Evernote as an inbox (i.e. as a temporary space) to jot down stuff when I’m not near my laptop.
• Calendar events were moved from Calendar app to a calendar.org file
• Habits were moved from Daily Deeds iPhone app to a reminders.org file with the habits org module enabled
• Files were moved from Dropbox to a files folder
• Diary entries (although rare) were moved from Evernote to a journal.org file with file+datetree configuration
• Pomodoros and time tracking have moved from Focus Time iPhone app to simple clocking commands: C-c C-x e to set a time estimate, C-c C-x C-i clocks in for the current task, and C-c C-x C-o clocks out of the current task and it’ll show me how much time I spent on a task and it’ll even aggregate subtasks and show how much total time I took on a higher-order project/task. And all this in plain text!
• Blog drafts have moved from WordPress to a blog_drafts folder full of .org files, including this post.
• Archiving (C-c C-x C-a) is one of the greatest OrgMode features, where it’ll take the current task and save it in an archive file for long-term storage, this is great when you want to just file a completed project’s notes and tasks and move it out of your current files.
• Backups : Instead of Dropbox or any auto syncing system, I use a combination of Git, a local folder and an external hard disk.
  • The files folder is not added to the Git repository (by adding it to the .gitignore file) because the list of files are potentially huge and I want my git history and diffs to be only about the .org files. Maybe someday, I’ll use git-annex for those files.
  • I have an alias in my bash profile, so a few times a day, I type gcb in my terminal and all the .org files get checkpointed.

    alias gcb='git add . && git commit -m "$(date +"%a, %d %b %Y")"'

  • Once in a few weeks, I zip the entire folder including the files subfolder and store it in an external hard disk. I have an alias for that as well.

    alias b#="cd && zip -rq org.$(date +%Y%m%d.%H%M).zip pimfolder && ls -lh *.zip"

  • Maybe once a year, I can push the org files to a private repository on BitBucket, and zip the files folder and put it in a private Amazon S3 bucket. Or something like that.

Notice how many apps have been merged and subsumed by OrgMode!

Best of all, I’m no longer constantly looking up my mobile phone. When I’m away from my laptop, I’m free to focus on the situation at hand.

Everyday morning, a simple agenda command C-c a a gets me started, all my info is in one place, no more looking at separate apps! Getting started with the day is simple and fun again.

Having a date-time type as well as custom types seems useful to me, so I was taking a look at the current Python implementations of the EDN format and I didn’t find them satisfactory, for example, one of the listed ones had all custom parsing code which was difficult to read, one was not even a real implementation, just boilerplate code, etc.

So I thought why not create a better implementation and I did – it is up on GitHub at https://github.com/swaroopch/edn_format.

It has been a long time since I did lex and yacc, so it was a fun weekend project :)