What modern web frameworks do for you

Motivation

If you are like I was, a former CS major whose programming experience consisted of writing command-line programs using C++, the world of web applications might seem mysterious and foreign.

You know how computers work i.e. you know processor architecture, you know the basics of operating systems, compilers, networking protocols… but this web stuff is something else. And it’s hard to get started – for one because there’s all this terminology, but even if you learn the terminology it feels like there’s all this magic going on.

This blog post is for people who were in the position I was in 6 months ago. It is not a tutorial – it is meant to be read before your tutorials so that things make sense.

Web Application – Basic Flow

First let’s break down the basic structure of a web application.

1. An application will expose multiple endpoints, which usually correspond to different parts of the application but don’t necessarily need to. An endpoint is represented by a URL, like www.myapp.com/register or www.myapp.com/logout.
2. Clients make requests to these endpoints. These use the HTTP protocol, which I won’t get into other than to say that a request is made up of:
1. Headers, some of which are used during transit and delivery, but you can add your own
2. The desired endpoint
3. An HTTP verb, like GET or POST
4. GET requests will often have a query string attached, like                                              /getInfo?name=Rohit&age=36
5. POST requests will usually have a body, which could be XML, JSON or something called “form-data”
3. When the application starts up, it registers its endpoints with the web server (a separate process running on that machine). Registration means providing callbacks for these endpoints.
4. When a request comes in, the web server matches the URL against the endpoints it has registered, to see if there is a listener available. If not, it returns an error message (following what the HTTP protocol defines, hopefully). Otherwise it passes it on to the registered handler.
5. The handler does what it needs to, and then sends an HTML response back to the web server
6. The web server sends this response back to the client

Ok, simple enough.

Web Frameworks

So what do web frameworks do for you? Keeping in mind the steps above, the framework should provide:

1. A simple endpoint & handler registration mechanism
2. Automatic parsing of the request, putting the information into some simple data structures
3. Simple ways for handlers to redirect a request (possible modified) to other endpoints

Another useful feature is an authentication framework, so you can mark certain endpoints as "protected" i.e. requiring a logged-in user. Of course to do this you need the concept of a user, so let's not get into that.

But an even more useful feature is layout templating. Think of your final page as being data placed in a layout container. The layout defines the colors, the text areas, navigation controls, where the scroll bars are, which buttons do what etc. Some of these don't change depending on the data being presented, and some do. The template will define the data-independent portions and indicate where the data is meant to go. It might be as simple as

<div id="userName" class="userNameDisp">

    {{userNameVar}}

</div>

The framework will read this HTML template, figure out that userNameVar is a variable whose value is required, and get that variable's value from the interpolation context. You can imagine this working like this:

getPage (templateName, dataContext):

    response = loadTemplateAsString (templateName)

    for every {{varName}} in the response string

        look up varName in the dataContext

        replace {{varName}} by this value in the response string

    return response

The two web frameworks which I know of which work this way are Django, written in Python, and Ruby on Rails, which is written in Ruby.

A different strategy

Nowadays everyone loves JavaScript, supposedly because browser support is so much better than it was before (I don't know how it used to be, this is just what I hear). In fact browsers are so good at running JavaScript now that applications are being written so that a large part just sits in the browser! 

One of these frameworks is called AngularJS, and I've actually written a small application using it. But I don't fully know how it works yet. Here's what I do know/ believe at this point:

1. Request to the web application loads a whole bunch of JavaScript code. This contains all the endpoint definitions and associated handlers.
2. As the user interacts with the application, requests will be triggered to different endpoints. These will, as far as possible, be trapped by the client-side application and NOT be translated into fresh requests to the server (since the application logic has already been loaded into the browser)
3. Of course certain requests do need to go to the server, like requests for data. The server will now just send back JSON or XML.
4. Layout templates will also be sent on request, though a framework might also provide a mechanism for pre-requesting and caching them locally. One would hope that a layout template is not loaded more than once though.

Now the server sends only three types of responses: the initial application code, layout templates, and raw data. Less network traffic, but more work for the browser.

I hope this helped. If you want to learn more about these JavaScript frameworks, you can Google 

1. AngularJS for the client side (and yes there are others)
2. Express for the server (again, there are others) 
3. and for solutions which combine both the client and server - Meteor and Sails! 

Tracking invitations via external services

Let's say you are writing a mobile application which allows users to invite their friends via Facebook, Gmail, and services like those. To make things concrete, let's say your application lets users create shopping lists for parties, and lets users link to their Facebook and Gmail accounts.

So Farah gets the app, links her Facebook account, creates a shopping list and invites her friend Priya to collaborate on it. By linking to Facebook, she is able to access her Facebook friend list of which Priya is a member, so that's how she sends the invitation.

Priya gets a message from Farah in her Facebook mailbox. Here's what we want to have happen:

1. If Priya already has the app, and has linked her Facebook account, she automatically sees the invitation when she next opens the app,
2. If she doesn't have the app, she will download it (by following the app-store link conveniently included in the Facebook message) and link her Facebook account. When she does that, the invitation will appear in the app.

How can we do this? Here's my solution which is simple and easy.

The app's server will maintain two types of user objects - active users and passive users. An active user is one who has downloaded the app and registered as a user. A passive user is someone to whom an invitation has been sent, but who may not have downloaded the app yet.

Active user objects have a (possibly empty) set of linked external accounts. A passive user object will contain exactly one linked account.

When Farah sends Priya the message, the system will look for an active user object containing a linked account for Priya@Facebook. If it finds one, it associates the invitation to that user object so that Priya sees it the next time she logs in.

If there is no active user, the system will look for a passive user. If it finds one, it will associate the invitation to it, and if not, it creates one and associates the invitation to the new object.

When Priya finds and downloads the app, she will hopefully link her Facebook account. At this time the system will look for passive users attached to her Facebook account. When it finds it (and there should be only one), it will link the newly created active object to this pre-existing passive object, and via the passive user will gain access to all associated invitations.

Since the passive user object is not removed from the system, no lists containing it need to be updated. So Farah's party list will continue to hold a reference to the passive object, and when the system asks for the user it will "follow the links" to the active object.

Since the passive object acts only as a forwarding mechanism now, the information about the linked Facebook account can be moved to the active object to avoid wastage of space. And in fact we don't need to hold on to the passive object at all - we could just maintain a mapping from the passive object IDs to the active ones. So a "getUser" function would look like

getUser(userId)
{
  if(userId in passiveToActive.keys)
    userId = passiveToActive[userId]
  return db.findUserObj(userId);
}

Note that multiple passive objects could link to a single active object - this would happen if someone received invitations via multiple services. When they link those services this mechanism will pull all that passive activity over to the active account, which is just what we want.