App Architecture and the Basics of Scale

Motivation
Recently a friend asked me about building applications. He is a graphics designer, and doesn't want to learn how to write code. He just wants to get a "big-picture" view of some of the common themes that arise when designing applications today. This blog post is aimed at people like him - interested but non-technical.

I'm going to start off by saying that what I'm about to describe does NOT describe all the different types of software being written today. In fact I know developers who never touch these issues. But my feeling is that this will be accurate for the vast majority of developers.

What Kind of Applications are We Talking About?
Here are some examples:

1. A blogging platform
2. An order and procurement system
3. A recruitment platform
4. A corporate HR system
5. A corporate finance system
6. A (software) bug tracking system
7. An email and chat platform

These are all very different, but let's see what's similar for most or all of them.

1. There are UI elements through which users can enter data. Very often these take the shape of forms.
2. There are UI elements which show data to users. This could be:
1. the data which they, or other users entered into the system,
2. data from external systems (machinery, sensors, the Internet),
3. the results of aggregation and computation, generally shown in a report. Think of "monthly sales totals", or your bank statement.
4. Often people want to see entire collections of similar data (your MakeMyTrip search results); these are displayed in a list.
3. There are processes which run in the background which actually do something with this data. These processes may only involve computers (generating your bank statement), or sometimes other people (processing your loan request). For something like email, one process would be the delivery of mail, which might involve looking up email addresses from nicknames; another process would be receiving mail, and running it through a spam filter.
4. All this data has to be stored somewhere, since it is generally not just sent from user to user. Even in the case of a chat system, users generally want a history of their chats stored for later. So there needs to be a database.

More and more applications today are being developed on the browser - whether Internet-facing or inside an organization, the browser is the portal of choice. We can book hotels, apply to university, check football scores... all through a single portal.

(I am completely ignoring stand-alone applications like Photoshop, Excel, Mathematica, StarCraft... I'm not understating their importance, but they're not relevant right now).

Another medium which is gaining in popularity is the mobile app. They don't run in browsers, but many of them provide functionality which would be natural on a browser. Many of them (LinkedIn, Facebook, Twitter) actually have browser counterparts.

How They Work

The following flow is widely applicable:

1. A request arrives at some endpoint. For web applications, an endpoint is just a URL, like http://www.my-awesome-app.com/login 
2. The web server decides whether or not the application can handle this request. For this to happen, the application needs to have told the web server what it can handle. Anything not on that list is a fail.
3. If the request is valid, and the endpoint exists, the web server forwards it to the appropriate handler in the application.
4. The handler takes the request and does what it needs to. This usually involves talking to the database. The list of tasks which needs to be performed fall into a workflow. Workflows will have portions which are straightfoward - "after checking that their ZIP code is valid, check their area code" - and portions which are conditional - "if their account balance is too low then deny their ATM request. Otherwise give them their money".
5. If the request has arrived "bearing data", it gets written to the database. Before that, there will usually be
1. validation e.g. the user entered their birth date. People from the future are not allowed.
2. clean-up or sanitization e.g. the user entered their name all in lowercase, so the application capitalizes the first letter
6. If the request is for data from the database, the application will ask the database for the data. When it comes back, maybe it gets re-formatted, or cleaned-up, before being sent back to the user
7. Many types of requests need to be authorized, so that not just anybody can get into the system and do just anything. Sometimes this is done using usernames and passwords. Sometimes it is outsourced to another service (sign in with Facebook!).

In summary (and since I'm too lazy to draw a picture):

    User --> Web Server --> Application --> Database --> Application --> Web Server --> User

Scaling Them Up

Now suppose you have more users - more customers, more employees, whatever. It might be obvious that the system will slow down, but how exactly does this happen?

Let's consider an analogy from the real world (which is actually more than an analogy, it really works just like this). Think of a fast-food restaurant, say one which makes sandwiches. They have 4 cashiers where customers place their orders (their requests) and the cashier forwards their order to the kitchen (the application, which starts the workflow). The workflow might look like this:

1. Get some bread
2. Put mustard on the bread
3. Put meat and cheese on the bread
4. Put the bread in the oven for 3 minutes
5. Put it on a plate with some chips and signal that its ready

Then the cashier would pick it up and give it to the user. We also assume that the restaurant hasn't figured out how to let a cashier serve more than one customer at a time - which means that until the sandwich has been made, that cashier's tied up.

Now imagine that the restaurant has a lot of customers one day. What happens?

Well, the cashiers work pretty fast, so they can handle the load. And the cooks are pretty fast too. But:

1. The oven can only handle one sandwich at a time, so there's more waiting there

Okay, so the customers wait. But if this happens repeatedly, the restaurant owners will want to do something about it. They have a few options, all of which will have an impact, but none of which will "solve" the problem permanently.

1. Buy a bigger oven
2. Buy more ovens
3. Hire more cooks

(Also, until the kitchen problem is solved, there's no point in hiring more cashiers).

Here's the analogy: The oven is the database server. A bigger oven is a more powerful server - more memory, faster CPU. More ovens means more database servers. The cooks are the application servers and the cashiers are the web servers.

The cashiers are all the same, and their job is easy. The cooks are all similar, but maybe some specialize in Reubens and some in pastrami on rye. 

The database is the real work-engine, that's where the time is spent. The performance of the application will eventually just be the performance of the database.

Terminology

Let me close with some terminology.

• Scaling up: This means making a component more powerful, generally by throwing CPU and RAM at it
• Scaling out: Add more components of the same type at that layer (like hiring more cashiers). Scaling out is cheaper (2 Marutis are cheaper than a Mercedes), but the application needs to have been designed to work that way.
• High Availability: A deployment that is fault-resistant due to intentional redundancy. Like having extra cooks waiting in case a working cook gets sick or injured.
• Disaster Recovery. How does the application react when something really bad happens e.g. the data center gets hit by a hurricane, or a virus attack. DR strategies generally involve setting up an entirely separate (and geographically separated) center, ready and waiting for such scenarios. Of course if they're ready and waiting, they cost money even while there's no disaster in-progress!
• Replication: Keep copies of the data, to protect against drive failure. Disk drives are mechanical objects, and they are the components which fail first and fail often. Data must always be stored in multiple locations

One part I left out is the entire portion of flow between the client and the web servers i.e. the network part. I left it out because one generally has no control over the network, especially if one is using the Internet. Your control starts at the web front-end, and that's where you start optimizing.