Programming with Callbacks

I wrote this as an answer on StackOverflow, and have decided to make a copy here as well. The question was "What is a callback function?"

Opaque Definition

A callback function is a function you provide to another piece of code, allowing it to be called by that code.

Contrived example

Why would you want to do this? Let's say there is a service you need to invoke. If the service returns immediately, you just:

1. Call it
2. Wait for the result
3. Continue once the result comes in

For example, suppose the service were the factorial function. When you want the value of 5!, you would invoke factorial(5), and the following steps would occur:

1. Your current execution location is saved (on the stack, but that's not important)
2. Execution is handed over to factorial
3. When factorial completes, it puts the result somewhere you can get to it
4. Execution comes back to where it was in [1]

Now suppose factorial took a really long time, because you're giving it huge numbers and it needs to run on some supercomputing cluster somewhere. Let's say you expect it to take 5 minutes to return your result. You could:

1. Keep your design and run your program at night when you're asleep, so that you're not staring at the screen half the time
2. Design your program to do other things while factorial is doing its thing

If you choose the second option, then callbacks might work for you.

End-to-end design

In order to exploit a callback pattern, what you want is to be able to call factorial in the following way:

factorial(really_big_number, what_to_do_with_the_result)

The second parameter, what_to_do_with_the_result, is a function you send along to factorial, in the hope that factorial will call it on its result before returning.

Yes, this means that factorial needs to have been written to support callbacks.

Now suppose that you want to be able to pass a parameter to your callback. Now you can't, because you're not going to be calling it, factorial is. So factorial needs to be written to allow you to pass your parameters in, and it will just hand them over to your callback when it invokes it. It might look like this:

factorial (number, callback, params)
{
    result = number!   // i can make up operators in my pseudocode
    callback (result, params)
}

Now that factorial allows this pattern, your callback might look like this:

logIt (number, logger)
{
    logger.log(number)
}

and your call to factorial would be

factorial(42, logIt, logger)

What if you want to return something from logIt? Well, you can't, because factorial isn't paying attention to it.

Well, why can't factorial just return what your callback returns?

Making it non-blocking

Since execution is meant to be handed over to the callback when factorial is finished, it really shouldn't return anything to its caller. And ideally, it would somehow launch its work in another thread / process / machine and return immediately so that you can continue, maybe something like this:

factorial(param_1, param_2, ...)
{
    new factorial_worker_task(param_1, param_2, ...);
    return;
}

This is now an "asynchronous call", meaning that when you call it, it returns immediately but hasn't really done its job yet. So you do need mechanisms to check on it, and to obtain its result when its finished, and your program has gotten more complex in the process.

And by the way, using this pattern the factorial_worker_task can launch your callback asynchronously and return immediately.

So what do you do?

The answer is to stay within the callback pattern. Whenever you want to write

a = f()
g(a)

and f is to be called asynchronously, you will instead write

f(g)

where g is passed as a callback.

This fundamentally changes the flow-topology of your program, and takes some getting used to.

Your programming language could help you a lot by giving you a way to create functions on-the-fly. In the code immediately above, the function g might be as small as print (2*a+1). If your language requires that you define this as a separate function, with an entirely unnecessary name and signature, then your life is going to get unpleasant if you use this pattern a lot.

If, on the other hand, you language allows you to create lambdas, then you are in much better shape. You will then end up writing something like

f( func(a) { print(2*a+1); })

which is so much nicer.

How to pass the callback

How would you pass the callback function to factorial? Well, you could do it in a number of ways.

1. If the called function is running in the same process, you could pass a function pointer
2. Or maybe you want to maintain a dictionary of fn name --> fn ptr in your program, in which case you could pass the name
3. Maybe your language allows you to define the function in-place, possible as a lambda! Internally it is creating some kind of object and passing a pointer, but you don't have to worry about that.
4. Perhaps the function you are calling is running on an entirely separate machine, and you are calling it using a network protocol like HTTP. You could expose your callback as an HTTP-callable function, and pass its URL.

You get the idea.

The recent rise of callbacks

In this web era we have entered, the services we invoke are often over the network. We often do not have any control over those services i.e. we didn't write them, we don't maintain them, we can't ensure they're up or how they're performing.

But we can't expect our programs to block while we're waiting for these services to respond. Being aware of this, the service providers often design APIs using the callback pattern.

JavaScript supports callbacks very nicely e.g. with lambdas and closures. And there is a lot of activity in the JavaScript world, both on the browser as well as on the server. There are even JavaScript platforms being developed for mobile.

As we move forward, more and more of us will be writing asynchronous code, for which this understanding will be essential.

Writing a small application using Parse and Backbone - Part 3

The Story So Far

In the last post, I set up a Backbone view and a model, and connected the two. I also connected the view to a route so that we could see it render with the data. This time I want to figure out:

1. How to query the Facebook Graph API
2. How to persist data to the Parse backend

The Graph API

The first thing you will need to hit the Graph API is an access token. This token as an associated set of permissions which will determine which of your queries succeed. When you play with the Graph API Explorer, you will see a button saying "Get Access Token", which brings up a window asking you which permissions you are going to ask the Facebook user for:

When you do this through Parse, you will ask for permissions in the call to Parse.FacebookUtils.logIn. If the user grants you the permission you want, the access token will be in Parse.User.current()._serverData.authData.facebook.access_token.

To use this token to make a query, you call the FB.api function. Which works fine... except when the FB object doesn't exist yet. Or the Parse.User hasn't signed in. Or the access token has expired. All of which are possible, especially if the initialization flow is still in progress.

To handle this I wrote a little helper:

 facebookQuery = function(queryString, callback) {  
     var doer = function () {  
       if ((!!window.FB) && (!!Parse.applicationId) && (!!Parse.User.current())) {  
         window.FB.api(queryString, { 'access_token': Parse.User.current()._serverData.authData.facebook.access_token }, function(response) {  
           // token expired! this logic needs to be a)understood, and b)put everywhere  
           if(response.hasOwnProperty('error') && response.error.code === 190) {  
             Parse.User.logOut();  
             Invite.loginUser(doer);  
           }  
           else {  
             callback(response);   
           }  
         });  
       }  
       else {  
         // wait 0.1s  
         setTimeout(doer, 100);  
       }  
     };  
     doer();  
   };  

The 190 error code is what Facebook returns if the access token has expired. What I do is I log out and log back in, and this seems to work.

Persisting to Parse

They have actually made this pretty easy - the Parse.User object has a save method you can call. Here is some code which shows a Graph query for the logged-in user's profile information - you can see that I am asking for only the name, gender, and cover photo. Then I call the set method to assign to the fields, followed by save to send it to their servers.

 facebookQuery('/me?fields=name,cover,gender', function(response) {  
       
       // information to populate the model  
       var info = {  
         fbId : response.id,  
         realname : response.name,  
         gender : response.gender,  
         privacy_non_fb_see_only_name : Parse.User.current().get("privacy_non_fb_see_only_name"),  
         photoURL : response.cover && response.cover.source  
       };  
       // overwrite existing values, if any, and send to Parse  
       _.each(info, function(val, key) {  
         Parse.User.current().set(key, val);  
       });  
       Parse.User.current().save();  
       //   
       next(info);  
     });  

Next Time

In the next post I'm going to build the core functionality of this application, which is the creation of an invite. The code is on GitHub if you want to follow along!

Update (May 10 2014)

I'm not going to get to the next post for a while, some other tasks have come up at higher priority! I'll get to it when I get to it.