Angularjs - $q, $q.all - Using deferred objects to avoid race conditions

Introduction

Since working with Ajax apps and even more so with a combination of Ajax and video players, my team has been plagued with race conditions. If the id of the video to be played is loaded via Ajax and we also need to wait for the player to be ready, how are we supposed to know which one will complete first?

We resorted to flags that only become true once both are complete. But this is an annoying task, very error prone. Deferred/promise objects are the (a) solution to this. Granted, deferred objects are in no way specific to Angular, but their implementation via $q is simple and extremely useful. In this simple app, we combine the loading of a YouTube player and a (fake) Ajax request  to load the actual video id to be played. Code can be found at: git@bitbucket.org:mchauvinc/referredtutorial.git

Note

We are still using an older version of Yeoman, so you will need to adapt the scaffolding to your own version ("yo" instead of "yeoman" in newer versions I think).
Also, we faced some issues with outdated FF Flash plugins, so this is best tested on Chrome.

Scaffolding

Using the extremely convenient Yeoman, let's put things in place. From a console:

yeoman init angular deferred
yeoman init angular:service playerService
yeoman install swfobject

You'll need to add the swfobject script to your index.html.

We'll stick to the "main" view and controller for this simple example. The service will create a swf object and return a promise object. It also defines the onYouTubePlayerReady callback which will be called by the YouTube player, when it is ready. Since the playerService factory is only called once, this should create the function only once (to be tested).

Broadcasting from $rootScope ensures that all children scopes will receive the event. In this case, the main controller's scope will be listening to it.

Service

The service does two things. Create the onYouTubePlayerReady function (once) and return an object that can be used to create (via the "create" function) any number of players which will call the onYouTubePlayerReady function when they're ready. Note that the we are loading a generic player, without any video.

Controller

In the controller, we call the service to create a player. The player variable is initialized without value, and will only be set to the <object> when the player is ready. We also initialize an empty videoId variable which gets set when the data is loaded via the Ajax call. We're faking the setting of the videoId, it does not depend on the actual result of the Ajax call.

Note the $scope.$apply call when resolving the deferred object. This is necessary because the YouTube event is "outside the world of Angular" and so a digest needs to be triggered. Though it still sometimes feels a bit magical, there is a discussion about this on the Angular 1.2 presentation video.

We then combine the two promises httpPromise and ytQ.promise into one using $q.all. It returns a new promise object, which will only be resolved once the two promises are resolved. The results variable in the callback function contains the resolve values of httpPromise and ytQ.promise, in the same order as the promise objects themselves.

By doing so, we ensure that the then callback will only occur after both deferred objects have been resolved, thus solving any form of race condition. By doing so, we make sure that when calling player.loadVideoById(videoId), both player and videoId have been set to their respecting value.
Here's the controller:

View

Using just the main view, we simply show a progress of the app's state using a messages array. Things happen in sequence, as they are expected to.

Conclusion

Deferred objects are not an Angular invention. Actually the $q implementation is clearly based on Kris Kowal's Q implementation. In any case, it allows for a simpler and less error prone solution to race conditions among other cases. $q.all is not limited to 2 promises as seen in this example but can accommodate more promise objects. 

Angularjs & Yeoman - Part 4: Global loading indicator

Introduction

In the previous posts, we've looked into Angularjs services and controllers. This time, we're going to show how to create a Loading indicator (like the "Loading..." box in your Gmail), that can be triggered on and off from within any part of the application. The app does nothing else than load some entries from an API via a $resource, and since that takes time, we want to inform the user that things are being loaded. Code can be pulled from ssh://git@bitbucket.org/mchauvinc/globalloader.git

Bootstrapping

Using yeoman, let's bootstrap this application:

yeoman init angular bubble

Let's also bootstrap a tweets route

yeoman init angular:route genres

And a service that will grab the genres from an API

yeoman init angular:service GenresService

One more item, will be a constant containing the API url, in case we want to globally change that later. This can be done via a service

yeoman init angular:service ApiUrl --value

Services

ApiUrl constant

The angular:service generator, doesn't seem to care for --value and we're getting a factory every time. Let's change /app/scripts/services/ApiUrl.js to:

bubbleApp.value('ApiUrl', "http://epg.suterastudio.com/api/genres");

GenresService

Since GenresService will return a $resource, we need to include ngResource into the app:

1. yeoman install angular-resource
2. Add the JS file to your index.html: <script src="components/angular-resource/angular-resource.js"></script>
3. Add the dependency into your app (app/scripts/app.js):
   
   var bubbleApp = angular.module('bubbleApp', ['ngResource'])
4. $resource and ApiUrl are now available everywhere in the app, so request them in the service

Here's what the service looks like (using JSONP since the API is on another domain):

View

Genres

 How things are displayed isn't really important in this post. Here's what it could look like, with a repeater:

Index.html

More importantly, since we want to display a "Loading" message globally, we will add a <div> straight in the body. The div has a ng-show="loading" directive, so it will show or hide depending on the value of $rootScope loading. The id on the element is only used for CSS purposes. Also note that we add an AppCtrl as the body's controller, it will be created next, and basically becomes the global controller.

Showing the body only:

Touch of CSS

For visual reason, let's just add the following to app/styles/main.css:

Controllers - Version 1: Using $rootScope

GenresCtrl

Using the following controller which request for $rootScope and GenresService, we can set the value of $rootScope.loading to true when landing on the page, and back to false in the complete callback of the $resource.query function:

AppCtrl

Actually not used in version 1, but since we added it to the index.html, Angular will crash if it doesn't exist. So for the time being, let's only bootstrap it:

yeoman init angular:controller app

And that should work. When going to the http://localhost:3051/#/genres url, the controller sets $rootScope.loading to true, and when the loading of $resource is complete, the callback is triggered, putting loading back to false. The loader, which follows the $rootScope.loading value is shown then hidden.

Note that $rootScope could have been accessed from $scope.$root instead.

Controllers - Version 2: Using events

The above works, but it expects GenresCtrl to be aware that the variable "loading" is controlling the loader. Also, if other components will not be informed that we are loading. In any case, this is a perfect use case for events. Angular listens using $on and triggers using $broadcast (top scope down) or $emit (bottom scope up). We decided to emit from the GenresCtrl scope up, bubbling to the AppCtrl:

Conclusion

Using events, we've managed to create a simple "Loading" box, accessible by emitting an event. The loading indicator can therefore be triggered from any part of the app.
Although we didn't include any extra scope, any parent scope of Genres, would've been able to catch the event bubbling towards the App controller. We could have used $rootScope.$broadcast to instead go from the top scope down, maybe in a next post.

Angularjs & Yeoman - Part 3

Intro

In the first post of this series, we had a look at features that we found impressive in Angular. In the second post we created a really simple edit in place "app" using the basics of Angular. This post takes things further, and, even though the app doesn't do much apart from displaying a list of genres / subgenres of TV programmes, it is well scaffolded and gives a good look at why Angular + Yeoman is awesome for creating web apps.
The code for this can be found on bitbucket: git@bitbucket.org/mchauvinc/genrespanel.git

Goal

The app will have 2 views, a "Control panel" view which holds links to the other views, and a Genres view, which displays a list of genres and subgenres which can be edited, added, removed.

Testing

Though it is highly encouraged to do so, we have not yet succeeded in getting the Jasmine tests running with dependency injection to fake the API calls. So skipping this at the moment. Any help is welcome :)

Scaffolding

As mentioned previously, one of the greatest things we found about Angular is it's scaffolding companion Yeoman. 

Yeoman is a command line tool that scaffolds your app (it can do several types of apps, but we're mostly interested in its Angular abilities), creates controllers, routes, views, services, includes corresponding js files into your main html file, helps you run tests, and more. It is an awesomely powerful tool, which should remind you of RoR scaffolding.

Furthermore, it includes a web server which watches for changes on your files and reloads automatically, a pretty neat feature.

Assuming that Yeoman is fully installed, creating an app is as simple as typing this in a console:

yeoman init angular

After answering a few questions, Yeoman creates a whole bunch of files. You can actually at this stage run

yeoman server

and view your app on port 3501 by default.
But going back to the console, we can actually scaffold a few more things.

Route

Using

yeoman init angular:route genres

creates the corresponding controller, html view, adds the route to the app, and the scripts to our HTML page. Isn't that awesome? Try browsing to http://localhost:3501/#/genres

Service

The service is the link to the backend API. You can have different types of definition of a service, for example one that simply returns a value (value definition), one that behaves like a constructor (service definition) or one that returns an object (factory definition). All services are singletons, and the factory ensures separation of concerns (other parts of the app don't need to know how the API connection is made). A good description of the difference between service and factory can be found here.
We will use a factory: type

yeoman init angular:service genresService --factory

and the genres service has been added to the app. We will get into the actual service in the next section.

Getting data

The service that was created by Yeoman currently only holds a someMethod function which returns 42. Yeoman has a good sense of humor. Let's wire it instead to grab data from an API. (app/scripts/services/genres/genresService.js)

If you try to save the above in the service and reload your page, you will see that the script fails. This is because $resource isn't available. $resource is an Angular service which lets you interact with RESTful APIs easily. Alternatively, you could use $http which handles all forms of AJAX requests (not limited to CRUD objects). Since $resource is not built in, you need to

1.  add the file angular-resource.js to your scripts. You can
   yeoman install angular-resource
   to get the latest from git. Somehow, in my case, the file did not get added to the html though, so I've added it manually in index.html using <script src="components/angular-resource/angular-resource.js"></script>
2. Tell angular that the app will be using $resource by adding 'ngResource' to the (currently empty) array of "imports" in the app.js definition:
   
   var nooApp = angular.module('nooApp', ['ngResource'])

Once this is done, $resource becomes available in any parts of your application, if you request it using $resource in the function parameters. This is handled by dependency injection.

Because we are calling an external API, JSONP is required. We can override the default http methods for each action of the $resource object when initializing the method as seen above. An awesome video by John Lindquist explains almost all you need to know about $resource. Just be careful when overriding the method for "query" to include "isArray" as true again if your query returns an array of objects (it does in our case). "JSON_CALLBACK" lets Angular keep track of which padding function to use.

So, what happens is that when the service is initialized, we create a $resource object with correct API url, and make it available to any part of the app that requests genresService using dependency injection.

Controller(s)

Though we have a main controller, it isn't really being used. So let's focus on the genres controller. It actually does 4 things only.

1. Query the API for a list of genres / subgenres
2. Add a subgenre (check that it has a unique id for given genre)
3. Remove a subgenre
4. Prepare to save (we won't implement save)

A pretty neat feature here, is that when query() is called on the genresService, an empty array is immediately returned, so that our table simply looks empty while loading.

When an ng-click event is fired, the context ("this") is the Angular enhanced DOM element. That element is bound to its subgenre and even one level up to its genre. This is how we handle deleting subgenres, by going from the DOM element to the Genre object's list of subgenres (and filtering out the model corresponding to the element we clicked on)

View

The genres view will show a table of genres, with their corresponding list of editable subgenres. To that effect we will use two levels of repeaters. How repeaters work is explained in the basic examples of angularjs.org.

Also used in the view are:

• ng-click directives on Add and remove buttons
• An input per genre to create a new subgenre. Note that ng-model is set to genre.newKey because each input "belongs" to a genre
• A hidden error message which will be shown if the key is already in use, using ng-show="genre.invalidKey". Again, showing the error message depends on each genre so we watch for genre.invalidKey
• Subgenre title is editable via an input bound to ng-model subgenre.title
• doSave does not actually save, only because the API point has not been implemented, but if you look at your network activity in Chrome dev tools, you'll see the call with all the JSON encoded data.

Conclusion

The list of genres and subgenres gets populated via the API, in a really easy manner.

Removing and adding elements to the array of subgenres automatically takes care of rendering and the final data can be read from the models (instead of having to go through the rows and input elements to read the values). I think this is absolutely awesome. Not one bit of DOM manipulation required. So we can focus on the logic instead.

Angularjs & Yeoman - Part 2

First experiment

In the first post in the series on Angular and Yeoman, we mentioned a few of the cool things about Angular, without actually getting into any concrete code. This here post is a first example of using Angular. So here's an extremely simple "edit in place". In the next post, we'll create a more complex and better scaffolded app.

Goal

Create a span which contains some text, allow to edit it in place, transforming the span into an input, and re-display the value in the span on "save"
Visually, it means:
Start with initial value, not editing

 Click the pencil to edit

Change the value

Click the ok icon and preserve value

Notes

We are not trying to:

• actually save that value to a db at this stage
• build a complex app, which would be done using Yeoman and async Angular bootstrapping
• use testing, although it is highly encouraged to do so for anything more complex than this

Bootstrap CSS was including purely to make things look a little bit nicer, but is absolutely not required.

HTML

Here's what the HTML looks like:

Everything is declarative:

• ng-app in the body (or anywhere else in the DOM) informs Angular that this is an app. Note that it will only affect the children (DOM-wise) of the element containing this directive
• ng-controller sets the controller for this specific part of our app
• ng-model on the input element, specifies that the value of the element shows/reads from the variable "configuration"
• {{configuration}} means that Angular should display the value of the variable "configuration" in this place
• ng-show is a directive that allows us to watch for the value of a variable (here "editing" and "!editing") and show or hide the element accordingly
• ng-click tells Angular to react to the click event, calling the specified function accordingly

Javascript

Assuming that you are correctly including angular.js, the javascript code behind this is extremely simple. We just need to define the controller, which is a simple function put into editinplace.js:

See how we request $scope in the definition of the controller, which is provided via dependency injection, another great feature of Angularjs. We could define it as EditCtrl( $http, $scope ) or anything else, as long as we provide the injector with the corresponding mapping of variable_name to an actual variable. The order does not matter (EditCtrl( $scope, $http ) works as well). Btw $http is a built-in resource of Angularjs used to handle Ajax requests.

Conclusion

Granted this is a really simple example, but it shows some of the basics of Angularjs, and as you can see, no interaction with the DOM is required at any stage. The HTML elements could be rearranged without affecting functionality. No jQuery or other framework is required.
This also shows that even if your goal is not to build a full one-page app, Angularjs can come in very handy for model-based DOM manipulation.

More complex and amazing things (dependency injection, repeaters, several views, routing and Yeoman scaffolding) to come in the next posts in this series.

Discussion

This does not handle focusing on the input when editing, or any action on pressing "Enter". Again this is a very simple use case.

Angularjs and Yeoman

This will be the first in a series of posts about Angularjs. Next will be the creation of a simple "edit in place". This post only gives first impressions and highlights what appears to make Angularjs + Yeoman a really different framework.

Angularjs

Angularjs is "HTML enhanced for apps" aka Superheroic Javascript MVW framework.

Quite a few articles can be found about this "new" javascript framework. It seems to be relatively new out here, but Google (they're behind it) has apparently been using if for a while.
In a word, from looking at the examples on the site, this looks awesome!

Yet another framework? Not in my opinion after playing around a bit with it. Enhanced with Yeoman for scaffolding, it becomes one of the most powerful tools ever to build web apps. Even if that's not your goal, it's very neat for creating interactivity within a single page (we'll get to an example in the second post of the series).

What makes it really different?

• Two way data binding - DOM elements become attached to your models, and vice-versa. So when you change the model, the DOM changes accordingly, and if you change e.g. the value of an input, your model is aware of it
• Separation of concerns: It becomes quickly clear what each part of the MVC is in charge of.
• Enhancing HTML with your own directives, forcing the browser to become "smarter"
• Yeoman (see below): generate your apps quickly and in a clean way
• Service/Factories for reusable parts of your app
• Dependency injection which allows you to "request" components and override those components temporarily when needed, for example faking a db connection while testing
• Test driven development: it's all up to you, but every single tutorial (from the creators anyway) comes with tests and they keep emphasizing on how you should do the same. Uses Jasmine and PhantomJS
• Scopes to separate parts of your application that do not need to interact
• Uses many other successful technologies, too many to list down, but see above Jasmine, PhantomJS, Grunt, etc.
• It's from Google. Sorry, have to say it, that does carry a certain amount of weight

It also does a lot of things that other frameworks provide:

• Routing
• DOM manipulation (mostly behind the scenes)
• Ajax via $http and $resource
• Promise objects
• JSONP support

Downsides:

• Doesn't work so well with other frameworks - (read it somewhere, need to find a reference here, because I've not even been needing other frameworks so far)
• Doesn't support some DOM events (blur, I think) but I'm sure it's a work in progress
• No animations (yet), not many plugins, though angularjs ui is a pretty good start
• Learning curve. If you come from a non-programming background, it might appear pretty scary after the basic examples
• No event delegation as far as we could see
• Many dependencies, for Yeoman anyway - though Yeoman takes you through the process of getting everything you need.

Yeoman

Yeoman helps you scaffold your apps in a  very very neat way. A simple command "yeoman init angularjs" builds your app and makes sure that you won't mix up elements of your code. Is also compiles SASS, coffeescript on the fly and has a built in web server for testing. The server watches for your changes and reloads the app automatically. A good overview is found in this video.

Conclusion

All we did in this post is mention some of the features of Angular, but have not gotten into any concrete application. So unless the words "Dependency injection" excite you, you mind find our here opinion a bit premature. This opinion was actually forged from really playing with Angular and Yeoman and will make much more sense once we've come up with part 3 in this series. But here it is.

Angular is not just another framework. It does not attempt to do the same things in a slightly different and maybe better way. It really takes a perpendicular trajectory to the common approach of grabbing HTML elements and modifying them; instead of that, it creates a permanent bond between DOM elements and models. Good coding practices and test-driven development make it a first class choice to encourage clean development.
Add Yeoman to that, and you have clean, easily maintainable code, extremely fast to scaffold.

Two User Interface handlers for SL4A WebView

I'm currently working on further projects of using WebView in SL4A to create a user interface. By using the two handler classes created for the SMS Merger, the communication to/from Python/Javascript is really simplified. The rest of the code is just HTML+Javascript to create the User Interface, Python to handle the background tasks.

Python handler

Code

How it works

General

The UI Handler will be listening (waiting) for an event that will be posted by the Javascript side. This event will include data which is a JSON encoded dict. The handler will then dispatch to the function corresponding to the "task" key in the data dict. Once the task has done its job, the handler will post a response event which will be caught by the Javascript side.

Init and dispatch

The UIHandler class is actually never used directly. Each script needs to extend it into another class (e.g. class CallUIHandler(UIHandler)) and then loop on the wait function. The dispatch dict of the UIHandler is empty, meaning that any call to a given task will lead nowhere. The dispatch dict of the extended class (CallUIHandler) must "point" from a task string (e.g. "callme") to a function of the CallUIHandler (e.g. dispatch = {"callme":self.callme}). The callme method is then defined in the CallUIHandler (e.g. def callme( self, data ): ).

Receiving event and responding

While looping on the wait() function, the handler is waiting for an event (by default "python"). Data from the event is directly decoded from JSON into a python dict so the data is available to each of the task functions. The task function must return the data it wishes to pass back to the javascript side - that data will be JSON encoded before passing it to the response event, which means that it should be an dict/list/tuple (otherwise JSON encoding might fail).

Miscellaneous stuff

There are a startLoad(title,message) and a stopLoad() functions available, that are simple ways of showing/hiding an Android spinner dialog.
There is a default log function that may be overridden in the extended class.
The 0.2 second thing, is due to a glitch in the droid.waitForEvent function: If Python catches an event and posts its response too quickly, the Javascript side will miss it. So we make sure that our processing takes at least 0.2 seconds. (value was found by trial and error)

Sample extended UIHandler

This could be an example of extended UIHandler.

Javascript UIHandler

Code

How it works

The javascript side basically works the same as the python side, except that it is usually the active side, the side that posts an event and waits for a response.

post and postAndWait

These are the functions used to send an event to the python handler. post simply sends the event including the data. postAndWait waits for a response (Javascript is blocked until then).

alert

A utility function to show an alert dialog including title and message

startLoad and stopLoad

Utility functions used to show/hide a spinner dialog

delay thingy

Again, as on the Python side, there will be a problem if events are received/posted too quickly. Therefore the UIHandler has a property called "last" that records the microtime() value of the last event post. If we try too fast, it will delay the next post.

droid

Note that a global variable droid is used which needs to be defined earlier as droid = Android();

Sample of application

This could be the HTML side:

SMS Merger reviewed with Javascript/Python file browser and Setup tab

Changes from previous version

• As a advised by Frank Westlake on the Android Scripting Google group, dropped the idea of using OI File Manager to select files, replacing it with a Javascript+Python file browser - Javascript does the display part and Python the listing of directories
• Setup tab (show configuration) replaces Preview tab Setup tab loads values from the configuration file and displays them. Values can be changed and saved. A "tooltip" appears to explain the meaning of the option when clicking its name
• Preview tab becomes part of the file tab
• Added the possibility for incompatibility of fields in the CSV compared to text file without crashing. Means that the text may contain 3 merge fields while CSV only two and vice-versa
• Added the possibility for incompatibility of fields in the CSV compared to text file without crashing. Means that the text may contain 3 merge fields while CSV only two and vice-versa
• Added a version check which compares the current installed version to the configuration file version and decides whether the user needs to re-download some of the user interface components

Screenshots

Files

Here is the zip zip file.