Project update: location

In my project, the mobile application is supposed to periodically obtain user's current location and record it into a database; this process should be done even when user quits the app. Previously, my solution is to set a recurring task using AlarmManager, which will send an alarm every 30 seconds after user starts it; another class, extends AlarmReceiver, will start a service every time it receives the alarm task; lastly, in the service class, I create a location manager to obtain location using getLastKnownLocation().

This implementation, technically, issues location tracking task exact every 30 seconds. Two problems with this. First is, although the alarm is generated every 30 seconds, the location tracking might not be finished within 30 seconds every time. There might a situation that the mobile phone has not got any location for the current alarm task when the next one is generated. I am not so sure if this is 100% correct, but I do find sometimes there will be delay in updating location, like no location update for 5 minutes, then several new location updates appear in database with the same coordinates. Anyway, I am suspecting it is because my Activity Alarm -> Service -> LocationManager process take too much time. Another problem is, every task the mobile phone creates a new service, which seems inefficient and unnecessary.

Several days ago I find this post writing about LocationManager and LocationListener, come to my rescue. Basically it explains how the method requestLocationUpdates(long minTime, float minDistance, Criteria criteria, PendingIntent intent) works. This methods aims at saving energy while giving accurate location tracking. The minTime parameter gives the time duration that the location provider will rest (status changes into unavailable) before it activates and obtain location again. The actual time interval will be equal or greater than minTime, due to several reasons: location update only be sent to the app if the difference between new and old location is greater than minDistance, also the provider might take some time to obtain the latest location. This definitely takes longer time than getLastKnownLocation(), since the latter one uses cached location.

Anyway, it seems I could use this request method directly for my periodically update, while it is not exact update, it has several advantages: there will only updates if necessary, if user does not move, my previous version will still update as alarms go on and off; also it is definitely more efficient than multiple services. Now my activity will start the service after user presses the start button, which will then register a LocationListener and request location update. Note this request will not stop until the service stops or I explicitly use removeUpdate method. Lastly, in my class of Location Listener, I put location into database in the onLocationChanged method: it gets invoked every time there is a location update from the location listener.

Some digress, I just find this Notification class which could perfectly display the app status in the notification bar. Previously, I use a textview to hold the status passed from service, but setting it globally visible is better, especially when user quits the app.


I use the sample code comes with SDK. But it is somehow deprecated, thus I change a little. This snippet of code would be put in the onCreate method of my service, so that every time my service starts there will be a notification in the bar. I also put a cancel method in the onDestroy method so that it gets killed when the service stops. Note Notification class is in API level 11 (Honeycomb), which means if you are using the same API level I use (2.3.3, level 10, GingerBread) or lower, you need to import support package in your project.

I will test this new version tomorrow.

Lab project update 3: asyncTask, db file export

This is a minor update, to add the function of exporting local db file onto SD card. The reason why we do export instead of directly dragging the db file using some file explorer app it because, when testing on device, unless you root your phone, you will not be able to access your db files using apps.

Looking through android doc (btw, newest official site looks absolutely sexy) and stack overflow, I decide to use AsyncTask class to do this background job. It looks like Service, but is easier to use to communicate with the main UI thread. Unlike Service class, in which you have to care about when to create, start, handle message, stop the thread, this class provides exact functions that wrap up those details, including one before you start the task (onPreExecute), one to do background job (doInBackground), one to update main thread if you want to (onProgressUpdate) and one to return some results after the task is finished (onPostExecute). Details could be found in the doc.

Normally, it requires to override at least the doInBackground method, also in most of the time the onPostExecute method. My second method is nothing new, just making a Toast to indicate whether file has been correctly exported. My first method:

Let's go through each step. Line 3 is to locate the db file you want to export. I use the Environment class to obtain path info. Note in android, the db file of an app is created in the path "/data/data/your.package.name/databases/your_db_name.db".  The method getDataDirectory() will return the first "/data" therefore for LOCTABLE_PATH you only need to add the path after it. Line 5 gets the external dic state, which I use in line 6 to detect if the SD card is writable, defined as MEDIA_MOUNTED. If SD card is not available, I will just return a Toast and finish the task.

Line 7 calls the getExternalStorageDirectory() to obtain the SD card dir, which should be "/mnt/sdcard", you could define the dir you would like to save your db file as EXPORT_PATH. The following if statement is to check if the path you want already exists, otherwise create it. Line 11 is to create a file object at your given path. Note currently you haven't created an actual file, you just create an object and make it ready to generate a file. Also, in order to write to the external disc, you have to add following permission in your AndroidManifest.xml:

Lastly the try block is to create the file and copy your db file to it. The copyfile method should be defined by you according to what kind of file you want. I recommend just copying raw content into a .db file and then open/read it in a db browser like this. Raw file copying method in java could be found here.

Lab project update 2

First, I didn't convert GET to POST as I said before. Turns out it's pretty difficult to use 3rd party POST request to communicate with a django app: django has banned it for safety issues, particularly CSRF attack. It would take a while if one wants to reenable it. Maybe another time I will sit down and get it over in the future.

Today I find another thing though. So Google Places API supports sorted result, i.e., return a list of places that is sorted based on either 'prominence' or 'distance'. Sorted by distance is just what I need because I need to predict where the user is and of course the geographically nearest place is a good start. However, when you use this feature, it requires you to put in at least one of other three options: keyword, name or types. The first two of course do not fit since I don't know where the user is; the last one makes sense only if we include all types it supports (it has a list). Well, unless there is another to sort by distance, I decide to include all types. This takes me 5 minutes using regular expression. But I do not update the function in the android app, therefore places it stores are still sorted by prominence (by default).

Previously the db in my web app only has fields for time, latitude and longitude, since I am gonna find the nearest place for each pair of coordinates, I decide to modify my db to add three fields: place name, place latitude and place longitude. In this case, it is the best time to learn South. South is a db migration tool for django. Db migration is to let you modify db attributes without wiping all current data. Use db is very simple and it has a great doc support. But one thing though, heroku has three environments, every one needs its own migration, but alway you sync between them. Therefore, you have to be careful that all dbs should be in the same stage as you develop. Otherwise, things could get pretty ugly.

Last thing is I finish the web app with functions to draw the original point, the nearest place obtained from google places, remove points and their places and some nice UI from twitter bootstrap. Twitter bootstrap is such an awesome project that no one would realize its awesomeness until you render your site and play with it. Currently my web app is just look-able, I will dig more from this bootstrap later.

Lab project update 1

Today I work on the project for my summer intern in the lab. Previously, I build it so that it could record and store user's location in the local db periodically in the background, also make it work with google places API and 4sq API. Since we are aiming at "guess" user's semantic location (like in a shop, in a campus, etc) based on their coordinates (either from GPS or cellular or wifi), although we haven't come up with a cool algorithm to guess, it would be better to draw our initial guess on the map so we could have a feel about it.

Instead of integrating a google map with the current android app, I choose to build a web app and let the android app send data to the web app so that it would draw the map. The major reason is we won't need a map, live or not, in the app; it would be overkill. The map is more for analyzing and improving our guess algorithm, not a part of implementation. Besides, I am more confident on my python skill in writing algorithms than Java.

Therefore, my second heroku app born with duty. I still use django. This app is expected to be able to receive data from http request ( generated from the android app), store into its own db, then draw markers using such data on google map. That's it. Might improve later but first let's get it going.

For the http communication part, I look into receiving http request first. It's easy in web framework because they are made to receive http request. For data transmission, coordinates, specifically, I choose to add them into the request url on the sender side. Then when the web app receives the url it will parse it and find correct data:
This method is used as the function described above. Track is the model I create to store coordinates data. After parsing, it will redirect to the home page, if necessary, because the request is only for transmitting data, not displaying web page. Yes you could yell at me because I use GET instead of POST to transmit data. But again, first I want to get it going. GET is fast to implement that is why. I promise I will fix it and return you a long long post about how good and safe to use POST instead of GET.

After this on the android app side, there are a lot of tutorials to tell you how to setup a http request in android/Java using bundled apache libs. Anyway, I insert the request function in my service, so that every time it updates the coordinates it will also send them to my web app.

So that's the first part. How to draw them on the map? after some googling I realize the fact that there is simply no easy and once-for-all effort to create a google map in python. Then I just use js API in my django app. Luckily google has a detailed dev guide. Another good news is, as long as you keep your js script within the html file, you could use django template language to call variables of your app in the js script; it will not work in an independent js file. The rest of story is easy, just do a for loop to extract each pair of coordinate out from the db and create new marker.

I am gonna test it maybe this weekend, after I make some improvements.

Using Google places API in Android

Google places API is a service provides information about places. Basically, it will take coordinates and other additional parameters (like city, limits, etc) as input, return information related to the input such as name, address, types of nearby places. Think about Google map and their street-view car; the accuracy of this API should not be bad. I take this as an alternative to the Foursquare API to obtain places information. Actually, I expect it outperforms 4sq because most of places in 4sq are created by users, which leads to inevitable noise.

Anyway, I look into the java library of places API since I need them in my android app. Turns out the bad news is they don't have any specific java lib; moreover, 3rd party java libs I find on github are poorly supported. The good news is, Google provides a general "Google API console" and also its java lib, google-api-java-client. Although this lib does not support places api specifically, build a wrapper using this and places api is sufficient for me. The more good news is, I find this awesome blog and its corresponding sample on github. However the blog is written one year ago thus some methods it uses are deprecated in today's new version. I modified the sample to make it compatible with the latest java client, find it on my github.

Since the blog actually tells us everything we need to know about Places API, I will just skip the basics and write down how the process works in case I forget. Generally, to build your own application using this API, you need to always look into three web pages: the blog, the java doc of Google API and the official doc of places API. 

Basically there are two steps: to request places information using Places API and selective parameters, then parse the output into strings we need. First step, use HttpFactory as parameter to generate a Http request; the HttpFactory object contains a json parser and an arbitrary header. Then we use request.getUrl() to put every option we want to customize our request and send it using request.execute(). Now the second step will automatically be executed when the server returns the result. The result is in json, the json parser in httpFactory would parse it into a java model. Then the blog propose a very clever method (at least to me, a java newbie): create some classes to catch some specific strings from models. With the decorator "@Key" we could define what part of that result we want, e.g. name, types, etc. This is really efficient. Then you do whatever you want with the result.

Places API provides three search URL (I only look into search part), general search (return a list of nearby places), search detail (return the detailed information of a place), search autocomplete (predict and return places based on input). I only need the first two, but the third one is really cool: adding appropriate processing blocks, it could become a real-time prediction search just like google instant search.

Finally, there is one thing holds me back several times. The AndroidManifest.xml file. NOW REMEMBER: you have to explicitly indicate them if you add following components to your project:

1. permissions, internet, gps, etc;
2. service, including intent services;
3. content providers

Every time I use these components I forget to add them and then stuck with it for a while. Now it is done here and I will never make the same mistake again. Also, any networking thing (http GET/POST, for example) is not allowed to be done in the main thread; you have to use a service or similar technique.