Lab project update 2

First, I didn't convert GET to POST as I said before. Turns out it's pretty difficult to use 3rd party POST request to communicate with a django app: django has banned it for safety issues, particularly CSRF attack. It would take a while if one wants to reenable it. Maybe another time I will sit down and get it over in the future.

Today I find another thing though. So Google Places API supports sorted result, i.e., return a list of places that is sorted based on either 'prominence' or 'distance'. Sorted by distance is just what I need because I need to predict where the user is and of course the geographically nearest place is a good start. However, when you use this feature, it requires you to put in at least one of other three options: keyword, name or types. The first two of course do not fit since I don't know where the user is; the last one makes sense only if we include all types it supports (it has a list). Well, unless there is another to sort by distance, I decide to include all types. This takes me 5 minutes using regular expression. But I do not update the function in the android app, therefore places it stores are still sorted by prominence (by default).

Previously the db in my web app only has fields for time, latitude and longitude, since I am gonna find the nearest place for each pair of coordinates, I decide to modify my db to add three fields: place name, place latitude and place longitude. In this case, it is the best time to learn South. South is a db migration tool for django. Db migration is to let you modify db attributes without wiping all current data. Use db is very simple and it has a great doc support. But one thing though, heroku has three environments, every one needs its own migration, but alway you sync between them. Therefore, you have to be careful that all dbs should be in the same stage as you develop. Otherwise, things could get pretty ugly.

Last thing is I finish the web app with functions to draw the original point, the nearest place obtained from google places, remove points and their places and some nice UI from twitter bootstrap. Twitter bootstrap is such an awesome project that no one would realize its awesomeness until you render your site and play with it. Currently my web app is just look-able, I will dig more from this bootstrap later.

Lab project update 1

Today I work on the project for my summer intern in the lab. Previously, I build it so that it could record and store user's location in the local db periodically in the background, also make it work with google places API and 4sq API. Since we are aiming at "guess" user's semantic location (like in a shop, in a campus, etc) based on their coordinates (either from GPS or cellular or wifi), although we haven't come up with a cool algorithm to guess, it would be better to draw our initial guess on the map so we could have a feel about it.

Instead of integrating a google map with the current android app, I choose to build a web app and let the android app send data to the web app so that it would draw the map. The major reason is we won't need a map, live or not, in the app; it would be overkill. The map is more for analyzing and improving our guess algorithm, not a part of implementation. Besides, I am more confident on my python skill in writing algorithms than Java.

Therefore, my second heroku app born with duty. I still use django. This app is expected to be able to receive data from http request ( generated from the android app), store into its own db, then draw markers using such data on google map. That's it. Might improve later but first let's get it going.

For the http communication part, I look into receiving http request first. It's easy in web framework because they are made to receive http request. For data transmission, coordinates, specifically, I choose to add them into the request url on the sender side. Then when the web app receives the url it will parse it and find correct data:
This method is used as the function described above. Track is the model I create to store coordinates data. After parsing, it will redirect to the home page, if necessary, because the request is only for transmitting data, not displaying web page. Yes you could yell at me because I use GET instead of POST to transmit data. But again, first I want to get it going. GET is fast to implement that is why. I promise I will fix it and return you a long long post about how good and safe to use POST instead of GET.

After this on the android app side, there are a lot of tutorials to tell you how to setup a http request in android/Java using bundled apache libs. Anyway, I insert the request function in my service, so that every time it updates the coordinates it will also send them to my web app.

So that's the first part. How to draw them on the map? after some googling I realize the fact that there is simply no easy and once-for-all effort to create a google map in python. Then I just use js API in my django app. Luckily google has a detailed dev guide. Another good news is, as long as you keep your js script within the html file, you could use django template language to call variables of your app in the js script; it will not work in an independent js file. The rest of story is easy, just do a for loop to extract each pair of coordinate out from the db and create new marker.

I am gonna test it maybe this weekend, after I make some improvements.

Using Google places API in Android

Google places API is a service provides information about places. Basically, it will take coordinates and other additional parameters (like city, limits, etc) as input, return information related to the input such as name, address, types of nearby places. Think about Google map and their street-view car; the accuracy of this API should not be bad. I take this as an alternative to the Foursquare API to obtain places information. Actually, I expect it outperforms 4sq because most of places in 4sq are created by users, which leads to inevitable noise.

Anyway, I look into the java library of places API since I need them in my android app. Turns out the bad news is they don't have any specific java lib; moreover, 3rd party java libs I find on github are poorly supported. The good news is, Google provides a general "Google API console" and also its java lib, google-api-java-client. Although this lib does not support places api specifically, build a wrapper using this and places api is sufficient for me. The more good news is, I find this awesome blog and its corresponding sample on github. However the blog is written one year ago thus some methods it uses are deprecated in today's new version. I modified the sample to make it compatible with the latest java client, find it on my github.

Since the blog actually tells us everything we need to know about Places API, I will just skip the basics and write down how the process works in case I forget. Generally, to build your own application using this API, you need to always look into three web pages: the blog, the java doc of Google API and the official doc of places API. 

Basically there are two steps: to request places information using Places API and selective parameters, then parse the output into strings we need. First step, use HttpFactory as parameter to generate a Http request; the HttpFactory object contains a json parser and an arbitrary header. Then we use request.getUrl() to put every option we want to customize our request and send it using request.execute(). Now the second step will automatically be executed when the server returns the result. The result is in json, the json parser in httpFactory would parse it into a java model. Then the blog propose a very clever method (at least to me, a java newbie): create some classes to catch some specific strings from models. With the decorator "@Key" we could define what part of that result we want, e.g. name, types, etc. This is really efficient. Then you do whatever you want with the result.

Places API provides three search URL (I only look into search part), general search (return a list of nearby places), search detail (return the detailed information of a place), search autocomplete (predict and return places based on input). I only need the first two, but the third one is really cool: adding appropriate processing blocks, it could become a real-time prediction search just like google instant search.

Finally, there is one thing holds me back several times. The AndroidManifest.xml file. NOW REMEMBER: you have to explicitly indicate them if you add following components to your project:

1. permissions, internet, gps, etc;
2. service, including intent services;
3. content providers

Every time I use these components I forget to add them and then stuck with it for a while. Now it is done here and I will never make the same mistake again. Also, any networking thing (http GET/POST, for example) is not allowed to be done in the main thread; you have to use a service or similar technique.