Nowadays, to use google api correctly in your web applications, most likely you have to use its own authentication oauth 2.0. They say it is simple and easy; to a newbie of web dev it is not. Anyway, I failed to integrate it with my web app, thusly I decide to learn auth from the scratch, and hope one day my web app will be popular enough to demand an authentication system (I mean it!).
Django has its own built-in auth package, which is ideal to use learn the concepts. Most importantly, built-in packages like this always come with built-in views, which could save you a lot of time to get started.
Let's see how to use built-in views: really simple.
In urls.py, we define three extra patterns besides admin ones. One is for the index view, the home page, which will require login authentication to see its content. The other two utilize the built-in views for login and logout. Note for login view you have to define a template to make it go if you don't want to copy the default template from django package; for logout I here use the logout_then_login view, which is pretty self-explained. Since it will redirect to login page automatically, we don't have to add any template for this view.
Now take a look at view.py:
Two things to know: @login_required decorator indicates that this view needs authentication, if currently the user is not logged in, it will take the user to the login view we defined above; if already logged in, it will render the index view as required.
I don't include the template it uses because you could find it in django doc. Only one thing is a little bit tricky: the default login view will pass several variables to the template, including one called next. This contains the url that the app will redirect to after successful login, which should be the url triggered this login (in our case it is "/" for index view). To make the redirect happens, in the template you have to define a redirect to the next, like this if your login page uses a form: <input type="hidden" name="next" value="{{ next }}"/>.
Lastly, remember to add a link to logout url in your index page in order to complete the whole login/logout process.
All done.
Thursday, June 28
Euler project 12: triangle number
I am writing this only because I find these simple math trick is like magic.
Triangle number is ones composed by adding previous number together, like the additive analog of factorial number: Tn = 1 + 2 + 3 + ... + n. Now it asks what is the first triangle number with more than 500 divisors?
There are several ways to generate a triangle number. One is use the plain equation to sum up; other one is clearly use the sum-up equation Tn = n*(n-1)/2. However, since what we basically need here is to iterate all triangle numbers starts with 1 (or some bigger yet still small number), I add a step every time using Tn = Tn-1 + n. I think latter two should be mostly at the same speed.
Another thing is to find the number of divisors for a number. This is just iterate from 1 to sqrt(n), according to many math properties. Also since divisors appear in pairs, one might add two every time it finds a fit one if one uses this method; but minus one if the divisor it finds happens to be the sqrt(n).
Codes:
Triangle number is ones composed by adding previous number together, like the additive analog of factorial number: Tn = 1 + 2 + 3 + ... + n. Now it asks what is the first triangle number with more than 500 divisors?
There are several ways to generate a triangle number. One is use the plain equation to sum up; other one is clearly use the sum-up equation Tn = n*(n-1)/2. However, since what we basically need here is to iterate all triangle numbers starts with 1 (or some bigger yet still small number), I add a step every time using Tn = Tn-1 + n. I think latter two should be mostly at the same speed.
Another thing is to find the number of divisors for a number. This is just iterate from 1 to sqrt(n), according to many math properties. Also since divisors appear in pairs, one might add two every time it finds a fit one if one uses this method; but minus one if the divisor it finds happens to be the sqrt(n).
Codes:
Monday, June 25
GRE helper project update 1
UPDATE: I cut this project. It turns out giving her a list generated by python script is much faster. Not everything needs a web app.
I start yet another django project on heroku today. It's a rather simple one, just to help my girlfriend memorize vocabulary and prepare her GRE test.
GRE is difficult for people whose first language is not English. The test involves so many non-daily English. You have to memorize, like hardcoded in your brain, thousands of words in order to just understand what this test is talking about, not mention other requirements. A typical preparation for GRE involves at least 4 months of memorizing words repeatedly. I have been through this before, it's a very stressful and boring period of time, especially when you spend several weeks and look back, realize you don't remember one damn word. Sadly I could not turn it into a happy thing; however, I think I could make it efficient and maybe shorten its duration somehow.
What I want it to do is let you type in how many vocabulary lists (do you want to know how many words Chinese have to memorize for just a single test?), then it will generate a calendar filled with each list as task, and distributed based on a predefined memory curve. Basically a customizable repetitive task generator hooked up with google calendar. I know it is still dull, but an organized and well-executed repetition on memorizing could efficiently reduce the amount of time you need to achieve your goal. For the calendar, I don't want to reinvent such a good calendar system, besides Google calendar API has a good python support.
Start with the model. It is pretty simple and common, but I intend to add maybe a list as a instance variable so that every list could have a list of dates on which they needs to be memorized. For the progress field, which is supposed to be within 0 to 100, I add two validators from django.core.validators to prevent overflow.
I make just one page, in which you will have a input area for you to type in how many lists you have, and a display area to show the calendar. The input with the submit button will issue a http POST request to server side, along with some parameters, including the number of your lists. Note django itself disable post method automatically to prevent CSRF attack. To reenable it:
TBD...
I start yet another django project on heroku today. It's a rather simple one, just to help my girlfriend memorize vocabulary and prepare her GRE test.
GRE is difficult for people whose first language is not English. The test involves so many non-daily English. You have to memorize, like hardcoded in your brain, thousands of words in order to just understand what this test is talking about, not mention other requirements. A typical preparation for GRE involves at least 4 months of memorizing words repeatedly. I have been through this before, it's a very stressful and boring period of time, especially when you spend several weeks and look back, realize you don't remember one damn word. Sadly I could not turn it into a happy thing; however, I think I could make it efficient and maybe shorten its duration somehow.
What I want it to do is let you type in how many vocabulary lists (do you want to know how many words Chinese have to memorize for just a single test?), then it will generate a calendar filled with each list as task, and distributed based on a predefined memory curve. Basically a customizable repetitive task generator hooked up with google calendar. I know it is still dull, but an organized and well-executed repetition on memorizing could efficiently reduce the amount of time you need to achieve your goal. For the calendar, I don't want to reinvent such a good calendar system, besides Google calendar API has a good python support.
Start with the model. It is pretty simple and common, but I intend to add maybe a list as a instance variable so that every list could have a list of dates on which they needs to be memorized. For the progress field, which is supposed to be within 0 to 100, I add two validators from django.core.validators to prevent overflow.
I make just one page, in which you will have a input area for you to type in how many lists you have, and a display area to show the calendar. The input with the submit button will issue a http POST request to server side, along with some parameters, including the number of your lists. Note django itself disable post method automatically to prevent CSRF attack. To reenable it:
- add 'django.middleware.csrf.CsrfViewMiddleware' to the MIDDLEWARE_CLASSES in your settings.py (this should be done by default in the latest version now)
- right after your <form> tag that involving POST action, add {% csrf_token %}
- in the view associated with this POST request, add context_instance=RequestContext(request) as a parameter in your http response.
TBD...
Sunday, June 24
国内翻墙用google drive, add line numbers for gist
[I decide to keep some tips and tricks in my blog in case I forget them. So if it is in Chinese, then it means you don't have to care about it if you are not one (I mean it).]
1. 翻墙用drive
很简单，目前修改host的方法就可以。打开相应系统的host文件，在尾部添加“74.125.224.231 drive.google.com” 。
host文件位置，第一行为Windows, 第二行为Mac：
低调一点。
2. add line numbers for gist
Ok, this is handy. Solution is here, just paste the code into your css file. Also, here provides the css code to let others copy your code without line numbers.
1. 翻墙用drive
很简单，目前修改host的方法就可以。打开相应系统的host文件，在尾部添加“74.125.224.231 drive.google.com” 。
host文件位置，第一行为Windows, 第二行为Mac：
低调一点。
2. add line numbers for gist
Ok, this is handy. Solution is here, just paste the code into your css file. Also, here provides the css code to let others copy your code without line numbers.
Lab project update 3: asyncTask, db file export
This is a minor update, to add the function of exporting local db file onto SD card. The reason why we do export instead of directly dragging the db file using some file explorer app it because, when testing on device, unless you root your phone, you will not be able to access your db files using apps.
Looking through android doc (btw, newest official site looks absolutely sexy) and stack overflow, I decide to use AsyncTask class to do this background job. It looks like Service, but is easier to use to communicate with the main UI thread. Unlike Service class, in which you have to care about when to create, start, handle message, stop the thread, this class provides exact functions that wrap up those details, including one before you start the task (onPreExecute), one to do background job (doInBackground), one to update main thread if you want to (onProgressUpdate) and one to return some results after the task is finished (onPostExecute). Details could be found in the doc.
Normally, it requires to override at least the doInBackground method, also in most of the time the onPostExecute method. My second method is nothing new, just making a Toast to indicate whether file has been correctly exported. My first method:
Let's go through each step. Line 3 is to locate the db file you want to export. I use the Environment class to obtain path info. Note in android, the db file of an app is created in the path "/data/data/your.package.name/databases/your_db_name.db". The method getDataDirectory() will return the first "/data" therefore for LOCTABLE_PATH you only need to add the path after it. Line 5 gets the external dic state, which I use in line 6 to detect if the SD card is writable, defined as MEDIA_MOUNTED. If SD card is not available, I will just return a Toast and finish the task.
Line 7 calls the getExternalStorageDirectory() to obtain the SD card dir, which should be "/mnt/sdcard", you could define the dir you would like to save your db file as EXPORT_PATH. The following if statement is to check if the path you want already exists, otherwise create it. Line 11 is to create a file object at your given path. Note currently you haven't created an actual file, you just create an object and make it ready to generate a file. Also, in order to write to the external disc, you have to add following permission in your AndroidManifest.xml:
Lastly the try block is to create the file and copy your db file to it. The copyfile method should be defined by you according to what kind of file you want. I recommend just copying raw content into a .db file and then open/read it in a db browser like this. Raw file copying method in java could be found here.
Looking through android doc (btw, newest official site looks absolutely sexy) and stack overflow, I decide to use AsyncTask class to do this background job. It looks like Service, but is easier to use to communicate with the main UI thread. Unlike Service class, in which you have to care about when to create, start, handle message, stop the thread, this class provides exact functions that wrap up those details, including one before you start the task (onPreExecute), one to do background job (doInBackground), one to update main thread if you want to (onProgressUpdate) and one to return some results after the task is finished (onPostExecute). Details could be found in the doc.
Normally, it requires to override at least the doInBackground method, also in most of the time the onPostExecute method. My second method is nothing new, just making a Toast to indicate whether file has been correctly exported. My first method:
Let's go through each step. Line 3 is to locate the db file you want to export. I use the Environment class to obtain path info. Note in android, the db file of an app is created in the path "/data/data/your.package.name/databases/your_db_name.db". The method getDataDirectory() will return the first "/data" therefore for LOCTABLE_PATH you only need to add the path after it. Line 5 gets the external dic state, which I use in line 6 to detect if the SD card is writable, defined as MEDIA_MOUNTED. If SD card is not available, I will just return a Toast and finish the task.
Line 7 calls the getExternalStorageDirectory() to obtain the SD card dir, which should be "/mnt/sdcard", you could define the dir you would like to save your db file as EXPORT_PATH. The following if statement is to check if the path you want already exists, otherwise create it. Line 11 is to create a file object at your given path. Note currently you haven't created an actual file, you just create an object and make it ready to generate a file. Also, in order to write to the external disc, you have to add following permission in your AndroidManifest.xml:
Lastly the try block is to create the file and copy your db file to it. The copyfile method should be defined by you according to what kind of file you want. I recommend just copying raw content into a .db file and then open/read it in a db browser like this. Raw file copying method in java could be found here.
Saturday, June 23
RSpec: TDD 1
Test Driven-Development, is used to write test for desired functions of your program before you write actual code. General steps would be:
In this case, your tests would lead you through development, to write your functions. Assuming your tests are consistent with your requirement, TDD will reduce bugs in your code because they ensure that you are building the program correctly.
In Ruby, RSpec is used to do this TDD task. Actually it is also involved in BDD, that is why I think TDD and BDD should be worked together. In fact I am not 100% sure what is TDD, what is BDD. Anyway, in rspec, tests are mostly like this:
This is a test for the function to search movie titles in the TMDB. Line 3 tells us it is a test for MoviesController, a controller file. Line 4 is to indicate our desired function, to "search TMDb". Then line 8, 18, 21, sentences that started with "it" are actually desired behaviors for our function. They will represent three blocks as shown, and each of the block will contains a series of smaller test. Note by this time we don't have any codes to realize such behaviors. It looks a lot like Cucumber syntax, but somehow more concise, more focus on general i/o than detail steps. Line 5-7 is a block that would be executed every time in the following three blocks. These are similar to background steps in Cucumber. Inside the before block, we create a fake result using mock method, which create 2 fake Movie object. I think this is another play of convention over configuration because the method does not explicitly state Movie, maybe this is test for MoviesController so it automatically creates Movie object. These fake objects are used to test whether there will be a method called.
In line 8-12, it is the first test, to "call the model method that perform TMDb search". Like I say above, to pass the test, we should have a model method, also if we pass some parameters it could return something, that is basically what line 9-10 says. Note although here is a model method, but we test on our controller. Thus, we have to include a call to a model method (it even does not need to exist in models.rb) in controller explicitly. What is more, even you have a model method with this name in the model file, it will get overwritten by RSpec during the test. All we do is only to pass the test. Line 11 is the action, to make a post request with given parameters.
After the first block, you could see line 13-24 is actually a nested block contains 2 tests. Because they have common steps so we create this to avoid duplicate code. Note in both 2 tests, the requirement turns from should_receive to stub. Stub also creates a model method, but it does not require it to be called. We use it because in the latter 2 tests we don't care about whether the method is called or not ( it is the 1st test's job). In 2nd test for example, we only care about if the corresponding search tmdb template is being rendered or not. For the 3rd test, we use assign method to get whatever the program send to the instance variable @movies (again, convention), because we want to know if search results are correctly sent to the template.
At lase, to make the test run we use rspec spec_file_name, or execute autotest at the project root, so that all tests would be executed automatically every time you change codes that would affect the test result. Also make sure there is a database for test; TDD of course belongs to the test environment.
Codes are not hard. But some concepts are tricky. I will continue tomorrow.
- write test code so that they will fail when executed (since you haven't implemented your function)
- write the simplest function code to make the test pass
- after passing all test, try to fill and refactor your function code
In this case, your tests would lead you through development, to write your functions. Assuming your tests are consistent with your requirement, TDD will reduce bugs in your code because they ensure that you are building the program correctly.
In Ruby, RSpec is used to do this TDD task. Actually it is also involved in BDD, that is why I think TDD and BDD should be worked together. In fact I am not 100% sure what is TDD, what is BDD. Anyway, in rspec, tests are mostly like this:
This is a test for the function to search movie titles in the TMDB. Line 3 tells us it is a test for MoviesController, a controller file. Line 4 is to indicate our desired function, to "search TMDb". Then line 8, 18, 21, sentences that started with "it" are actually desired behaviors for our function. They will represent three blocks as shown, and each of the block will contains a series of smaller test. Note by this time we don't have any codes to realize such behaviors. It looks a lot like Cucumber syntax, but somehow more concise, more focus on general i/o than detail steps. Line 5-7 is a block that would be executed every time in the following three blocks. These are similar to background steps in Cucumber. Inside the before block, we create a fake result using mock method, which create 2 fake Movie object. I think this is another play of convention over configuration because the method does not explicitly state Movie, maybe this is test for MoviesController so it automatically creates Movie object. These fake objects are used to test whether there will be a method called.
In line 8-12, it is the first test, to "call the model method that perform TMDb search". Like I say above, to pass the test, we should have a model method, also if we pass some parameters it could return something, that is basically what line 9-10 says. Note although here is a model method, but we test on our controller. Thus, we have to include a call to a model method (it even does not need to exist in models.rb) in controller explicitly. What is more, even you have a model method with this name in the model file, it will get overwritten by RSpec during the test. All we do is only to pass the test. Line 11 is the action, to make a post request with given parameters.
After the first block, you could see line 13-24 is actually a nested block contains 2 tests. Because they have common steps so we create this to avoid duplicate code. Note in both 2 tests, the requirement turns from should_receive to stub. Stub also creates a model method, but it does not require it to be called. We use it because in the latter 2 tests we don't care about whether the method is called or not ( it is the 1st test's job). In 2nd test for example, we only care about if the corresponding search tmdb template is being rendered or not. For the 3rd test, we use assign method to get whatever the program send to the instance variable @movies (again, convention), because we want to know if search results are correctly sent to the template.
At lase, to make the test run we use rspec spec_file_name, or execute autotest at the project root, so that all tests would be executed automatically every time you change codes that would affect the test result. Also make sure there is a database for test; TDD of course belongs to the test environment.
Codes are not hard. But some concepts are tricky. I will continue tomorrow.
Friday, June 22
Lab project update 2
First, I didn't convert GET to POST as I said before. Turns out it's pretty difficult to use 3rd party POST request to communicate with a django app: django has banned it for safety issues, particularly CSRF attack. It would take a while if one wants to reenable it. Maybe another time I will sit down and get it over in the future.
Today I find another thing though. So Google Places API supports sorted result, i.e., return a list of places that is sorted based on either 'prominence' or 'distance'. Sorted by distance is just what I need because I need to predict where the user is and of course the geographically nearest place is a good start. However, when you use this feature, it requires you to put in at least one of other three options: keyword, name or types. The first two of course do not fit since I don't know where the user is; the last one makes sense only if we include all types it supports (it has a list). Well, unless there is another to sort by distance, I decide to include all types. This takes me 5 minutes using regular expression. But I do not update the function in the android app, therefore places it stores are still sorted by prominence (by default).
Previously the db in my web app only has fields for time, latitude and longitude, since I am gonna find the nearest place for each pair of coordinates, I decide to modify my db to add three fields: place name, place latitude and place longitude. In this case, it is the best time to learn South. South is a db migration tool for django. Db migration is to let you modify db attributes without wiping all current data. Use db is very simple and it has a great doc support. But one thing though, heroku has three environments, every one needs its own migration, but alway you sync between them. Therefore, you have to be careful that all dbs should be in the same stage as you develop. Otherwise, things could get pretty ugly.
Last thing is I finish the web app with functions to draw the original point, the nearest place obtained from google places, remove points and their places and some nice UI from twitter bootstrap. Twitter bootstrap is such an awesome project that no one would realize its awesomeness until you render your site and play with it. Currently my web app is just look-able, I will dig more from this bootstrap later.
Today I find another thing though. So Google Places API supports sorted result, i.e., return a list of places that is sorted based on either 'prominence' or 'distance'. Sorted by distance is just what I need because I need to predict where the user is and of course the geographically nearest place is a good start. However, when you use this feature, it requires you to put in at least one of other three options: keyword, name or types. The first two of course do not fit since I don't know where the user is; the last one makes sense only if we include all types it supports (it has a list). Well, unless there is another to sort by distance, I decide to include all types. This takes me 5 minutes using regular expression. But I do not update the function in the android app, therefore places it stores are still sorted by prominence (by default).
Previously the db in my web app only has fields for time, latitude and longitude, since I am gonna find the nearest place for each pair of coordinates, I decide to modify my db to add three fields: place name, place latitude and place longitude. In this case, it is the best time to learn South. South is a db migration tool for django. Db migration is to let you modify db attributes without wiping all current data. Use db is very simple and it has a great doc support. But one thing though, heroku has three environments, every one needs its own migration, but alway you sync between them. Therefore, you have to be careful that all dbs should be in the same stage as you develop. Otherwise, things could get pretty ugly.
Last thing is I finish the web app with functions to draw the original point, the nearest place obtained from google places, remove points and their places and some nice UI from twitter bootstrap. Twitter bootstrap is such an awesome project that no one would realize its awesomeness until you render your site and play with it. Currently my web app is just look-able, I will dig more from this bootstrap later.
Subscribe to:
Posts (Atom)