Step by step https client/server building

Server side with apache, mysql and python scripting on ec2 ubuntu server 12.04; client side with android 2.3 and sqlite built-in. Also, both of them have secure tls connection ability.

I will go directly into the topic. Start with server side. For a basic http server on ec2 using apache, refer here. Note in this article the environment is actually different, instead of using source code to build the apache, I install apache directly using command apt-get install apache2. It surprisingly takes good care of all details and works well at least for now. The configuration is different in these two installation and I suggest you use apt-get.

When done installing, apache is already up to go. Default http configuration file is located at /etc/apache2/sites-available/default. If you fire up http://localhost you should be seeing "It works" page. This is page, index.html is located at /var/www, it serves as your site, where you could put all html files in and if your computer has an ip address, others could see your site by accessing the ip address. Another easier way to test if the server is working well is to use curl command, curl http://localhost and it will return the response of the server, which in this case the default index.html. Curl is easier to use when you want to test the response of server, you don't need any other clients to fire the request.

Now let's go into tls. I assume you already have the key file and cert file on your server. Put them into /etc/ssl/private and /etc/ssl/certs respectively, they are the default dir apache is looking at for key and cert files. Then follow this excellent doc to setup ssl module for apache. There is a default ssl configuration file you could customize: /etc/apache2/sites-available/default-ssl, it includes file directories of ssl request and so on. The default dir is the same as http connection, which is /var/www. If you put a different index.html in it, when you test using curl -k -3 https://localhost, k means accessing without any cert files and 3 is the version number of ssl protocol you are gonna use. This will give this page so that you know you are in a https connection.

Ok, now we have this ssl server working pretty well. There is one more step to go on the server side which is add the handler to deal with different requests. Now all we could request is the default page. We want more. Particularly, I need a handler that takes in a POST method, extract its data, and then put them into a table in mysql db on the server.

First to install mysql. apt-get install mysql-server mysql-client. Note, in ec2, you could directly sudo su, without typing in any password, go into root. just to make things easier, cause most of configurations and commands here need to root. Then, I install mysql python interface, python-mysqldb, you could install whatever you like, php, etc.

Then we will see how to use python script to handle http/https request. We use CGI (Common gateway interface); it is a way to make executable file like scripts request-able at client side. The default dir for cgi scripts is /usr/lib/cgi-bin. Put your scripts there and they should be immediately up for http request. Here is my echo python script:

Basically for a POST request with several key/value pairs, it will print out # of pairs and every pair. It uses a python module called cgi, and cgitb is another module to enable debugging function. Note the line 9 is necessary because it tells the server and client this is valid html text, otherwise client would probably throw out "invalid response" error. In fact CGI is not the best way to do script request, it is highly unstable when scripts get complex. But it is the easiest way to get it going. Now curl --data "key1=value1&key2=value2" https://localhost/cgi-bin/yourscript.py it will and should return the result of the script. Note --data is how you send POST request via curl.

To this point, the server setup should reach a happy ending. Now let's look at the client side. The very first thing you need to do is, in your android project, be sure to include the cert file of your server, maybe at /res/raw; it is required in tls connection. Details could be found here.

Assuming you know how to use httpClient in android, you should be already connecting your server and client. Have fun!

http/tls connection in python, android, EC2

Have a side project to build a standard tls package for the team. Ive never tried socket, so start with python just to get a feel. Following is my own experiment, to connect the server side code on EC2, and the client side code on my local laptop.

1. simple http
I use sample code from official doc. It is really simple. All you need to do other than code is configure the port for EC2 instance.

For the instance you are running, configure its security group so that the specific port you want to communicate on is open like 2727 above.

2. simple https
Things get rough with security. Basically, what I know about https, i.e. tls, is that it utilizes a public key identification system to secure the communication via http. The server has a private key, which is only known to itself. It also has a corresponding public key ready to distribute to anyone need to communicate with it. In order for the other side to trust it, the server has to have its public key certified by trusted 3rd party, called a Certificate Authority. Same with client side.

However, if we just want a connection between our own server and client, we could generate keys and certificates ourselves without paying for CA cert file. This is called self-signed certificate, or root CA certificate.

openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout cert.pem

If you have openssl installed on your computer, you could use it to generate keys. In this case, I generate private key and certificate in the same file. Then I just copy it to the other side. Both sides use the same keys. Things are simpler here, for which most cases you might wanna use a more secure authority to certify for you.

Then both sides I use sample scripts from official doc again. Note for https connection you also have to open the port for ec2 instance.

3. simple https with android
With android things are bit complex with certificates. I have this cert.pem file, which is not enough for android. Bouncy Castle encryption is supported well by android, which is the one we are gonna use to generate client side key file.

First is to install Bouncy Castle. Note android is using a different version of it, version 145, not 146 from official site. Find one, download it, a jar file. Put it in the directory '/usr/libexec/java_home/lib/ext', where on mac should be '/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/lib/ext'. Second, add following sentence into the jave.security file also located in lib folder:


Having keytool in your machine, do following with the cert.pem file:
Now you will have mykeystore.bks file in raw directory. I here use a der file because android returns 'wrong version of certificate' error. To generate der file from pem:


We are almost done here. Just grab any sample code for https connection in android, using whether httpURLconnection or httpclient, put correct password and file name into place, everything should be fine now.