Showing posts with label database. Show all posts
Showing posts with label database. Show all posts

Monday, March 3

Basic mongodb administration

Recently picked up mongodb for the ongoing project. It is a popular NoSQL database; I prefer to call it schema-less database, and that is also why I use it: the data goes into the database does not have to follow any predefined schema like they do in SQL ones. This is perfect for any exploratory data analysis like we do when we have to constantly add/delete items/attributes for your data. Previously I use SQL database, in which I have to rebuild the database every time if any new stuff is added; it is painful.

In this post I include a few commands to quickly get you started with regular administration with mongodb.
mongod --dbpath /srv/mongodb/

This initiates the mongodb server with predefined local database path.
mongodump --dbpath /data/db/ --out /data/backup/
This uses mongodump creates a backup for the entire database you point to.
mongorestore --dbpath <database path> <path to the backup>
This restores/dumps a backup copy into defined database.

I will update once I come across necessary commands.

Friday, July 6

Learning postgresql pt2

This actually should be the first part; because now Id like to look into db administration and management, etc.

Postgresql also use client/server model. In order to manipulate any database or its tables, we have to first connect to the database using client applications, for example, psql. This is an interactive query tool in terminal. We use it to connect to our database:

-U option provides the username that we would like to use to connect to the database. After issuing this the terminal will ask password corresponding to the username. By default, when freshly installed, postgresql will create several predefined stuff for you to get started: superuser role postgres, database postgres, etc. Therefore for the first time you could use postgres to connect to the database. After you login, the prompt will change into the name of the database you are currently logged into, now you could use various commands to know about this database.

\d displays all the tables in this database, \du displays all roles across all database clusters: in postgresql, every role acts like an account, with different privileges. Use \q to quit the interactive client. For more usage, please refer to the official manual.

I remember I come across some problems logging into the database as the default superuser, but I cannot remember exactly what the problem is. Anyway, you could always log in as the default superuser as long as you use sudo. To create your own username and password using createuser. Also createdb and dropdb is other commands that are pretty self-explanatory. Note all these commands are issued in the terminal before logging into psql.

Learning postgresql pt1

My graduate course database system teaches me many interesting things, like working in a push environment just like that in a real company, building web app, using AWS, but not database itself. In fact, until now I am still unable to write down a complex SQL query and assure you it is correct. Considering building web app alone requires me constantly manipulate database, its models and write queries,  it is time for me to roll back and study database from the scratch. I know MySQL is just a popular choice; but since Heroku uses postgresql, I might as well start with it.

As far as I know, it is an open source object relational database management system, using SQL. Relational database actually means database with tables. Other database systems like noSQL, does not require tables. ORDMS should work like: columns -> rows -> tables -> databases -> database cluster, which is stored on the database server.

Assuming you know what is relational database, what is "select xx from xx where xx", and other basics in database, let's begin. Most of codes below is from the official postgresql manual. It is great and easy to read, by the way.

Create a new table using standard sql:

Every column has a type, int, varchar, real (for floating number), etc. Brackets indicate the maximum number of this type of value could be put in this column; dashing provides description.

Line 2 indicates the primary key of the weather table. Primary key is just a constraint, which is equal vent to UNIQUE NOT NULL, i.e. the value of this column should be unique and should not be blank. By indicating this constraint, the table gives us this column as the unique identifier to identify any single row in it. One table should have at most one primary key; but a key could be a combination of several columns, as long as they do not violate the constraint. Also, it is optional for a table.

Line 7 indicates a relationship between two tables, called a foreign key. A relationship is another constraint. Now weather table will refer to the cities table for available city name. For example, now if you create a row in weather table with a city name that is not in the cities table, it will throw out an error.

Now we have two empty tables. To insert values into them:

Note now you have to first create a city in cities table and then refer it in weather table. If you have a large amount of data, you could format it in a file and use COPY command to import them into the table at once. Similarly, use COPY * TO file you could export your data in the table to a local file. Note first is the file should be publicly accessible, and second this is only allowed in the server side, i.e. you could not use COPY in a client query.

There are various ways to query data from a table:

They should be pretty self-explanatory. Note the 3rd and 4th queries join two tables to fetch data, which means they are querying multiple tables or multiple rows in one table at the same time. The 4th one is a special join, which means every row in the table mentioned on the left side of the join operator will be displayed at lease once. The output should be something like this:

Note the first row, it does not meet the constraint that "city = name" but still present, because it is in the left table. There are also some aggregate functions you could use up and go like max, min, count. However they are not allowed in the WHERE clause unless it is nested in a subquery like line 15.

To remove data from your table, you could indicate rows you want to delete by WHERE clause in a DELETE FROM query like line 22. If you don't assign any constraints, it will simply delete all rows in the table.

TBD.