Always learning

Installed django on ubuntu today. Do not make the mistake of installing django from the synaptic package manager. Firstly, its an older version than the current 1.0.2 and secondly, it does not install it in your python "site-packages" folder.

The best way to do it is as they say on the django site. Download the tar file and install using python setup.py install. The cool thing about this is is that I found that the symlinks were also created for you. I could access django-admin.py from any directory in my machine.

I also installed postgreSQL and psycopg2 (the python interface for postgres) from the synaptic package manager and I was good to go with django in a matter of minutes! For now I am developing with the webserver bundled inside django but I hope to be switching to apache with mod_wsgi soon.

I hope to be tinkering a lot with django over the next few weeks.

Theres a dictionary dict1 with keys key1, key2, etc such that $dict1{keyn} has a value which itself is a dictionary. Lets call this dictionary %subdict. Now subdict has different keys subkey1, subkey2 and so on such that $subdict{subkey5} = subvalue5.

Problem: How do I sort dict1 such that the keys are in the ascending order of subvalue5?

This would not be easy to do in a single line of code, unless of course the language is Perl! Ready? here it is:

dict1_keys_sorted = sort {$dict1{$a}{subkey5} cmp $dict1{$b}{subkey5}} keys %dict1

Lets take this apart. Going from the right side, you start of with dict1. Now, keys %dict1 returns a list of the keys. With the sort command being invoked, Perl feeds the first two values of the keys into $a and $b. So, $dict1{$a}{subkey5} gives the value of subvalue5 for that key value of dict1. The values are compared and depending on true or false value, it puts the value into the new list dict1_keys_sorted which is sorted.

Note: "cmp" is for sorting strings. To sort number use "<=>".

Printing and scanning in Ubuntu proved to be much easier than expected. Ubuntu easily recognized the HP Photosmart C3500 series that I had, and right away I was able to print documents off the text editor. The Scanning application (XSANE) was a breeze to operate and I was able to scan documents to various formats.

However, my favorite lightweight editor on windows was SciTe, so I installed it in ubuntu also, and tried to print from it. Turns out SciTe uses the "a2ps" program to print, and it was not able to direct the output to the printer. A bit of googling told me that replacing a2ps with "enscript" would be able to do it. So I replaced the print options in SciTeGlobal.properties file, but now I get an error that its not able to find the printer class. Guess I will have to look into this more.

But besides that, the printing and scanning functionality works like a charm in Ubuntu.

I just upgraded my ubuntu distro to Hardy Heron (8.0.4) from Gutsy Gibbon (7.10). I did not use it a lot when I had Gutsy but now, hearing about how much better the distros are getting everytime theres an upgrade, I am setting out to explore it more. The major difference I noticed in Hardy was that I am able to make ubuntu work on dual monitors directly without running additional scripts. In Gutsy, I actually had to run an XRANDR command everytime I logged in, so that I could switch it from clone mode to dual monitor mode. Not anymore in Hardy.

Further things I will be exploring on Ubuntu in the coming days:
- Printing and Scanning
- IPOD and music play
- Python, Django
- Java Development with Eclipse
- Ant, CVS, Tomcat
- MySQL

If I find that the above are easily setup and software development in that environment is as easy as windows, theres no reason why I should not completely switch to Ubuntu.

Oh I will have to go back to windows frequently though, since Cricket games for PC do not run on Linux yet!

Over the past few years, I have observed the agile movement and I feel that I have to put down some of my thoughts regarding this. The main reason for me writing it is to clarify my thoughts, and also, but not necessarily, to invite some feedback.
I cannot dispute the fact that in the process of evangelizing it, the people have talked about some software practices that contribute to making better software. But somehow I feel that the useful practices are all those that have to be part of writing software in general, rather than being attributed to a process.
Its my belief that there are mostly only two qualities that a developer has to have in order to deliver successful software. This is of course only in context on enterprise software development where the developer in employed to create and maintain internal software systems within the company. The two qualities are: one - not being afraid to go in and change code and two - constant communication of the status of his work.
The second one is an auxillary capability, since it solely exists to put people mind's at rest, so that they don't bother him/her while he's actually trying to solve a problem. I would think its an equally necessary quality to possess as the primary one because when it comes to deploying a fix or a system, its never only about the code. There's always a dynamic between the current users that have to look for a workaround or need to be informed about systems that will be down, etc.
In such an environment, if the developer possesses the confidence of being able to tackle changing the codebase or developing something new with the confidence that it can always be changed later, there's not many more practices that he needs to religiously follow to deliver working software.

This confidence comes from constantly learning from a good amount of experience as well as possessing a couple of tools that come in handy. I will delve on this aspect more in the next post.

Say you had to read a file line by line and change the delimiter from comma to semi-colon. The python code would look like this:

f = open("c:\\testfile.txt") 
for line in f.readlines(): 
    ';'.join(line.split(','))

Now say you wanted to do the same thing, but only to the first 5 lines of the file. The impulse for a Python newbie would be to write the following code:

f = open("c:\\testfile.txt") 
counter = 0 
for line in f.readlines(): 
  if counter <> 5 
    ';'.join(line.split(',')) 
    counter += 1 

A more savvy python newbie would write the following code taking the advantage of the beautiful enumerate() function:

f = open("c:\\testfile.txt") 
for counter,line in enumerate(f.readlines()): 
  if counter <> 5 
    ';'.join(line.split(',')) 

The problem with is code is that it has two concepts intermingled, one to read 5 lines and the other for the actual process of changing delimiters. If we wanted to keep the concepts separate, we would have to be able to write something like this:

f = open("c:\\testfile.txt") 
for line in f.readfirst5lines(): 
  ';'.join(line.split(',')) 

Here the process of stopping after 5 lines is encapsulated by the readfirst5lines() method in the file object and the method body only does the changing of delimiters as before. Now this code will actually work! The reason that we can write code like this is due the python generators feature. readfirst5lines will look like this:

import itertools 
class myfile(file): 
  def readfirst5lines(self): 
    for i in itertools.count(): 
      if i == 5: 
        break 
      next = self.readline() 
      i += 1 
      yield next 

This is of course more lines of code, but the concept is abstracted away nicely. We are separating the conditions for processing the file from the actual processing. And whats more, this method can be slightly changed to take number of lines as the argument. So if you want choose to read the first 2, 3, 5 or how many ever lines, the method will look like this:

import itertools 
class myfile(file): 
  def readfirstfewlines(self, n): 
    for i in itertools.count(): 
      if i == n: 
        break 
      next = self.readline() 
      i += 1 
      yield next 

And you can use it like this:

f = myfile("c:\\testfile.txt") 
for line in f.readfirstfewlines(3): 
  ';'.join(line.split(',')) 

This will process the first 3 lines.

(Note that we used the 'myfile' constructor method instead of the 'open' function to open the file, since we need an object of type 'myfile' and not 'file'. There are other ways to downcast in python but this is probably the simplest way to do it in this case.)

Python generators go a long way in making the code more elegant and encapsulating separate concepts. I will blog about the mechanics of how generators work and its other uses as and when I learn more about them. For Rubyists, this concept of generators is pretty much similar to "blocks".

Oddly enough, I have not found a clear explanation of how to make ActiveRecord work with a table that has a non-autogenerated primary key. In other words, the table has a column as a primary key for which the value has to be inserted manually. (Of course this is not good practice but what other better reason to do this than to say such a table already exists and needed a user interface for maintaining its values!)
So how do we go about doing this? Simple. First, set the primary key in the Model object to be the column in the table (since ActiveRecord always assumes primary key is a column called "id"). If we just run this and try to save the model object by firing up the rails console, rails gives an error saying there are no attributes "col1". This is because, rails assumes that the primary key is not editable. So all we have to do is define attributes for the primary key column col1. The overall changes are shown below:

class MyModel < ActiveRecord::Base

set_primary_key "col1"
def col1
read_attribute "col1"
end
def col1=(value)
write_attribute "col1", value
end

end

Now fire up the rails console and create and save the model object, setting values for the required columns. Voila!

Note: since you are departing a bit from rails convention of primary keys, you will need to make sure that the required validations are added in the model so that you dont get errors from the database.

I had to update some data in a production database for a new functionality. I had already done that in the test database during development, but it was not just a simple insert. The steps that I did went something like this:

1. Generate a file with a set of insert statements (used Python for this!).
2. Manually change some data in the insert statements.
3. Run the insert statements

Now since I had to do the same in production, I wanted to make sure that after this insert, the data would look exactly the same as it did now in test. In other words, I wanted to be able to repeatedly run the process above, and make sure that the steps produced always the same set of data. This would guard against any mistakes being done during step 2, which is a manual edit of a file and prone to error.
Sounds simple - I could generate the file of insert statements, and then compare it against the file that was already changed and used to load the data in the table. Problem is, I no longer had that file I could compare against. So instead of comparing insert statements, I would have to compare data. The process I did went something like this:

1. Export current data in the table to a file, say file1.txt
2. Delete the inserted rows from the table.
3. Generate insert statements again, manually edit, and insert rows into the table.
4. Export data in the table to a file, say file2.txt

Now all I had to do, was compare file1.txt and file2.txt. If they are equivalent, it means my manual edits were on target.

So how did I compare the two files? Python to the rescue! This is all it took:

import filecmp
filecmp.cmp('file1.txt', 'file2.txt')

If they are equal it returns true, else false.

I have to admit that that the only reason I thought about comparing data rather than the insert statements was because I was sure Python would have something neat for file comparison. The database-oriented guys might have fired up some magic by using stored procedures and temporary tables, and it would have been equally satisfying to them.

Its all a matter of what you are comfortable with, I guess.