Debugging and Profiling in Python

To debug hello.py, do

$ python -m pdb hello.py

You will immediately get the debug prompt, with the execution stopped before the first line of the program. You can now use:

l - to display source code around the line to be executed next
n - to execute the next line of code
s - to step into the next line of code
b <line no.> - to set breakpoints
b - to list breakpoints
clear <breakpt no.> - to clear breakpoints
c - to continue execution
tbreak <line no.> followed by c - to execute till a particular line
p <var> - to print contents of variables,
vars(obj) - to print contents of objects (same as obj.__dict__)
dir(obj) - to list attributes/methods of objects
help - to get more help.


 To profile hello.py, do
 $ python -m cProfile hello.py

The program will be run and you will get a profiler report on the console.

To debug a particular part of the code, do
import cProfile
profile = cProfile.Profile()
profile.enable()

# do something

profile.disable()
profile.dump_stats('something.prof')

To view the dump file, use cprofilev.

Lagrangian duality

Given an minimization problem

min f(x)
s.t. g_i(x)  = 0,     i=1..m
      h_j(x) <=  0,  j=1..p,

the Lagrangian function is

L = f(x) + \sum_i \alpha_i g_i(x) + \sum_j \beta_j h_j(x)

where \alpha_i is unconstrained, and \beta_j >= 0.

See more on Wikipedia.

Literal strings in Python

Some of the many ways one can construct strings in Python:

s = "qwe\nasd\nzxc"  #gives 'qwe\nasd\nzxc'
s = 'qwe\nasd\nzxc'  #gives 'qwe\nasd\nzxc'
s = 'qwe\

\nasd\

\nzxc'  #gives 'qwe\nasd\nzxc'

s = '''qwe

asd

zxc'''  #gives 'qwe\nasd\nzxc'

s = '''qwe\\n

asd

zxc'''  #gives 'qwe\\n\nasd\nzxc'

s = r'qwe\n\

asd\

zxc'  #gives 'qwe\\n\\\nasd\\\nzxc'; => \n in the string is always accompanied by a '\'.

s = u'qwe\u0020asd'  #gives a Unicode string 'qwe asd' (the Unicode code for space is 20)

s = ur'qwe\u0020asd'  #gives a Unicode string 'qwe asd'

s = ur'qwe\\u0020asd'  #gives a Unicode string 'qwe\\\\u0020asd' (the desired format for use in regular expressions.

s = u'qwe\u0020asd'.encode('utf-8')  #gives an encoding of the Unicode string in the specified encoding ('utf-8')

s1 = unicode(s, 'utf-8')  #gives a Unicode string after decoding the content of s according to specified encoding

Scheduling a regular job in Linux

To execute the shell script myjob.sh every Friday and Saturday of July at 0,20 and 40 minutes past 10pm, do

crontab -e 

In the editor that opens up, enter

0,20,40 22-23 * 7 fri-sat /absolute/path/to/myjob.sh 

and save. You are done!
The format is for specifying jobs is

Min Hour DoM MoY DoW Cmd.

See Ian's article for more.

Reading ISO files in Linux

Given the file

/path/to/iso/abc.iso

do

mkdir /path/to/mount/dir/abcdir
mount -o loop /path/to/iso/abc.iso /path/to/mount/dir/abcdir
cd /path/to/mount/dir/abcdir

You can now start using the contents of the ISO file.

A `best practice' for writing papers in LaTeX

To handle complexity and versions, I do the following when I start writing a paper in LaTeX.

• Create new directory Conf1.
• Within this, create a new file Conf1.tex, and two new directories: tex and images.
• Conf1.tex contains: document class, title, author, bibliography style, etc., which are specific to the conference/journal. This is followed by \input{tex/main}.
• Create main.tex in directory tex. The tex directory contains all the LaTeX code, and could have further subdirectories if required (e.g., I usually have a directory for Experiments, another for Algorithm, etc.). All tex files are included in the paper by adding \input{tex/subdir/abc.tex} to main.tex in the appropriate place.
• All package inclusions (\usepackage{}), command declarations, etc. required for the paper are included in tex/include.tex.
• The first line of main.tex is \input{tex/include} followed by \begin{document} and \maketitle. Include all your LaTeX code after this. End with \bibliography{tex/biblio} followed by \end{document}, where tex/biblio.bib is the bibliography file.
• All images are stored in the directory images.

Some things to note:

• You can use git/svn to do versioning. Until I get used to them, I am using the following method. When I have a significant frozen version, I copy the directory tex to texN where N is 1,2,3.... The timeline of versions from the earliest to the current is tex1, tex2, ..., texN, tex.
• I put the .bib file into tex because I found the bibliography changes as the paper develops.
• On the other hand, images usually accumulate. So, only the directory tex needs versioning.
• In order to use the material for another conference/journal, follow the steps above to create Conf2 and Conf2.tex. Then copy images and tex from Conf1 and compile. Your first draft of Conf2.pdf is ready!
• I read somewhere that using PDFLaTeX gives more compact files, also avoids some font problems which arise due to DVI to PDF conversion when using LaTeX. I also had problems using Beamer with LaTeX. 
• If you decide to use PDFLaTeX, remember that it does NOT accept images in .eps format. I found that .png files did not render text well on conversion. Creating / converting all images in / to .pdf format (using Inkscape) worked for me.
• I use Inkscape for most drawings, and Gimp for screen capture (and also for format conversion).

Find and Replace across multiple files in Linux

For example, do this (Courtesy Rushi):
find mydir -name "*.tex" | xargs sed -i 's/foo/bar/g'
This replaces `foo' by `bar' in all .tex files in the directory mydir and its subdirectories.

This is useful for changes to latex code stored in a directory structure, since
find: gets the correct recursive file listing
-i: edit the files in place
/g: replace all occurrences in the file (globally)
foo: is a regular expression (POSIX.2 BRE)

[search tex]