Copy file preserving directory structure in Linux

Use cp --parents /d1/d2/d3/f4 /e1/e2/e3 to get the copy /e1/e2/e3/d1/d2/d3/f4.

[Courtesy: this]

Finding files after excluding some directories

Normally, to find a file recursively under a directory, we do something like

find path/to/directory -name "*filename*"

To exclude finding in some sub-directories under directory, we can do

find path/to/directory -name "*filename*" -not -path "*subdir1*" -not -path "*subdir2*"

[Courtesy: this]

Restarting Nautilus on Ubuntu

Sometimes Nautilus does not work/open. One quick thing to try is to do killall nautilus in a terminal and then try to open it in the usual way. (Courtesy: this)

Check whether the fonts are embedded in a PDF document

To check whether the fonts are embedded in a PDF document, do the following.

On Ubuntu

Run pdffonts mydoc.pdf at the terminal. It will show a table listing all fonts in the PDF document, and the 'emb' field is 'yes' if the font is indeed embedded.

Anywhere

Open the document in Adobe Reader, and select File...Properties. Under the 'Fonts' tab, you can see the list of fonts. Embedded fonts are followed by the phrase 'Embedded' or 'Embedded subset' within brackets.

Using bc in calculations

To use the linux command bc for calculation in shell programs, do the following:

• Add the line scale=2 to the file ~/.bcrc.
• Then use var1=`echo "($var2+$var3)/$var4" | bc` to set var1.

Here

• The scale parameter tells bc to use 2 decimal digits.

Piping stderr

To pipe stderr (instead of stdout) use `|&' instead of `|'. This works from Bash 4 onwards. It is shorthand for `2>&1 |', which redirects bytes sent to stderr (2) to stdout (1), and then pipes it (|).

[Courtesy stackoverflow.]

LaTeX to HTML

To generate a HTML version of a TeX document:

• First ensure that all metadata files are up-to-date (e.g. if you have references in the tex file, run latex; bibtex; latex; latex to ensure that the .bbl file is ok).
• Run latex2html -split +1 -show_section_numbers -no_footnode myfile.tex. This will create a directory myfile containing myfile.html (among other things).
• -split +1 causes all subsections within a section to be in a single page (-split 0 will cause the whole document to be in a single page)
• -no_footnode causes footnotes to be included in the same page where they are referenced.
• See man latex2html for more options.

Random reordering of lines in a file

To generate a random reordering or shuffling of lines in a file, use
shuf inp.txt > out.txt

Grep tips

To get all lines not beginning with a tab, do
grep -vP "^\t" abc.txt
-P turns on Perl-style regex (otherwise tab cannot be specified as \t). "^\t" matches lines beginning with a tab. -v inverts the selection.

For most cases though, using -E (extended regex) instead of -P works, and is better supported.

Using `less'

View text files using less, including very large files. It also opens gzip-ed files. Use less -S to truncate long lines (instead of wrapping around).

Information about your Linux system

Here are some commands that give information about your linux system.

To get  the number of CPU's/cores in the system, use

less /proc/cpuinfo

To see if your processor supports 64-bit OS, you can Google the processor name (got using /proc/cpuinfo). To see if your OS is 64-bit, use

uname -a

If it says x_86_64 somewhere in the output, then it is a 64-bit OS, else it is probably a 32-bit OS.

To see where (in which directories) a package is installed in Ubuntu, use
dpkg -L <packagename>

Scheduling a regular job in Linux

To execute the shell script myjob.sh every Friday and Saturday of July at 0,20 and 40 minutes past 10pm, do

crontab -e 

In the editor that opens up, enter

0,20,40 22-23 * 7 fri-sat /absolute/path/to/myjob.sh 

and save. You are done!
The format is for specifying jobs is

Min Hour DoM MoY DoW Cmd.

See Ian's article for more.

Reading ISO files in Linux

Given the file

/path/to/iso/abc.iso

do

mkdir /path/to/mount/dir/abcdir
mount -o loop /path/to/iso/abc.iso /path/to/mount/dir/abcdir
cd /path/to/mount/dir/abcdir

You can now start using the contents of the ISO file.

Using tar and gzip

To tar a list of files/directories, do

tar -cf mytar.tar file1 file2 dir3 file4

To add more files to this, do

tar -rf mytar.tar file5 dir6 dir7

To list the contents of the tar, do
tar -tf mytar.r
To delete files from the tar, list the contents, do

tar --delete -f mytar.tar file1 dir2

using the exact path as given in the `list contents' step.

To compress the tar, do

gzip -c mytar.tar > mytar.tar.gz

To decompress the file, do

gunzip -c mytar.tar.gz > mytar.tar

To extract files from the tar, do

tar -xf mytar.tar file2 dir3

To extract everything, do

tar -xf mytar.tar

To simultaneously tar and compress, do

tar -czf mytar.tar.gz file1 dir2 file3

To simultaneously decompress and extract, do

tar -xzf mytar.tar.gz

(Note: Additions, using tar -rf, are not allowed for compressed tar files)

Using wget

Full Site
To download a full website, do

wget -c -w 15 --mirror -p -k -P /path/to/dir http://my.website.com -a my.log

This will download the entire website, waiting 15 seconds between retrievals. It will copy everything needed to show the page correctly (-p), convert links for local viewing (-k), store the files in /path/to/dir (-P), and write (append) messages to my.log (instead of the console). If stopped in between, run this command again, and it will start where it left off (-c).

Add --spider at the end to do a dry run first.
Add --random-wait to vary the wait time (for sites that block automated downloading).
Add --limit-rate=20k to limit download speed to 20 kilobytes per second (an example politeness limit; use your discretion).

Courtesy: this, this and this.

Small part of site
To download a small part (e.g. a given page and its first-level links) to the current directory, do

wget -c -w 3 -a my.log -r -l 1 -p -k http://somesite.com/interesting/link.html

This will download the page link.html and all links in it (-r), but will not recurse further to links of links (-l 1).

A URL list
To download a list of files specified by URLs, do
wget -c -w 15 -i urls.txt
This will download all URLs listed in the file urls.txt.

Backup and Restore using rsync

I backup my data onto a second hard disk in my PC, like this:

rsync -av --delete /disk1/my/data /disk2/backup > backup.log 2> backup.err

This creates a folder called data in /disk2/backup and copies the contents of the folder /disk1/my/data to it.

If I rerun this command, it will copy only the changed files/directories from /disk1/my/data into /disk2/backup/data.

Restore is simple; /disk2/backup/data is like a mirror. Just browse and copy back whatever is needed.

Note: Using /disk1/my/data/ (with the trailing slash) instead of /disk1/my/data does not cause the directory data to be created. The contents of data are copied directly into backup.

For a one-time copy to an external hard disk, I did this:

rsync -avW --no-compress --delete --modify-window=2 /path/to/src  /path/to/dst > bakup.log 2> bakup.err


Compression is useful for network transfers but can be skipped for local copying. The 'modify-window' option was required since the destination was on an NTFS drive, and there could be time stamp issues when copying from ext3 to NTFS. The 'W' option (copy whole files) might speed things up since partial file transfer need not be supported.

Using xargs to redirect to command line arguments

To delete all .bak files in a directory, do
rm *.bak or ls *.bak | xargs rm
Essentially, rm is executed once with the whole of the piped input as argument.

To move the .bak files to another place, do
mv *.bak diffdir or ls *.bak | xargs -I var1 mv var1 diffdir.
Here, mv is executed separately for each file listed by ls. var1 is a placeholder. It is replaced each time by a file, and mv is executed.

To rename the .bak files to .bakup files, do
ls *.bak | xargs -0 -I {} mv {} {}up.
{} is another way of specifying a placeholder. -0 takes care of white space anomalies in the piped input.

Linux commands for gzip and bzip2

These examples are for compressing/decompressing, and preserving the original file.

Gzip-ing and gunzip-ing -

cat abc.txt | gzip > abc.gz

gzip -cd zbc.gz > abc.txt (-d decompress, -c send to console)

and similarly for bzip2 and bunzip2 -

cat abc.txt | bzip2 > abc.txt.bz2

gzip -cd abc.txt.bz2 > abc.txt

Some commands that operate directly on gzip-ped files -
zcat abc | less (abc can be a text file or a gzip-ped file)
zless def (the file need not have extension .gz)

zdiff abc.gz def

zgrep "pattern" abc.gz

and similarly for bzip2, we have bzcat, bzless, bzdiff and bzgrep.

Opening/Creating .zip files in linux

Opening
To extract contents of abc.zip (keeping original zip file) --- unzip abc
To just list the contents --- unzip -l abc
To extract a specific file def.txt --- unzip -p abc def.txt > mydef.txt (where def.txt was listed in previous step)

Creating
To create abc.zip (keeping original files) --- zip abc def.txt xyz.txt ...
To zip a directory /path/to/dir1 --- cd /path/to; zip -r abc dir1
To add a file to existing zip file abc.zip --- zip /path/to/abc.zip lmn.txt (Do not give /path/to/lmn.txt)

Mounting drives using HAL in Ubuntu

Some drives in Ubuntu are mounted using HAL. To change the mount options for these drives, do the following -

• Run gnome-mount -u -d /dev/sdb1.
• Run gnome-mount -d /dev/sdb1 --display-settings to view current settings.
• Run gnome-mount -d /dev/sdb1 --write-settings --mount-options umask=000,opt2,opt3=foo to set the mount options. 
• If you want to keep the existing options and change/add only a few, then use gnome-mount -d /dev/sdb1 --write-settings --extra-mount-options opt4,opt5=foo.
• Now mount the drive in the usual way (from Nautilus).

To see complete details about the drives mounted by HAL, run lshal.

(In the /dev/sdb2 example above, remember that umask=000 gives everyone all permissions; so it is the inverse of what you would give for chmod.)