Copy file preserving directory structure in Linux

Use cp --parents /d1/d2/d3/f4 /e1/e2/e3 to get the copy /e1/e2/e3/d1/d2/d3/f4.

[Courtesy: this]

Grepping specific files recursively

To search the pattern PAT in all csv and csv.bak files in the directory DIR and its subdirectories, do

grep --include=*.csv* -ER "PAT" DIR

[Courtesy: this]

Finding files after excluding some directories

Normally, to find a file recursively under a directory, we do something like

find path/to/directory -name "*filename*"

To exclude finding in some sub-directories under directory, we can do

find path/to/directory -name "*filename*" -not -path "*subdir1*" -not -path "*subdir2*"

[Courtesy: this]

Python regex to match decimal numbers

Here is a Python regex to match regular decimal numbers (without the e or exponent part):
r'([+-]?([0-9]+(\.[0-9]*)?|\.[0-9]+))'

Lenovo Motorola G5 Plus wifi issue after update

After a recent update on my Lenovo Motorola G5 Plus phone, I started facing wireless connectivity issues. It would randomly get disconnected from wifi and connect again. My laptop was connected to the same wifi and did not have this problem, so it was not a problem with the wifi. I tried restarting the phone, the modem and the router. I tried resetting the phone wireless settings, forgetting the network, and restarting (phone, modem, router). None of these worked. Finally, the following action worked for me.

• Open the wifi router's settings page in the browser (for me, this was at http://192.168.0.1). 
• Login and check the Wireless settings. Under "Channel", the value will "auto" or some particular channel (e.g. channel 1, channel 2, etc.).
• Change the value to channel 10.

This fixed the problem for me.

[Courtesy: this]

Taking screenshot on Moto G5 Plus

To take a screenshot on the Moto G5 Plus phone, press the Volume-Down button and the Power button together. To see the screen shot, go to Photos --> Albums --> Screenshots.

Timing code in Python

To time a particular code block in Python, do this:

import time

t = time.time()
#<code_block>
print(f'code-block-name:{time.time()-t})

[Courtesy: this]

Pandas Dataframe Tips

When dropping rows with NaN's in a Pandas Dataframe, this

df = df.dopna()

may be faster than this

df.dropna(inplace=True).

---

Casting converting a column of a dataframe to float (for example) can be very slow if done like this:

df[col1] = pd.to_numericdf(df[col1])

or

df[col1] = df[col1].apply(pd.to_numeric)

or

df[col1] = df[col1].astype(float)

If you know how to do this efficiently, please tell me. As of now, my only work around is to avoid doing this operation if possible.

Pandas DataFrame Jupyter notebook display: increasing column width

To increase the column width when displaying a Pandas dataframe in Jupyter notebook, do this:

pd.set_option('display.max_colwidth', -1)

[Courtesy: this]

Specify data types when creating Pandas dataframe from csv

To specify data types for various columns when creating a Pandas dataframe, do this:

df = pd.read_csv(filename, dtype={'col1':np.int64, 'col2':str, ...})

If converters are given, they will be used instead of data type conversion.

Convert Python / numpy datetime to Unix timestamp

To convert Python datetime to Unix / POSIX timestamp (float), do this:

mytime.timestamp()

[Courtesy: this]

To convert numpy.timestamp to Unix / POSIX timestamp (float), do this:

mytime.astype('uint64')

[Courtesy: this]

Changing dimensions of plot in matplotlib

To specify dimensions of a plot in matplotlib (e.g. to get longer x or y axis), do this:

fig = plt.figure(figsize=(20,3)) #x-axis = 20", y-axis=3"
ax = fig.add_subplot(111)
ax.plot(x, y)

[Courtesy: this]

Show summary statistics for a Pandas dataframe

To see the summary statistics for a Pandas dataframe, do this:

df.describe()

This will show things like: count, mean, standard deviation, etc.

To see column details (data types), do this:

df.info()

Cast a Pandas dataframe column to timestamp

To cast / convert a column in a Pandas dataframe to the timestamp datatype, do this:

df['timestamp'] = pd.to_datetime(df['timestamp'])

Jupyter-notebook: plot matplotlib graphs inline

To plot matplotlib graphs inline in Jupyter-notebook, do this:

import matplotlib.pyplot as plt
%matplotlib inline

Now, plt.plot(...) while show the graph inline.

Jupyter-notebook module reload

To reload modules being used in a jupyter-notebook (when they have been changed outside), do this:

%load_ext autoreload
%autoreload 2

This will automatically reload the module every time it is changed.

[Courtesy: this]

Restarting Nautilus on Ubuntu

Sometimes Nautilus does not work/open. One quick thing to try is to do killall nautilus in a terminal and then try to open it in the usual way. (Courtesy: this)

Passing lists in numba

If you plan to @jit() a Python function using numba, and one of the arguments is a list, it will be treated as an object, and the jit-ted function will probably be slower than the original. Instead, explicitly specify the argument data types (e.g. int32, double, unit64, etc.) within the @jit([data types]) declaration and, importantly, when calling the function, remember to convert the list into a numpy array of the data type specified in the jit declaration. For all your efforts, you should be rewarded with a good speed-up (if you have long-running loops).

Check whether the fonts are embedded in a PDF document

To check whether the fonts are embedded in a PDF document, do the following.

On Ubuntu

Run pdffonts mydoc.pdf at the terminal. It will show a table listing all fonts in the PDF document, and the 'emb' field is 'yes' if the font is indeed embedded.

Anywhere

Open the document in Adobe Reader, and select File...Properties. Under the 'Fonts' tab, you can see the list of fonts. Embedded fonts are followed by the phrase 'Embedded' or 'Embedded subset' within brackets.

Debugging in iPython

To debug a function func in module mod.py, do the following at the iPython prompt.

from IPython.core.debugger import Pdb
ipdb = Pdb()
import mod
ipdb.runcall(mod.func, [args, for, func])

This will start the iPython debugger at the first line of func.