Unix as IDE

Text Processing

Echo a file without the first and last lines

sed '1d;$d' file.txt

How to remove all blank lines in fie

grep -v '^$' old-file.csv > new-file.csv
# or
grep -v '^$' file.csv | sponge file.csv
# or if lines may contain spaces
egrep -v '^[[:space:]]*$' old-file.csv > new-file.csv

Search

String search

egrep -r "findmestr" .

Search and replace in file

sed 's/replaceme/byme/g' ffff > TMPFILE && mv TMPFILE ffff

run grep with multiple AND patterns?

awk '/regexp1/ && /regexp2/ && /regexp3/ { print; }'

How to search a PDF file from command line?

pdftotext document.pdf - | grep -C5 -n -i "search term"

Bash

Shortcuts
Export enviroment variable for a single command

PANTS=kakhi PLANTS="ficus fern" env | grep 'P.*NTS'
if [ -e afile ] ; then VAR=val ./program_using_VAR ; fi

Bash auto completion

For Django commands http://docs.djangoproject.com/en/dev/ref/django-admin/#bash-completion

echo '. /path/to/django-trunk/extras/django_bash_completion' >> .bashrc

Filesystem

Change file extension

cp /opt/nginx/conf/nginx.conf{,.bak}

Search and remove all *.pyc files

find . -name "*.pyc" -exec rm -rf {} \;

Sync dirs

rsync -avuzr <from_dir> <to_dir>

Find files by modification date and copy them to folder

find . -mtime -1 -maxdepth 1 -exec cp "{}" /some/dist_dir/ \;

Archives

tar.bz2

tar -cjf ../ffff.tar.bz2 *
tar -xjf ffff.tar.bz2

tar

tar xvf ffff.tar
tar xvfz ffff.tar.gz

Navigation

Dirs

dirs -v
pushd # / popd

Send file(s) via ssh

cat filename | ssh -l login ip_address 'cat > filename'
# get folder
tar -cjf - /path/to/files/ | ssh -l root ip_address 'cd needed_directory; tar -xjf -'

Administration

How can I get my external IP address in bash?

curl http://automation.whatismyip.com/n09230945.asp

Routing table

route

How can I find which process sends data to a specific port?

netstat -npl | grep 5432

Repeat a unix command every x seconds

watch -n1  command

How to log all the events performed on directory or file?

inotifywait -m -r /dir

Look for a process with a given name

ps -C python

Find the process ID of a running program:

pidof java

Email

How to send email from the Linux command line

man mail
man sendmail

Monitoring

Network usage

sudo apt-get install bwm-ng
bwm-ng

Measuring Web Server Performance

sudo apt-get install httperf
httperf --server 127.0.0.1 --port 8000 --uri / --num-conns 200 --add-header='Accept-Encoding: gzip,deflate\n'