Posts Categorized: shell

Hostname Completion


Courtesy of Pete in work. To enable hostname auto completion in Bash, do the following: dig pop @dns.server.tld axfr | awk ‘$4 ~ /\yA\y/ && gsub(/.$/,””,$1) {print $1}’ > ~/.hosts If you would prefer to filter out hosts e.g. dev and swi, change awk to: awk ‘$4 ~ /\yA\y/ && ! /dev/ && ! /swi/… Read more »

Exclude Directories for Disk Usage (du)


I sometimes want to omit directories from my disk usage (du -sh *) checks. This includes virtual files systems such as proc / dev and also mounts to other filesystems (data in the example below). This is what works for me: du –exclude={proc,sys,dev,data} -sh * The equals sign appears to be optional and worked both… Read more »

Joining Multi-line Output with Paste


I was checking the memory on approximately 140 servers using ansible: ansible –user root -k -i servers.list -m shell -a “free -h; mysql” test but the output appears over several lines e.g. | SUCCESS | rc=0 >> total used free shared buffers cached Mem: 94G 44G 49G 59M 203M 4.6G -/+ buffers/cache: 39G 54G… Read more »

Tab Separated File Returns Wrong Number of Fields


So we had a CSV/TSV file with tabs separating each field but one of the fields was a description field that contained spaces. When parsing this TSV file some programs interpreted the spaces as field delimiters (TSV might be a bad idea!). Input file similar to: NULL NULL 7.0 Huayang Time Warner Cable 4430 The… Read more »

Command Line Optimise JPEG image

This command significantly improved file size on disk although the quality value can be increased to adjust the image quality versus file size: convert input-file.jpeg -sampling-factor 4:2:0 -strip -quality 60 -interlace JPEG -colorspace RGB output-file-60.jpeg  

Batch Auto-Levels on Images

I had a thousand image files the kids had photographed when creating a stop motion video. An ‘auto-levels’ on many of the images was required to brighten them up so at the command line I used the following (with ImageMagick already installed of course): for FILE in $(ls /home/jonny/tmp/stopmotion/*.JPG); do convert -auto-level $FILE /home/jonny/tmp/stopmotion-fixed/$(basename $FILE)… Read more »

Slurm & Environment Modules Bash Error

I had the following error message when trying to assign a job via Slurm (with Environment Modules installed). Since this has come up a few times for me I thought I would detail the solution: bash: BASH_FUNC_module(): line 0: syntax error near unexpected token `)’ bash: BASH_FUNC_module(): line 0: `BASH_FUNC_module() () {  eval `/usr/bin/modulecmd bash… Read more »

Retrieving folder.jpg thumbnails: PHP Script

When using XBMC we usually use the ‘Files’ menu. The folder icons under ‘Files’ do not show a preview thumbnail unless there is an image in the folder named ‘folder.jpg’. I could manually search for folder images for each movie but decided that would be time-consuming so I put together a wee PHP script –… Read more »

Shell Script Example Squid Logs

I was trying to explain to a colleague a few days ago how a few shell commands can be really useful, when today I came across an example to try to illustrate. My problem was that I had 245 log files each about 70-80MB in size – roughly 4 million lines in each log file…. Read more »