User Tools

Site Tools


aoe:awk

This is an old revision of the document!


Awk Command tips

multiple field separators

awk 'BEGIN { FS="[/()\" ]" } {print $1 " " $2 " " $3 }' data.txt

printing quotes

awk -v q="'" '{print "value is" q $2 q}' inputfile

or

awk '{print "value is \x27" $2 "\x27"}' inputfile 

or

awk '{ print "value is","'\''" $2 "'\''" }' input.txt

multiple field separators

http://bashshell.net/utilities/using-variables-with-awk

grep -i brown /var/log/maillog |awk -F'[<>]' '/whitelisted/ {print $4}' |sort |uniq -c

Filter on column

tail -100 /var/log/httpd/access_log |awk -F '[ "]' '$11~/20[06]/ {print $8,$11,$12}'
tail -100 /var/log/httpd/access_log |awk -F '[ "]' '$11~/20[06]/ {sum +=$12;print $8,$11,$12} END {print sum/1024/1024 "MB"}'

Count web hits

http://www.unix.com/shell-programming-scripting/65529-using-uniq-awk.html

cat /var/log/httpd/access_log |awk -F '[ "]' '$11~/200/ {hits[$8]++} END {for (i in hits) print hits[i], i}' |sort -n

udp firewall hits on athena

watch 'grep DROPPED /var/log/messages |grep -v 0.0.0.0|grep -v SRC=128.173|grep UDP|grep -v DPT=137|grep -v DPT=67|grep -v DPT=17500|tail'

pick a udp port being hit. e.g., 56846 Place a sniffer on the port:

tcpdump -i eth1 -vnn -s0 -X port 56846 -w port56846
tcpdump -nn -v -s0 -X -r port56846 |less

collect the addresses from the logs:

awk -F '[ =]' '/56846/ {print $14}' /var/log/messages >>athena-udp-src

Dropbox machines

awk 'BEGIN { FS="[ =]" } /DPT=17500/ && /^Jun 21/ {print $14}' /var/log/messages |sort -n |uniq |while read line; do echo -n $line " "; host $line ;done
Be sure to change the date from Jun 21 to your desired date

bootp UDP from 0.0.0.0 addresses

awk -F ":" '/FIREWALL/ && /DPT=67/ && /SRC=0.0.0.0/ {print $10":"$11":"$12":"$13":"$14":"$15}' /var/log/messages |sort|uniq -c|sort -n
lvdisplay |awk '/LV Name/ || /LV Size/ || /VG Name/ || /Block device/ {print $0}'

on one line:

lvdisplay |awk '/LV Name/ {NAME=$3} /LV Size/ {SIZE[NAME]=$3 ; } END {for (x in SIZE) print x,SIZE[x]}'

Add on the mountpoint found in /etc/fstab

lvdisplay | cat - /etc/fstab |awk '/LV Name/ {NAME=$3} /VG Name/ {VG[NAME]=$3 } /LV Size/ {SIZE[NAME]=$3$4} /Block device/ {DEV[NAME]=$3} /ext3/ {for (i in SIZE) {if (i == $1) {MOUNT[i]=$2}}} END {for (x in SIZE) print VG[x],SIZE[x],DEV[x],x,MOUNT[x]}' |sort

format for the wiki (for non-mapper version of /etc/fstab names and ext3)

lvdisplay | cat - /etc/fstab |awk '/LV Name/ {NAME=$3} /VG Name/ {VG[NAME]=$3 } /LV Size/ {SIZE[NAME]=$3$4} /Block device/ {DEV[NAME]=$3} /ext3/ {for (i in SIZE) {if (i == $1) {MOUNT[i]=$2}}} END {for (x in SIZE) print "| | |"VG[x]"| |"SIZE[x]"|"DEV[x]"|"x"|"MOUNT[x]"|"}' |sort
[root@alexandria ~]# cat linkPartitions.awk 
BEGIN{
	FS = "/"
}

{
	print "ln -s /home/"$6"/"substr($4,0,length($4)-1)"/ /home/"$3"/"substr($4,0,length($4)-1)
}
aoe/awk.1020206.txt.gz · Last modified: 1970/01/12 19:23 by 127.0.0.1 · Currently locked by: 172.17.0.1,216.73.216.138