Automatic tracking of changes for a web page with email notification

Εδώ θα βρείτε θέματα και λύσεις για τον διαγωνισμό εκπαιδευτικών του ΑΣΕΠ του έτους 2005, για την πληροφορική (ΠΕ19 & ΠΕ20)

I have created a script that helps me monitor a web page for specific changes and notify me when those changes have occurred! So if you want to monitor a web page for changes you can use this script in conjunction with a scheduled cron job which will periodically do the check.

# is the root shell (to gain root priviledges just type "sudo su -" at the command line

Install mailx and postfix (instructions for debian/ubuntu)

# apt-get install mailx postfix

Setup postfix to be able to send mails

You may be asked during installation about postfix configuration. If you are not asked or you already have postfix issue the following command:
# dpkg-reconfigure postfix
and select "Internet site".
If you have more info about setting up postfix send me the link by mail.

We create file urlwatch.

You can get it here: urlwatch.sh
or

• View script
# editor /etc/myscripts/urlwatch (instead of editor put your favourite editor nano, pico, vi or even kedit or gedit from KDE or Gnome)

Add the following lines to the file:
#!/bin/bash
# urlwatch script created by jtsop
# easily monitor a web page for specific changes
# visit http://tsopokis.gr for more info
# Note: you need to have mailx and postfix installed on your system
# beware that postfix must be installed as internet site, so as
# to be able to send outgoing messages

GW=$1
if [ "$GW" == "-s" ] || [ "$GW" == "--show" ]; then
ps -f -C ssh | grep "ssh -L"
exit;
fi
if [ $# -eq 0 ] || [ $# -eq 1 ] || [ $# -eq 2 ]; then
GW="--help"
fi
if [ "$GW" == "--help" ] || [ "$GW" == "-h" ] || [ "$GW" == "?" ]; then
echo "tunnel: create an ssh tunnel through a gateway."
echo " ssh tunnel script by jtsop"
echo
echo "tunnel usage:"
echo " tunnel [user@]gateway target port [source-port [ssh-port] ]"
echo " tunnel -s || --show"
echo
echo "for more details see the presentation at http://tsopokis.gr"
exit;
fi
SRC_PORT=$4
if [ "$#" -eq 3 ]; then
SRC_PORT=$3
fi
SSH_PORT=""
if [ "$#" -eq 5 ]; then
SSH_PORT="-p "$5
fi
ssh $SSH_PORT -L $SRC_PORT:$2:$3 $GW -Nfg




Μετά κάνουμε το αρχείο εκτελέσιμο.

# chmod +x /etc/myscripts/urlwatch

Then we create a link to /usr/bin, in order to be able to run the script from everywhere.

# ln -s /etc/myscripts/urlwatch /usr/bin/

Then you can run the script against a web page

$ urlwatch kernel http://www.kernel.org/ kernel.html "new" "linus@linux.org,me@mymail.com"


• View Notes
This command will get the page at "http://www.kernel.org/" and will use files kernel.html and kernel.html.old which will be compared and if the differences have the word "new" it will send a notification mail to linus@linux.org and me@mymail.com with subject "kernel and the current date/time". Of course if it is run for the first time you won't get a notification, but it will only create the .old file for the next run. Then you can manually run the script when you want to check for changes or proceed to the next steps for automated monitoring.