For now, I’m writing a script for a customer, where I got to inject those manipulated dates in an SQL query.
The problem is that the script I’m talking in my previous post does not put the leading 0 when the day or the month is less than 10.

Example :

datecalc -a 1960 12 31 + 7
1961 1 7


In the sql query, the dates MUST be with format YYYY-MM-DD!
It’s pretty annoying when the dates returned are with format YYYY-M-D or YYYY-MM-D or YYYY-M-DD, isn’t it ?

As usual, I googled (Google knows all!) and found the easy answer!


#!/bin/sh
# by Jakukyo Friel under GPL v2.

sed -e 's/\(-\)\([0-9]-\)/-0\2/' | sed -e 's/\(-\)\([0-9]$\)/-0\2/'

Source

I’ve fought for hours with this line, cause it ain’t worked! The mistake was in the second sed substitution command, where the $ was included in the last range of characters (the last 2 digits representing the day), and it shouldn’t be there, but behind the mark!

Finally here is the correct version :


### This script convert date formats:
# YYYY-MM-DD => YYYY-MM-DD
# YYYY-M-DD => YYYY-MM-DD
# YYYY-MM-D => YYYY-MM-DD
# YYYY-M-D => YYYY-MM-DD

sed -e 's/\(-\)\([0-9]-\)/-0\2/' | sed -e 's/\(-\)\([0-9]\)$/-0\2/'


Let’s try the same command line :


echo "1967-1-7" | sed -e 's/\(-\)\([0-9]-\)/-0\2/' | sed -e 's/\(-\)\([0-9]\)$/-0\2/'
1967-01-07


YAY!!!

I’m angry at wasting so much time for a small misplaced $ sign. So I’m sharing with you all…
Unlike me, It may save you some time

I had to implement date manipulation in an existing unix shell script (ksh).
The easiest way to add or substract days (or months, or years) from the current date (assuming that you are logged on a Linux host) is :

user@host:~$ date –date “now +3 days”
lundi 12 mars 2012, 09:55:17 (UTC+0100)

user@host:~$ date –date “now -3 days -4 month”
dimanche 6 novembre 2011, 09:58:20 (UTC+0100)

But this time I’m dealing with solaris 9 & 10 servers! “date –date” is not available.

So how to add X days or substract Y days from the current date ?

I’ve found this very useful script :

#! /usr/bin/ksh

# datecalc — Perderabo’s date calculator
#

USAGE=”\
datecalc -a year month day – year month day
datecalc -a year month day [-|+] n
datecalc -d year month day
datecalc -D year month day
datecalc -j year month day
datecalc -j n
datecalc -l year month
use \”datecalc -help\” use for more documentation”

DOCUMENTATION=”\
datecalc Version 1.1

datecalc does many manipulations with dates.
datecalc -a is for date arithmetic
datecalc -d or -D converts a date to the day of week
datecalc -j converts to date to or from julian day
datecalc -l outputs the last day of a month

All dates must be between the years 1860 and 3999.

datecalc -a followed by 7 parameters will calculate the
number of days between two dates. Parameters 2-4 and 6-8
must be dates in ymd form, and parameter 5 must be a minus
sign. The output is an integer. Example:

> datecalc -a 1960 12 31 – 1922 2 2
14212

datecalc -a followed by 5 parameters will calculate the
a new date offset from a given date, Parameters 2-4 must
be a date in ymd form, paramter 5 must be + or -, and
paramter 6 must be an integer. Output is a new date.
Example:

> datecalc -a 1960 12 31 + 7
1961 1 7

datecalc -d followed by 3 parameters will convert a date
to a day-of-week. Parameters 2-4 must be a date in ymd
form. Example:

> datecalc -d 1960 12 31
6

datecalc -D is like -d except it displays the name of
the day. Example:

> datecalc -D 1960 12 31
Saturday

datecalc -j followed by 3 parameters will convert a date
to Modified Julian Day number. Example:
> datecalc -j 1960 12 31
37299

datecalc -j followed by a single parameter will convert
a Modified Julian Day number to a date. Example:
> datecalc -j 37299
1960 12 31

datecalc -l followed by year and month will output the last
day of that month. Note that by checking the last day of
February you can test for leap year. Example:
> datecalc -l 2002 2
28″

lastday() {
integer year month leap
# ja fe ma ap ma jn jl ag se oc no de
set -A mlength xx 31 28 31 30 31 30 31 31 30 31 30 31

year=$1
if ((year<1860 || year> 3999)) ; then
print -u2 year out of range
return 1
fi
month=$2
if ((month<1 || month> 12)) ; then
print -u2 month out of range
return 1
fi

if ((month != 2)) ; then
print ${mlength[month]}
return 0
fi

leap=0
if ((!(year%100))); then
((!(year%400))) && leap=1
else
((!(year%4))) && leap=1
fi

feblength=28
((leap)) && feblength=29
print $feblength
return 0
}

date2jd() {
integer ijd day month year mnjd jd lday

year=$1
month=$2
day=$3
lday=$(lastday $year $month) || exit $?

if ((day<1 || day> lday)) ; then
print -u2 day out of range
return 1
fi

((standard_jd = day – 32075
+ 1461 * (year + 4800 – (14 – month)/12)/4
+ 367 * (month – 2 + (14 – month)/12*12)/12
– 3 * ((year + 4900 – (14 – month)/12)/100)/4))
((jd = standard_jd-2400001))

print $jd
return 0
}

jd2dow()
{
integer jd dow numeric_mode
set +A days Sunday Monday Tuesday Wednesday Thursday Friday Saturday

numeric_mode=0
if [[ $1 = -n ]] ; then
numeric_mode=1
shift
fi

jd=$1
if ((jd<1 || jd>782028)) ; then
print -u2 julian day out of range
return 1
fi

((dow=(jd+3)%7))

if ((numeric_mode)) ; then
print $dow
else
print ${days[dow]}
fi
return
}

jd2date()
{
integer standard_jd temp1 temp2 jd year month day

jd=$1
if ((jd<1 || jd>782028)) ; then
print julian day out of range
return 1
fi
((standard_jd=jd+2400001))
((temp1 = standard_jd + 68569))
((temp2 = 4*temp1/146097))
((temp1 = temp1 – (146097 * temp2 + 3) / 4))
((year = 4000 * (temp1 + 1) / 1461001))
((temp1 = temp1 – 1461 * year/4 + 31))
((month = 80 * temp1 / 2447))
((day = temp1 – 2447 * month / 80))
((temp1 = month / 11))
((month = month + 2 – 12 * temp1))
((year = 100 * (temp2 – 49) + year + temp1))
print $year $month $day
return 0
}

#
# Parse parameters and get to work.
case $1 in
-a) if (($# == 8)) ; then
if [[ $5 != - ]] ; then
print -u2 – “$USAGE”
exit 1
fi
jd1=$(date2jd $2 $3 $4) || exit $?
jd2=$(date2jd $6 $7 $8) || exit $?
((jd3=jd1-jd2))
print $jd3
exit 0
elif (($# == 6)) ; then
jd1=$(date2jd $2 $3 $4) || exit $?
case $5 in
-|+) eval ‘((‘jd2=${jd1}${5}${6}’))’
jd2date $jd2
exit $?
;;
*)
print -u2 – “$USAGE”
exit 1
;;
esac

fi
;;

-d|-D) if (($# != 4)) ; then
print -u2 – “$USAGE”
exit 1
fi
jd1=$(date2jd $2 $3 $4) || exit $?
numeric=-n
[[ $1 = -D ]] && numeric=”"
eval jd2dow $numeric $jd1
exit $?
;;

-j) if (($# == 4)) ; then
date2jd $2 $3 $4
exit $?
elif (($# == 2)) ; then
jd2date $2 $3 $4
exit $?
else
print -u2 – “$USAGE”
exit 1
fi
;;

-l) if (($# == 3)) ; then
lastday $2 $3
exit $?
else
print -u2 – “$USAGE”
exit 1
fi
;;

-help) print – “$USAGE”
print “”
print – “$DOCUMENTATION”
exit 0
;;

*) print -u2 – “$USAGE”
exit 0
;;

esac

#not reached
exit 7

Source : http://www.unix.com/unix-dummies-questions-answers/4870-days-elapsed-between-2-dates.html?postid=16559#post16559

I took my inspiration from it, and I could make it easily!

Have a nice day,
Wangee

The dream for the geeks would be to use a single port for several protocols. Yves Rutschle (aka WhiteRabbit) started from a simple observation :

• When you’re browsing on a secured website, the https client (your web browser) talks the first.
• A ssh client (putty for example) is waiting for a response from the SSH server.  The server talks first (login prompt)

Imagine a daemon that listens on a TCP port (ex : port 443), and it waits.

• When no packet is received and after 2 seconds, it means it’s a ssh connection and it forwards to the ssh server.
• When it gets an incoming packet, and depending of the header, it’s either a https, openvpn, xmpp or even a tinc request and it forwards to the right server!

This amazing proxy exists and is called sslh.   sslh is a SSL/SSH multiplexer pretty easy to use and install.

For Fedora Core 16, there is no rpm package yet, that’s why I made one, and you can download it : sslh-1.10-1.fc16.i686.rpm

Please note that this package is unofficial for now, I’ve submitted it for being part of the official Fedora repositories.

Another IMPORTANT note: For now, sslh is launched under the user root, and it’s not a good thing at all on a security point of view.  I’m looking forward to solve this as soon as possible.

How to install sslh on Fedora Cora 16 ?

• Install the package

rpm -ivh sslh-1.10-1.fc16.i686.rpm

• check the file /etc/sysconfig/sslh and the ports where SSH and https (SSL) will listen. I want sslh to listen on port 443, ssh listen on its original port (22) and https to listen on 8443. Mine file looks like this :

LISTEN=ifname:443
SSH=localhost:22
SSL=localhost:8443

• check the file /etc/init.d/sslh. Locate the line OPTIONS= and set up the ports for your needs. Eventually replace –user=root by a restricted existing user name. This will workaround the root security flaw. Mine looks like this :

OPTIONS="-v --user=root -p 0.0.0.0:443 --ssl 127.0.0.1:8443 --ssh 127.0.0.1:22"

• configure the web server. Edit the file /etc/httpd/conf.d/ssl.conf and locate every line containing the string “443″ and replace by “8443″
• configure the ssh server : Edit the file /etc/ssh/sshd_config and modify the line starting by “Port 22″ according to your needs. Mine didn’t require any change.
• restart the syslog daemon, ssh and httpd

systemctl restart rsyslog.service
systemctl restart httpd.service
systemctl restart sshd.service

• Finally start sslh

systemctl start sslh.service

• Enjoy!

If you have applied this mini how-to, please leave a comment if you found something wrong or if you want to propose improvements.

But do you know HTOP ? Htop is similar to top but with nice features.   It requires ncurses.

Instead of just showing numbers, htop will show you in a graphic way the CPU Usage, Memory Usage, Swap Usage.

Among those nice features (non-exhaustive list) :

• Horizontal and vertical Scrolling.
• Easier to kill a task.  No need to type the PID of the process you want to kill.   Just scroll to the process you want to kill and press “k” or F9, then the signal you want to send.
• Same if you want to renice a process : Press F7 to lower or F8 to raise the priority level.
• You want to see the parents/child processes ?  No problem, press F5 and you got a nice treeview.
• Sorting the processes by CPU%, MEM%, PID ?  F6 will do it!
• Finding the open files by a process ? pressing “l” will make it!
• Setting CPU Affinity (“a” key)

Tree View

Support for a large number of processors

 

Setting up CPU Affinity

 

htop running on a machine with 128 cores and 1TB of RAM

The father of “htop” is Hisham Muhammad (now you know where the “h” comes from ^^), and is hosted on sourceforge.

Packages available for Fedora, Debian and Ubuntu.

Next episode  in those obvious things a backup administrator *must* know is about the tape rotation schemes.

I’m filling my wiki slowly and today I wrote a page about the GFS (Grandfather Father Son) tape rotation.  This tape rotation scheme is very often described as an industry standards.  Right or Wrong, I don’t know?  What I’m sure of, is that I used it many times.   And this GFS scheme is very interesting and can be adapted to most business need.  Depending of the constraints of data retention you might have, depending on the number of tapes you have, depending on the amount of data to backup, GFS is the first scheme to look at.

Click here to read the wiki page about the GFS Tape rotation scheme

 I’ve been around the block in this fantastic world of data backup.  I know amanda for more than 10 years.  I’ve worked on using amanda for saving a beta site when I worked at Isabel .  The constraint was to avoid paying a lot of money for buying Legato Networker (Now called  EMC Networker) licences.  I used amanda with a Digital Equipment 8-tapes DLT Library.  Again using amanda, I reworked the full backup strategy for another of my system engineer  jobs (Molis [now called vision4health]) but the interest for backups came back for good when I worked in a Google Datacenter. Those huge libraries of several thousand of tapes made me very excited to embrace again data saving.

In the cases I mentioned above, Amanda had all the features that were fitting perfectly with my needs.  Why ?

• Amanda is an Open Source software.  It’s released under the GNU GPL Licence.  In two  words, you can use it for private purposes, but also for your business needs.   You can even charge a fee for the service you provide. I would recommend you to browse to http://www.gnu.org/licenses/licenses.html for more details on the GNU Public License
• Amanda can handle a single tape drive, but also a 10.000 tapes library.
• Amanda offers a command line interface, which was very insteresting because I could monitor it more easily (Especially with Nagios, the Open Source Monitoring software).   This is also a *very* useful feature when you’re working remotely!
• Amanda can backup and restore using the network.  In other words, one tape library could backup several servers.   This may seem obvious, but in the late 90’s, that was purely awesome!

Today, I don’t remember much about amanda.  Because I just left it aside because I had no need and opportunity to use it. But as soon as I found an interest in this software again, I was pleasantly surprised it was still there and that it had improved!

The goal of this post is to tell you that I’m happy and excited to have a new takeover of amanda as I had done in the past.
Then, I’ll plan to explore the new features this fabulous backup software offers.

So I think to do all that more in a spirit of re-learning, with details on how I configured it.

This is the time to introduce you to my new personal wiki where I’ll cover the topics I’m interested in. My wish is to make this wiki my personal knowledge base, but also for anyone who might be interested in those topics as well.  I hope they will find relevant information for their private or business use and share theirs.

So, what is Amanda ?

As the official website says, AMANDA, is the acronym of “Advanced Maryland Automatic Network Disk Archiver”.  It consists of a backup solution that allows the IT administrators to set up a single master backup server to back up multiple hosts over network to tape drives/changers or disks or optical media. Amanda uses native utilities and formats (e.g. dump and/or GNU tar) and can back up a large number of servers and workstations running multiple versions of Linux or Unix. Amanda uses also a native Windows client to back up Microsoft Windows desktops and servers.  This may seem little, but believe me, having the possibility to save any type of Unix, or Windows server is purely great, especially when used in a professional environment.

As I’ve retrieved 2 old good SCSI DAT-40 drives, I said to myself it’s the right time to spend time on Amanda and start a topic called Amanda Cookbook on my wiki

The first topic is pretty obvious, and talks about installing Amanda on a RPM based linux distribution using Yum. More infos will come with the time.

Thanks for reading

This is my first post, and I’m starting up a technical blog.  The goal of this blog is simple : Keeping my readers updated on some technical stuffs I wrote.

What I plan is a new takeover of the skills I had in the past.  I think to do all that more in a spirit of re-learning, with details on how I made it.  I hope the readers of this technical blog will find relevant information for their private or business use.

In addition to this blog, I’ve set up a wiki that you can reach at the following URL : http://wiki.opsyx.com/

Thanks for reading, and you can already bookmark this website

Wangee