Icinga

Icinga with IDOUtils Quickstart

Introduction

[Note] Note

Instead of installing Icinga from scratch you may want to use a package which might be available for your OS. Please take a look at the table of packages.

If you are planning to install from source then please use the official release tarball.

[Important] Important

Please don't use git snapshots unless you have an issue which might be solved in the current developer version.

This guide is intended to provide you with simple instructions on how to install Icinga from source (code) and have it monitoring your local machine within 30 minutes.

No advanced installation options are discussed here - just the basics that will work for most of the users who want to get started.

This guide will give you examples for currently three different Linux distributions: Fedora, Ubuntu and openSuSE. Similar distributions may work as well. That should include RedHat, CentOS, Debian and SLES.

Other distributions may inherit from these examples.

If you are planning to use Icinga without IDOUtils please read “Icinga Quickstart” instead!

What You'll End Up With

If you follow these instructions, here's what you'll end up with:

Prerequisites

During portions of the installation you'll need to have root access to your machine.

Make sure you've installed the following packages on your system before continuing. IDOUtils use the libdbi and the libdbi-drivers for several databases. The development libraries are also required. The following examples will show how to install the IDOUtils with the libdbi using MySQL or PostgreSQL.

Optional

At one time or another you may need to use SNMP-based checks so it is a good idea to install the required packages now. Otherwise some plugins are not compiled i.e. not available when you need them and it would require a recompile of the plugins.

New features for the IDOUtils:

SSL-Encryption between idomod and ido2db

If you want to use SSL-encryption you'll need openssl and openssl-devel/libssl-dev (Ubuntu) to be installed !

Oracle Database Support

If you want Oracle as RDBMS you'll need to install ocilib instead of libdbi.

[Note] Note

Using IDOUtils 1.5.0 with Oracle requires at least OCILIB 3.9.2 - don't install 3.9.0 or 3.9.1 as they remain buggy.

[Note] Note

More detailed instructions using IDOUtils with Oracle you can find in our wiki howto series about Oracle.

Get it from http://orclib.sourceforge.net/ and point configure to your Oracle libraries and header files e.g. from the Oracle instant client:

 #> ./configure --with-oracle-headers-path=/path/to/instantclient/sdk/include \
    --with-oracle-lib-path=/path/to/instantclient/
 #> make
 #> make install

Icinga 1.4

As of Icinga 1.4 the minimum expected Oracle version is Oracle 10gR2. Older versions may work, but are not supported. Oracle scripts are designed to split data, index and lobs into different tablespaces. For this reason there is a new prerequisite to define the tablespace names you want to use. If you are working in a small environment, you can set all defines to the same real tablespace. You will find a new script icinga_defines.sql which you have to adapt for your needs before applying oracle.sql. For your convenience there is a new script create_oracle_sys.sql included, which should help you to create the necessary tablespaces and an Icinga application user and must run as SYS. It will make use of icinga_defines.sql as well. Object creation has been moved from oracle.sql to the script create_icinga_objects_oracle.sql.

The former oracle.sql has been redesigned as a master script to record all other scripts as includes and expects these includes within the current directory. For this reason you should start sqlplus executing oracle.sql within this directory. This way the creation of user and tablespaces and the creation of Icinga tables runs in one step. As an all-in-one sample you will find a new script db/scripts/create_oracledb.sh. Edit variables to suit your needs and enjoy. If you prefer to do SYS steps for yourself, please uncomment create_oracle_sys.sql and make sure that your database Icinga user and tables exist and are defined with the same (or more) rights and that the correct settings have been applied to icinga_defines.sql.

Timezone support

Starting with Icinga 1.6 all dates are stored as local timestamps in the database. Before that there was a different behaviour storing and retrieving dates for each database system which was difficult to handle for frontend apps. Therefore datatypes have been changed for MySQL from DATETIME to TIMESTAMP and for Oracle from DATE to LOCAL TIMESTAMP (0). PostgreSQL was already using TIMESTAMP but has been changed to TIMESTAMP WITH TIME ZONE. IDO2DB will set session timezone to UTC and store all unix timestamps (which are UTC per definition) as UTC based values. Please make sure your system returns unix timestamps as real UTC based values (like "date -u '+%s'").

[Note] Note

Make sure your database session runs in the same timezone in which the existing dates have been stored (check your local timezone e.g. Oracle:"select sessiontimezone from dual;") if you are running the upgrade script. Additionally for your convenience in Oracle you should set your session timestamp format to the value you want, e.g "alter session set nls_timestamp_format='YYYY-MM-DD HH24:MI:SS';" or similar. Now you can view entries from different Icinga installations in different time zones which are stored in the same database and regardless from where you accessing it, all entries are returned in your local time zone.

Install Packages

You can install your packages by running the following commands (as root or sudo).

[Note] Note

Unfortunately sometimes the names of packages change between different releases of the same distribution so if you get a message that one of the packages cannot be found then please use the search option of your package manager to get the new name:

  • yum search <package name> ( Fedora/RHEL/CentOS )

  • apt-cache search <package name> ( Debian/Ubuntu )

  • zypper search <package name> ( openSuSE/SLES )

Fedora/RHEL/CentOS:

 #> yum install httpd gcc glibc glibc-common gd gd-devel
 #> yum install libjpeg libjpeg-devel libpng libpng-devel
[Note] Note

You may have to use libjpeg-turbo and libjpeg-turbo-devel instead

Debian/Ubuntu:

 #> apt-get install apache2 build-essential libgd2-xpm-dev
 #> apt-get install libjpeg62 libjpeg62-dev libpng12 libpng12-dev
[Note] Note

The numbers <62/12> might differ, depending on your distribution

[Note] Note

Starting with Debian 6.0 / Ubuntu 10.10 the package is called libpng-12-0, the name of the dev-package hasn't changed.

openSuSE:

Please use YaST to install at least the packages gd, gd-devel, libjpeg, libjpeg-devel, libpng, libpng-devel and, optionally, net-snmp, net-snmp-devel and perl-Net-SNMP.

Using zypper should work as well:

 #> zypper install gd gd-devel libjpeg libjpeg-devel libpng libpng-devel
 #> zypper install net-snmp net-snmp-devel perl-Net-SNMP
[Note] Note

The devel packages might be placed on the SDK DVDs.

Using old OpenSuSE (SLES) versions including version 10 it is most likely that there aren't any libdbi packages so you have to download and compile the sources. Replace <rdbm> with your desired RDBM like mysql or pgsql. Remember that the Oracle driver is not yet working and read the appropriate section with ocilib instead of libdbi.

  1. Download and extract the tar.gz files

    http://libdbi.sourceforge.net/download.html

    http://libdbi-drivers.sourceforge.net/download.html

     #> tar xvzf libdbi-0.8.3.tar.gz
     #> tar xvzf libdbi-drivers-0.8.3-1.tar.gz
  2. Install libdbi. Maybe you have to specify additional options with configure (set --prefix=/usr ... )

     #> cd libdbi-0.8.3
     #> ./configure --disable-docs
     #> make
     #> make install
  3. Install libdbi-drivers

     #> cd libdbi-drivers-0.8.3-1
     #> ./configure --with-<rdbm> --disable-docs
     #> make
     #> make install
    [Note] Note

    Using the 64-bit-versions you have to specify the paths to the include- and lib-dir explicitly:

     #> ./configure --with-<rdbm> \
        --with-<rdbm>-incdir=/usr/include/<rdbm>/ \
        --with-<rdbm>-libdir=/usr/lib64/ --disable-docs

Create Account Information

Become the root user.

 $> su -l

Create a new Icinga user account and give it a password.

 #> /usr/sbin/useradd -m icinga 
 #> passwd icinga 

On some distributions you'll need to add the group in a single step:

 #> /usr/sbin/groupadd icinga

For sending commands from the classic web interface to Icinga, you'll need to create a new group icinga-cmd. Add the webuser and the Icingauser to this group:

 #> /usr/sbin/groupadd icinga-cmd
 #> /usr/sbin/usermod -a -G icinga-cmd icinga
 #> /usr/sbin/usermod -a -G icinga-cmd www-data

(or www, wwwrun, apache depending on the distribution)

[Note] Note

Some usermod-versions (e.g in OpenSuSE 11 and SLES 11, resp.) are lacking the option -a. In this case please omit the option -a.

[Important] Important

Solaris only supports groupnames with max. 8 characters, please use icingcmd instead of icinga-cmd.

Download Icinga and the Plugins

Change to your local source directory i.e. /usr/src

 #> cd /usr/src

Get the current source from the Icinga Website. Don't forget to download the Nagios Plugins

Compile and Install Icinga with IDOUtils

Extract the Icinga source code tarball

 #> cd /usr/src/ 
 #> tar xvzf icinga-1.7.tar.gz 
 #> cd icinga-1.7
[Note] Note

This absolute path is meant when we refer to '/path/to/icinga-src/' in the following text.

Run the Icinga configure script and enable IDOUtils. You will get help by using the --help flag:

 #> ./configure --with-command-group=icinga-cmd --enable-idoutils 

Also with PostgreSQL you should not use the argument --enable-pgsq because it is not supported yet.

[Important] Important

Compiling on Solaris might fail upon unresolved library dependencies on gethostbyname. In that case run this before running configure:

 #> export LIBS=-lsocket -lnsl

With SSL-Encryption:

 #> ./configure --with-command-group=icinga-cmd --enable-idoutils --enable-ssl

With Oracle Database Support:

 #> ./configure --with-command-group=icinga-cmd --enable-idoutils --enable-oracle 

If you didn't install Oracle libraries to PATH, you can point configure there:

 #> ./configure --with-command-group=icinga-cmd \
    --enable-idoutils --enable-oracle \
    --with-oracle-lib=/path/to/instantclient

If you didn't install ocilib to the default path (/usr/local) you can point configure to the lib/inc directories:

 #> ./configure --with-command-group=icinga-cmd \
    --enable-idoutils --enable-oracle \
    --with-ocilib-lib=/path/to/ocilib/lib \
    --with-ocilib-inc=/path/to/ocilib/include
[Note] Note

If you want to change RDBM from Oracle to others, you need to recompile and reinstall IDOUtils!

 #> make distclean
 #> ./configure --enable-idoutils

Compile and Install

Compile the Icinga source code. There is also an extra option for IDOUtils (make idoutils) if you need to recompile only this module. To see available options, only use "make".

 #> make all

Install binaries, init script, sample config files, some eventhandlers, and set permissions on the external command directory.

 #> make install
 #> make install-init
 #> make install-config
 #> make install-eventhandlers
 #> make install-commandmode 

or shorter

 #> make fullinstall
 #> make install-config
[Note] Note

Starting with Icinga 1.5.0 make install-config is NOT included in make fullinstall anymore to avoid accidently overwriting of your config files.

[Note] Note

Starting with Icinga 1.7.0 make install-eventhandlers will install some event handler routines. To prevent undesired behaviour it is only included when you use make fullinstall.

[Note] Note

Install IDOUtils and applicable event broker modules only using the primary "make install" target. Do not manually copy and overwrite the existing module as this will result in a segfault on icinga core which is using idomod.so directly preventing usage of a temporary copy explicitly. This is useful for OMD.

[Note] Note

The Icinga-API served as a database abstraction layer written in PHP. Starting with Icinga 1.5 it is replaced by an internal database abstraction layer so Icinga-API doesn't have to be installed anymore.

Don't start Icinga yet - there's still more that needs to be done...

Customise Configuration

Sample configuration files have been installed by using "make install-config" into /usr/local/icinga/etc/. You'll need to make just one change before you proceed...

Edit the /usr/local/icinga/etc/objects/contacts.cfg config file with your favourite editor and change the email address associated with the icingaadmin contact definition to the address you'd like to use for receiving alerts.

 #> vi /usr/local/icinga/etc/objects/contacts.cfg
 #> cd /usr/local/icinga/etc/
 #> mv idomod.cfg-sample idomod.cfg
 #> mv ido2db.cfg-sample ido2db.cfg

If you want to enable SSL-encryption and you configured the IDOUtils with ./configure --enable-ssl, you have to change idomod.cfg as follows:

 use_ssl=1
 output_type=tcpsocket
 output=127.0.0.1

(don't forget to adjust the ip-address if your database isn't located on localhost) and ido2db.cfg in the following way:

 use_ssl=1
 socket_type=tcp 
[Note] Note

If SSL is enabled in ido2db but not in the different idomod clients - data from those instances will be lost - that's guaranteed! SSL configuration has to be the same on all nodes!!!

Enable the idomod event broker module

[Important] Important

Under normal circumstances the following module definition is already present in the modules subdirectory so there is no need to edit the main config file

 define module{
        module_name    ido_mod
        path           /usr/local/icinga/lib/idomod.so
        module_type    neb
        args           config_file=/usr/local/icinga/etc/idomod.cfg
        }

This also means that a broker_module entry must not be enabled!

Creation of Database and IDOUtils

[Note] Note

If you just installed a new database system then you have to start the database server before you can create a database. In case of MySQL you might use /etc/init.d/mysqld start (or /etc/init.d/mysql, depending on your distribution).

Configure the Classic Web Interface

Icinga ships with the classic web interface ("the CGIs") which can be installed via

 #> make cgis
 #> make install-cgis
 #> make install-html

If you are interested in the new Icinga Web, please refer to Install Icinga Web Interface.

Install the Icinga classic web config file in the Apache conf.d directory.

 #> make install-webconf

Create an icingaadmin account for logging into the Icinga classic web interface. Remember the password you assign to this account - you'll need it later.

 #> htpasswd -c /usr/local/icinga/etc/htpasswd.users icingaadmin

To change the password of an existing user or to add a new user, take this command:

 #> htpasswd /usr/local/icinga/etc/htpasswd.users <USERNAME>
[Note] Note

Depending on your distribution/Apache-version you may have to use htpasswd2 instead.

Reload/Restart Apache to make the new settings take effect.

Fedora/RHEL/CentOS:

 #> service httpd restart

Ubuntu/openSuSE:

 #> service apache2 restart

Debian:

 #> /etc/init.d/apache2 reload

Compile and Install the Nagios Plugins

Extract the Nagios plugins source code tarball.

 #> cd /usr/src 
 #> tar xvzf nagios-plugins-1.4.15.tar.gz 
 #> cd nagios-plugins-1.4.15  

Compile and install the plugins by changing install directory to /usr/local/icinga

 #> ./configure --prefix=/usr/local/icinga \
    --with-cgiurl=/icinga/cgi-bin --with-htmurl=/icinga \
    --with-nagios-user=icinga --with-nagios-group=icinga
 #> make 
 #> make install 

Adjusting the SELinux settings

RHEL and derived distributions like Fedora and CentOS are shipped with activated SELinux (Security Enhanced Linux) running in "enforcing" mode. This may lead to "Internal Server Error" messages when you try to invoke the Icinga-CGIs.

Check if SELinux runs in enforcing mode

 #> getenforce

Set SELinux in "permissive" mode

 #> setenforce 0

To make this change permanent you have to adjust this setting in /etc/selinux/config and restart the system.

Instead of deactivating SELinux or setting it into permissive mode you can use the following commands to run the CGIs in enforcing/targeted mode:

 #> chcon -R -t httpd_sys_script_exec_t /usr/local/icinga/sbin/
 #> chcon -R -t httpd_sys_content_t /usr/local/icinga/share/
 #> chcon -t httpd_sys_script_rw_t /usr/local/icinga/var/rw/icinga.cmd

Start IDOUtils and Icinga

IDOUtils must be started and running before Icinga is started.

Start IDOUtils:

Fedora/RHEL/CentOS/Ubuntu/openSuSE:

 #> service ido2db start

Debian:

 #> /etc/init.d/ido2db start

Stop IDOUtils:

Fedora/RHEL/CentOSUbuntu/openSuSE:

 #> service ido2db stop

Debian:

 #> /etc/init.d/ido2db stop

Verify the sample Icinga configuration files.

 #> /usr/local/icinga/bin/icinga -v /usr/local/icinga/etc/icinga.cfg 

If there are no errors, start Icinga.

Fedora/RHEL/CentOS/Ubuntu/openSuSE:

 #> service icinga start

Debian:

 #> /etc/init.d/icinga start

Configure Icinga Startup

Add Icinga to the list of system services and have it automatically start when the system boots (make sure you have installed the init script before).

Fedora/RHEL/CentOS/openSuSE:

 #> chkconfig --add icinga chkconfig icinga on

Debian/Ubuntu:

 #> update-rc.d icinga defaults

Login to the Classic Web Interface

You should now be able to access the Icinga classic web interface at the URL below. You'll be prompted for the username ( icingaadmin ) and password you specified earlier.

 http://localhost/icinga/

or

 http://yourdomain.com/icinga/ 

Click on the "Service Detail" navbar link to see details of what's being monitored on your local machine. It will take a few minutes for Icinga to check all the services associated with your machine.

Other Modifications

Make sure your system's firewall rules are configured to allow access to the web server if you want to access the Icinga classic interface remotely.

 #> iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT

Setting up your mail transfer agent (MTA) like exim, sendmail or postfix to allow Icinga sending notification emails won't be explained here. Please refer to the Nagios-Wiki for more resources.

There are several tweaks for the IDOUtils (former NDOUtils of Nagios) available.

You're Done

Congratulations! You successfully installed Icinga with IDOUtils. Your journey into monitoring has just begun. You'll no doubt want to monitor more than just your local machine, so check out the chapter on "Getting Started" about "Monitoring ..."