Root mail

By default, root mail is just saved on the system and tends to never be read.

/etc/aliases

# Person who should get root's mail
root: test@gmail.com

Run

sudo newaliases

sistema

nome della macchina

hostname -f
uptime

processes

top
htop
ps aux

RAM Memory

free

spazio su disco

# usage of each partition in the server:
df -h
 
#To list the size of a specific directory, run the following command:
du -sh /home/
 
#If you want to list all the subdirectory sizes under /home, run the following command:
du -sh /home/*

incron/inotify

permette di agganciare un listener su un nodo filesystem che notifica in caso di modifica

fail2ban

impedisce attacchi bruteforce chiudendo le connessioni dopo $maxretry

see also:

  • DenyHosts
  • sshguard

configuration: ignoreip: This is a space-separated list of IP addresses that cannot be blocked by fail2ban. bantime: Time in seconds that a host is blocked (600 seconds = 10 minutes) maxretry: Max. number of failed login attempts before a host is blocked filter: Refers to the appropriate filter file in /etc/fail2ban/filter.d logpath: The log file that fail2ban checks for failed login attempts.

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo /etc/init.d/fail2ban start
sudo vi /etc/fail2ban/jail.local
 
** by default all jails are disabled! enable relevant jails only in .local file(not overridden by updates) **
 
[DEFAULT]
bantime  = 900
destemail = test@gmail.com
action = %(action_mwl)s
ignoreip = 127.0.0.1 x.x.x.x
maxretry = 4
 
[apache]
enabled = true
 
[apache-noscript]
enabled = true
 
[apache-overflows]
enabled = true
 
[apache-badbots]
enabled = true
port    = http,https
filter  = apache-badbots
bantime = 86400
logpath = /var/log/apache*/*access.log
maxretry = 2
# applica cambiamenti
sudo /etc/init.d/fail2ban restart
 
# monitorare il funzionamento
/var/log/fail2ban.log
/var/log/fail2ban.log | grep WARNING
 
sudo /etc/init.d/fail2ban status
 
# controllare il firewall
iptables -L
 
# ispezione manuale dei logs
zcat /var/log/auth.log* | grep 'Failed password' | grep sshd | awk '{print $1,$2}' | sort -k 1,1M -k 2n | uniq -c
 
cat /var/log/apache2/error.log

settare monitoring del log di fail2ban, un ip bannato viene bannato per 1 anno /etc/fail2ban/jail.conf

[fail2ban]
enabled = true
filter = fail2ban
action = iptables-allports[name=fail2ban]
logpath = /path/to/fail2ban.log
# findtime: 1 day
findtime = 86400
# bantime: 1 year. Use -1 for forever
bantime = 31536000
# number of failures before banning
maxretry = 2

Setting up the script to extract the frequently banned IPs and then putting them into /etc/hosts.deny is what I'd recommend.

bannare tutto

ALL : ALL

report dei ban applicati

sudo fail2ban-client status apache

apticron

Make The System Send Notifications About Available Updates

apt-get install apticron
nano /etc/apticron/apticron.conf
 
# test:
sudo apticron --cron

specify one or more email addresses (separated by a space) to be notified about available updates:

# set EMAIL to a space separated list of addresses which will be notified of
# impending updates
#
EMAIL="root@localhost test@gmail.com"

Log monitor

Inviare Log Files per email

mail -s "a subject" test@gmail.com < /var/log/apache2/error.log

ssmtp can use a Gmail account as SMTP and send emails from the command line.

echo "Hello, User!" | mail user@domain.com

/etc/ssmtp/ssmtp.conf

root=***E-MAIL***
mailhub=smtp.gmail.com:587
rewriteDomain=
hostname=smtp.gmail.com:587
UseSTARTTLS=YES
UseTLS=YES
AuthUser=***E-MAIL***
AuthPass=***PASSWORD***
AuthMethod=LOGIN
FromLineOverride=YES

LogWatch

It normally runs nightly, parses through a bunch of logs and will email a nice report of the daily activity. Things like user logins, sudo commands, disk utilization, as well as generally weird log messages. the default config provides sufficiently good results.

systemd distros(debian,ubuntu) will no longer automatically install a syslog server that logwatch needs (due to systemd relying on its own journal), you need to install and enable rsyslog to make sure your /var/log is not empty before logwatch will be of any use.

# sudo
 
Output = mail
Format = html
MailTo = test@gmail.com
MailFrom = test@gmail.com
 
# To make Logwatch send you daily reports, set the time range to "Today". Other choices are "All" and "Yesterday."
Range = Today
# Now set your desired detail level for your reports:
Detail = High
Save your changes, and run Logwatch to send you a report:
# logwatch
# logwatch --print --detail High --archives --range All
Stampa a video (--print) invece che inviare via mail, con il massimo dettaglio (--detail High),
includendo anche i log archiviati (--archives) tutti i messaggi di ogni data (--range All)

there should be a cronjob for it:

sudo nano /etc/cron.daily/00logwatch
# m h dom mon dow user  command
  0 1  * * *   root       /usr/sbin/logwatch

logrotate

sudo vi /etc/logrotate.d/appname.log
/var/log/appname.log {
        rotate 2
        weekly
        size 250k
        compress
}
 
sudo vi /etc/logrotate.d/$service_name
chmod 644 /etc/logrotate.d/$service_name
chown root.root /etc/logrotate.d/$service_name
 
# call log rotation (outside of the usual cron job)
logrotate -f /etc/logrotate.d/$service_name
 
/usr/local/zend/var/log/*.log {
        rotate 7
        daily
        weekly
        size 5M
        missingok
        notifempty
        compress
        delaycompress
        copytruncate
}

Logsentry (formerly logcheck)

LogWatch gives summary of the state of the system each day, logsentry when something out of the ordinary happened. is strictly a log message parser, runs hourly. It contains a whitelist of normal messages and assumes that anything else is bad. bad messages then get compiled together and emailed out, This require quite a bit of tuning. You must make sure that it both monitors all the log files you want,as well as making sure it knows what is normal in your environment.

Apache Monitoring

seo_analytics

APC tool

php_apc

network monitoring systems

Monit

riavvia daemon processes, For instance sshd, apache and mysql.

Monit provides a built-in HTTP(S) interface and you can use a browser to access info.

In difference to many monitoring systems, Monit can act if an error situation occur, e.g.; if apache is using too much resources (e.g. if a DoS attack is in progress) Monit can stop or restart apache and send you an alert mail message.

conf: /etc/monit/monitrc

check system localhost
    if loadavg (1min) > 4 then alert
    if loadavg (5min) > 2 then alert
    if memory usage > 75% then alert
    if swap usage > 25% then alert
    if cpu usage (user) > 70% then alert
    if cpu usage (system) > 30% then alert
    if cpu usage (wait) > 20% then alert
 
#set mailserver smtp.gmail.com port 587
#    username "MYUSER" password "MYPASSWORD"
#    using tlsv1
 
check process mysqld with pidfile /var/run/mysqld/mysqld.pid
group database
start program = "/etc/init.d/mysql start"
stop program = "/etc/init.d/mysql stop"
if failed host 127.0.0.1 port 3306 then restart
if 5 restarts within 5 cycles then timeout
if cpu > 60% for 2 cycles then alert
if cpu > 80% for 5 cycles then restart
if totalmem > 200.0 MB for 5 cycles then restart
if children > 250 then restart
if loadavg(5min) greater than 10 for 8 cycles then stop
 
check process sshd with pidfile /var/run/sshd.pid
start program "/etc/init.d/ssh start"
stop program "/etc/init.d/ssh stop"
if failed host 127.0.0.1 port 22 protocol ssh then restart
if 5 restarts within 5 cycles then timeout
 
check process httpd with pidfile /var/run/apache2.pid
group apache
start program = "/etc/init.d/apache2 start"
stop program = "/etc/init.d/apache2 stop"
if failed host 127.0.0.1 port 80
protocol http then restart
if 5 restarts within 5 cycles then timeout
if cpu > 60% for 2 cycles then alert
if cpu > 80% for 5 cycles then restart
if totalmem > 200.0 MB for 5 cycles then restart
if children > 250 then restart
if loadavg(5min) greater than 10 for 8 cycles then stop
#    if failed host www.tildeslash.com port 80 protocol http
#       and request "/somefile.html"
#       then restart
#    if failed port 443 type tcpssl protocol http
#       with timeout 15 seconds
#       then restart
#    if 3 restarts within 5 cycles then timeout

supervisord

alternativa leggera a monit di "Process supervision"

apt-get install supervisor
 
sudo apt-get install supervisor
sudo vi /etc/supervisor/conf.d/beanstalk.conf
[program:shoptest0_worker]
command=php /var/zend-applications/shoptest0.it/tests/btd_test_worker.php
autostart=true
autorestart=true
stdout_logfile=/var/zend-applications/shoptest0.it/logs/shoptest0_worker.log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
stdout_capture_maxbytes=1MB
stderr_logfile=/var/zend-applications/shoptest0.it/logs/shoptest0_worker-err.log
stderr_logfile_maxbytes=1MB
stderr_logfile_backups=10
stderr_capture_maxbytes=1MB
sudo /etc/init.d/supervisor stop
sudo /etc/init.d/supervisor start

Pannelli di controllo

  • phpsysinfo: script di monitoraggio

serverstats

semplicissimo e basic. usa cron e sysstat RRDtool

  1. need to modify configuration files in the "/etc/serverstats"
  2. sources.php - this file contains information about all active modules/sources
  3. ServerStats directory view-able from the web
 
# graph/ must be writable for the webserver user
ls -la /var/lib/serverstats/graph/
rm /var/lib/serverstats/graph/*.png
 
# cache/ and rrd/ must be writable by the user updating the stats (cronjob)
ls -la /var/lib/serverstats/
ls -la /var/lib/serverstats/cache/
ls -la /var/lib/serverstats/cache/graph
ls -la /var/lib/serverstats/rrd/
chown nobody.nogroup /var/lib/serverstats/rrd/*.rrd
ls -la /var/lib/serverstats/rrd/
 
# add update.php to the crontab:
sudo nano /etc/cron.d/serverstats
 
# fa girare il log dati macchina, senza notifica email, in root crontab
*/3 * * * *    /usr/bin/php /usr/share/serverstats/update.php >/dev/null 2>&1
rm /etc/cron.d/serverstats # il default potrebbe non funzionare, permessi errati
# oppure togliere user nobody o dare permessi in scrittura sulla cartella dei database

collectd

collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. can then be used to find current performance bottlenecks.

sysstat

basato su sar, produce log binari molto dettagliati. vedi pacchetto distribuzione

  • sar: collects and reports system activity information;
  • iostat: reports CPU utilization and disk I/O statistics;
  • mpstat: reports global and per-processor statistics;
  • pidstat: reports statistics for Linux tasks (processes);
  • sadf: displays data collected by sar in various formats;
  • nfsiostat: reports I/O statistics for network filesystems;
  • cifsiostat: reports I/O statistics for CIFS filesystems.

isag: consultazione grafici

altri:

  • http://www.observium.org/wiki/Main_Page
  • Ganglia: pensato i cluster. interfaccia web php. non usare
  • munin: complicato client server multi server. non usare
  • FreeNATS
  • Zabbix
  • cacti: multi server. non usare