CLI monitoring basic commands

  • man/apropos
  • free: ram control
  • kill/killall/pkill (pattern kill): chiude processi
  • ps/pgrep: find processes
  • cd/pwd
  • nice: run a program with modified scheduling priorit
  • top/htop
  • vmstat: Report virtual memory statistics(processes, memory, paging, block IO, traps, disks and cpu activity)
  • watch: runs command repeatedly, displaying its output and errors
  • w
  • cat/less/more/grep log file

Gruppi

groupadd

aggiunge un gruppo

groupadd $group

groupdel

groupmod

groups

Tells which group you DON'T belong to (opposite of command "groups")

sed -e "/$USER/d;s/:.*//g" /etc/group | sed -e :a -e '/$/N;s/\n/ /;ta'
add user to group

aggiunge $username al grupppo

sudo usermod -aG $group $username

Utenti

list users:

list user that can potentially access the system:

cat /etc/passwd | grep /bin/sh
cat /etc/passwd | grep -v nologin | grep -v /bin/false

to prevent system user login:

chsh -s /bin/false <username>
id

mostra le informazioni sui gruppi di un utente

  id tony
->uid=1123(tony) gid=1124(developers) groups=1124(developers)

per vedere tutti gli utenti del sistema

  cat /etc/passwd

A user who has a valid shell and no password can still log in by non-password-based methods, the most common being an ssh key. A valid shell is necessary to run cron jobs.

  cat /etc/shadow

mostra utenti con password

passwd

Setup password for user $user:

passwd $user
useradd

aggiungere un utente ESISTENTE ad un gruppo

sudo usermod -a -G $group $username
sudo usermod -a -G www-data  $username
 
id $username
groups $username

aggiungere un NUOVO utente ad un gruppo

useradd -g $group $username
id tony
->uid=1123(tony) gid=1124(developers) groups=1124(developers)
 
useradd -G $group $username

Create a new user with no shell. Useful to provide other services without giving shell access

useradd -s /usr/sbin/nologin $username

in alternativa

chsh -s /sbin/nologin <username>

assicurarsi che la shell muta esista e se non esistesse crearla

less /etc/shells
echo "/usr/sbin/nologin" >> /etc/shells

assicurarsi che l'utente abbia la shell bash, non la shell sh

echo $0
# if "sh" instead of "bash"
sudo chsh -s /bin/bash <username>
userdel
usermod

aggiungere un utente a un gruppo

adduser <username> www-data

oppure

usermod -a -G <groupname> <username>

ensure that the group has write permission:

sudo chown www-data:www-data /var/www/file
chmod g+w /var/www/file

List all groups and the user names that were in each group

for u in `cut -f1 -d: /etc/passwd`; do echo -n $u:; groups $u; done | sort

Sticky bit

If your program creates directories that behave like /tmp, in that lots of people put things there but shouldn't be able to remove each other's files, then you should set the sticky bit on the directory.You can set the sticky bit on a directory with the chmod command by invoking the following:

% chmod o+t directory

This name is anachronistic; it goes back to a time when setting the sticky bit caused a pro- gram to be retained in main memory even when it was done executing.The pages allocated to the program were "stuck" in memory.

Servizi, autostart, upstart

sudo update-rc.d tomcat7 disable 0123456

Performance Monitoring

proc - process information pseudo-file system

man proc 5

basic server monitor script:

$msg .= "<hr>";
    
$msg .= "<b>DISKS(df):</b>";
    
$msg .= sprintf('<pre style="font-family: monospace;">%s</pre>', `df -h /`);
    
$msg .= "<b>RAM(free):</b>";
    
$msg .= sprintf('<pre style="font-family: monospace;">%s</pre>', `free -h`);
    
$msg .= "<b>CPU(actual load avg.,5min,10min,currently running processes,last PID):</b>";
    
$msg .= sprintf('<pre style="font-family: monospace;">%s</pre>', `cat /proc/loadavg`);
iostat + iotop

To get more details of disk IO, use iostat -x. Again, you want to give it a sampling interval, and ignore the first set of output. iostat also gives you some cpu information but top does that better; let's focus on the Device section: Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 9.80 0.20 36.60 0.40 5326.40 4.80 144.09 0.06 1.62 1.41 5.20 There are 3 easy ways to tell if a disk is a probable bottleneck here, and none of them show up without the -x flag, so get in the habit of using that. "avgqu-sz" is the size of the io request queue; if it is large, there are lots of requests waiting in line. "await" is how long (in ms) the average request took to be satisfied (including time enqueued); recall that on non-SSDs, a single seek is between 5 and 10ms. Finally, "%util" is Linux's guess at how fully saturated the device is.

top / htop

To learn more about per-process CPU and RAM usage, use "top." I won't paste top output here because everyone is so familiar with it, but I will mention a few useful things to know: "P" and "M" toggle between sorting by cpu usage and sorting by memory usage "1" toggles breaking down the CPU summary by CPU core SHR (shared memory) is included in RES (resident memory) Amount of memory belonging to a process that has been swapped out is VIRT - RES a state (S column) of D means the process (or thread, see below) is waiting for disk or network i/o "steal" is how much CPU the hypervisor is giving to another VM in a virtual environment; as virtual provisioning becomes more common, avoiding noisy neighbors is increasingly important

VIRT stands for the virtual size of a process, which is the sum of memory it is actually using, memory it has mapped into itself (for instance the video card's RAM for the X server), files on disk that have been mapped into it (most notably shared libraries), and memory shared with other processes. VIRT represents how much memory the program is able to access at the present moment.

RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (This also corresponds directly to the %MEM column.) This will virtually always be less than the VIRT size, since most programs depend on the C library.

SHR indicates how much of the VIRT size is actually sharable (memory or libraries). In the case of libraries, it does not necessarily mean that the entire library is resident. For example, if a program only uses a few functions in a library, the whole library is mapped and will be counted in VIRT and SHR, but only the parts of the library file containing the functions being used will actually be loaded in and be counted under RES.

vmstat / dstat

is your server is swapping? run vmstat with "vmstat sampling-period", e.g.,

vmstat -SM 5

output:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0   4428    317   1345    0    0    96    71  199  776 10  2 88  1  0
 0  0      0   4428    317   1345    0    0     0     0  225  414  0  0 99  1  0

The first line is your total system average since boot; typically this will not be very useful, since you are interested in what is causing problems NOW. Then you will get one line per sample period; most of the output is self explanatory. The reason to start with vmstat is the "swap" section: si and so are swap in (memory read from disk) and swap out (memory written to disk). si (swap in) and so (swap out). Remember that a little swapping is normal, by default, Linux will swap infrequently used pages of application memory to disk to free up more room for disk caching, even if there is enough ram to accommodate all applications. Numbers in the hundreds or thousands indicate your server is swapping heavily. This consumes a lot of CPU and other server resources and you would get a very (!) significant benefit from adding more memory to your server.

The bi (bytes in) and bo (bytes out) column show disk I/O (including swapping memory to/from disk) on your server.

The us (user), sy (system) and id (idle) show the amount of CPU your server is using. The higher the idle value, the better.

Temperature
cat /proc/acpi/thermal_zone/THRM/temperature
acpi -t
Files

Kill processes that have been running for more than a week

find /proc -user myuser -maxdepth 1 -type d -mtime +7 -exec basename {} \; | xargs kill -9

encript file

openssl des3 -salt -in unencrypted-data.tar -out encrypted-data.tar.des3
gpg --allow-multiple-messages --decrypt-files *

concatenate avi files

avimerge -o output.avi -i file1.avi file2.avi file3.avi
RAM Memory
free -h
top
cat /proc/meminfo

The Linux free command extracts and formats pertinent information from the meminfo file and displays it as follows:

free -h
free -mot
free -mt

look at is the buffers/cache row used value:This is how much space your applications are currently using. Linux likes to use any spare memory to cache disk blocks(colonna "cached")

ps aux --sort -rss

ps legge la memoria usata dai singoli processi, con l'opzione --sort si indicano campi per cui ordinare, separati da virgola, +/- per ordinamento ascendente discendente

Find Out The Top 10 Memory Consuming Process

  ps -auxf | sort -nr -k 4 | head -10
CPU monitor

comando linux per monitorare il server:

cat /proc/loadavg
0.22 0.12 0.09 1/68 12621

first three columns measure the CPU and IO utilization( actual, five and 10 minute periods ). The fourth column shows the number of currently running processes and the total number of processes. The last column displays the last process ID used.

Find Out top 10 CPU Consuming Process

ps -auxf | sort -nr -k 3 | head -10
controllare cosa sta facendo un programma

tecnica spiegata da Rasmuss a verona 2009, permette di verificare le operazioni + lente effettuate da un programma come php

Each line in the trace contains the system call name, followed by its arguments in parentheses and its return value.

Run strace against /bin/foo and capture its output to a text file in output.txt:

  strace -o output.txt /bin/foo

You can strace the webserver process and see what it's doing. For example, strace php5 fastcgi process, enter:

    strace -p 22254 -s 80 -o /tmp/debug.lighttpd.txt

To see only a trace of the open, read system calls, enter :

    strace -e trace=open,read -p 22254 -s 80 -o debug.webserver.txt

files aperti da un programma

strace -e trace=open  mc

-o filename : Write the trace output to the file filename rather than to screen (stderr). -p PID : Attach to the process with the process ID pid and begin tracing. The trace may be terminated at any time by a keyboard interrupt signal (hit CTRL-C). strace will respond by detaching itself from the traced process(es) leaving it (them) to continue running. Multiple -p options can be used to attach to up to 32 processes in addition to command (which is optional if at least one -p option is given). -s SIZE : Specify the maximum string size to print (the default is 32).

W

w command displays information about the users currently on the machine, and their processes.

  w <username>
uptime
uptime
w
top
Hot Key Usage t Displays summary information off and on. m Displays memory information p Display Processor information A Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system. f Enters an interactive configuration screen for top. Helpful for setting up top for a specific task. o Enables you to interactively select the ordering within top. r Issues renice command. k Issues kill command. z Turn on or off color/mono

vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.

  vmstat 3

Display Memory Utilization Slabinfo

  vmstat -m

Get Information About Active / Inactive Memory Pages

  vmstat -a

Find Out Who Is Logged on And What They Are Doing w command displays information about the users currently on the machine, and their processes.

  w <username>

sysstat command

iostat - Average CPU Load, Disk Activity The command iostat report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).

  iostat

sar command is used to collect, report, and save system activity information. To see network counter, enter:

  sar -n DEV | more

To display the network counters from the 24th:

  sar -n DEV -f /var/log/sa/sa24 | more

You can also display real time usage using sar:

  sar 4 5
Storage

spazio disponibile

df -m

spazio usato

du -sh .

Network

linux_networking

Posta locale

molti programmi riferiscono del loro funzionamento a root via email aggiungi un alias di posta per il tuo utente per ricevere al posta inviata a root

nano /etc/aliases
root: myuser
# root: test@test.com
 
# run update
newaliases
# reload smtp
service postfix restart
# test msg
echo test | mail -s "test message" root@domain.com

start up scripts

# To run a daemon or any command as root before login, add it to /etc/rc.local @see /etc/init.d/skeleton
# ~/.bash_profile will run a command after login
# ~/.bashrc will be executed by login shells only
# a cron job with @reboot as its time field

date time zone

for t in America/New_York Australia/Perth Japan Asia/Kolkata; do zdump ${t}; done

screencast in GIF

  • usa Recordmydesktop per catturare un video .mkv
  • genera la palette che ffmpeg userà per creare la GIF:
  • creare la GIF vera e propria
ffmpeg -y -ss 30 -t 3 -i video1.mkv \
-vf fps=10,scale=320:-1:flags=lanczos,palettegen palette.png
 
ffmpeg -ss 30 -t 3 -i video1.mkv -i palette.png -filter_complex \
"fps=10,scale=320:-1:flags=lanczos[x];[x][1:v]paletteuse" video1.gif

inotify wait

monitor change of file and exec a program each time a file is modified

 
BASE_DIR=$(cd $(dirname "$0"); pwd)/..
inotifywait -e close_write -m $BASE_DIR/public/* |
while read -r directory events filename; do
  #echo "$filename change!";
  $BASE_DIR/bin/compile file $filename
done