Memory Management in Linux

Memory is very important in linux. We need to be very cautios while managing our memories. We need to figure out bottleneck and make sure we have enough memory so that application running in Linux runs smooth. Few are the parameters we need to keep in mind and tune memory.

vmstat - swap in and swap out should be less.
procps - rpm needed for vmstat
dstat --top-oom -- First process kernal is going to kill so that memory is freed
getconf PAGE_SIZE - To view page size(Memory)
Resident Memory - Is actual memory
Virtual Memory - Each process cant access direct physical memory, so it uses virtual memory. Each process have its own virtual space. Physical memory address space is mapped to virtual memory address space. It is sort of security restriction where process cant directly indulge with Physical RAM. MMU - Memory Management Unit does the mapping between physical address space to virtual address space. pmap 1 shows mapping between VAS to PAS.
pmap 1 - 1st column in virtual memory / 2nd column is Physical memory
Resint column when added comes out more then actual physical memory.
less /proc/1/smaps - It shows dirty read and dirty buffer

Architecture:
1. Free
2. Inactive Clean - File is opened but dint modify. It will read from disk always
3. Inactive dirty - File is opened and made some changes. It will be in RAM, if any power cut, there will be loss, so kernal keep pushing them to disk. BDI is responisble for moving dirty buffers to disk.
4. Inactive

CREATING SWAP in LINUX

FIRST WAY:
Create extra partition and make that partition as SWAP.

Check the present partition disk - fdisk -l
To see the current SWAP - swapon -s
/etc/fstab - We need to make an entry here of partitions so that after reboot, we dont loose the partition created.
To remove Swap - swapoff -v /dev/xvd3
To read from fstab - swapon -a

Step 1:
1. Create partition
   fdisk /dev/sdc   -- main disk name where we will create partition
   After that follow what is on screen, it will help to create a partition for SWAP
   p - will show the partitions
   m - will show menu where all commands and switches are mentioned
   n - add a new partitions."p" will create primary partitions
   partition number - Which number of partitions
   rest few paramter as default enter, enter
   Mention how much partition we need - +2G
   p - will show the new partitions
   t - change partition system ID
   Mention partition number for which we want to change system ID - 3
   Enter HEX code - L and then 82 (Linux SWAP/solaris). L will list the codes
   Then enter W - Write table to disk and exit
   fdisk -l /dev/sdc - Will show the existing partitons on the disk
   
2. Create Swap Partition
   mkswap /dev/sdc3  - It will create the partitions
   swapon -v /dev/sdc3 - It will add the partitions
   swapon -s - will show partition has been added
3. Make entry in Fstab
   device_name mount_point(swap) partition_type(swap) defaults 0 0
4. Check swap
   swapon -a -- It will read from /etc/fstab

SECOND WAY:
Create swap with the existing partition.

#In this tutorial we will learn how to create new swap file on CentOS 7 / RHEL 7. Generally on Cloud Servers ( AWS/ DigitalOcean) or VPS, swap space is not created by default.
#Prerequisites
#Must have free space on mounted disk. You can check by using df -Th command.
#Steps to create/add new swap file on Linux

1. Create swapfile-additional file with dd command in / (root). You can select any other partition but it should be mounted (For eg. /opt, /usr ,/NewMountedPartition)

dd if=/dev/zero of=/swapfile-additional bs=1M count=4048
dd = It is a unix command used for convert and copy a file
if = read from FILE instead of stdin
/dev/zero = /dev/zero is a special file in Unix-like operating systems that provides as many null characters (ASCII NUL, 0x00) as are read from it
of = write to FILE instead of stdout
/swapfile-additional = file named swapfile-additional will be created in /
bs = Read and write bytes at a time but if you do not mention MB or GB like only number it will read as bytes. for eg. bs=1024 means 1024 bytes
count = Copy input blocks in our case it is 1024 (1M * 4048 = 4GB)

2. Run mkswap command to make swap area
mkswap /swapfile-additional

3. Change the permission of file swapfile-additional
chmod 600 /swapfile-additional

4. Permanent mounting the swap space by editing the /etc/fstab file .
Use your file editor, I generally use vi editor.
vi /etc/fstab
Paste below given content in /etc/fstab file
/swapfile-additional swap swap    0   0

5. Now mount the swap area, run below given command.
mount -a

6. Enable the swap area
swapon -a

7. Check the number swap space mounted on your system
swapon -s

8. To check how much is swap space available on system. Run below given command
free -m

TOP Command in Linux

Top is an utility provided by Linux. It gives completes analysis of all the process running in the system like all the active process. Also it tells about system performance like CPU, Memory utilisation. Some of the commands are:

shift + M - List process according to memory usage
shift + P - List the process according to CPU usage
Shift + W - For saving top output to a file.

EXAMPLE:

top - 04:38:00 up 10 days, 13:23,  1 user,  load average: 0.00, 0.01, 0.05
Tasks:  90 total,   2 running,  88 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  1014976 total,   116512 free,    93008 used,   805456 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   628588 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
14446 postgres  20   0  279896   2236   1472 S  0.3  0.2   5:04.70 postgres
    1 root      20   0  128092   6700   3944 S  0.0  0.7   0:27.88 systemd
    2 root      20   0       0      0      0 S  0.0  0.0   0:00.10 kthreadd
    3 root      20   0       0      0      0 S  0.0  0.0   0:01.29 ksoftirqd/0
    7 root      rt   0       0      0      0 S  0.0  0.0   0:00.00 migration/0
    8 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcu_bh
    9 root      20   0       0      0      0 R  0.0  0.0   1:26.88 rcu_sched
   10 root      rt   0       0      0      0 S  0.0  0.0   0:03.87 watchdog/0
   12 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 khelper
   13 root      20   0       0      0      0 S  0.0  0.0   0:00.00 kdevtmpfs
   14 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 netns
   15 root      20   0       0      0      0 S  0.0  0.0   0:00.01 xenwatch
   16 root      20   0       0      0      0 S  0.0  0.0   0:00.00 xenbus
   18 root      20   0       0      0      0 S  0.0  0.0   0:00.14 khungtaskd
   19 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 writeback
   20 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 kintegrityd
   21 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 bioset
   22 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 kblockd
   23 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 md
   28 root      20   0       0      0      0 S  0.0  0.0   0:00.24 kswapd0
   29 root      25   5       0      0      0 S  0.0  0.0   0:00.00 ksmd


Below are the few description of the contents of TOP Command:

Job - Present time, how much time system is up, how many users connected, load average.

Task - How many total tasks, how many tasks running, how many tasks sleeping, how many tasks stopped, how many zombie process.

CPU - % of CPU used by users, % of CPU used by system, % of CPU used by nice value, idle CPU %, % of CPU wating for IO(to write), % of CPU used by Hardware Interupt, % of CPU used by Software Interupt, Steal time - used when there is a virtual box( delay in getting the actual CPU by virtual CPU in virtual systems)

Memory - Total RAM, Memory Used (Actually used - buffer), Memory Free( Actual + buffer + cache from swap) , Buffer( when system have free memory, it extracts few pages and keep it in buffer for caching, if system needs, it will give back the pages from buffer)

Swap- Total Swap, Swap Used, free, caching

PID - Process ID
USER - user through which process are running
PR - Priority - Process priority. 0 - highest priority 40 - Lowest
NI - Nice value - CPU Scheduling Priority. Utility through which we can set the priority manually. -19 -highest priority 20 - lowest priority
Virtual Memory - Each process cant access direct physical memory, so it uses virtual memory. Each process have its own virtual space. Physical memory address space is mapped to virtual memory address space. It is sort of security restriction where process cant directly indulge with Physical RAM. MMU - Memory Management Unit does the mapping between physical address space to virtual address space. pmap 1 shows mapping between VAS to PAS.
Resident Memory - Total Physical Memory that is non swapped.
Shared Memory - Memory that is being shared by other processes
S - state - Sleeping or running
% CPU - % of CPU used by the process
% MEM - % of Memory Used by the process
Time - How long is the process running
Command - What is the command that is being executed.

C - Process with obsolete path
k <pid> - to kill a process
z - Change color of top
r - for re nice a value
1 - shows the number of CPU
d - change refresh rate - Default 3 sec
A - Splits screen into multiple part/screen. Top Right Side says which split screen I am. So accordingly we can sort per split screen. Goto different split screen using a. Shift + a will toggle screen from small to big.
h - help
n - Reduce the number of listed process to n
I - To hide load average line
t - to hide CPU states. 2nd and 3rd line.
m - to hide/show the memory line. 4th and 5th.
top -u postgres or top and press u
top -n 2 stops refresh after 2 sec
top -p pid, pid
top -b -n 1
top -d3 -b -n 2  > top_output.txt.
top -H -p pid
Shift F - add extra column as required.