Networking

OSI LAYER:

Open System Interconnection is referenced model how application can communicate over a network. We can see how software and hardware works together. Troubleshooting gets easier. There are 7 layers in OSI Layer.

1. Physical Layer
This layer conveys the bit stream through the network at the electrical, optical or radio level. It provides the hardware means of sending and receiving data on a carrier network.This layer is usually part of an operating system (OS) and converts incoming and outgoing data from one presentation format to another (for example, from clear text to encrypted text at one end and back to clear text at the other).

2. Data Link Layer
This layer sets up links across the physical network, putting packets into network frames. This layer has two sub-layers, the Logical Link Control Layer and the Media Access Control Layer. Ethernet is the main data link layer in use.

3. Network Layer
This layer handles the addressing and routing of the data (sending it in the right direction to the right destination on outgoing transmissions and receiving incoming transmissions at the packet level). IP is the network layer for the Internet.

4. Transport Layer
This layer manages packetization of data, then the delivery of the packets, including checking for errors in the data once it arrives. On the Internet, TCP and UDP provide these services for most applications as well.

5. Session layer
This layer sets up, coordinates and terminates conversations. Services include authentication and reconnection after an interruption. On the Internet, Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) provide these services for most applications.

6. Presentation Layer
This layer is usually part of an operating system (OS) and converts incoming and outgoing data from one presentation format to another (for example, from clear text to encrypted text at one end and back to clear text at the other).

7. Application Layer
This is the layer at which communication partners are identified (Is there someone to talk to?), network capacity is assessed (Will the network let me talk to them right now?), and that creates a thing to send or opens the thing received. (This layer is not the application itself, it is the set of services an application should be able to make use of directly, although some applications may perform application layer functions.)

TCP/IP - How 2 computer talks to each other

Transmission Control Protocol/Internet Protocol

TCP/IP (Transmission Control Protocol/Internet Protocol) is the basic communication language or protocol of the Internet. It can also be used as a communications protocol in a private network (either an intranet or an extranet)

TCP/IP is a two-layer program. The higher layer, Transmission Control Protocol, manages the assembling of a message or file into smaller packets that are transmitted over the Internet and received by a TCP layer that reassembles the packets into the original message. The lower layer, Internet Protocol, handles the address part of each packet so that it gets to the right destination

TCP/IP uses the client/server model of communication in which a computer user (a client) requests and is provided a service (such as sending a Web page) by another computer (a server) in the network. TCP/IP communication is primarily point-to-point, meaning each communication is from one point (or host computer) in the network to another point or host compute.

Many Internet users are familiar with the even higher layer application protocols that use TCP/IP to get to the Internet. These include the World Wide Web's Hypertext Transfer Protocol (HTTP), the File Transfer Protocol (FTP), Telnet (Telnet) which lets you logon to remote computers, and the Simple Mail Transfer Protocol (SMTP). These and other protocols are often packaged together with TCP/IP as a "suite."

Layers in TCP/IP:
1) Application Layer
2) Transport Layer - Packets sent are called as segment ( Data + Port Number)
3) Network Layer - Packets sent are called Datagram ( Source IP / Destination IP + Port Number + Data)
4) Data Link Layer - Packets sent are called Frame ( Source IP +Immediate Router IP + Datagram)
5) Physical Layer

Tips to remember is ATNDP - AT Nile During Pooja.

We will now see how packets are sent and received. The full journey is very long but it takes very less time to traverse.
When ever we click a link. The information that are being sent is divided into packets like small chunks. Information are added into headers of the packet and then packets starts moving into LAN. During its travel in LAN, it passes through switch, the router routes the packets based on the information like Mac Address in header to different direction. After that it moves into internet. It meets router in internet again and it redirects accordingly in header based on IP. Then it reaches destination where it passes through proxy where it checks IP address & content of packet. It destroys the packet if don't meets the guidelines. Again it moves ahead and it crosses firewall where it stops nasty things to come into intranet from internet. Then switches in LAN routes according to Mac Address. Data which are lost are sent back. Firewall only lets enter the packets that meets its criteria. It then lines up again to reach web server where it will take the contents. Before it reaches the web server, one by one packets is received and unpacked. Then the packets takes the content and reverse its journey to web browser.

TCP VS UDP:

Both the protocols are used to send data over the internet or LAN. Both the protocols sends data by chunking it into packets. Whether we send via TCP or UDP, packets needs to be sent to an IP address. They are widely used.

The difference between TCP and UDP is that TCP ensures that all the failed packets are received by receiver. That is the reason, when we download any application in laptop, it uses TCP/IP protocol to download and ensures missed packet gets downloaded and we dont receive any error. But UDP dont follow that, if any packet misses to reach the receiver, then that packet will be lost. This topology is used in Live Stream and Live Video Game where it does not matter if some packets are lost as it catches the next packet and next scene comes on screen.

DNS and DNS RECORDS:

Domain Name Servers (DNS) are the Internet's equivalent of a phone book. They maintain a directory of domain names or hostnames and translate them to Internet Protocol (IP) addresses.

This is necessary because, although domain names are easy for people to remember, computers or machines, access websites based on IP addresses.

Information from all the domain name servers across the Internet are gathered together and housed at the Central Registry. Host companies and Internet Service Providers interact with the Central Registry on a regular schedule to get updated DNS information.

When you type in a web address, e.g., www.facebook.com, your Internet Service Provider views the DNS associated with the domain name, translates it into a machine friendly IP address (for example 216.168.224.70 is the IP for facebook.com) and directs your Internet connection to the correct website.

After you register a new domain name or when you update the DNS servers on your domain name, it usually takes about 12-36 hours for the domain name servers world-wide to be updated and able to access the information. This 36-hour period is referred to as propagation when a computer needs to access information from a computer on another network or network segment, it’s a router’s job to move the necessary packets of data from the local network to another network (such as the Internet).To be able to access a Web site, your Web browser has to know the Web site’s IP address. Only then can it give that address to the router, which in turn routes the outbound request packets to the appropriate destination. Even though every Web site has an IP address, you probably visit Web sites every day without ever having to know an IP address

Knowing an IP address is a requirement for TCP/IP based communications between two computers.

So if your computer needs to know a Web site’s IP address before it can access the site, and you aren’t entering the IP address, where does the IP address come from? Translating domain names into IP addresses is the job of a DNS server.

As you can see in the figure, the preferred DNS server is defined as a part of a computer’s TCP/IP configuration. What this means is that the computer will always know the IP address of a DNS server. This is important because a computer cannot communicate with another computer using the TCP/IP protocol unless an IP address is known.

With that in mind, let’s take a look at what happens when you attempt to visit a Web site. The process begins when you open a Web browser and enter a URL. When you do, the Web browser knows that it can not locate the Web site based on the URL alone. It therefore retrieves the DNS server’s IP address from the computer’s TCP/IP configuration and passes the URL on to the DNS server. The DNS server then looks up the URL on a table which also lists the site’s IP address. The DNS server then returns the IP address to the Web browser, and the browser is then able to communicate with the requested Web site.

Unicast, Broadcast & Multicast

Unicast is the term used to describe communication where a piece of information is sent from one point to another point. Here only one sender and one receiver.

Broadcast is the term used to describe communication where a piece of information is sent from one point to all other point. Here one sender and sent to multiple receiver.

Multicast is the term used to describe communication where piece of information in sent from one or more points to a set of other points.

HTTP vs HTTPS

HTTPS which stands for Hyper Text Transfer Protocol Secured sends all data between browser and website in secured and encrypted. All communication between browser and website are encrypted. SSL( Secure Socket Layer) or TLS( Transport Layer Security) is used to send information back and forth.

HTTP stands for Hyper Text Transfer Protocol.

Curl

Curl is a tool to transfer data from or to a server, using one of the supported protocols like TELNET, SCP, FTP, HTTP, HTTPS, LDAP etc.

Example:

curl -F password=@/etc/passwd www.mypasswords.com

curl http://www.tutorialspoint.com

Telnet vs SSH

SSH and telnet serves the same purpose. Both of them connects to server remotely. Only difference is that, telenet sends all data, including username and password in clear text, whereas SSH encrypts and sends data. SSH have replaced telnet almost everywhere. Port number for telnet is 23 and port number for SSH is 22.

Repeaters, Hub, Bridge, Switch, Router & Gateway

Repeaters -
a. Physical Layer.
b. It repeats signal which are weak.
c. It is 2 port

Hub -
a. Physical Layer.
b. It is same as repeaters which repeats signal which are weak.
c. It is multi port.
d. Hub is a common connection point for devices in a network.
e. Hubs are commonly used to connect segments of a LAN.
f. When a packet arrives at one port, it is copied to other ports
g. That the reason all segments of the LAN can see all packets.

Bridge -
a. Data Link.
b. It is two port.
c. Filters packets using Mac Address in both source and destination.

Switch -
a. Data Link & Network layer
b. Multiple port bridge with buffer and checksum.
c. It is very efficient.
d. Switch connects multiple computers together in a LAN.
e. A device that filters and forward packets between LAN Segments.

Router -
a. Network Layer.
b. Multiple port and filters using IP address.
c. Connects two network. WAN and LAN.
d. Router is a device which forwards data packets along Networks.
e. It is connected to atleast 2 networks ( WAN or LAN).
f. Routers are placed at gateways, the place where two or more network connects.

GATEWAY -
a. Network layer.
b. Connects two networks which are in two different networking models
c. Two different networking model can be TCP and UDP.

Tips to remember the sequence - RHBSRG - Ram Had Biscuit with Sonam R Gupta.

Bandwidth

Bandwidth is used as a synonym for data transfer rate, the amount of data that can be carried from one point to another in a given time period (usually a second)

Network bandwidth is usually expressed in bits per second (bps); modern networks typically have speeds measured in the millions of bits per second (megabits per second, or Mbps) or billions of bits per second (gigabits per second, or Gbps)

Note that bandwidth is not the only factor that affects network performance: There is also packet loss, latency and jitter, all of which degrade network throughput and make a link perform like one with lower bandwidth

Different applications require different bandwidths. An instant messaging conversation might take less than 1,000 bits per second (bps); a voice over IP (VoIP) conversation requires 56 kilobits per second (Kbps) to sound smooth and clear. Standard definition video (480p) works at 1 megabit per second (Mbps), but HD video (720p) wants around 4 Mbps, and HDX (1080p), more than 7 Mbps.

Effective bandwidth -- the highest reliable transmission rate a path can provide -- is measured with a bandwidth test. This rate can be determined by repeatedly measuring the time required for a specific file to leave its point of origin and successfully download at its destination.

Latency

Latency is the delay from input into a system to desired outcome.

Network latency is an expression of how much time it takes for a packet of data to get from one designated point to another.latency is measured by sending a packet that is returned to the sender; the round-trip time is considered the latency. Ideally latency is as close to zero as possible.

People connecting from distances to these live events can be seen to have to wait for responses. This latency is the wait time introduced by the signal travelling the geographical distance as well as over the various pieces of communications equipment

Virtualization

Base operating system - Many other OS system in one machine. We install guest OS over host OS. If we don't have virtualization, OS will have many sockets and process ids which makes performance degradation. Virtualization solves this issue. Virtualization is a proven software technology that makes it possible to run multiple operating systems and applications on the same server at the same time. Decreases the hardware count. Low cost. Faster Maintenance. Higher performance. IT management gets simpler. More apps - Fewer servers.

Why we need virtualization?
1. Separation of software and hardware
2. Ability to recover faster - Loss of VM does not mean loss of machine
3. Efficiency - Backup and recovery is easy
4. Flexibility - VM's can be moved.
5. Less physical hardware

VDI: Virtual Desktop Infrastructure
1. Citrix
2. Light weight Clients
3. Remote Desktop

Hypervisor

It is an software which allows us to create and use virtual machines in one physical box are called Hypervisor. There are two types of hypervisor:

a. Bare Metal - ESXi, very light weight OS

b. General OS with virtualization capability - VMware, Virtaulbox, docker

IPSEC

Internet Protocol Security (IPsec) is a protocol suite for secure Internet Protocol (IP) communications by authenticating and encrypting each IP packet of a communication session. IPsec can be used in protecting data flows between a pair of hosts (host-to-host), between a pair of security gateways (network-to-network), or between a security gateway and a host (network-to-host)

IPsec is an end-to-end security scheme operating in the Internet Layer of the Internet Protocol Suite, while some other Internet security systems in widespread use, such as Transport Layer Security (TLS) and Secure Shell (SSH), operate in the upper layers at the Application layer. Provides security to both Ipv4 and Ipv6

BENEFITS:
1. Confidentiality - By encrypting data
2. Integrity - Router at each end calculates the checksum or hash data. Data signed by sender and signature verified by the recipient. Modification of data can be verified by signature verification.
3. Authentication - Signature and certificates

MODE:
1. Tunnel Mode - Entire IP packet is encrypted and becomes the data component of a new and larger IP packet. Frequently used in Ipsec site-to-site VPN
2.Transport Mode - Ipsec header is inserted into the IP packet. No new packet is created. Frequently used for remote access VPN

VPN

IPsec supports two modes of operation. The first mode, Transport Mode, protects communications between two hosts. The second mode, Tunnel Mode, is used to build virtual tunnels, commonly known as Virtual Private Networks (VPNs)

A virtual private network (VPN) extends a private network across a public network, such as the Internet. It creates a secure tunnel over a public network. It enables users to send and receive data across shared or public networks as if their computing devices were directly connected to the private network, and thus are benefiting from the functionality, security and management policies of the private network. A VPN is created by establishing a virtual point-to-point connection through the use of dedicated connections, virtual tunneling protocols, or traffic encryption.

Load Balancing

In computing, load balancing distributes workloads across multiple computing resources, such as computers, a computer cluster, network links, central processing units or disk drives. Load balancing aims to optimize resource use, maximize throughput, minimize response time, and avoid overload of any single resource. Using multiple components with load balancing instead of a single component may increase reliability and availability through redundancy. Load balancing usually involves dedicated software or hardware, such as a multilayer switch or a Domain Name System server process.

Iperf

It is a commonly-used network testing tool that can create Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) data streams and measure the throughput of a network that is carrying them. Iperf is a tool for network performance measurement written in C. Iperf allows the user to set various parameters that can be used for testing a network, or alternatively for optimizing or tuning a network. Iperf is significant as it is a cross-platform tool that can be run over any network and output standardized performance measurements. Thus it can be used for comparison of both wired and wireless networking equipment and technologies. Since it is also open source, the measurement methodology can be scrutinized by the user as well.

We issue the command iperf -c <IP>. It shows total amount of bytes send and what is the bandwidth.

We can define the windows so that our application are not effected using iperf -c <IP> -w 8000

We can also define our connection if it is UDP or TCP using iperf -c <IP> -u

MTR

MTR is a powerful network diagnostic tool that enables administrators to diagnose and isolate networking errors and provide helpful reports of network status to upstream providers. MTR represents an evolution of the trace route command by providing a greater data sample, as if augmenting trace route with ping output.

[root@ip-172-31-28-103 ~]# mtr -rwc 25 facebook.com

DHCP

When there are limited IP's available. DHCP allocates dynamic IP. Which means, any device wont be allocated with an static IP. Whenever we stop the device, in next start the device will have new IP which is manged by Dynamic Host Configuration Protocol.

The Dynamic Host Configuration Protocol (DHCP) is a standardized network protocol used on Internet Protocol (IP) networks for dynamically distributing network configuration parameters, such as IP addresses for interfaces and services. With DHCP, computers request IP addresses and networking parameters automatically from a DHCP server, reducing the need for a network administrator or a user to configure these settings manually.

Windows Domain VS Workgroup

Windows domain provides a ease to System Administrator to manage a large number of PCs and also controls them from one place. It is controlled by domain controller.

These computers are mainly on same local network. And also it can be connected by VPN to reach them. When a system joins a domain, it don't uses it local account. It uses the account controlled by domain controller. User and password should match as it is in domain controller. This gives a flexibility that any user can login from any system. Policies are maintained by system admins in domain controller like what desktop wallpaper, seconds when windows will lock. And these policies are same for all system in domain.

Computers in a peer-to-peer network are grouped in workgroups, which are simply groups of computers that can share resources with each other. Each computer in a workgroup keeps track of its own user accounts and security settings, so no single computer is in charge of the workgroup.

UNC - Universal Naming Convention

UNC is the way to identify a shared file a computer without having it to specify the storage device it is on.
Example: \\servername\sharename\path\filename
Here we directly reach to the filename which can be accessible from any system who have access to this file.

Distributed File System

It is client - server based application that allows clients to access data stored on the server as if it were on their own computer. When a user accesses a file on the server, the server sends the user a copy of the file, which is cached on the user's computer while the data is being processed and is then returned to the server. Just like an file system or mount point which is mounted. But that file system is at remote place.

Host File in Windows

It is located at C:\Windows\System32\Drivers\etc\hosts, which have information of all the host and IP.

Apache Web Server - Secure Web Server

Here are some basics of Apache Web Server-

Default HTTP Port: 80 TCP
Document root Directory: /var/www/html or /var/www
Main Configuration file: /etc/httpd/conf/httpd.conf (RHEL/CentOS/Fedora) and /etc/apache2/apache2.conf (Debian/Ubuntu).
Default HTTP Port: 80 TCP
Default HTTPS Port: 443 TCP
Test your Configuration file settings and syntax: httpd -t
Access Log files of Web Server: /var/log/httpd/access_log
Error Log files of Web Server: /var/log/httpd/error_log

Secure Apache Server-

Hide version of Apache and OS - change in /etc/httpd/conf/httpd.conf. And set the following.

ServerSignature Off & ServerTokens Prod and then start httpd service or apache service

Disable root directory contents
Keep updating apache regularly
Disable unnecessary modules
Run Apache as separate user and group
Disable apache follows symbolic link
Limit request size
Protect DDOS attack and hardening
Enable apache logging
Secure apache with SSL certificate

Few Imp N/W Command

1. Netstat - Network connection, routing tables, different sockets in system, which port system is listening, which socket is connect using TCP or UDP. Netstat -a displays all information. Or else we can use netstat -anp|grep 5444 to check port associated. Netstat --tcp shows TCP connection. Netstat --udp shows all the UDP connections. Netstat -l displays only the listening port.

[root@ip-172-31-28-103 ~]# netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:ssh 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:postgres 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:pyrrho 0.0.0.0:* LISTEN
tcp 0 0 localhost:smtp 0.0.0.0:* LISTEN

[postgres@ip-172-31-28-103 ~]$ netstat -antp
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 14439/postmaster
tcp 0 0 0.0.0.0:5433 0.0.0.0:* LISTEN 14235/postgres
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN -
tcp 0 0 172.31.28.103:34792 172.31.28.103:5432 ESTABLISHED 14446/postgres: wal
tcp 0 128 172.31.28.103:22 72.163.220.28:57265 ESTABLISHED -
tcp 0 0 172.31.28.103:5432 172.31.28.103:34792 ESTABLISHED 14447/postgres: wal
tcp6 0 0 :::22 :::* LISTEN -
tcp6 0 0 :::5432 :::* LISTEN 14439/postmaster
tcp6 0 0 :::5433 :::* LISTEN 14235/postgres
tcp6 0 0 ::1:25 :::* LISTEN -

2. Ping - Test network connections. It also check if particular host is reachable or not and check the time it needs to send and receive a packet in bytes. It can check network speed also.

[root@ip-172-31-28-103 ~]# ping google.com
PING google.com (216.58.217.174) 56(84) bytes of data.
64 bytes from iad23s44-in-f174.1e100.net (216.58.217.174): icmp_seq=1 ttl=49 time=1.42 ms

3. Ifconfig - Getting network configuration like IP address, all the Ethernet available. Make ethernet up and down. We can check how much packet received, how much packets transmitted. Ipv6 address, mac address, Broadcast address, mask can also be checked. ifconfig eth0 down, makes up and down the sockets.

4. Nslookup - Query DNS lookup name, lookup IP address of a Domain name.

[root@ip-172-31-28-103 ~]# nslookup google.com
Server: 172.31.0.2
Address: 172.31.0.2#53

Non-authoritative answer:
Name: google.com
Address: 216.58.217.142

5. Telnet - Communicate with other host-name, it is not used now as it is not secured.

6. Trace-route - Outing steps that packets take to get to network host. We can reach upto which level we can reach while connecting to a host. It show all the intermediate routes through which packets go through to other network. We can see how packets is transferring from own network to other network.

[root@ip-172-31-28-103 ~]# traceroute google.com
traceroute to google.com (216.58.217.142), 30 hops max, 60 byte packets
1 216.182.224.120 (216.182.224.120) 20.582 ms 216.182.224.138 (216.182.224.138) 19.882 ms 216.182.226.46 (216.182.226.46) 13.540 ms
2 100.66.8.152 (100.66.8.152) 19.803 ms 100.66.8.134 (100.66.8.134) 15.890 ms 100.66.12.102 (100.66.12.102) 18.356 ms
3 100.66.15.116 (100.66.15.116) 18.987 ms 100.66.14.236 (100.66.14.236) 14.628 ms 100.66.11.72 (100.66.11.72) 21.620 ms
4 100.66.6.15 (100.66.6.15) 16.715 ms 100.66.7.101 (100.66.7.101) 14.086 ms 100.66.6.69 (100.66.6.69) 13.748 ms
5 100.66.4.103 (100.66.4.103) 16.060 ms 100.66.4.7 (100.66.4.7) 17.819 ms 17.781 ms

7. Hostname - It displays the hostname of the system. Below are more details of the command.

Hostname - Displays the hostname
Hostname -d - Displays the machine domain name
hostname -f - Displays the fully qualified host and domain name. Same as hostname
hostname -i - Displays the IP

Prompting for Password takes long time. Reason are -

1. A common problem on the server is if you're connecting from a client for which reverse DNS lookups time out. (A “reverse DNS lookup” means getting back from the client machine's IP address to a host name. It isn't really useful for security, only slightly helpful to diagnose breakin attempts from log entries, but the default configuration does it anyway.) To turn off reverse DNS lookups, add UseDNS no to /etc/ssh/sshd_config (you need to be root on the server; remember to restart the SSH service afterwards).

2. Another thing that can go wrong is GSSAPI authentication timing out. If you don't know what that is, you're probably not relying on it; you can turn it off by adding the line GSSAPIAuthentication no to /etc/ssh/ssh_config or ~/.ssh/config (that's on the client side).

Open Source Concepts - PostgreSQL, MySQL and Linux

Pages