OS and Linux Basics

This guide contains the basics of how the OS works and certain basic Linux commands.

What is an OS?

• The OS is the middleman between apps and hardware.
• It manages resources among apps.
• It isolates contents of apps.

Tasks the OS performs

• Resource allocation and management

  • Process management: Manages CPU resources

  • Memory management

    • Allocating working memory
    • RAM is limited on hardware
    • When you ran out of memory, the OS does memory swapping
    • It swaps memory between apps
    • Swapping slows down the computer
    • It swaps out the process that is not being used and saves its data to the HD
    • It then loads the new process onto the RAM
    • This process takes some time and slows down the computer

  • Storage management

    • Called secondary memory
    • Persistent long-term data
• Manage file structure
  • Unix: Tree file system
  • Windows: Multiple root folders
• Management of I/O Devices
  • eg. monitors, keyboards, etc
• Security and networking
  • Managing users and permissions
  • Each user has his own space
  • Each user has permissions
  • Ports and IP addresses assignment

OS Components

The Kernel

• It's the part that loads first
• It manages hardware components
• It also manages device drivers (programs that allow external devices to connect to computer)
• Different OS's have different kernels
• The kernel is the part interacting between the hardware and the applications
• The Kernel starts the process for the app
• Allocates resources to the app
• Cleans up resources when app shuts down
• Kernel is a program which consists of device drivers, dispatcher, scheduler, File System, etc
• There's an application layer on top of the kernel
  • eg. Different Linux distributions: Ubuntu, Mint, CentOs, Debian
  • Different application layers but based on the same Linux kernel
  • Android is also based on the Linux kernel
  • MacOS and iOS are based on a different kernel: Darwin
• Server Operating Systems
  • Mostly Linux based
  • More light-weight and performant
  • No GUI or other user applications
• How to interact with Kernel:
  • GUI
  • CLI
• For Android
  • User Apps
  • Android (Application Layer)
  • Linux Kernel
  • Hardware
• 3 Main OS: Mac, Windows, Linux
  • Each has many versions
  • Kernel stays the same but the Application Layer is being developed and improved
• Client OS vs Server OS
  • Linux and Windows have server distributions
• MacOS vs Linux
  • CLI and File Structure is similar
  • Unix
    • Codebase for many OSes
    • Developed independent of Linux
    • Many OS built on top of Unix
    • MacOS Kernel Darwin is based on Unix
    • To keep them compatible, standards were introduced
    • POSIX is the most popular standard
      • Portable OS Interface
  • Linux
    • Linux was developed parallel to UNIX based OS
    • Created by Linus Torvalds
    • No source code of UNIX
    • Clone of UNIX or also called "unix like"
    • Linux and macOS both POSIX compliant
    • Mostly used for servers
    • Ubuntu is the most popular Linux distribution

Introduction to virtualization & virtual machines

• No separate hardware needed
• You can achieve virtualization with a hypervisor
• Hypervisor
  • Allows hosting multiple virtual computers on a physical computer on top of your OS
  • VirtualBox from Oracle is one of the most used
  • Takes hardware resources from Host OS
  • Creates virtual CPU, virtual RAM, virtual storage for each virtual machine for your virtual computer
  • You can only give resources you actually have
  • Hardware resources are shared
  • Virtual machines are isolated
  • If something breaks inside the VM, it doesn't affect the host machine
• Benefits of a VM
  • Learning and experimentation
  • Test app on different OS

Hypervisor Types

• Type 1 hypervisor and Type 2 hypervisor

1. Type 1

   • For big companies
   • The hypervisor is installed directly on the hardware, not the host OS
   • Also called Bare Metal hypervisors
   • Controls hardware resources itself
   • e.g. vmware ESXi or Microsoft's Hyper-v
   • For big servers
     • Have one physical server with a Bare Metal hypervisor installed on it and multiple VMs running on the hypervisor
   • Big cloud platforms use Type-1 hypervisors
   • On platforms like AWS, when you create an EC2 instance for example, you're creating that VM on a physical server
   • Another instance created by someone else may get a VM on the same server as yours, but they are isolated
   • Benefits
     • Efficient usage of hardware resources

2. Type 2: Hosted Hypervisors

   • Creates VMs on top of existing OS
   • We have the host OS
   • Install hypervisor on OS
   • Use hypervisor to install guest OS that borrows resources from host OS
   • Personal computers

Why companies are adopting virtualization

• Abstraction of the OS from the hardware
  • Without virtualization
    • OS is tightly coupled to the hardware
  • With virtualization
    • OS as a portable file you can move around
    • Files are virtual machine images
    • Images will have OS, editors etc
    • Can have backups of your entire OS

Linux File System

• FS is used to handle the data management of the storage
• Hierarchical tree structure with 1 root folder
• Windows
  • Has multiple root folders
  • e.g. Local Disk, Removable Disks, etc
• Everything in Linux is a file
• Everything in the system is represented by a file descriptor
  • Text documents, pictures
  • Commands, like pwd, ls etc
  • Devices like printer, keyboard, usb
  • Even directories. Linux makes no difference between a file and a directory, since a directory is just a file containing names of other files
• Users on OS
  • Possible to have multiple user accounts
  • Each user has its own space
  • Each user can have own configurations

Some important directories

Introduction to the CLI

alfredasare@alfredasare-ubuntu:~$

• alfredasare : username
• alfredasare-ubuntu : computer name
• :~ : separator
• $ : regular user
• # : root user

Basic Linux Commands

# Rename
mv test.ts test-2.ts

# List
ls -R
ls -R Documents

# Copy dirs
cp -r doc doc2

# History
history
history 20

# Search history
CTRL + r

# Copy
SHIFT + CTRL + c

# Paste
SHIFT + CTRL + v

# OS Info
uname -a
cat /etc/os-release
lscpu
lsmem

# Switch User
su - alfred

Introduction to package managers

• A software package is a compressed archive, containing all required files for a software to run
• Apps usually have dependencies
• Package manager
  • For Linux files are spread across multiple folders
  • Managing apps is difficult
  • PM downloads, installs or updates existing software from a repo
  • Ensures the integrity and authenticity of the package
  • Knows where to put all the files in the Linux file system
  • Easy upgrading of software
• A package manager is already bundled in every Linux distribution
• In Ubuntu, we use apt
  • APT - Advanced Package Tool

# apt Search
sudo apt search openjdk

Copy - Add Shift when copying

# Install
apt install <package_name1> <package_name2>

# Remove
apt remove <package_name>

• apt vs apt-get
  • apt-get is also available on Ubuntu
  • apt is more user-friendly than apt-get
    • You get stuff like progress bars
  • search query not available with apt-get

apt-get install <package>

Repositories

• Linux uses repositories
  • Storage location with thousands of programs
  • Package Manager fetches the packages from these repositories
  • Always update the package index before upgrading or installing new packages

sudo apt update

• Pulls the latest changes from APT repos

• The APT package index is basically a database holding records of available packages from the repos

• /etc/apt

  • sources.list - Contains all the repos the APT package manager will use to find packages and fetch from them

• On Linux, you'll mostly use apt

• But sometimes there are packages not available in those repos

• Sometimes available but not the latest

• Alternatives

  • Ubuntu Software Center
  • Snap package manager
  • Add repository to official list of repos
    • When installing relatively new applications which are not in official repos yet
    • Repository will be added to /etc/apt/sources.list
    • The next "apt install" will also look into the newly added repo
    • Repos
      • PPA - Personal Package Archive
      • Not verified by Linux distribution
      • Beware of risks

Snap

• Self-contained - dependencies contained in the package
• Supports universal Linux packages (package type .snap)
• Automatic Updates
• Larger installation size

APT

• Dependencies are shared
• Only for specific Linux distributions (package type .deb)
• Manual Updates
• Smaller installation size

Other distros

• Ubuntu is just one distro
• Linux Distros are grouped based on same source code
• Distros in the same category use the same package manager

Package manager categories

• Debian based: use APT
  • Ubuntu
  • Debian
  • Mint
• Red Hat based: use YUM
  • RHEL
  • CentOS
  • Fedora

Working with VIM

• UNIX text editor
  • Vi
  • Vim - improved version of Vi
• Modes
  • command mode
  • insert mode

# Install
sudo apt install vim

# Save
:wq

# Quit without saving
:q!

# Delete line (in command mode)
dd

# Delete 10 lines
d10 and d again

# Undo
u

# Jump to end of line
$

# Jump to end of line and switch to insert mode
Shift + A

# Jump to beginning of line
0

# Show or hide line numbers
:set number
:set nonumber

# Jump to line
12G - Go to line 12

# Search
/pattern
Use n to jump to next occurrence
Use N - search in opposite reaction


# Replace
:%s/nginx/web-app

Linux account groups and user permissions

User categories

• Root User - unrestricted access and permissions

• User Account (regular users) - Users we create to log in to system

  • Each user will have its own dedicated space
    • /home/alfred

• Service Account - Relevant on Linux server distributions

  • Db server, web app server, etc
  • Each app will get its own user
  • Each app is isolated in terms of permissions
  • Don't run services with root

• Always one root user per computer

• Multiple user accounts and service users

• Why multiple standard users

  • So they can share a computer

• Windows can centrally manage Users

  • All computers are connected to the central admin system
  • Only access your home folder

• On Linux

  • Users accounts are managed on that specific hardware

• Multiple users on a server is important for Servers

  • Permissions per team member
  • Traceability
    • who did what on the system
  • Admin will create users with permissions

Groups and permissions

• We can do it on 2 levels
  1. User Level
     • Give permissions directly to user
  2. Group Level
     • Group users into Linux Groups
     • Give permissions to the Group
     • Can add a user to multiple groups

/etc/passwd - User info
# Stores user account info

USERNAME:PASSWORD:UID:GID:GECOS:HOMEDIRE:SHELL
# x means that encrypted password is stored in /etc/shadow file

# Root user has UID of 0

# The primary group ID is stored in /etc/group file

sudo adduser tom

• Whenever we create a user with adduser, it creates a group with the same name as the user and sets that as the primary group

sudo adduser tom

# Regular user
su - riddle

# Root
su -

• Create new group
  • By default, system assigns the next available GID from the range of group IDS specified in the login.defs file

sudo groupadd devops

# Verify group creation
cat /etc/group

Different user and group commands

• adduser and addgroup
  • Interactive
  • More User Friendly
  • Easier to use
• useradd and groupadd
  • Need to provide infos yourself
  • Low-level utilities
• User
  • adduser and addgroup when executing manually
  • User useradd and groupadd in scripts (automated)
• Same for delete
  • deluser/delgroup and userdel/groupdel

# Change user group
sudo usermod -g devops tom
sudo usermod -g <group> <user>

# Delete group
delgroup tom

• User can have multiple secondary groups

# Overwrites secondary groups list
sudo usermod -G admin, othergroup tom

# Append new group to existing groups
sudo usermod -aG groupname riddle

# Print groups
groups
groups riddle

# Add user with group
sudo useradd -G devops nicole

# Remove user from group
sudo gpasswd -d nicole devops

Users and Permissions: File Ownership Permissions & User Permissions

• User permissions are related to reading, writing and executing files.

ls -l

Ownership

• Who owns the file/directory?
• Each file has two owners
  • The user: User that created the file
  • The group: Primary group of user
• chown - Change ownership

sudo chown tom:admin test.txt

# Only user
sudo chown user test.txt

# Only group
sudo chgrp devops test.txt

File Permissions

• Example: drwxrwxr-x
  • File type
    • -- : regular file
    • d : directory
    • c : character device file
    • l : symbolic link
  • First character is the file type
  • First 3 chars after file type are permissions for the owner
    • rwx
      • r - read
      • w - write
      • x - execute: For running / executing scripts
      • -- - No permission
  • Next 3 characters are permissions for the group
  • Next 3 chars are permissions for Other
    • Other users that are not the user or part of the group

Modifying Permissions

1. Symbolic Mode

sudo chmod -x api

# u - owner
# g - group
# o - other
# a - all

sudo chmod g-w config.yaml
sudo chmod g+x config.yaml

2. Set permission

sudo chmod g=rwx config.yaml
sudo chmod g=r-- config.yaml

3. Numeric mode

sudo chmod 777 script.sh

Basic Linux Commands: Pipes, Redirects and CLI

• Every program has
  • Input
  • Output
• The output from one command can become the input of another command
• You can chain multiple commands
• pipe command |
  • Pipes the output of the previous command as an input to the next command
• NB
  • less: Displays the contents of a file or a command output, one page at a time

cat /var/log/syslog | less

• Pipes output to less command

Pipe and Less

• Space - next page
• b - Previous page

ls /usr/bin | less
history | less

Pipe and Grep

• grep
  • Globally Search for Reg Ex and Print out
  • Searches for a particular pattern of characters and displays all lines that contain that pattern

history | grep sudo
history | grep "sudo chmod"

history | grep sudo | less
ls /usr/bin/ | grep python

cat Documents/java-app/config.yaml | grep nginx

Redirects in Linux

• > character is the redirect operator
• Takes the output from the previous command and sends it to a file that you give

history | grep sudo > sudo-commands.txt

• >> append

history | grep rm >> sudo-rm-commands.txt

• Can be used for collecting logs

Standard Input and Standard Output

• Every program has 3 built-in streams:
  • STDIN (0) = Standard Input
  • STDOUT (1) = Standard Output
  • STDERR (2) = Standard Error

Chaining commands

clear; sleep 1; echo "Hello"

Shell Scripting: Introduction

• Shell scripts have a .sh extension
• Shell
  • The program that interprets and executes the various commands we type in the terminal
  • Translates our command that the OS Kernel can understand
• Different shell implementations
  • sh (Bourne shell) - /bin/sh
    • used to be the default shell
  • Bash
    • Bourne again shell
    • Newer versions of Linux distributions
    • Improved version of sh
• Shell and Bash are terms that are often used interchangeably
• We're talking about Bash scripting & Bash commands
• Bash is a shell program
• Bash is a programming language

Shebang

• All shell script files have the same .sh file extension
  • sh
  • bash
  • zsh
• How does the OS know which shell to use?
  • We need to tell the OS by writing a shebang line as the first line of the script
    • #!/bin/sh
    • #!/bin/bash
    • #!/bin/zsh
  • It's a declaration of the syntax we'll use
  • # - sharp
  • ! - bang
  • #! - shebang
• One advantage of using shell (#!/bin/sh) is that it can be executed on any Unix system because Bourne shell is always installed
• To use #!/bin/bash, bash has to be installed on server.

Write and Execute Script

#!/bin/bash

echo "Configure server"

Execute

./setup.sh
bash setup.sh

• By default, when you create a file, you don't have execute permissions on that file.

sudo chmod u+x setup.sh

Shell Scripting: Concepts and Syntax

Conditions

• We have file test operators

#!/bin/bash
echo "Setup and configure server"
file_name=config.yaml
if [ -d "config" ]
then
  echo "reading config directory contents"
  config_files=$(ls config)
else
  echo "config dir not found. Creating one"
  mkdir config
fi
echo "using file $file_name to configure something"
echo "here are all configuration files: $config_files"

File test operators

if [ -g/-w/-r/-x "config.yaml" ]

Number comparisons

• -eq
• -ne
• -gt
• -lt
• -ge
• -le

num_files=xx
if [ "$num_files" -eq 10 ]

String comparisons

• =

• !=

• -z String operand size is zero

• -n String operand size is non-zero

• str Checks if string is not empty. If empty, return false

• For POSIX, you can use =

• == is Bash-specific

user_group=xx
if [ "$user_group" == "alfredasare"  ]
then
  echo "configure the server"
elif [ "$user_group" == "admin" ]
then
  echo "administer server"
else
  echo "No permission to configure server. Wrong user group"
fi

Passing Arguments To A Script

• Args are processed in the same order in which they are sent
• The indexing of arguments starts at 1

user_group=$1
if [ "$user_group" == "alfredasare"  ]
then
  echo "configure the server"
elif [ "$user_group" == "admin" ]
then
  echo "administer server"
else
  echo "No permission to configure server. Wrong user group"
fi

• Parameters start from 1 to 9

#!/bin/bash
echo "Setup and configure server"
file_name=config.yaml
config_dir=$1
if [ -d "$config_dir" ]
then
  echo "reading config directory contents"
  config_files=$(ls "$config_dir")
else
  echo "config dir not found. Creating one"
  mkdir "$config_dir"
  touch "$config_dir/config.sh"
fi
user_group=$2
if [ "$user_group" == "alfredasare"  ]
then
  echo "configure the server"
elif [ "$user_group" == "admin" ]
then
  echo "administer server"
else
  echo "No permission to configure server. Wrong user group"
fi
echo "using file $file_name to configure something"
echo "here are all configuration files: $config_files"

Read user input

#!/bin/bash
echo "Reading user input"
read -p "Please enter your password: " user_pwd
echo "thanks for your password $user_pwd"

Unknown number of arguments

• $* represents all the arguments as a single string

#!/bin/bash
echo "all params $*"
echo "number of params $#"

Shell Loops

#!/bin/bash
echo "all params $*"
echo "number of params $#"
for param in $*
  do
   echo $param
  done

#!/bin/bash
echo "all params $*"
echo "number of params $#"
for param in $*
  do
   if [ -d "$param" ]
   then
    echo "executing scripts in the config folder"
    ls -l "$param"
   fi
   echo $param
  done

while Loop

• Execute repeatedly till some condition is met

#!/bin/bash
echo "all params $*"
echo "number of params $#"
for param in $*
  do
   if [ -d "$param" ]
   then
    echo "executing scripts in the config folder"
    ls -l "$param"
   fi
   echo $param
  done
sum=0
while true
do
 read -p "enter a score: " score
 if [ "$score" == "q" ]
 then
  break
 fi
 sum=$((sum+$score)) --- Arithmetic
 echo "total score: $sum"
 done

• [[ - Bash

  • Don't have to enclose variable names in quotes

• [ - POSIX

• Better alternatives for automation

  • Python
  • Ansible

Shell Scripting: Functions

#!/bin/bash
echo "all params $*"
echo "number of params $#"
for param in $*
  do
   if [ -d "$param" ]
   then
    echo "executing scripts in the config folder"
    ls -l "$param"
   fi
   echo $param
  done
function score_sum () {
  sum=0
  while true
  do
   read -p "enter a score: " score
   if [ "$score" == "q" ]
    then
     break
   fi
  sum=$((sum+$score))
  echo "total score: $sum"
  done
}
score_sum

Parameters and Args

• Don't use too many parameters
  • More than 5 params per function is too much.
• Apply the Single Responsibility Principle: a function should only do one thing.

function create_file () {
  file_name=$1
  is_shell_script=$2
  touch $file_name
  echo "file $file_name created"
  if [ "$is_shell_script" = true  ]
  then
   chmod u+x $file_name
   echo "Added execute permission"
  fi
}
create_file test.txt
create_file myfile.yaml
create_file myscript.sh true

Returning values from functions

• $? command captures value returned from the last command

function sum() {
 total=$(($1 + $2))
 return $total
}
#result=$(sum 2 10)
sum 2 10
result=$?
echo "sum of 2 and 10 is $result"

Environmental Variables

• Each user can configure its own environment/account settings preferences
• These OS configs should be isolated from other user environments
• OS stores configs in environmental variables
  • Key-value pairs
  • Variables store information
  • Are available for the whole environment
  • By convention names are defined in UPPER CASE
• egs
  • SHELL - Default shell program
  • HOME - User's home dir
  • USER - Currently logged in user
• Bash is the default shell program but the user can change it

printenv
printenv | less

printenv USER
printenv | grep USER
echo $USER

Application Environmental Variables

• Makes app secure
• Makes app more flexible

export DB_USERNAME=dbuser
export DB_PASSWORD=jsdlsadkl
export DB_NAME=mydb

unset DB_NAME

Persisting environmental variables

• When you set envs with export, they are available only in the current session
• There is a file in the home directory generated for the user
  • ~/.bashrc
  • Variables set in this file are loaded whenever a bash login is entered

export DB_NAME=mydb

• After making changes to file, run

source ~/.bashrc

Persisting environmental variables (system wide)

• ~/.bashrc is for a specific user
• System wide - /etc/environment
• PATH env variable
  • List of directories to executable files separated by :
  • Tells the shell which directories to search for the executable in response to our executed command

Add a custom command / program

#!/bin/bash  c
echo "Welcome to Devops Bootcamp $USER"

chmod a+x welcome.sh

PATH=$PATH:/home/alfredasare

• Append to PATH variable

source ~/.bashrc

Networking

• LAN - Collection of devices connected together in one physical location
• Each device has a unique IP address
• Devices communicate via IP addresses
  • eg 172.16.0.0
    • 32 bit value
    • Every 8 bits are grouped into an Octet
    • Each octet can be represented as a decimal
  • Range from 0.0.0.0 to 255.255.255.255
• Switch
  • Sits within the LAN
  • Facilitates the connection between all the devices within the LAN
• How to talk to devices outside the LAN?
  • We use a router
• Sits between the LAN and outside network
• Connects devices on LAN and WAN
• Allows networked devices to access the Internet
• The IP address of the router is called a Gateway
• Router and network gateway are the same thing

Subnet

• How to know whether the other device is inside or outside the LAN
• It's decided by the IP of the target device
• Devices in the LAN belong to the same IP address range
• Subnet
  • Logical subdivision of an IP network
  • Subnet sets the range
• Subnetting
  • Process of diving a network into two or more networks
• Example of IP address range:
  • 192.168.0.0 - IP address: Starting point
  • Subnet mask: 255.255.255.0
  • The subnet sets the range
  • Whenever an IP address starts with 192.168.0.x, it belongs to the same LAN
• Example 2
  • Subnet mask: 255.255.0.0
  • The IP range will start with 192.168.x.x
  • All IP addresses starting with 192.168.x.x belong to the same LAN
• 255.255.0.0 - means that 16 bits are fixed
• 255.255.255.0 - means that 24 bits are fixed
• Value 255 fixates the Octet
• Value 0 means free range
• Subnet defines how many bits in the IP are fixed making the rest flexible

CIDR Block

• Subnet mask dictates how many bits are fixed

• CIDR: Classless Inter-Domain Routing

  • Shorthand
  • It's a method of representing IP addresses and their associated network prefixes.
  • 192.168.0.0/16 - 16 fixed
  • 192.168.0.0/24 - 24 fixed

• Any device needs 3 pieces of data for communication

  • IP Address
  • Subnet
  • Gateway

NAT: Network Address Translation

• IP Address range chosen by an administrator
• Each device gets a unique IP from that range
• How do you make sure IPs don't overlap?
• IP addresses within the LAN are not visible to the outside network or internet
• When requests are made by devices in the LAN, the IP of the device is replaced by the IP of the router when making the request
• This functionality of the router is called Network Address Translation or NAT
• The router makes requests on the devices' behalf
  • Has public and private IP
• Benefits
  • Security and Protection of devices within LAN
  • Reuse IP address ranges
• Limited number of IPv4 addresses available
  • 4294967296 addresses

Firewall

• If outside device wants to talk to LAN
• Or if you have a server in your LAN and an external device wants to connect, it's not possible by default
• Firewall
  • Set of rules or a system that prevents unauthorized access from entering a private network.
  • Using Firewall rules you can define which requests are allowed.
• Firewall rules
  • Which IP address in your network is accessible
  • Which IP address can access your server
  • You can allow any device access your server
    • e.g. when running a web app
  • You define which ports are accessible on the device you want to allow access to
• Ports
  • Every device has a set of ports
  • You can allow specific ports and specific IP addresses
  • Different apps start or listen on different ports
  • Standard ports for many apps
    • 80 - Web servers
    • 3306 - MySQL
    • 5432 - Postgres
  • Every app needs a port
  • Each port is unique on a device
• Firewalls allow device IP addresses at ports to be accessed
• Monitors who can access which port. That config is called Port Forwarding.

DNS

• Domain Name Service
• Every computer is uniquely identified
• IP addresses are mapped to domain names
• The domain name is translated to the IP address. The DNS handles this translation
• Domain names follow a hierarchical structure
• There are 13 root domains across the globe
• There are also Top Level Domain
  • 6 original TLDs
    • .mil
    • .edu
    • .com
    • .org
    • .net
    • .gov
• There are also geographical domains
  • .us
  • .uk
• Others
  • .biz
  • .dev
• Can buy a domain name
• ICANN - Internet Corp for Assigned Names and Numbers
  • Manages names
  • Authorizes Domain Name Registrars, which register and assign domain names
  • Keeps track of available names
• Subdomains
  • Can assign subdomains for other apps within the same organization
• Fully Qualified Domain Names
  • eg. courses.alfred.com.
  • The dot at the end represents the root domain
  • Not necessary to specify the dot
• Every computer has a DNS client pre-installed
• DNS entries are cached for efficiency

Networking Commands

• ifconfig - Computer's IP, Subnet Mask, Gateway
• netstat - Active connections on machine
• ps aux - Active processes
• nslookup - Check domain's IP
• ping - Checks whether a service or app is accessible

SSH - Secure Shell

• Is a network protocol that gives users a secure way to access a computer over the internet
• 2 ways to authenticate
  • Username and password
    • Admin creates user on remote server
    • User can then connect with the username and password
  • SSH Key Pair
    • More secure
    • Client creates an SSH key pair
    • Key pair = Private Key + Public Key
    • Private Key = Secret Key - Stored securely on client machine
    • Public Key = Public. Can be shared e.g. with the remote server
    • Client can unlock the public key with his private key
• Each user creates key-pair locally
• Admin copies public keys to server
• User can SSH with the private key
• If public key of a user is not registered on the remote server, the user can't connect to it

SSH For Services

• Services like Jenkins often need to connect to another server via SSH
• E.g for Jenkins
  • Create Jenkins User on application server
  • Create SSH Key Pair on Jenkins Server
  • Add public SSH key to authorized_keys on application server

Firewall and Port 22

• Communication between machines must be explicitly allowed through a Firewall rule
• By default, all communication is blocked
• SSH Authentication comes AFTER the connection to the machine
• So connection to port 22 must be allowed before authentication can happen
• The firewall allows access to port 22 for specific IP addresses
  • Source = Who can access the server on the port
• Most OS have an SSH service running by default
• SSH is powerful and needs to be restricted to specific IP addresses

SSH in action

• On Digital Ocean
  • Create droplet with password (not ssh)

ssh username@hostname

With SSH Key Pair

ssh-keygen -t rsa

• RSA: Type of key pair
  • Type of algorithm used to encrypt key
• Private key remains on local machine
• Public key is copied to remote server

Add Public Key To authorized_keys

• Find .ssh on server

• We add the public key in the authorized_keys file

• Open with vim

• Copy contents of public key and paste in authorized_keys

• known_hosts: Lets the client authenticate the server to check that it isn't connecting to an impersonator

• authorized_keys: Lets the server authenticate the user or client

ssh username@hostname

• By default it uses id_rsa

ssh -i .ssh/id_rsa username@hostname

Copy bash scripts and execute

• Use scp
  • scp: Secure Copy

vim test.sh
scp test.sh username@hostname:/root

scp -i .ssh/id_rsa test.sh username@hostname:/root