Apache deep dive

Apache Hypertext Transfer Protocol Server (httpd)

Apache:
Open Source cross-platform web server software
Services 57% of all active websites
Name comes from respect for apache Native American tribe(A PATCH SERVER)

http://httpd.apache.org/

https://en.wikipedia.org/wiki/List_of_Apache_modules

https://httpd.apache.org/docs/2.4/mod/

Background

Most of the functionality of the Apache web server is provided by modules. A module can be either:

static, meaning that it is built into the Apache executable at compile time (and is therefore always available).
shared, meaning that it is loaded at run time by a LoadModule directive within the Apache configuration file.

It is thus possible for a shared module to be installed on a machine, but not loaded by Apache and therefore not usable. This is one of the more common reasons for Apache failing to start or failing to behave as expected.

In short, the modules extend the Apache server. An administrator can easily configure Apache by adding and removing the modules according to required needs. Apache comes with a set or pre-installed modules.

[root@ansible ~]# yum install httpd httpd-manual 

[root@ansible ~]# service httpd restart

http://192.168.183.128/manual/

[root@ansible ~]# rpm -qa httpd
httpd-2.4.6-45.el7.centos.4.x86_64

[root@ansible ~]# grep LoadModule /etc/httpd/conf/httpd.conf
# have to place corresponding `LoadModule’ lines at this location so the
# LoadModule foo_module modules/mod_foo.so

Commonly used Apache modules

The below list shows few commonly used Apache modules.
1) Mod_security
2) Mod_rewrite
3) Mod_deflate
4) Mod_cache
5) Mod_proxy
6) Mod_ssl

Find list of compiled modules in Apache
[root@ansible ~]# httpd -l
Compiled in modules:
core.c
mod_so.c
http_core.c

Find list of loaded modules in Apache
[root@ansible ~]# httpd -M
Loaded Modules:
core_module (static)
so_module (static)
http_module (static)
access_compat_module (shared)
actions_module (shared)
alias_module (shared)
allowmethods_module (shared)
auth_basic_module (shared)
auth_digest_module (shared)
authn_anon_module (shared)
authn_core_module (shared)
authn_dbd_module (shared)
authn_dbm_module (shared)
authn_file_module (shared)
authn_socache_module (shared)
authz_core_module (shared)
authz_dbd_module (shared)
authz_dbm_module (shared)
authz_groupfile_module (shared)
authz_host_module (shared)
authz_owner_module (shared)
authz_user_module (shared)
autoindex_module (shared)
cache_module (shared)
cache_disk_module (shared)
data_module (shared)
dbd_module (shared)
deflate_module (shared)
dir_module (shared)
dumpio_module (shared)
echo_module (shared)
env_module (shared)
expires_module (shared)
ext_filter_module (shared)
filter_module (shared)
headers_module (shared)
include_module (shared)
info_module (shared)
log_config_module (shared)
logio_module (shared)
mime_magic_module (shared)
mime_module (shared)
negotiation_module (shared)
remoteip_module (shared)
reqtimeout_module (shared)
rewrite_module (shared)
setenvif_module (shared)
slotmem_plain_module (shared)
slotmem_shm_module (shared)
socache_dbm_module (shared)
socache_memcache_module (shared)
socache_shmcb_module (shared)
status_module (shared)
substitute_module (shared)
suexec_module (shared)
unique_id_module (shared)
unixd_module (shared)
userdir_module (shared)
version_module (shared)
vhost_alias_module (shared)
dav_module (shared)
dav_fs_module (shared)
dav_lock_module (shared)
lua_module (shared)
mpm_prefork_module (shared)
proxy_module (shared)
lbmethod_bybusyness_module (shared)
lbmethod_byrequests_module (shared)
lbmethod_bytraffic_module (shared)
lbmethod_heartbeat_module (shared)
proxy_ajp_module (shared)
proxy_balancer_module (shared)
proxy_connect_module (shared)
proxy_express_module (shared)
proxy_fcgi_module (shared)
proxy_fdpass_module (shared)
proxy_ftp_module (shared)
proxy_http_module (shared)
proxy_scgi_module (shared)
proxy_wstunnel_module (shared)
systemd_module (shared)
cgi_module (shared)
php5_module (shared)

[root@ansible ~]# httpd -M |wc -l
84

[root@ansible ~]# ls -l /usr/lib64/httpd/modules/ |wc -l
102

How to add new module ex: mod_ssl

[root@ansible ~]# yum install mod_ssl -y

[root@ansible ~]# httpd -M |wc -l
85

[root@ansible ~]# httpd -M |grep ssl
ssl_module (shared)

[root@ansible ~]# ls -l /usr/lib64/httpd/modules/ |grep -i ssl
-rwxr-xr-x 1 root root 219464 Apr 13 02:34 mod_ssl.so

[root@ansible ~]# ls -l /usr/lib64/httpd/modules/ |wc -l
103

####################################
[root@ansible ~]# cat /etc/httpd/conf/httpd.conf |wc -l
353
[root@ansible ~]# grep “#” /etc/httpd/conf/httpd.conf |wc -l
259

[root@ansible ~]# grep -v “#” /etc/httpd/conf/httpd.conf
ServerRoot “/etc/httpd”
Listen 80
Include conf.modules.d/*.conf
User apache
Group apache
ServerAdmin root@localhost
AllowOverride none
Require all denied

DocumentRoot “/var/www/html”

<Directory “/var/www”>
AllowOverride None
Require all granted

<Directory “/var/www/html”>
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
DirectoryIndex index.html

<Files “.ht*”>
Require all denied

ErrorLog “logs/error_log”
LogLevel warn

LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”” combined
LogFormat “%h %l %u %t \”%r\” %>s %b” common

LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\” %I %O” combinedio

CustomLog “logs/access_log” combined
ScriptAlias /cgi-bin/ “/var/www/cgi-bin/”

<Directory “/var/www/cgi-bin”>
AllowOverride None
Options None
Require all granted
TypesConfig /etc/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml

AddDefaultCharset UTF-8

MIMEMagicFile conf/magic

EnableSendfile on
IncludeOptional conf.d/*.conf

apache1

 What is a module in Apache?
In computing, Apache, an open-source HTTP server, comprises a small core for HTTP request/response processing and for Multi-Processing Modules (MPM) which dispatches data processing to threads and/or processes. Many additional modules (or “mods” ) are available to extend the core functionality for special purposes.

What is a directive in Apache?
Apache directives are a set of rules which define how your server should run, number of clients that can access your server, etc. you can change them by editing the httpd.conf and related files to meet your requirements. shareimprove this answer. ex:Listen

Loading Modules
The default config file loads a large number of modules

LoadModule ldap_module modules/mod_ldap.so
LoadModule status_module modules/mod_status.so
LoadModule proxy_module modules/mod_proxy.so

Defining Multi-Process Settings
To improve response times, apache manages a pool of “spare” server processes

These numbers control the size of the pool:
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256

Containers
Container directives use XML-style opening / closing tags
Restrict the scope of the directives they contain

<Directory “/var/www/cgi-bin”>
AllowOverride None
Options None
within the specified
Order allow,deny
Allow from all

Other containers include <Location> and <VirtualHost>

 

Boot procedure

The stages involved in Linux Booting Process are:
BIOS
Boot Loader
– MBR
– GRUB
Kernel
Init
Runlevel scripts
BIOS
This is the first thing which loads once you power on your machine.
When you press the power button of the machine, CPU looks out into ROM for further instruction.
The ROM contains JUMP function in the form of instruction which tells the CPU to bring up the BIOS
BIOS determines all the list of bootable devices available in the system.
Prompts to select bootable device which can be Hard Disk, CD/DVD-ROM, Floppy Drive, USB Flash Memory Stick etc (optional)
Operating System tries to boot from Hard Disk where the MBR contains primary boot loader.
Boot Loader
To be very brief this phase includes loading of the boot loader (MBR and GRUB/LILO) into memory to bring up the kernel.
MBR (Master Boot Record)
It is the first sector of the Hard Disk with a size of 512 bytes.
The first 434 – 446 bytes are the primary boot loader, 64 bytes for partition table and 6 bytes for MBR validation timestamp.

NOTE: Now MBR directly cannot load the kernel as it is unaware of the file-system concept and requires a boot loader with file system driver for each supported file systems, so that they can be understood and accessed by the boot loader itself.
To overcome this situation GRUB is used with the details of the filesystem in /boot/grub.conf and file system drivers

GRUB (GRand Unified Boot loader)
This loads the kernel in 3 stages

GRUB stage 1:
The primary boot loader takes up less than 512 bytes of disk space in the MBR – too small a space to contain the instructions necessary to load a complex operating system.
Instead the primary boot loader performs the function of loading either the stage 1.5 or stage 2 boot loader.

GRUB Stage 1.5:
Stage 1 can load the stage 2 directly, but it is normally set up to load the stage 1.5.
This can happen when the /boot partition is situated beyond the 1024 cylinder head of the hard drive.
GRUB Stage 1.5 is located in the first 30 KB of Hard Disk immediately after MBR and before the first partition.
This space is utilized to store file system drivers and modules.
This enabled stage 1.5 to load stage 2 to load from any known location on the file system i.e. /boot/grub

GRUB Stage 2:
This is responsible for loading kernel from /boot/grub/grub.conf and any other modules needed
Loads a GUI interface i.e. splash image located at /grub/splash.xpm.gz with list of available kernels where you can manually select the kernel or else after the default timeout value the selected kernel will boot

The original file is /etc/grub.conf of which you can observe a symlink file at /boot/grub/grub.conf
Sample /boot/grub/grub.conf
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-194.26.1.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.26.1.el5 ro root=/dev/VolGroup00/root clocksource=acpi_pm divisor=10
initrd /initrd-2.6.18-194.26.1.el5.img
title Red Hat Enterprise Linux Server (2.6.18-194.11.4.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.11.4.el5 ro root=/dev/VolGroup00/root clocksource=acpi_pm divisor=10
initrd /initrd-2.6.18-194.11.4.el5.img
title Red Hat Enterprise Linux Server (2.6.18-194.11.3.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.11.3.el5 ro root=/dev/VolGroup00/root clocksource=acpi_pm divisor=10
initrd /initrd-2.6.18-194.11.3.el5.img
Kernel
This can be considered the heart of operating system responsible for handling all system processes.
Kernel is loaded in the following stages:
Kernel as soon as it is loaded configures hardware and memory allocated to the system.
Next it un-compresses the initrd image (compressed using zlib into zImage or bzImage formats) and mounts it and loads all the necessary drivers.
Loading and unloading of kernel modules is done with the help of programs like insmod, and rmmod present in the initrd image.
Looks out for hard disk types be it a LVM or RAID.
Unmounts initrd image and frees up all the memory occupied by the disk image.
Then kernel mounts the root partition as specified in grub.conf as read-only.
Next it runs the init process

Init Process
Executes the system to boot into the run level as specified in /etc/inittab
Sample output defining the default boot runlevel inside /etc/inittab
# Default runlevel. The runlevels used by RHS are:
# 0 – halt (Do NOT set initdefault to this)
# 1 – Single user mode
# 2 – Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 – Full multiuser mode
# 4 – unused
# 5 – X11
# 6 – reboot (Do NOT set initdefault to this)
#
id:5:initdefault:
As per above O/P system will boot into runlevel 5
You can check current runlevel details of your system using below command on the terminal
# who -r
run-level 3 Jan 28 23:29 last=S
Next as per the fstab entry file system’s integrity is checked and root partition is re-mounted as read-write (earlier it was mounted as read-only).
Runlevel scripts
A no. of runlevel scripts are defined inside /etc/rc.d/rcx.d
Runlevel Directory
0 /etc/rc.d/rc0.d
1 /etc/rc.d/rc1.d
2 /etc/rc.d/rc2.d
3 /etc/rc.d/rc3.d
4 /etc/rc.d/rc4.d
5 /etc/rc.d/rc5.d
6 /etc/rc.d/rc6.d
Based on the selected runlevel, the init process then executes startup scripts located in subdirectories of the /etc/rc.d directory.
Scripts used for runlevels 0 to 6 are located in subdirectories /etc/rc.d/rc0.d through /etc/rc.d/rc6.d, respectively.
Lastly, init runs whatever it finds in /etc/rc.d/rc.local (regardless of run level). rc.local is rather special in that it is executed every time that you change run levels.
NOTE: rc.local is not used in all the distros as for example Debian.
Next if everything goes fine you should be able to see the Login Screen on your system.

########################################################

Introducing process basics
A running instance of a program is called as process. A program stored in the hard disk or pen drive is not a process. When that stored program starts executing, then we say that process has been created and is running. Let’s very briefly understand the Linux operating system boot-up sequence:

1. In PCs, initially the BIOS chip initializes system hardware, such as PCI bus, display device drivers, and so on.
2.Then the BIOS executes the boot loader program.
3.The boot loader program then copies kernel in memory, and after basic checks, it calls a kernel function called start_kenel() .
4.The kernel then initiates the OS and creates the first process called init .
5.You can check the presence of this process with the following command:
$ ps –ef
6.Every process in the OS has one numerical identification associated with it. It is called a process ID. The process ID of the init process is 1. This process is the parent process of all user space processes.
7.In the OS, every new process is created by a system call called fork() .
8.Therefore, every process has a process ID as well as the parent process ID.
9.We can see the complete process tree using the following command:
    $ pstree

https://rafishaikblog.wordpress.com/introduction-to-linux-and-os-installations/

https://rafishaikblog.wordpress.com/2017/02/13/list-block-device-information/

 

Linux Kernel Fundamentals

What is a the Linux kernel?
The kernel is a program, Often with a name vmlinuz-<version>(vmlinuz-3.10.0-514.26.2.el7.x86_64), and that program needs to loaded into memory and run and that operation is a boot loader. With Linux we often have a boot-loader called grub, so grub reads the kernel file from disk into memory and transfer control to it.

The kernel program like other program has command-line parameters and grub is responsible for passing those parameters to the kernel. The Linux kernel is a API it provides an programming interface. The functions we call from user space into the kernel we call system calls, but linux kernel also provides virtual file system entries say proc,sys,debugfs., and through those virtual file systems we can interact directly with kernel. Like getting information from kernel and changing things in kernel . Also our file system as Device-File’s (system calls) .

The kernel is a Gate Keeper: that Enforces priviliges(capabilities), Executes superior instructions, Implements security policies(SElinux),Controls access to h/w and other resources.

Kernel is a Modular: Kernel image is relatively small(In MB),Kernel image is sufficient to boot to user space, Optional functionality is added after booting,It allows for alternatives; for example, =>loading only drivers required for present h/w.

[root@cms ~]# ls -l /boot/
total 140568
-rw-r–r–. 1 root root 126426 Nov 20 2015 config-3.10.0-327.el7.x86_64
-rw-r–r–. 1 root root 137701 Jul 4 20:45 config-3.10.0-514.26.2.el7.x86_64
drwxr-xr-x. 2 root root 26 Jan 26 01:44 grub
drwx——. 6 root root 4096 Jul 15 06:07 grub2
-rw-r–r–. 1 root root 43565957 Jan 26 01:48 initramfs-0-rescue-63f5c6cb01084bad905d0c80fe992973.img
-rw——-. 1 root root 20081417 Jul 15 06:08 initramfs-3.10.0-327.el7.x86_64.img
-rw-r–r–. 1 root root 17663887 Jan 26 01:50 initramfs-3.10.0-327.el7.x86_64kdump.img
-rw——-. 1 root root 20815122 Jul 15 06:08 initramfs-3.10.0-514.26.2.el7.x86_64.img
-rw——- 1 root root 18595423 Jul 15 06:15 initramfs-3.10.0-514.26.2.el7.x86_64kdump.img
-rw-r–r–. 1 root root 613842 Jul 15 06:07 initrd-plymouth.img
-rw-r–r–. 1 root root 252612 Nov 20 2015 symvers-3.10.0-327.el7.x86_64.gz
-rw-r–r–. 1 root root 277955 Jul 4 20:47 symvers-3.10.0-514.26.2.el7.x86_64.gz
-rw——-. 1 root root 2963044 Nov 20 2015 System.map-3.10.0-327.el7.x86_64
-rw——-. 1 root root 3114352 Jul 4 20:45 System.map-3.10.0-514.26.2.el7.x86_64
-rwxr-xr-x. 1 root root 5156528 Jan 26 01:48 vmlinuz-0-rescue-63f5c6cb01084bad905d0c80fe992973
-rwxr-xr-x. 1 root root 5156528 Nov 20 2015 vmlinuz-3.10.0-327.el7.x86_64
-rwxr-xr-x. 1 root root 5397008 Jul 4 20:45 vmlinuz-3.10.0-514.26.2.el7.x86_64

1.What kernel version is your Linux system running?
[root@cms ~]# uname -a
Linux cms.example.com 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

2.What is the size of the kernel file that corresponds to the kernel your system is running?
[root@cms ~]# ls -lh /boot/vmlinuz-3.10.0-327.el7.x86_64
-rwxr-xr-x. 1 root root 5.0M Nov 20 2015 /boot/vmlinuz-3.10.0-327.el7.x86_64
[root@cms ~]# ls -lh /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64
-rwxr-xr-x. 1 root root 5.2M Jul 4 20:45 /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64
[root@cms ~]# ls -lh /boot/vmlinuz-0-rescue-63f5c6cb01084bad905d0c80fe992973
-rwxr-xr-x. 1 root root 5.0M Jan 26 01:48 /boot/vmlinuz-0-rescue-63f5c6cb01084bad905d0c80fe992973

Commands for H/W info?
lshw & lspci
lsusb & lsblk
lscpu & lsdev

Command for H/W control and config
hdparm
write (eg: echo)tp proc,dev,or sys files.
inb & outb
setpci

System call Mechanics
Standard library uses architecture-dependent means to invoke the system call mechanism.
suitably sized parameters are usually put in registers.
The Kernel is invoked, determined which system call, and calls it.

man 2 read
dmesg shows RAM buffer message from kernel.

dmesg |wc -l
1471

dmesg |head
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 4.4.0-83-generic (buildd@lgw01-10) (gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3) ) #106~14.04.1-Ubuntu SMP Mon Jun 26 18:10:19 UTC 2017 (Ubuntu 4.4.0-83.106~14.04.1-generic 4.4.70)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.4.0-83-generic root=UUID=50baebd2-73f6-4d7c-b951-e3a3853c9cbd ro quiet splash vt.handoff=7
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Centaur CentaurHauls
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256

/proc
The proc filesystem is mounted on /proc at boot. proc gers its name from “process”. proc conatins lots of process info and lots more. kernel tunable variable are an important part of proc.

/sys
The sys filesystem is mounted on /sys at boot. sysyfs is for “kernel project” info. In particular it is h/w info.

3.How much RAM is available to your running kernel? Note: It may or may not be the amount of physical RAM on your system.
mohammedrafi@NOC-RAFI:~$ free -m
total used free shared buffers cached
Mem: 3885 2105 1779 300 24 566
-/+ buffers/cache: 1514 2371
Swap: 4028 620 3408
mohammedrafi@NOC-RAFI:~$ head /proc/meminfo
MemTotal: 3978556 kB
MemFree: 1820000 kB
MemAvailable: 1937948 kB
Buffers: 25468 kB
Cached: 583228 kB
SwapCached: 33116 kB
Active: 1482844 kB
Inactive: 485312 kB
Active(anon): 1298280 kB
Inactive(anon): 368968 kB

4.The command strace will display the system calls that a process makes as it runs. Using the man command, determine what option for strace will show a summary, with a count, of the number of times a process called each system call. Using that option, what system call is called the most by the command date ?
mohammedrafi@NOC-RAFI:~$ strace -c date
Sat Jul 22 23:44:33 IST 2017
% time seconds usecs/call calls errors syscall
—— ———– ———– ——— ——— —————-
0.00 0.000000 0 3 read
0.00 0.000000 0 1 write
0.00 0.000000 0 4 open
0.00 0.000000 0 6 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 10 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 3 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 3 3 access
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 arch_prctl
—— ———– ———– ——— ——— —————-
100.00 0.000000 46 3 total

5.Can you determine, using strace , what system call is used to change the directory?
mohammedrafi@NOC-RAFI:~$ strace ls /home/

6.Run a sleep 100 with & (to put it in the background). What files does its process have open?
mohammedrafi@NOC-RAFI:~$ sleep 100&
[1] 2602
mohammedrafi@NOC-RAFI:~$ jobs
[1]+ Running sleep 100 &
mohammedrafi@NOC-RAFI:~$ cd /proc/2602/
mohammedrafi@NOC-RAFI:/proc/2602$ ls
attr clear_refs cpuset fd limits mem net oom_score projid_map sessionid stat task
autogroup cmdline cwd fdinfo loginuid mountinfo ns oom_score_adj root setgroups statm timers
auxv comm environ gid_map map_files mounts numa_maps pagemap sched smaps status uid_map
cgroup coredump_filter exe io maps mountstats oom_adj personality schedstat stack syscall wchan
mohammedrafi@NOC-RAFI:/proc/2602$ ls -l fd/
total 0
lrwx—— 1 mohammedrafi mohammedrafi 64 Jul 22 23:55 0 -> /dev/pts/2
lrwx—— 1 mohammedrafi mohammedrafi 64 Jul 22 23:55 1 -> /dev/pts/2
lrwx—— 1 mohammedrafi mohammedrafi 64 Jul 22 23:55 2 -> /dev/pts/2
mohammedrafi@NOC-RAFI:/proc/2602$ tty
/dev/pts/2

7.Does your system have a PCI Ethernet device?
mohammedrafi@NOC-RAFI:~$ lspci |grep -i ethernet
25:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)

8.Is the kernel variable ip_forward (under /proc/sys/ …) set to 1 or 0 on your system?
mohammedrafi@NOC-RAFI:~$ sudo find /proc/ -name ip_forward
/proc/sys/net/ipv4/ip_forward
mohammedrafi@NOC-RAFI:~$
mohammedrafi@NOC-RAFI:~$ cat /proc/sys/net/ipv4/ip_forward
1
mohammedrafi@NOC-RAFI:~$ sudo sysctl -a |grep ip_forward
net.ipv4.ip_forward = 1
net.ipv4.ip_forward_use_pmtu = 0
sysctl: reading key “net.ipv6.conf.all.stable_secret”
sysctl: reading key “net.ipv6.conf.default.stable_secret”
sysctl: reading key “net.ipv6.conf.docker0.stable_secret”
sysctl: reading key “net.ipv6.conf.eth0.stable_secret”
sysctl: reading key “net.ipv6.conf.lo.stable_secret”
sysctl: reading key “net.ipv6.conf.virbr0.stable_secret”
sysctl: reading key “net.ipv6.conf.vmnet1.stable_secret”
sysctl: reading key “net.ipv6.conf.vmnet8.stable_secret”
sysctl: reading key “net.ipv6.conf.wlan0.stable_secret”

mohammedrafi@NOC-RAFI:~$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

9.According to /sys/block , do you have a block device (disk) sda ? If so, do you have device files for partitions of sda ? How many? Using strace , does the command fdisk -l (run it as root), open any files under /sys/dev/block ?
mohammedrafi@NOC-RAFI:~$ cd /sys/block/
mohammedrafi@NOC-RAFI:/sys/block$ ls
loop0 loop1 loop2 loop3 loop4 loop5 loop6 loop7 sda sr0

10.Using dmesg and grep , do you see the kernel reporting the kernel command line? If not, can you determine if the boot messages from the kernel were lost? Does your system have a log file that recorded the boot messages? You can grep for BOOT_IMAGE under /var/log to look.
dmesg

11.What other device files are character devices and share the same major number with /dev/null ?
mohammedrafi@NOC-RAFI:~$ ls -l /dev/ |grep ^c |grep “1,”
crw-rw-rw- 1 root root 1, 7 Jul 21 17:27 full
crw-r–r– 1 root root 1, 11 Jul 21 17:27 kmsg
crw-r—– 1 root kmem 1, 1 Jul 21 17:27 mem
crw-rw-rw- 1 root root 1, 3 Jul 21 17:27 null
crw-r—– 1 root kmem 1, 4 Jul 21 17:27 port
crw-rw-rw- 1 root root 1, 8 Jul 21 17:27 random
crw——- 1 root root 251, 0 Jul 21 17:27 rtc0
crw-rw—- 1 root disk 21, 0 Jul 21 17:27 sg0
crw-rw—-+ 1 root cdrom 21, 1 Jul 21 17:27 sg1
crw-rw-rw- 1 root root 1, 9 Jul 21 17:27 urandom
crw-rw—-+ 1 root video 81, 0 Jul 21 17:27 video0
crw-rw-rw- 1 root root 1, 5 Jul 21 17:27 zero

Run grub2-mkconfig to generate a new config file.

Grub passes paramertes
The kernel process command-line arguments

Initial Root FileSystem
Linux systems frequently start up by a mounting a filesystem from RAM. The filesystem that contains “/” is called root file system. This initial RAM disk or RAM filesystem(initrd) is used to provide drivers and support for mounting the system’s real root file system. The initrd has an init that takes tuns first.

The First process(from disk)
when the init from the initrd terminates, the Linux kernel init again; this time from the real file system, which is commonly on disk. Historically that program was called “init” . Now, init may be a link to systemd. This process is responsible for starting for starting up services such as daemon like a web server.

mohammedrafi@NOC-RAFI:~$ cd /lib/modules
mohammedrafi@NOC-RAFI:/lib/modules$ ls
4.2.0-27-generic 4.4.0-59-generic 4.4.0-63-generic 4.4.0-70-generic 4.4.0-75-generic 4.4.0-83-generic
4.2.0-42-generic 4.4.0-62-generic 4.4.0-64-generic 4.4.0-72-generic 4.4.0-78-generic

mohammedrafi@NOC-RAFI:/lib/modules$ cd 4.4.0-83-generic/
mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic$ ls
build kernel modules.alias modules.builtin modules.dep modules.devname modules.softdep modules.symbols.bin vdso
initrd misc modules.alias.bin modules.builtin.bin modules.dep.bin modules.order modules.symbols updates
mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic$ cd kernel/
mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel$ ls
arch crypto drivers fs kernel lib mm net sound ubuntu virt

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel$ cd ..

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic$ head modules.dep
kernel/arch/x86/kernel/cpu/mcheck/mce-inject.ko:
kernel/arch/x86/kernel/msr.ko:
kernel/arch/x86/kernel/cpuid.ko:
kernel/arch/x86/crypto/glue_helper.ko:
kernel/arch/x86/crypto/aes-x86_64.ko:
kernel/arch/x86/crypto/des3_ede-x86_64.ko: kernel/crypto/des_generic.ko
kernel/arch/x86/crypto/camellia-x86_64.ko: kernel/crypto/xts.ko kernel/crypto/lrw.ko kernel/crypto/gf128mul.ko kernel/arch/x86/crypto/glue_helper.ko
kernel/arch/x86/crypto/blowfish-x86_64.ko: kernel/crypto/blowfish_common.ko
kernel/arch/x86/crypto/twofish-x86_64.ko: kernel/crypto/twofish_common.ko
kernel/arch/x86/crypto/twofish-x86_64-3way.ko: kernel/arch/x86/crypto/twofish-x86_64.ko kernel/crypto/twofish_common.ko kernel/crypto/xts.ko kernel/crypto/lrw.ko kernel/crypto/gf128mul.ko kernel/arch/x86/crypto/glue_helper.ko

mohammedrafi@NOC-RAFI:~$ lsmod |head
Module Size Used by
hid_generic 16384 0
usbhid 49152 0
hid 118784 2 hid_generic,usbhid
ipt_REJECT 16384 2
nf_reject_ipv4 16384 1 ipt_REJECT
xt_CHECKSUM 16384 1
iptable_mangle 16384 1
xt_tcpudp 16384 6
xt_addrtype 16384 2

mohammedrafi@NOC-RAFI:~$ cd /lib/modules/4.4.0-83-generic/kernel/

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel$ find . -name camellia*
./arch/x86/crypto/camellia-aesni-avx2.ko
./arch/x86/crypto/camellia-x86_64.ko
./arch/x86/crypto/camellia-aesni-avx-x86_64.ko
./crypto/camellia_generic.ko

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel$ cd arch/x86/crypto/

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel/arch/x86/crypto$ insmod camellia-x86_64.ko
insmod: ERROR: could not insert module camellia-x86_64.ko: Operation not permitted

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel/arch/x86/crypto$ sudo !!
sudo insmod camellia-x86_64.ko
insmod: ERROR: could not insert module camellia-x86_64.ko: Unknown symbol in module

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel/arch/x86/crypto$ dmesg |tail
[69485.394393] input: PixArt USB Optical Mouse as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.1/2-1.1:1.0/0003:0461:4E2A.0004/input/input52
[69485.422727] userif-3: sent link down event.
[69485.422738] userif-3: sent link up event.
[69485.450243] hid-generic 0003:0461:4E2A.0004: input,hidraw0: USB HID v1.11 Mouse [PixArt USB Optical Mouse] on usb-0000:00:1d.0-1.1/input0
[69486.505699] userif-3: sent link down event.
[69486.505761] userif-3: sent link up event.
[73281.057882] wlan0: AP 80:37:73:a8:d5:78 changed bandwidth, new config is 2437 MHz, width 2 (2447/0 MHz)
[80996.450650] wlan0: AP 80:37:73:a8:d5:78 changed bandwidth, new config is 2437 MHz, width 1 (2437/0 MHz)
[83418.473374] wlan0: AP 80:37:73:a8:d5:78 changed bandwidth, new config is 2437 MHz, width 2 (2447/0 MHz)
[85129.918108] camellia_x86_64: Unknown symbol xts_crypt (err 0)

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel/arch/x86/crypto$ sudo modprobe camellia-x86_64.ko

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel/arch/x86/crypto$ lsmod |head

mohammedrafi@NOC-RAFI:/lib/modules/4.4.0-83-generic/kernel/arch/x86/crypto$ modinfo camellia-x86_64.ko
filename: /lib/modules/4.4.0-83-generic/kernel/arch/x86/crypto/camellia-x86_64.ko
alias: crypto-camellia-asm
alias: camellia-asm
alias: crypto-camellia
alias: camellia
description: Camellia Cipher Algorithm, asm optimized
license: GPL
srcversion: 60B293291F8592BCEB105DD
depends: xts,lrw,glue_helper
intree: Y
vermagic: 4.4.0-83-generic SMP mod_unload modversions
parm: force:Force module load, ignore CPU blacklist (int)

Boot Procedure
https://rafishaikblog.wordpress.com/introduction-to-linux-and-os-installations/

 

Continuous Delivery Using Docker And Ansible

Continuous Delivery
Release Often
Release Faster
Great Reliable

Continuous Delivery workflow/pipeline

With which we can Test,Build,Release,& Deploy a simple application.

Application will be of any technology ex:python based.

The work flow will be Based upon using docker & docker-compose which is a emerging technology. With a goal to release our application as docker image.

So that it can be tested, building application artefacts, creating docker-release images, and running acceptance test, by external functionality, and finally publish docker image.

First we can run it locally, & then set in popular jenkins continuous delivery system, we will configure integration with GitHub, allowing continuous delivery work flow to be triggered on each application source code commit.

Assuming application passes all unit,integration & acceptance tests. Our pipeline will release images to docker hub, which will deployed to Aws with ansible using IAAC(S) approach with aws cloud formation and leveraging ec2 container service for running docker container is production.

 

AWS Certified DevOps Engineer – Professional Level Objectives(samples)

1) ___ is an alternative to a rolling deployment and it ensure that changes which require replacing instances are applied correctly and efficiently.
Immutable environments

Explanation
Immutable is an alternative to a rolling deployment and it ensures that changes which require replacing instances are applied correctly and efficiently. For example, you create a separate environment when using an immutable deployment which adds instances to a different auto-scaling group behind the same load balancer. The new instances and the old instances will serve traffic alongside each other. Once we figure out that the new instances are healthy, then the original auto-scaling group can be terminated or shut down. So, with this, you are using two auto-scaling groups but the same load balancer.

2) What are examples of requirements to consider when selecting a deployment method?
Downtime, Cost , Re-usability

Explanation
Deployments are not just about deploying application and code, it’s also deploying changes to the architecture that you are using. You need to think about requirements: Can there be downtime if there is a failed deployment? Is it alright if our deployments take longer and require more resources which means they cost more, but maybe they reduce the chances of having downtime? Can we re-use instances or do we want to destroy those and start with brand new ones?

3) Which of the following is the main problem with “All at Once” deployment?
If deployment fails, you will have downtime

Explanation
One main problem with this approach is that if the deployment fails, the application will be unavailable and you will have downtime. For you to roll back, you must revert and re-deploy to all of the instances.

4) With this deployment method, we create two identical stacks of the application, running in separate environments.
Blue/Green

Explanation
In Blue/Green deployment, you might have two elastic load balancers, two auto-scaling groups, and each of those would run the independent instances that have separate versions of our application. Other than that, they are identical. We do this to test the application in the same environment but isolated from production, until we have thoroughly checked the new application version and we are ready to switch over to production.

5) True or False: DevOps is a process or methodology around deploying and managing environments.
True

Explanation
DevOps isn’t a piece of software. Instead, it’s a process and a combination of concepts around that process. That combination increases our ability to deliver applications or services faster than before.

6) True or False: In-place deployment method is used for configuration management such as Chef, Puppet, Salt, or Ansible.
True

Explanation
In-place involves performing application updates on an instance, while disposable involves rolling out new instances and terminating older ones. For example, when thinking about in-place deployments, think about configuration management such as Chef, Puppet, Salt, or Ansible. Now, in an in-place upgrade environment where we want to maintain configuration over the lifetime of the instance, we might use something like OpsWorks. Instead, disposable upgrades involve rolling out new instances and terminating the old ones. Making changes and creating a new AMI from those changes to deploy new instances, for example. Essentially rolling out those changes using an immutable or blue/green deployment, and then destroying the old deployments.

7) How many questions are on the AWS DevOps Exam?
80

Explanation
The DevOps Engineer exam is three hours long with 80 scenario-based questions which are much more advanced than the associate-level exams.

8) Which method would you use when making changes and creating a new AMI from those changes to deploy new instances. In other words, essentially rolling out those changes using an immutable or blue/green deployment and then destroying the old deployments.
Disposable

Explanation
Disposable upgrades involve rolling out new instances and terminating the old ones.

9) Select all deployment methods that never require a DNS change:
“All at Once”, Rolling, Immutable environments

Explanation
All the above do not require a DNS change except for Blue/Green. The key word is never. In Blue/Green we need to change DNS settings to point to the new environment, especially if you are using Elastic Beanstalk which currently uses DNS changes. But this is not necessary all the time. If we’re not using Elastic Beanstalk, we can reuse the ELB and only switch over the load balancer to your new auto scaling deployment once it is tested. The reason we usually want to avoid DNS changes is because they can be messy. There are multiple levels of DNS clients, and they don’t always obey time to live rules, which can cause weird issues and send users to the wrong application.

10) Which deployment method rolls out changes to a specific number or percentage of instances at a time?
Rolling

Explanation
Rolling deployment rolls out changes to a specific number or percentage of instances at a time. For example, 20% of instances will get the new code while 80% will still run the previous application version. This approach does require a little bit more time to deploy changes, since we are not doing it all at the same time. However, this can provide zero downtime because if we notice an issue with the newly deployed code, we still have functioning instances with functioning code and we can simply roll back the 20% of instances affected by the changes. We can have the load balancer stop serving traffic to those instances as well so that no one gets sent to them and gets affected by downtime.

How to make hostname permanent on google cloud engine (GCE)

syntax:

gcloud compute instances add-metadata <instance-name> –zone <instance-zone> –metadata startup-script=’#! /bin/bash hostname <hostname>’

Open gcloud shell :

gcloud compute instances add-metadata cms –zone us-central1-c –metadata startup-script=’#! /bin/bash hostname your.hostname.com’

Welcome to Cloud Shell! Type “help” to get started.
mshaikdevops@rising-artifact-164906:~$ gcloud compute instances add-metadata cms –zone us-central1-c –metadata startup-script=’#! /bin/bash hostname cms.puppethub.in’
Updated [https://www.googleapis.com/compute/v1/projects/rising-artifact-164906/zones/us-central1-c/instances/cms].

$ sudo crontab -e
@reboot hostname $(curl –silent “http://metadata.google.internal/computeMetadata/v1/instance/attributes/hostname&#8221; -H “Metadata-Flavor: Google”)

vim /etc/rc.local
hostname *your.hostname.com*
chmod +x /etc/rc.d/rc.local

Ansible Tower adding-user & hosts(Inventory)

Creating New User,& modifying admin users

Go to settings on top right which is with gear symbol

Click on ORGANIZATIONS

tower1.png

Select Users tab
tower2.png

Select +ADD tab on right side

tower3.png

Click on admin user
add extra details and then save
toawer4.png

Check permissions on same window
tower5.png

SYSTEM ADMINISTRATORS HAVE ACCESS TO ALL PERMISSIONS

Create normal user
Create a new user
click on +add & add fields and save
tower6.png

Logout and login as test user for validation

tower7tower8

Adding Credentials to test user

tower9.png

Click on +add

Fill name & choose type as Machine

tower10.png

Adding Credentials to test user

tower10tower11

logging out as test user and logging in as admin

Click on projects

tower13.png
Click on start scm update

clikon demo project(sucess!)
tower16.png

U can edit the project at any point of time
Go to home page

Select template tab
tower17.png

click on `Demo Job Template`
start the job click cliking rocket icon

tower18.png

Click on jobs tab
tower19.png

Select Inventories
tower20.png

Demo Inventory

Adding host to inventory

Come to home page
Click on Inventories
Add New Inventory
ex:myinventory and save it
tower20.png

Add host
ex:104.154.116.136

tower21tower22tower23

add group
tower25.png

move or copy node to new group(centos)
click on edit option for node and check centos box and then move/copy ans save
tower26.png

Check running with command on centos group by checking centos group and select RUN COMMANDS option

EX:ping module

 

click on launch

tower29.png

 

tower30.png

Need to know before starting scripting

Scripts
•Contain a series of commands
•An interpreter executes commands in the script
•Anything you can type at the command line, you can put in a script
•Great for automating tasks

Basic Syntax
•#! /bin/bash
# Commands
•Shebang / HashBang • #! /bin/bash • Informs Linux which command line interpreter to use for the script; in this example, the Bourne Again Shell

Shell
Global Shell Configuration Files
•/etc/profile
•/etc/profile.d
•/etc/bashrc
•/etc/bash.bashrc
•/etc/skel • Contents of this directory are copied to new users directories when a new user is created User Shell Configuration Files
•~/.bash_login • Executes whatever commands are within the file ( ~/.bash_login ) when a user logs in

•~/.profile • User-specific bash configuration file
•~/.bash_profile • User-specific bash configuration file
•~/.bashrc • User-specific bash configuration file; executes whatever commands are within the file (~/.bash_login) when a user logs in.
•~/.bash_logout • Executes whatever commands are within the file ( ~/.bash_logout ) when a user logs out

Shell Variables
•set • Shows shell variables for the current instance of the running shell
•Set your own shell variables • EXAMPLE=VAR ; echo $EXAMPLE
» » Creates the shell variable EXAMPLE and sets the value to VAR, then prints the variable’s value
• Remove shell variables • unset EXAMPLE ; echo $EXAMPLE
» » Removes the shell variable EXAMPLE; echo will show no display since $EXAMPLE is no longer set to any value

Environment Variables
•env • Shows all environment variables
•env | grep EXAMPLE • Prints current environment variables and then greps the result for the term EXAMPLE
•export EXAMPLE=VAR • Exports shell variable EXAMPLE to the environment variables
•EXAMPLE=VAR ; export EXAMPLE • Export a previously-defined shell variable to the environment variables
•After you log off, the environment variables you set will restore to default; to permanently set an environment variable, you must either edit the user configuration files or global configuration files for bash
•Add to .bashrc (for user):
» » ABC=”123″; export ABC
•Add to /etc/.bash.bashrc (for system):
» » ABC=”123″; export ABC

Common Environment Variables
•DISPLAY • X display name
•EDITOR • Name of default text editor
•HISTCONTROL • History command control options
•HOME • Path to home directory
•HOSTNAME • Current hostname
•LD_LIBRARY_PATH • Directories to look for when searching for shared libraries
•MAIL • Holds the location of the user mail spools
•PATH • Executable search path
•PS1 • Current shell prompt
•PWD • Path to current working directory
•SHELL • Path to login shell
•TERM • Login terminal type
•USER / USERNAME • Current user’s username
•VISUAL • Name of visual editor

Changing the Shell Prompt
•Basic syntax • PS1=’\[“\] ‘
•Prompt variables:
» » \h • hostname
» » \w • current working directory
» » \u • username
» » \@ • 12 hour am/pm date
» » \t • 24 hour hh:mm:ss
» » \T • 12 hour hh:mm:ss
» » \j • Number of jobs running on the shell
» » \d • Date (day of week, month, day of month)
» » \H • Full hostname (hostname.domain.com)
» » \n • New line

•Example • PS1='[`pwd`]$ ‘
» » Makes the shell prompt the path to current directory followed by the $ sign

File Tests
•-a • True if exists; may cause conflicts
•-e • True if exists
•-f • True if exists and is a regular file
•-d • True if exists and is a directory
•-c • True if exists and is a character special file
•-b • True if exists and is a block special file
•-p • True if exists and is a named pipe (FIFO)
•-S • True if is a socket file
•-L • True if exists and is a symbolic link
•-h • True if exists and is a symbolic link
•-g • True if exists and has sgid bit set
•-u • True if exists and has suid bit set
•-r • True if exists and is readable
•-w • True if exists and is writable
•-x • True if exists and is executable
•-s • True if exists and has size bigger than 0
•-t • True if file descriptor is open and refers to a terminal
• -nt • True, if is newer than
• -ot • True if is older than
• -ef • True if and refer to the same device and inode numbers

String Tests
•-z • True if is empty
•-n • True if is not empty; this is the default operation
• = • True if the strings are equal
• != • True if the strings are not equal
• < • True if sorts before lexicographically; remember to escape •
• > • True if sorts after lexicographically; remember to escape •

Arithmetic Tests
• -eq • True if the integers are equal
• -ne • True if the integers are NOT equal
• -le • True if the first integer is less than or equal second one
• -ge • True if the first integer is greater than or equal second one
• -lt • True if the first integer is less than second one
• -gt • True if the first integer is greater than second one

Misc Syntax
• -a • True if and are true; -a may also be used as a file test
• -o • True if either or is true
•! • True if is false
•( ) • Group a test (for precedence); in normal shell-usage, parentheses must be escaped; use “\(” and “\)”
•-o • True if the shell option is set
•-v • True if the variable has been set; use var[n] for array elements
•-R • True if the variable has been set and is a nameref variable (since 4.3-alpha)

##################################################
If Statements
Basic Syntax
•if [ condition ];
then
else
fi
#commands to be run if true
#commands to be run if false

##################################################
Else If Syntax
•When using else if within an if statement, you want to use elif
•if [ condition ];
then
fi
#commands to be run if true
elif [ condition ];
then
#commands to be run if true
else
#commands to be run if false

If Statement with Multiple Conditions
•if [ condition ] OPERATOR [ condition ];
if [ condition ] || [ condition ];
if [ $g == 1 && $c == 123 ] || [ $g == 2 && $c == 456 ];

•if [[ ( Condition ) OPERATOR ( Condition ) ]];
if [[ ( Condition ) || ( Condition ) ]];
if [[ ( $g == 1 && $c == 123 ) || ( $g == 2 && $c == 456 ) ]];

##################################################
Case Statements
•Case statements are used to check the value of a parameter and execute code depending on the value
•This is similar to the switch statement in other languages with some slight differences:
» » Instead of the word switch , use the word case
» » Where you would use case , instead list the pattern followed by a closing parenthesis
» » To break the command chain, use ;;
Basic Syntax
•case “$VAR” in
pattern_1 )
# Commands to be executed
;;
pattern_2 )
# Commands to be executed
;;
* )
# Default
;;
esac
##################################################
While Loop
Basic Syntax
•while [ condition ] do
#command(s)
#increment
• Example:
» » x=1
while [ $x -le 5 ]
do
echo “Welcome $x times”
x=$(( $x + 1 ))
done
» » The above loop will run a command while x is less than or equal to 5
» » The last line adds 1 to x on each iteration
##################################################
For Loop
Basic Syntax
•for arg in [list]
do
done
#command(s)
•Any variable name can be used in place of arg
•Brace-expanded {1..5} items can be used in place of [list]
•During each pass through the loop, arg takes on the value of each successive variable in the list
•Example:
» » for COLOR in red green blue do
done
echo “COLOR: $COLOR”
» » # Output:
# Color: red
# Color: green
# Color: blue
##################################################

Few sample scripts

#!/bin/more
==============================
SCRIPT NAME: arrayex.sh
==============================
#!/bin/bash
# simple array list and loop for display
SERVERLIST=(“websrv01” “websrv02” “websrv03” “websrv04”)
COUNT=0
for INDEX in ${SERVERLIST[@]}; do
echo “Processing Server: ${SERVERLIST[COUNT]}”
COUNT=”`expr $COUNT + 1`”
done
==============================
SCRIPT NAME: casesample.sh
==============================
#!/bin/bash
# demo of the case statement
clear
echo
echo
echo
echo
echo
echo
echo
read
“MAIN MENU”
“=========”
“1) Choice One”
“2) Choice Two”
“3) Choice Three”
“”
“Enter Choice: ”
MENUCHOICE
case $MENUCHOICE in
1)
echo “Congratulations for Choosing the First Option”;;
2)
echo “Choice 2 Chosen”;;
3)
echo “Last Choice Made”;;
*)
echo “You chose unwisely”;;
esac
==============================
SCRIPT NAME: checkargs2.sh
==============================
#!/bin/bash
: ${3?”USAGE: $1 ARGUMENT $2 ARGUMENT $3 ARGUMENT”}
echo “I got all three!”
==============================
SCRIPT NAME: checkargs.sh
==============================
#!/bin/bashif [ “$#” != “3” ]; then
echo “USAGE: checkargs.sh [parm1] [parm2] [parm3]”
exit 300
fi
echo “I live! I got what I needed!”
==============================
SCRIPT NAME: cmdlinevar.sh
==============================
#!/bin/bash
# demo of command line values passed in with our shell script
USERNAME=$1
PASSWORD=$2
echo “The following Username is $USERNAME and Password is $PASSWORD”
==============================
SCRIPT NAME: comments.sh
==============================
#!/bin/bash
# This line is intended to be used as a general description of the script
# and anything that it does
clear
# clears the screen
# MYUSERNAME=”Terry”
# the username for this application
MYUSERNAME=”Don” # new username added later
echo “We are using the default user called: $MYUSERNAME” # display to the
console
DATETIMESTAMP=`date`
echo “This is when the script was run: $DATETIMESTAMP” # this is the
timestamp of run
==============================
SCRIPT NAME: env.sh
==============================
#!/bin/bash
clear
echo “This script will give us environment information”
echo “================================================”
echo “”
echo “Hello Username: $USER”
echo “”
echo “Your Home Directory is: $HOME”
echo “”
echo “Your History File Will Ignore: $HISTCONTROL”
echo “”
echo “Your Terminal Session Type is: $TERM”
echo “”
==============================
SCRIPT NAME: errorexit.sh==============================
#!/bin/bash
# demo of using error handling with exit
echo “Change to a directory and list the contents”
DIRECTORY=$1
cd $DIRECTORY 2>/dev/null
if [ “$?” = “0” ]; then
echo “We can change into the directory $DIRECTORY, and here are the
contents”
echo “`ls -al`”
else
echo “Cannot change directories, exiting with an error and no listing”
exit 111
fi
==============================
SCRIPT NAME: errors.sh
==============================
#!/bin/bash
# this is to show exit status types
set -e
expr 1 + 5
echo $?
rm doodles.sh
echo $?
expr 10 + 10
echo $?
==============================
SCRIPT NAME: execops.sh
==============================
#!/bin/bash
# execution operators example
echo “Enter a number between 1 and 5: ”
read VALUE
if [ “$VALUE” -eq “1” ] || [ “$VALUE” -eq “3” ] || [ “$VALUE” -eq “5” ];
then
echo “You entered the ODD value of $VALUE”
else
echo “You entered a value of $VALUE”
fi
==============================
SCRIPT NAME: expressions.sh
==============================
#!/bin/bash
# expression evaluation
expr 2 + 2expr 2 + 2 \* 4
expr \( 2 + 2 \) \* 4
==============================
SCRIPT NAME: ex.sh
==============================
#!/bin/bash
FINDUSER=`find /home -user user`
alias finduser=”find /home -user user”
echo “Variable: $FINDUSER”
VARFIND=`finduser`
echo “ALIAS: $VARFIND”
==============================
SCRIPT NAME: filedesc.sh
==============================
#!/bin/bash
# demo of reading and writing to a file using a file descriptor
echo “Enter a file name to read: ”
read FILE
exec 5<>$FILE
while read -r SUPERHERO; do
echo “Superhero Name: $SUPERHERO”
done <&5
echo “File Was Read On: `date`” >&5
exec 5>&-
==============================
SCRIPT NAME: forsample.sh
==============================
#!/bin/bash
# this is a demo of the for loop
echo “List all the shell scripts contents of the directory”
SHELLSCRIPTS=`ls *.sh`
for SCRIPT in “$SHELLSCRIPTS”; do
DISPLAY=”`cat $SCRIPT`”
echo “File: $SCRIPT – Contents $DISPLAY”
done
==============================
SCRIPT NAME: funcparms.sh
==============================
#!/bin/bash
# this demo is for functional parameter passing
# global variableUSERNAME=$1
# function definitions – start
# calculate age in days
funcAgeInDays () {
echo “Hello $USERNAME, You are $1 Years Old.”
echo “That makes you approximately `expr $1 \* 365` days old…”
}
# function definitions – stop
# scrip – start
clear
echo “Enter Your Age: ”
read USERAGE
# calculate the number of days
funcAgeInDays $USERAGE
==============================
SCRIPT NAME: funcstruct.sh
==============================
#!/bin/bash
# demo of functions within a shell script structure
# script or global variables
CMDLINE=$1
# function definitions – start
# displays a message
funcExample () {
echo “This is an example”
}
# display another message
funcExample2 () {
echo “This is another example”
}
# function definitions – stop
# beginning of the script
echo “This is the start…”
funcExample2
funcExample
funcExample
==============================
SCRIPT NAME: ifexpr.sh
==============================
#!/bin/bash
# test multiple expressions in single if statementFILENAME=$1
echo “Testing for file $FILENAME and readability”
if [ -f $FILENAME ] && [ -r $FILENAME ]
then
echo “File $FILENAME exists AND is readable”
fi
==============================
SCRIPT NAME: ifsdelim.sh
==============================
#!/bin/bash
# delimiter example using IFS
echo “Enter filename to parse: ”
read FILE
echo “Enter the Delimiter: ”
read DELIM
IFS=”$DELIM”
while read -r CPU MEMORY DISK; do
echo “CPU: $CPU”
echo “Memory: $MEMORY”
echo “Disk: $DISK”
done <“$FILE”
==============================
SCRIPT NAME: ifthenelse.sh
==============================
#!/bin/bash
# simple example of if then else and nested if statements
clear
echo “Enter a number between 1 and 3:”
read VALUE
if [ “$VALUE” -eq “1” ] 2>/dev/null; then
echo “You entered #1”
elif [ “$VALUE” -eq “2” ] 2>/dev/null; then
echo “You successfully entered #2”
elif [ “$VALUE” -eq “3” ] 2>/dev/null; then
echo “You entered the 3rd number”
else
echo “You didn’t follow the directions!”
fi
==============================
SCRIPT NAME: makedoc.sh
==============================
#!/bin/bash
DOCFILE=”script_listing”echo “#!/bin/more” > “$DOCFILE”
ls *.sh > tmplisting.txt
while IFS= read -r FILENAME; do
if [ -f “$FILENAME” ]; then
echo “==============================” >> “$DOCFILE”
echo “SCRIPT NAME: $FILENAME ” >> “$DOCFILE”
echo “==============================” >> “$DOCFILE”
echo “”
echo “`cat $FILENAME`” >> “$DOCFILE”
fi
done < tmplisting.txt
chmod 755 “$DOCFILE”
rm tmplisting.txt
==============================
SCRIPT NAME: nested.sh
==============================
#!/bin/bash
# demo of nested functions and some abstraction
# global variable
GENDER=$1
# function definitions – start
# create a human being
funcHuman () {
ARMS=2
LEGS=2
echo “A Human has $ARMS arms and $LEGS legs – but what gender are we?”
echo “”
funcMale () {
BEARD=1
echo “This man has $ARMS arms and $LEGS legs, with $BEARD
beard(s)…”
echo “”
}
funcFemale () {
BEARD=0
echo “This woman has $ARMS arms and $LEGS legs, with $BEARD
beard(s)…”
echo “”
}
}# function definitions – stop
# script – start
clear
echo “Determining the characteristics of the gender $GENDER”
echo “”
# determine the actual gender and display the characteristics
if [ “$GENDER” == “male” ]; then
funcHuman
funcMale
else
funcHuman
funcFemale
fi
==============================
SCRIPT NAME: null.sh
==============================
#!/bin/bash
# redirect to /dev/null example
echo “This is displaying on the console”
echo “This is going into the black hole” >> /dev/null
==============================
SCRIPT NAME: override2.sh
==============================
#!/bin/bash
# override/trap the system exit and execute a custom function
# global variables
TMPFILE=”tmpfile.txt”
TMPFILE2=”tmpfile2.txt”
trap ‘funcMyExit’ EXIT
# function declarations – start
# run this exit instead of the default exit when called
funcMyExit () {
echo “Exit Intercepted…”
echo “Cleaning up the temp files…”
rm -rf “tmpfil*.txt”
exit 255
}
# function declarations – stop
# script – start
echo “Write something to tmp file for later use…” > $TMPFILE
echo “Write something to tmp file 2 for later user…” > $TMPFILE2
echo “Trying to copy the indicated file before processing…”cp -rf $1 newfile.txt 2>/dev/null
if [ “$?” -eq “0” ]; then
echo “Everything worked out ok…”
else
echo “I guess it did not work out ok…”
exit 1
fi
# script – stop
==============================
SCRIPT NAME: overriding.sh
==============================
#!/bin/bash
# override/trap the system exit and execute a custom function
# global variables
TMPFILE=”tmpfile.txt”
TMPFILE2=”tmpfile2.txt”
trap ‘funcMyExit’ EXIT
# function declarations – start
# run this exit instead of the default exit when called
funcMyExit () {
echo “Exit Intercepted…”
echo “Cleaning up the temp files…”
rm -rf tmpfil*.txt
exit 255
}
# function declarations – stop
# script – start
echo “Write something to tmp file for later use…” > $TMPFILE
echo “Write something to tmp file 2 for later user…” > $TMPFILE2
echo “Trying to copy the indicated file before processing…”
cp -rf $1 newfile.txt 2>/dev/null
if [ “$?” -eq “0” ]; then
echo “Everything worked out ok…”
else
echo “I guess it did not work out ok…”
exit 1
fi
# script – stop
==============================
SCRIPT NAME: readfile.sh
==============================
#!/bin/bash# simple file reading (non-binary) and displaying one line at a time
echo “Enter a filename to read: ”
read FILE
while read -r SUPERHERO; do
echo “Superhero Name: $SUPERHERO”
done < “$FILE”
==============================
SCRIPT NAME: readsample.sh
==============================
#!/bin/bash
# interactive script for user input
echo
read
echo
read “Enter Your First Name: ”
FIRSTNAME
“Enter Your Last Name: ”
LASTNAME
echo
echo
echo
echo
read “”
“Your Full Name is: $FIRSTNAME $LASTNAME”
“”
“Enter Your Age: ”
USERAGE
echo “In 10 Years, You will be `expr $USERAGE + 10` years old.”
==============================
SCRIPT NAME: returnval.sh
==============================
#!/bin/bash
# demo of return values and testing results
# global variable
YES=0
NO=1
FIRST=$1
SECOND=$2
THIRD=$3
# function definitions – start
# check the command line parameters passed in
funcCheckParms () {
# did we get three
if [ ! -z “$THIRD” ]; then
echo “We got three parms…”
return $YES
else
echo “We did not get three parms…”
return $NO
fi
}
# function definitions – stop# script – start
funcCheckParms
RETURN_VALS=$?
# did we get three or not?
if [ “$RETURN_VALS” -eq “$YES” ]; then
echo “We received three parms and they are: ”
echo “Parm 1: $FIRST”
echo “Parm 2: $SECOND”
echo “Parm 3: $THIRD”
echo “”
else
echo “Usage: returnval.sh [parm1] [parm2] [parm3]”
exit 1
fi
==============================
SCRIPT NAME: simpledialog.sh
==============================
#!/bin/bash
# demo of a dialog box that will display a menu
# global variables / default values
MENUBOX=${MENUBOX=dialog}
# function declarations – start
# function to display a simple menu
funcDisplayDialogMenu () {
$MENUBOX –title “[ M A I N
M E N U ]” –menu “Use UP/DOWN Arrows to
Move and Select or the Number of Your Choice and Enter” 15 45 4 1
“Display Hello World” 2 “Display Goodbye World” 3 “Display Nothing” X
“Exit” 2>choice.txt
}
# function declarations – stop
# script – start
funcDisplayDialogMenu
case
1)
2)
3)
X)
esac
“`cat choice.txt`” in
echo “Hello World”;;
echo “Goodbye World”;;
echo “Nothing”;;
echo “Exit”;;
# script – stop
==============================
SCRIPT NAME: simplefunc.sh
==============================
#!/bin/bash
# this is a simple function exampleecho “Starting the function definition…”
funcExample () {
echo “We are now INSIDE the function…”
}
funcExample
==============================
SCRIPT NAME: simpleif.sh
==============================
#!/bin/bash
# simple if script for guessing a number
echo
echo
echo
echo
read
“Guess the Secret Number”
“=======================”
“”
“Enter a Number Between 1 and 5: ”
GUESS
if [ $GUESS -eq 3 ]
then
echo “You Guessed the Correct Number!”
fi
==============================
SCRIPT NAME: simpleinfobox.sh
==============================
#!/bin/bash
# demo of a simple info box with dialog and ncurses
# global variables / default values
INFOBOX=${INFOBOX=dialog}
TITLE=”Default”
MESSAGE=”Something to say”
XCOORD=10
YCOORD=20
# function declarations – start
# display the infobox and our mesage
funcDisplayInfoBox () {
$INFOBOX –title “$1” –infobox “$2” “$3” “$4”
sleep “$5”
}
# function declarations – stop
# script – start
if [ “$1” == “shutdown” ]; then
funcDisplayInfoBox “WARNING!” “We are SHUTTING DOWN the System…” “11”
“21” “5”
echo “Shutting Down!”else
funcDisplayInfoBox “Information…” “You are not doing anything fun…”
“11” “21” “5”
echo “Not doing anything…”
fi
# script – stop
==============================
SCRIPT NAME: simpleinputbox.sh
==============================
#!/bin/bash
# simple demo of an input dialog box
# global variables / default values
INPUTBOX=${INPUTBOX=dialog}
TITLE=”Default”
MESSAGE=”Something to display”
XCOORD=10
YCOORD=20
# function declarations – start
# display the input box
funcDisplayInputBox () {
$INPUTBOX –title “$1” –inputbox “$2” “$3” “$4” 2>tmpfile.txt
}
# function declarations – stop
# script – start
funcDisplayInputBox “Display File Name” “Which file in the current
directory do you want to display?” “10” “20”
if [ “`cat tmpfile.txt`” != “” ]; then
cat “`cat tmpfile.txt`”
else
echo “Nothing to do”
fi
# script – stop
==============================
SCRIPT NAME: simplemsgbox.sh
==============================
#!/bin/bash
# demo of a message box with an OK button
# global variables / default variables
MSGBOX=${MSGBOX=dialog}
TITLE=”Default”
MESSAGE=”Some Message”
XCOORD=10
YCOORD=20# function declarations – start
# display the message box with our message
funcDisplayMsgBox () {
$MSGBOX –title “$1” –msgbox “$2” “$3” “$4”
}
# function declarations – stop
# script – start
if [ “$1” == “shutdown” ]; then
funcDisplayMsgBox “WARNING!” “Please press OK when you are ready to
shut down the system” “10” “20”
echo “SHUTTING DOWN NOW!!!”
else
funcDisplayMsgBox “Boring…” “You are not asking for anything fun…”
“10” “20”
echo “Not doing anything, back to regular scripting…”
fi
# script – stop
==============================
SCRIPT NAME: substitution.sh
==============================
#!/bin/bash
# This script is intended to show how to do simple substitution
shopt -s expand_aliases
alias TODAY=”date”
alias UFILES=”find /home -user user”
TODAYSDATE=`date`
USERFILES=`find /home -user user`
echo “Today’s Date: $TODAYSDATE”
echo “All Files Owned by USER: $USERFILES”
A=`TODAY`
B=`UFILES`
echo “With Alias, TODAY is: $A”
echo “With Alias, UFILES is: $B”
==============================
SCRIPT NAME: testfile.sh
==============================
#!/bin/bash
# tests for existence of indicated file name
FILENAME=$1
echo “Testing for the Existence of a File called $FILENAME”
if [ ! -f $FILENAME ]
thenecho “File $FILENAME Does NOT Exist!”
fi
==============================
SCRIPT NAME: test.sh
==============================
#!/bin/bash
clear
echo “Hello World”
==============================
SCRIPT NAME: trapex.sh
==============================
#!/bin/bash
# example of trapping events and limiting the shell stopping
clear
trap ‘echo ” – Please Press Q to Exit..”‘ SIGINT SIGTERM SIGTSTP
while [ “$CHOICE” != “Q” ] && [ “$CHOICE” != “q” ]; do
echo “MAIN MENU”
echo “=========”
echo “1) Choice One”
echo “2) Choice Two”
echo “3) Choice Three”
echo “Q) Quit/Exit”
echo “”
read CHOICE
clear
done
==============================
SCRIPT NAME: varexample.sh
==============================
#!/bin/bash
MYUSERNAME=”username”
MYPASSWORD=”password123″
STARTOFSCRIPT=`date`
echo “My login name for this application is: $MYUSERNAME”
echo “My login password for this application is: $MYPASSWORD”
echo “I started this script at: $STARTOFSCRIPT”
ENDOFSCRIPT=`date`
echo “I ended this script at: $ENDOFSCRIPT”
==============================
SCRIPT NAME: varscope.sh
==============================
#!/bin/bash
# demonstrating variable scope# global variable declaration
GLOBALVAR=”Globally Visible”
# function definitions – start
# sample function for function variable scope
funcExample () {
# local variable to the function
LOCALVAR=”Locally Visible”
echo “From within the function, the variable is $LOCALVAR…”
}
# functions definitions – stop
# script – start
clear
echo
echo
echo
echo
echo
echo
echo
“This step happens first…”
“”
“GLOBAL variable = $GLOBALVAR (before the function call)”
“LOCALVAR variable = $LOCALVAR (before the function call)”
“”
“Calling Function – funcExample()”
“”
funcExample
echo “”
echo “Function has been called…”
echo “”
echo “GLOBAL variable = $GLOBALVAR (after the function call)”
echo “LOCALVAR variable = $LOCALVAR (after the function call)”
==============================
SCRIPT NAME: whilesample.sh
==============================
#!/bin/bash
# while loop example
echo “Enter the number of times to display the ‘Hello World’ message”
read DISPLAYNUMBER
COUNT=1
while [ $COUNT -le $DISPLAYNUMBER ]
do
echo “Hello World – $COUNT”
COUNT=”`expr $COUNT + 1`”
done