Ubuntu Server Runup Script

Ubuntu Server Post Installation Runup

Following on from my CentOS 7 runup script is a series of Ubuntu scripts I have written over time. They're presented as a "Work in Progress" which may or may not end. 

I started the Ubuntu scripts using the CentOS scripts as a base, however they diverged quickly to become both unique and specialised. With Ubuntu, the goal was to standardise server builds so they could be duplicated by others. So the scripts began to evolve into a partially menu based build system.

But as I said - it's a work in progress, and the comments reflect areas that need improvement. So, for better or worse, here's the best of about two dozen iterations.

Base Ubuntu 20.04 LTS Runup Script

The script follows after the commentary given here. The script was designed to be edited before use. It is divided loosely into several sections. While some parts have been improved to be interactive, the majority requires serious modification before you should even think of running it on a server.

1. Variables

The first section includes many variables. Most should be self-explanatory. Check these carefully and make any modifications you need. Only those variables used in sections you are using need to be used, the rest can be commented out. In particular, make sure the following is properly declared:

MAILFWD IP address of the email relay

EMAILINST Your email address
EMAILADMIN The server administrators email address
SNMPRO The SNMPv2 public community
SNMPRW The SNMPv2 private community
SNMPSRV IP address of the SNMP server

2. Updates & Cockpit

Next updates are applied and cockpit is installed. This will take a few minutes to run. Choose between cockpit and webmin - do not install both. Cockpit has a smaller footprint and is easier to use, but webmin is more versatile.

3. Active Directory Integration

If you are integrating with Active Directory, study this section carefully as it is a little buggy and doesn't trap a lot of exceptions. I'd welcome feedback here!

4. PCP Logging tools & Grafana

This section is very experimental. Use with care. Grafana is a powerful tool that is not utilised enough. However it does take significant time to configure.

5. Firewall & Fail2ban

UFW (Uncomplicated Firewall) is then setup to allow access to ssh and cockpit. Rate limiting is applied to ssh. Fail2ban is also installed.

6. Swap

Default swappiness for Ubuntu server is 60. This is way too high. This section changes it to 20. This is a simple change, but an important one.

7. Tools

Several system utilities are installed here. Comment or uncomment according to what you find useful. The utilities are:

• wget - get files via http

• telnet - telnet client
• bind9utils - utlities for querying dns (such as dig)
• nmap - network analysis tool
• mlocate - Faster and more efficient file locator
• mc - Midnight Commander (XTree like file system interface)
• elinks - Text based broswer (has dependencies)
• systat - Statistical tools such as iostat

8. vm-tools & hyper-V

VMWare tools are installed next. Comment this section if not using vmware. An adjacent section for Hyper-V is provided to enable LIS and enhanced session mode.

9. Mail Relay

This section sets up s-nail for sending email via an email relay defined previously. S-nail is the preferred mailer for Ubuntu and provides the same service that mailx does.

10. SNMP (incomplete)

Install snmpd. This section is not finished. The snmpd.conf file must be manually edited.

11. Unattended updates (incomplete)

Ubuntu server installs updates automatically by default. This section must be configured to provide notification and control over the update process. The config files need to be manually edited afterwards as this section still needs some work.

12. Webmin

Webmin is still the best server application administration tool. There are hundreds of plugins for webmin. Cockpit is the best tool for system administration. If you don’t need webmin, comment out these lines. If you do, you may want to consider commenting out the cockpit installation lines to have just one tool.

Next we have the server applications

13. VSFTP

If you need ftp, this is the version to use. The downside is there is no webmin plugin for it. If you really need to administer ftp via webmin, install proftpd instead. If you don’t need an ftp server, comment out this section.

14. MariaDB (MySQL Server)

This will install mariadb server and client and harden the service. This section is interactive.

15. PERL

PERL is required for many web based applications. If not required, comment this section. This section is highly interactive and takes several minutes to install. As well as installing PERL, it installs cpan, cpan minus and numerous perl modules and keeps them updated as much as possible. Error messages in this section are not unusual. Ignore them.

16. Apache

Installs the Apache 2.4 web server. Configuration of httpd.conf is required. Comment out if installing NGINX.

17. PHP for Apache

Installs basic php modules and PEAR. Other modules can be installed by uncommenting as required.

18. NGINX

Installs NGINX instead of Apache.

19. WordPress

Requires Apache. Some heavy editing foo required here or you will be unhappy with the result...

20. phpMyAdmin

If you don't know what this is for, don't install it. 

21. Security evaluation

This section installs security evaluation and reporting tools and prepares a security report for the system.

22. Finish

Final section cleans up and reboots.

The Future

Plans for updates to the post-installation script include:

1. Make it interactive, so you don’t have to edit the file before running it.

2. Add logging to file.

3. Add auto-document capability. This will be the basis for as-built documentation.

4. Add a silent unattended mode.

The Script

#!/bin/bash
#
# Post installation script for Ubuntu Server 20.04 LTS
#
# Written by Wayne Doust 02 September 2021
#

## Needed for ifconfig to work
apt -y -qq install net-tools

SCTL="/etc/sysctl.conf"
STAMP=`date +%Y%m%d`
SWPP=20
HOST=`hostname -s`
FQDN=`hostname -d`
FQDN=site.local
MAILFWD=<IP of email forwarder>
EMAILINST=installer@$FQDN
EMAILADMIN=admin@$FQDN
SNMPRO=public
SNMPRV=private
SNMPSRV=<IP of SNMP server>
ALLOWFRM="127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fc00::/7, fe80::/10"
ADDOM=ADdomainname
ADFQDN=ADFQDN
ADUSER=Administrator


IP4="`ifconfig | sed -En 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p'`"
#IP6="`ifconfig | sed -En 's/::1//;s/.*inet6 (addr:)?(([[:xdigit:]]*::){,4}[[:xdigit:]]*::{,4}[[:xdigit:]]*::{,4}[[:xdigit:]]*::{,4}[[:xdigit:]]*).*/\2/p'`"

###
### The Following section asks for user input to modify some of the above variables
###
clear
echo
echo Ubuntu Server 20.04 LTS Runup Script
echo =========================================================================
echo
echo Enter parameters of this server. Press enter to accept the default.
echo
printf "Hostname [%s]:" $HOST
read ANSWER
if [ "$ANSWER" != "" ]
then
    HOST=$ANSWER
fi
echo Hostname: $HOST

printf "Domain [%s]:" $FQDN
read ANSWER
if [ "$ANSWER" != "" ]
then
    FQDN=$ANSWER
fi
echo Domain: $FQDN

echo Continuing will commence post-installation work of Ubuntu Server
echo ----------------------------------------------------------------
echo
echo Press any key to continue
read ANSWER

### Set Timezone and hostname
## Should use variables here
timedatectl set-timezone Australia/Melbourne
hostnamectl set-hostname $HOST
echo $HOST.$FQDN > /etc/hostname
sleep 3

### Apply updates and install cockpit & optionally pcp
## Note: Whilst you can install both, pick either Cockpit or webmin
##
echo
echo Apply updates
echo
sleep 3
apt -y -qq update && apt -y -qq upgrade
#echo
#echo Installing cockpit
#echo
#sleep 3
#apt -y -qq install cockpit
#apt -y install cockpit-pcp
#systemctl enable cockpit
#systemctl start cockpit

### (Optional) Add server to Actice Directory Domain
echo "deb http://au.archive.ubuntu.com/ubuntu/ bionic universe" >> /etc/apt/sources.list
echo "deb http://au.archive.ubuntu.com/ubuntu/ bionic-updates universe" >> /etc/apt/sources.list
hostnamectl set-hostname $HOST.$ADFQDN
hostnamectl
echo Check Name servers are correct
cat /etc/resolv.conf | grep nameserver
echo
sleep 10
systemctl disable systemd-resolved
systemctl stop systemd-resolved
apt -y update
apt -y install realmd libnss-sss libpam-sss sssd sssd-tools adcli samba-common-bin oddjob oddjob-mkhomedir packagekit
echo
echo Discover AD Domain
echo
realm discover $ADDOM
realm join -U $ADUSER $ADDOM
realm list $ADDOM
pam-auth-update --enable mkhomedir
## Do the following if the previous line doesn't work
#cp /usr/share/pam-configs/mkhomedir /usr/share/pam-configs/mkhomedir.org
#echo "Name: activate mkhomedir" > /usr/share/pam-configs/mkhomedir
#echo "Default: yes" >> /usr/share/pam-configs/mkhomedir
#echo "Priority: 900" >> /usr/share/pam-configs/mkhomedir
#echo "Session-Type: Additional" >> /usr/share/pam-configs/mkhomedir
#echo "Session:" >> /usr/share/pam-configs/mkhomedir
#echo "        required                        pam_mkhomedir.so" >> /usr/share/pam-configs/mkhomedir
#echo "umask=0022 skel=/etc/skel" >> /usr/share/pam-configs/mkhomedir
pam-auth-update
systemctl restart sssd
realm permit $ADUSER@$ADFQDN
realm permit 'Domain Admins' 'sysadmins'
echo "$ADUSER@$ADFQDN    ALL=(ALL)    ALL"          > /etc/sudoers.d/domain_admins
echo "%Domain\ Admins@ADFQDN    ALL=(ALL)    ALL"     >> /etc/sudoers.d/domain_admins
echo "%sysadmins@ADFQDN    ALL=(ALL)    ALL"         >> /etc/sudoers.d/domain_admins

### Optional logging tools based around pcp
## Don't install these unless you know what you're doing
# apt -y install pcp
# systemctl enable pmcd
# systemctl start pmcd
# systemctl enable pmlogger
# systemctl start pmlogger
# systemctl enable pmie
# systemctl start pmie
## use 'pcp atop' 'pmstat' 'pmiostat' etc
## Following is for web API for Grafana
# systemctl enable pmproxy
# systemctl start pmproxy
# wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
# add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
# apt update
# apt -y install grafana
# systemctl enable grafana-server
# systemctl start grafana-server
# ufw allow 3000/tcp
## Securing Grafana using NGINX Reverse Proxy (more here)
## See https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-grafana-on-ubuntu-20-04

    
### Setup ufw
echo
echo Setting up UFW
echo
sleep 3
ufw default deny incoming
## Allow SSH
ufw allow ssh
ufw limit ssh
## Allow cockpit
#ufw allow 9090/tcp
echo y | ufw enable
ufw status

### Install fail2ban
echo
echo Installing fail2ban
echo
sleep 3
apt -y -qq install fail2ban
#configure fail2ban as required

### Change Swappiness from 60 to 20
## Need to add check for current value in config file
## This could be done better
echo
echo Change swappiness to 20
echo
sleep 3
printf "\nCurrent swappiness="
cat /proc/sys/vm/swappiness
cat $SCTL | grep swappiness | sed -e 's/[^0-9]//g'
echo Current setting=$CSWP
echo Swap Details
echo ------------
swapon --show
sleep 3
printf "\n"
echo 20 > /proc/sys/vm/swappiness
if [ -e $SCTL.org ];
then
    cp -p $SCTL $SCTL.$STAMP;
else
    cp -p $SCTL $SCTL.org;
fi
echo "" >> $SCTL
echo "#Set swappiness to $SWPP" >> $SCTL`
echo "vm.swappiness = $SWPP" >> $SCTL`

### Install useful tools
## wget       - get files via http
## telnet     - telnet client
## bind9utils - utlities for querying dns (such as dig)
## nmap       - network analysis tool
## mlocate    - Faster and more efficient file locator
## mc         - Midnight Commander (XTree like file system interface)
## elinks     - Text based broswer (has dependencies)
## systat     - Statistical tools such as iostat
echo
echo Installing tools
echo
sleep 3
apt -y -qq install wget telnet bind9-utils nmap mlocate mc sysstat
apt -y -qq install elinks

### Install VMware tools if running on VMware
apt -y -qq install open-vm-tools
vmware-toolbox-cmd -v

## ALT: Install Hyper-V LIS if running on Hyper-V
#echo -e "hv_vmbus" >> /etc/initramfs-tools/modules
#echo -e "hv_storvsc" >> /etc/initramfs-tools/modules
#echo -e "hv_blkvsc" >> /etc/initramfs-tools/modules
#echo -e "hv_netvsc" >> /etc/initramfs-tools/modules
#apt -y install linux-virtual linux-cloud-tools-virtual linux-tools-virtual
#update-initramfs -u
## ALT: Install Hyper-V Enhanced Session Mode (xRDP)
## See https://www.kali.org/docs/virtualization/install-hyper-v-guest-enhanced-session-mode/
#apt -y install git
#git clone https://github.com/Microsoft/linux-vm-tools.git ~/linux-vm-tools
#cd ~/linux-vm-tools/ubuntu/
#chmod +x install.sh
#./install.sh
##edit /etc/xrdp/xrdp.ini Change port=vsock://-1:3389 to use_vsock=false
#systemctl enable xrdp.service
#systemctl start xrdp.service
## On host in Admin PS: Set-VM -VMName <vmname> -EnhancedSessionTransportType HvSo

### Setup email relay
echo
echo Setup email relay
echo
sleep 3
apt -y -qq install s-nail
ln -s /usr/bin/s-nail /bin/email
## This next bit could be done better
echo -e "set mta=smtp://$MAILFWD " >> /etc/mail.rc
echo -e "set mailx-extra-rc=/etc/mail.rc" >> /etc/s-nail.rc
echo 'Testing Email relay' | s-nail --subject='Email test 1'  -r "$HOST<$HOST@$FQDN>" $EMAILINST

### Setup SNMP (Not finished)
echo
echo Setup SNMP
echo
sleep 3
apt -y install snmpd snmp
ufw allow snmp
ufw status
#add lines for editing /etc/snmp/snmpd.conf
#change rocommunity public ro6community public etc
#SNMPDOPTS='-LS 0-4 d -Lf /dev/null -p /var/run/snmpd.pid'
cp /etc/snmp/snmpd.conf /etc/snmpd.conf.org
systemctl enable snmpd
systemctl restart snmpd
systemctl status snmpd
snmpwalk -v 2c -c $SNMPRO localhost

### Setup Unattended Updates (Not finished)
echo
echo Setup unattended updates
echo
sleep 3
apt -y -qq install unattended-upgrades
dpkg-reconfigure -plow unattended-upgrades ### Requires intervention
apt-config dump APT::Periodic::Unattended-Upgrade
cat /etc/apt/apt.conf.d/50unattended-upgrades | grep -v '//' | grep '[A-Aa-z]'
apt -y -qq install apt-listchanges
sed -i "/\b\(Unattended-Upgrade\:\:Mail\)\b/d" /etc/apt/apt.conf.d/50unattended-upgrades
echo -e "Unattended-Upgrade::Mail \"$EMAILADMIN\";" >> /etc/apt/apt.conf.d/50unattended-upgrades

### Install Webmin
## If using cockpit, comment this section out
## This also could be done better
echo
echo Installing Webmin
echo
sleep 3
apt -y install wget apt-transport-https software-properties-common
wget -q http://www.webmin.com/jcameron-key.asc -O- | sudo apt-key add -
add-apt-repository "deb [arch=amd64] http://download.webmin.com/download/repository sarge contrib"
apt -y install webmin
ufw allow webmin
ufw limit webmin

###
### Application Section
###

### Install, secure and run MySQL
##
#echo
#echo Installing MariaDB (MySQL)
#echo
#sleep 3
#apt -y install mariadb-server mariadb-client
# alternate install in case the above doesn't work
#apt -y install mariadb-client-10.3
#apt -y install mariadb-server-10.3
# Secure MySQL
#ufw allow mysql
#systemctl start mariadb
#mysql_secure_installation
#systemctl enable mariadb.service

### Install CPAN Minus and update PERL modules (some will fail on dependencies)
echo
echo Installing CPAN and PERL modules (This will take a while and requires interaction)
echo
sleep 5
apt -y install make
apt -y install libnet-ssleay-perl perl-IO-Zlib
cpan App::cpanminus
cpanm Net::FTPSSL
cpanm App::cpanoutdated
cpan-outdated -p | cpanm

### Install Apache web server (needs work)
echo
echo Installing Apache web server
echo
sleep 3
apt -y install apache2
apachectl -v
#<change httpd.conf listen to 0.0.0.0:80>
#be sure to set FQDN
ufw allow http
ufw allow https
ufw status
apachectl graceful
apachectl configtest

### Install PHP for Apache, MySQL and PEAR
echo
echo Installing PHP
echo
sleep 3
apt -y install php php-pear php-mysql
## Enable the following as required: Postgres, ODBC (MS SQL), LDAP, SOAP
#apt -y install php-pgsql php-odbc php-ldap php-soap
## Enable the following to install all PHP related development tools (this is a huge list > 60 packages)
## Only install this on test/dev servers. Don't install on stage, canary or prod servers.
#apt -y install pkg-php-tools
##
echo -e "<?php phpinfo(); ?>" > /var/www/html/info.php
# Test with http://server/info.php
systemctl restart httpd.service
## Setup dedicated Apache2 user

### Install NGINX instead of Apache (Needs lots more work)
#echo
#echo Installing NGINX
#echo
#sleep 3
#apt -y install nginx php php-common php-fpm
## Install as required
#apt -y install php-cli php-json php-pdo php-mysql php-zip php-gd php-mbstring php-curl php-xml php-pear php-bcmath
#ufw allow 'nginx http'
#ufw allow 'nginx https'
#ufw reload
#systemctl stop httpd
#systemctl stop apache2
#systemctl disable --now httpd
#systemctl disable --now apache2
#systemctl enable nginx
#systemctl start nginx  
#nginx -v
#nginx -t
#mkdir -p /var/www/<website>/public_html
#mkdir /var/www/<website>/logs
#chown -R nginx:nginx /var/www/<website>
## edit /etc/nginx/sites-available/default

### Install Wordpress (assumes Apache)
#echo
#echo Installing WordPress
#echo
#sleep 3
#apt -y install php-gd
#systemctl restart httpd.service
#wget http://wordpress.org/latest.tar.gz
#tar xzvf latest.tar.gz
#rsync -avP ~/wordpress/ /var/www/html/
#mkdir /var/www/html/wp-content/uploads
#chown -R apache:apache /var/www/html/*
## Setup WordPress Database
#mysql -u root -p <password>
#CREATE DATABASE wordpress;
#CREATE USER wordpressuser@localhost IDENTIFIED BY 'password'
#GRANT ALL PRIVILEGES ON wordpress.* TO wordpressuser@localhost IDENTIFIED BY 'password';
#FLUSH PRIVILEGES;
#exit
## Configure WordPress
#cd /var/www/html
#cat wp-config-sample.php | sed 's/database_name_here/wordpress/g' | sed 's/username_here/wordpressuser/g' | sed 's/password_here/password/g' > wp-config.php

### Installs phpMyAdmin
#echo
#echo Installing phpMyAdmin
#echo
#sleep 3
#apt -y install php-mbstring php-zip php-gd php-json php-myadmin
#cp /etc/phpMyAdmin/config.inc.php /etc/phpMyAdmin/config.inc.php.orig
## Harden PHPMyAdmin
#cat /etc/phpMyAdmin/config.inc.php.orig | sed -e 's/AllowRoot\'\]\ \=\ TRUE/AllowRoot\'\]\ \=\ FALSE/g' > /etc/phpMyAdmin/config.inc.php  
## Test with http://server/phpMyAdmin

###
### Add security compliance scanning to setup Script for Ubuntu 20.04
###
echo
echo Perform Security Evaluation
echo
sleep 3

##
## Apply updates and install openscap
##
apt -y -qq update && apt -y -qq upgrade
apt -y -qq install libopenscap8
apt -y -qq install ssg-base ssg-debderived ssg-debian ssg-nondebian ssg-applications

##
## Perform default and standard security policy compliance scans for Ubuntu 20.04
##
oscap xccdf eval /usr/share/xml/scap/ssg/content/ssg-ubuntu1604-ds.xml
oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_standard /usr/share/xml/scap/ssg/content/ssg-ubuntu1604-ds.xml

##
## Perform CVE scans
##

snap install cvescan
cvescan -p all
cvescan -p all --show-links > cvescan.out
echo >> cvescan.out
touch cvescan.csv
cvescan --csv -p all > cvescan.csv
## Also add open ports to report
netstat -tulpn | grep LISTEN | grep -v 127.0.0. | grep -v ::1
netstat -tulpn | head -n 2 >> cvescan.out
netstat -tulpn | grep LISTEN | grep -v 127.0.0. | grep -v ::1 >> cvescan.out
echo >> cvescan.out
echo Firewall Status >> cvescan.out
echo >> cvescan.out
ufw status >> cvescan.out
echo >> cvescan.out

##
## Perform OVAL scan and generate report
##
wget -c https://security-metadata.canonical.com/oval/com.ubuntu.$(lsb_release -cs).usn.oval.xml.bz2
bunzip2 com.ubuntu.$(lsb_release -cs).usn.oval.xml.bz2
oscap oval eval --report report.html com.ubuntu.$(lsb_release -cs).usn.oval.xml
## Email the report
cat cvescan.out | s-nail --subject="Security Report for $HOST" -a report.html -a cvescan.csv -r "$HOST<$HOST@$FQDN>" $EMAILINST

##

## Remove security eval prodcuts
##
echo
echo Removing security scan software
echo
sleep 3
snap remove cvescan
apt -y -qq remove libopenscap8

###
### Finish installation
###
echo
echo Cleanup installation and reboot
echo
sleep 3
apt -y -qq update && apt -y -qq upgrade
apt -y -qq autoremove --purge
echo
echo Rebooting in 60 seconds
echo
sleep 10
shutdown -r +1 Server Rebooting in 1 minute
echo
echo
sleep 60