Ubuntu Server Runup Script

Ubuntu Server Post Installation Runup

Following on from my CentOS 7 runup script is a series of Ubuntu scripts I have written over time. They're presented as a "Work in Progress" which may or may not end. 

I started the Ubuntu scripts using the CentOS scripts as a base, however they diverged quickly to become both unique and specialised. With Ubuntu, the goal was to standardise server builds so they could be duplicated by others. So the scripts began to evolve into a partially menu based build system.

But as I said - it's a work in progress, and the comments reflect areas that need improvement. So, for better or worse, here's the best of about two dozen iterations.

Base Ubuntu 20.04 LTS Runup Script

The script follows after the commentary given here. The script was designed to be edited before use. It is divided loosely into several sections. While some parts have been improved to be interactive, the majority requires serious modification before you should even think of running it on a server.

1. Variables

The first section includes many variables. Most should be self-explanatory. Check these carefully and make any modifications you need. Only those variables used in sections you are using need to be used, the rest can be commented out. In particular, make sure the following is properly declared:

MAILFWD IP address of the email relay

EMAILINST Your email address
EMAILADMIN The server administrators email address
SNMPRO The SNMPv2 public community
SNMPRW The SNMPv2 private community
SNMPSRV IP address of the SNMP server

2. Updates & Cockpit

Next updates are applied and cockpit is installed. This will take a few minutes to run. Choose between cockpit and webmin - do not install both. Cockpit has a smaller footprint and is easier to use, but webmin is more versatile.

3. Active Directory Integration

If you are integrating with Active Directory, study this section carefully as it is a little buggy and doesn't trap a lot of exceptions. I'd welcome feedback here!

4. PCP Logging tools & Grafana

This section is very experimental. Use with care. Grafana is a powerful tool that is not utilised enough. However it does take significant time to configure.

5. Firewall & Fail2ban

UFW (Uncomplicated Firewall) is then setup to allow access to ssh and cockpit. Rate limiting is applied to ssh. Fail2ban is also installed.

6. Swap

Default swappiness for Ubuntu server is 60. This is way too high. This section changes it to 20. This is a simple change, but an important one.

7. Tools

Several system utilities are installed here. Comment or uncomment according to what you find useful. The utilities are:

• wget - get files via http

• telnet - telnet client
• bind9utils - utlities for querying dns (such as dig)
• nmap - network analysis tool
• mlocate - Faster and more efficient file locator
• mc - Midnight Commander (XTree like file system interface)
• elinks - Text based broswer (has dependencies)
• systat - Statistical tools such as iostat

8. vm-tools & hyper-V

VMWare tools are installed next. Comment this section if not using vmware. An adjacent section for Hyper-V is provided to enable LIS and enhanced session mode.

9. Mail Relay

This section sets up s-nail for sending email via an email relay defined previously. S-nail is the preferred mailer for Ubuntu and provides the same service that mailx does.

10. SNMP (incomplete)

Install snmpd. This section is not finished. The snmpd.conf file must be manually edited.

11. Unattended updates (incomplete)

Ubuntu server installs updates automatically by default. This section must be configured to provide notification and control over the update process. The config files need to be manually edited afterwards as this section still needs some work.

12. Webmin

Webmin is still the best server application administration tool. There are hundreds of plugins for webmin. Cockpit is the best tool for system administration. If you don’t need webmin, comment out these lines. If you do, you may want to consider commenting out the cockpit installation lines to have just one tool.

Next we have the server applications

13. VSFTP

If you need ftp, this is the version to use. The downside is there is no webmin plugin for it. If you really need to administer ftp via webmin, install proftpd instead. If you don’t need an ftp server, comment out this section.

14. MariaDB (MySQL Server)

This will install mariadb server and client and harden the service. This section is interactive.

15. PERL

PERL is required for many web based applications. If not required, comment this section. This section is highly interactive and takes several minutes to install. As well as installing PERL, it installs cpan, cpan minus and numerous perl modules and keeps them updated as much as possible. Error messages in this section are not unusual. Ignore them.

16. Apache

Installs the Apache 2.4 web server. Configuration of httpd.conf is required. Comment out if installing NGINX.

17. PHP for Apache

Installs basic php modules and PEAR. Other modules can be installed by uncommenting as required.

18. NGINX

Installs NGINX instead of Apache.

19. WordPress

Requires Apache. Some heavy editing foo required here or you will be unhappy with the result...

20. phpMyAdmin

If you don't know what this is for, don't install it. 

21. Security evaluation

This section installs security evaluation and reporting tools and prepares a security report for the system.

22. Finish

Final section cleans up and reboots.

The Future

Plans for updates to the post-installation script include:

1. Make it interactive, so you don’t have to edit the file before running it.

2. Add logging to file.

3. Add auto-document capability. This will be the basis for as-built documentation.

4. Add a silent unattended mode.

The Script

#!/bin/bash
#
# Post installation script for Ubuntu Server 20.04 LTS
#
# Written by Wayne Doust 02 September 2021
#

## Needed for ifconfig to work
apt -y -qq install net-tools

SCTL="/etc/sysctl.conf"
STAMP=`date +%Y%m%d`
SWPP=20
HOST=`hostname -s`
FQDN=`hostname -d`
FQDN=site.local
MAILFWD=<IP of email forwarder>
EMAILINST=installer@$FQDN
EMAILADMIN=admin@$FQDN
SNMPRO=public
SNMPRV=private
SNMPSRV=<IP of SNMP server>
ALLOWFRM="127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fc00::/7, fe80::/10"
ADDOM=ADdomainname
ADFQDN=ADFQDN
ADUSER=Administrator


IP4="`ifconfig | sed -En 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p'`"
#IP6="`ifconfig | sed -En 's/::1//;s/.*inet6 (addr:)?(([[:xdigit:]]*::){,4}[[:xdigit:]]*::{,4}[[:xdigit:]]*::{,4}[[:xdigit:]]*::{,4}[[:xdigit:]]*).*/\2/p'`"

###
### The Following section asks for user input to modify some of the above variables
###
clear
echo
echo Ubuntu Server 20.04 LTS Runup Script
echo =========================================================================
echo
echo Enter parameters of this server. Press enter to accept the default.
echo
printf "Hostname [%s]:" $HOST
read ANSWER
if [ "$ANSWER" != "" ]
then
    HOST=$ANSWER
fi
echo Hostname: $HOST

printf "Domain [%s]:" $FQDN
read ANSWER
if [ "$ANSWER" != "" ]
then
    FQDN=$ANSWER
fi
echo Domain: $FQDN

echo Continuing will commence post-installation work of Ubuntu Server
echo ----------------------------------------------------------------
echo
echo Press any key to continue
read ANSWER

### Set Timezone and hostname
## Should use variables here
timedatectl set-timezone Australia/Melbourne
hostnamectl set-hostname $HOST
echo $HOST.$FQDN > /etc/hostname
sleep 3

### Apply updates and install cockpit & optionally pcp
## Note: Whilst you can install both, pick either Cockpit or webmin
##
echo
echo Apply updates
echo
sleep 3
apt -y -qq update && apt -y -qq upgrade
#echo
#echo Installing cockpit
#echo
#sleep 3
#apt -y -qq install cockpit
#apt -y install cockpit-pcp
#systemctl enable cockpit
#systemctl start cockpit

### (Optional) Add server to Actice Directory Domain
echo "deb http://au.archive.ubuntu.com/ubuntu/ bionic universe" >> /etc/apt/sources.list
echo "deb http://au.archive.ubuntu.com/ubuntu/ bionic-updates universe" >> /etc/apt/sources.list
hostnamectl set-hostname $HOST.$ADFQDN
hostnamectl
echo Check Name servers are correct
cat /etc/resolv.conf | grep nameserver
echo
sleep 10
systemctl disable systemd-resolved
systemctl stop systemd-resolved
apt -y update
apt -y install realmd libnss-sss libpam-sss sssd sssd-tools adcli samba-common-bin oddjob oddjob-mkhomedir packagekit
echo
echo Discover AD Domain
echo
realm discover $ADDOM
realm join -U $ADUSER $ADDOM
realm list $ADDOM
pam-auth-update --enable mkhomedir
## Do the following if the previous line doesn't work
#cp /usr/share/pam-configs/mkhomedir /usr/share/pam-configs/mkhomedir.org
#echo "Name: activate mkhomedir" > /usr/share/pam-configs/mkhomedir
#echo "Default: yes" >> /usr/share/pam-configs/mkhomedir
#echo "Priority: 900" >> /usr/share/pam-configs/mkhomedir
#echo "Session-Type: Additional" >> /usr/share/pam-configs/mkhomedir
#echo "Session:" >> /usr/share/pam-configs/mkhomedir
#echo "        required                        pam_mkhomedir.so" >> /usr/share/pam-configs/mkhomedir
#echo "umask=0022 skel=/etc/skel" >> /usr/share/pam-configs/mkhomedir
pam-auth-update
systemctl restart sssd
realm permit $ADUSER@$ADFQDN
realm permit 'Domain Admins' 'sysadmins'
echo "$ADUSER@$ADFQDN    ALL=(ALL)    ALL"          > /etc/sudoers.d/domain_admins
echo "%Domain\ Admins@ADFQDN    ALL=(ALL)    ALL"     >> /etc/sudoers.d/domain_admins
echo "%sysadmins@ADFQDN    ALL=(ALL)    ALL"         >> /etc/sudoers.d/domain_admins

### Optional logging tools based around pcp
## Don't install these unless you know what you're doing
# apt -y install pcp
# systemctl enable pmcd
# systemctl start pmcd
# systemctl enable pmlogger
# systemctl start pmlogger
# systemctl enable pmie
# systemctl start pmie
## use 'pcp atop' 'pmstat' 'pmiostat' etc
## Following is for web API for Grafana
# systemctl enable pmproxy
# systemctl start pmproxy
# wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
# add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
# apt update
# apt -y install grafana
# systemctl enable grafana-server
# systemctl start grafana-server
# ufw allow 3000/tcp
## Securing Grafana using NGINX Reverse Proxy (more here)
## See https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-grafana-on-ubuntu-20-04

    
### Setup ufw
echo
echo Setting up UFW
echo
sleep 3
ufw default deny incoming
## Allow SSH
ufw allow ssh
ufw limit ssh
## Allow cockpit
#ufw allow 9090/tcp
echo y | ufw enable
ufw status

### Install fail2ban
echo
echo Installing fail2ban
echo
sleep 3
apt -y -qq install fail2ban
#configure fail2ban as required

### Change Swappiness from 60 to 20
## Need to add check for current value in config file
## This could be done better
echo
echo Change swappiness to 20
echo
sleep 3
printf "\nCurrent swappiness="
cat /proc/sys/vm/swappiness
cat $SCTL | grep swappiness | sed -e 's/[^0-9]//g'
echo Current setting=$CSWP
echo Swap Details
echo ------------
swapon --show
sleep 3
printf "\n"
echo 20 > /proc/sys/vm/swappiness
if [ -e $SCTL.org ];
then
    cp -p $SCTL $SCTL.$STAMP;
else
    cp -p $SCTL $SCTL.org;
fi
echo "" >> $SCTL
echo "#Set swappiness to $SWPP" >> $SCTL`
echo "vm.swappiness = $SWPP" >> $SCTL`

### Install useful tools
## wget       - get files via http
## telnet     - telnet client
## bind9utils - utlities for querying dns (such as dig)
## nmap       - network analysis tool
## mlocate    - Faster and more efficient file locator
## mc         - Midnight Commander (XTree like file system interface)
## elinks     - Text based broswer (has dependencies)
## systat     - Statistical tools such as iostat
echo
echo Installing tools
echo
sleep 3
apt -y -qq install wget telnet bind9-utils nmap mlocate mc sysstat
apt -y -qq install elinks

### Install VMware tools if running on VMware
apt -y -qq install open-vm-tools
vmware-toolbox-cmd -v

## ALT: Install Hyper-V LIS if running on Hyper-V
#echo -e "hv_vmbus" >> /etc/initramfs-tools/modules
#echo -e "hv_storvsc" >> /etc/initramfs-tools/modules
#echo -e "hv_blkvsc" >> /etc/initramfs-tools/modules
#echo -e "hv_netvsc" >> /etc/initramfs-tools/modules
#apt -y install linux-virtual linux-cloud-tools-virtual linux-tools-virtual
#update-initramfs -u
## ALT: Install Hyper-V Enhanced Session Mode (xRDP)
## See https://www.kali.org/docs/virtualization/install-hyper-v-guest-enhanced-session-mode/
#apt -y install git
#git clone https://github.com/Microsoft/linux-vm-tools.git ~/linux-vm-tools
#cd ~/linux-vm-tools/ubuntu/
#chmod +x install.sh
#./install.sh
##edit /etc/xrdp/xrdp.ini Change port=vsock://-1:3389 to use_vsock=false
#systemctl enable xrdp.service
#systemctl start xrdp.service
## On host in Admin PS: Set-VM -VMName <vmname> -EnhancedSessionTransportType HvSo

### Setup email relay
echo
echo Setup email relay
echo
sleep 3
apt -y -qq install s-nail
ln -s /usr/bin/s-nail /bin/email
## This next bit could be done better
echo -e "set mta=smtp://$MAILFWD " >> /etc/mail.rc
echo -e "set mailx-extra-rc=/etc/mail.rc" >> /etc/s-nail.rc
echo 'Testing Email relay' | s-nail --subject='Email test 1'  -r "$HOST<$HOST@$FQDN>" $EMAILINST

### Setup SNMP (Not finished)
echo
echo Setup SNMP
echo
sleep 3
apt -y install snmpd snmp
ufw allow snmp
ufw status
#add lines for editing /etc/snmp/snmpd.conf
#change rocommunity public ro6community public etc
#SNMPDOPTS='-LS 0-4 d -Lf /dev/null -p /var/run/snmpd.pid'
cp /etc/snmp/snmpd.conf /etc/snmpd.conf.org
systemctl enable snmpd
systemctl restart snmpd
systemctl status snmpd
snmpwalk -v 2c -c $SNMPRO localhost

### Setup Unattended Updates (Not finished)
echo
echo Setup unattended updates
echo
sleep 3
apt -y -qq install unattended-upgrades
dpkg-reconfigure -plow unattended-upgrades ### Requires intervention
apt-config dump APT::Periodic::Unattended-Upgrade
cat /etc/apt/apt.conf.d/50unattended-upgrades | grep -v '//' | grep '[A-Aa-z]'
apt -y -qq install apt-listchanges
sed -i "/\b\(Unattended-Upgrade\:\:Mail\)\b/d" /etc/apt/apt.conf.d/50unattended-upgrades
echo -e "Unattended-Upgrade::Mail \"$EMAILADMIN\";" >> /etc/apt/apt.conf.d/50unattended-upgrades

### Install Webmin
## If using cockpit, comment this section out
## This also could be done better
echo
echo Installing Webmin
echo
sleep 3
apt -y install wget apt-transport-https software-properties-common
wget -q http://www.webmin.com/jcameron-key.asc -O- | sudo apt-key add -
add-apt-repository "deb [arch=amd64] http://download.webmin.com/download/repository sarge contrib"
apt -y install webmin
ufw allow webmin
ufw limit webmin

###
### Application Section
###

### Install, secure and run MySQL
##
#echo
#echo Installing MariaDB (MySQL)
#echo
#sleep 3
#apt -y install mariadb-server mariadb-client
# alternate install in case the above doesn't work
#apt -y install mariadb-client-10.3
#apt -y install mariadb-server-10.3
# Secure MySQL
#ufw allow mysql
#systemctl start mariadb
#mysql_secure_installation
#systemctl enable mariadb.service

### Install CPAN Minus and update PERL modules (some will fail on dependencies)
echo
echo Installing CPAN and PERL modules (This will take a while and requires interaction)
echo
sleep 5
apt -y install make
apt -y install libnet-ssleay-perl perl-IO-Zlib
cpan App::cpanminus
cpanm Net::FTPSSL
cpanm App::cpanoutdated
cpan-outdated -p | cpanm

### Install Apache web server (needs work)
echo
echo Installing Apache web server
echo
sleep 3
apt -y install apache2
apachectl -v
#<change httpd.conf listen to 0.0.0.0:80>
#be sure to set FQDN
ufw allow http
ufw allow https
ufw status
apachectl graceful
apachectl configtest

### Install PHP for Apache, MySQL and PEAR
echo
echo Installing PHP
echo
sleep 3
apt -y install php php-pear php-mysql
## Enable the following as required: Postgres, ODBC (MS SQL), LDAP, SOAP
#apt -y install php-pgsql php-odbc php-ldap php-soap
## Enable the following to install all PHP related development tools (this is a huge list > 60 packages)
## Only install this on test/dev servers. Don't install on stage, canary or prod servers.
#apt -y install pkg-php-tools
##
echo -e "<?php phpinfo(); ?>" > /var/www/html/info.php
# Test with http://server/info.php
systemctl restart httpd.service
## Setup dedicated Apache2 user

### Install NGINX instead of Apache (Needs lots more work)
#echo
#echo Installing NGINX
#echo
#sleep 3
#apt -y install nginx php php-common php-fpm
## Install as required
#apt -y install php-cli php-json php-pdo php-mysql php-zip php-gd php-mbstring php-curl php-xml php-pear php-bcmath
#ufw allow 'nginx http'
#ufw allow 'nginx https'
#ufw reload
#systemctl stop httpd
#systemctl stop apache2
#systemctl disable --now httpd
#systemctl disable --now apache2
#systemctl enable nginx
#systemctl start nginx  
#nginx -v
#nginx -t
#mkdir -p /var/www/<website>/public_html
#mkdir /var/www/<website>/logs
#chown -R nginx:nginx /var/www/<website>
## edit /etc/nginx/sites-available/default

### Install Wordpress (assumes Apache)
#echo
#echo Installing WordPress
#echo
#sleep 3
#apt -y install php-gd
#systemctl restart httpd.service
#wget http://wordpress.org/latest.tar.gz
#tar xzvf latest.tar.gz
#rsync -avP ~/wordpress/ /var/www/html/
#mkdir /var/www/html/wp-content/uploads
#chown -R apache:apache /var/www/html/*
## Setup WordPress Database
#mysql -u root -p <password>
#CREATE DATABASE wordpress;
#CREATE USER wordpressuser@localhost IDENTIFIED BY 'password'
#GRANT ALL PRIVILEGES ON wordpress.* TO wordpressuser@localhost IDENTIFIED BY 'password';
#FLUSH PRIVILEGES;
#exit
## Configure WordPress
#cd /var/www/html
#cat wp-config-sample.php | sed 's/database_name_here/wordpress/g' | sed 's/username_here/wordpressuser/g' | sed 's/password_here/password/g' > wp-config.php

### Installs phpMyAdmin
#echo
#echo Installing phpMyAdmin
#echo
#sleep 3
#apt -y install php-mbstring php-zip php-gd php-json php-myadmin
#cp /etc/phpMyAdmin/config.inc.php /etc/phpMyAdmin/config.inc.php.orig
## Harden PHPMyAdmin
#cat /etc/phpMyAdmin/config.inc.php.orig | sed -e 's/AllowRoot\'\]\ \=\ TRUE/AllowRoot\'\]\ \=\ FALSE/g' > /etc/phpMyAdmin/config.inc.php  
## Test with http://server/phpMyAdmin

###
### Add security compliance scanning to setup Script for Ubuntu 20.04
###
echo
echo Perform Security Evaluation
echo
sleep 3

##
## Apply updates and install openscap
##
apt -y -qq update && apt -y -qq upgrade
apt -y -qq install libopenscap8
apt -y -qq install ssg-base ssg-debderived ssg-debian ssg-nondebian ssg-applications

##
## Perform default and standard security policy compliance scans for Ubuntu 20.04
##
oscap xccdf eval /usr/share/xml/scap/ssg/content/ssg-ubuntu1604-ds.xml
oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_standard /usr/share/xml/scap/ssg/content/ssg-ubuntu1604-ds.xml

##
## Perform CVE scans
##

snap install cvescan
cvescan -p all
cvescan -p all --show-links > cvescan.out
echo >> cvescan.out
touch cvescan.csv
cvescan --csv -p all > cvescan.csv
## Also add open ports to report
netstat -tulpn | grep LISTEN | grep -v 127.0.0. | grep -v ::1
netstat -tulpn | head -n 2 >> cvescan.out
netstat -tulpn | grep LISTEN | grep -v 127.0.0. | grep -v ::1 >> cvescan.out
echo >> cvescan.out
echo Firewall Status >> cvescan.out
echo >> cvescan.out
ufw status >> cvescan.out
echo >> cvescan.out

##
## Perform OVAL scan and generate report
##
wget -c https://security-metadata.canonical.com/oval/com.ubuntu.$(lsb_release -cs).usn.oval.xml.bz2
bunzip2 com.ubuntu.$(lsb_release -cs).usn.oval.xml.bz2
oscap oval eval --report report.html com.ubuntu.$(lsb_release -cs).usn.oval.xml
## Email the report
cat cvescan.out | s-nail --subject="Security Report for $HOST" -a report.html -a cvescan.csv -r "$HOST<$HOST@$FQDN>" $EMAILINST

##

## Remove security eval prodcuts
##
echo
echo Removing security scan software
echo
sleep 3
snap remove cvescan
apt -y -qq remove libopenscap8

###
### Finish installation
###
echo
echo Cleanup installation and reboot
echo
sleep 3
apt -y -qq update && apt -y -qq upgrade
apt -y -qq autoremove --purge
echo
echo Rebooting in 60 seconds
echo
sleep 10
shutdown -r +1 Server Rebooting in 1 minute
echo
echo
sleep 60

CentOS 7 post installation script

I like automation. Anything I can do to speed up repetitious tasks is worth the effort. The following shell script is a work-in-progress. Each time I run up a server, I use this script to automate the post installation tasks. I just uncomment the sections I need and run the script. Parts of it take some time and require some human intervention, but usually that just involves pressing 'enter' a few times. The script doesn't do much in the way of hardening the server - that's the stuff of another blog entry.

To use the script, cut and paste it into an editor, make it executable and run it as root. Unfortunately sudo doesn't cut it for some steps. Make sure you uncomment the sections you need first. There are modifications in there for RHEL if you're using that instead of CentOS. I have created a version for CentOS 8, but the it now EOL at the end of this year, I figure it's a little pointless. If anyone wants it, just ask and I'll make it available.

Briefly, the script does the following things:

1. Runs a yum update using deltas
2. Adds the epel repository and optionally the remi repository
3. Installs selinux utilities
4. Installs PERL, CPAN and useful utilities for PERL.
5. Installs Apache or NGINX
6. Installs php for apache
7. Optional upgrade for php 5.6 
8. Installs webmin
9. Installs vsftpd
10. Installs and secures mysqld
11. Installs vmtools
12. Installs miscellaneous utilities
13. Configure to remove old kernels automatically
14. Setup email forwarding
15. Setup and configure WordPress

As I runup more servers with different services, I will update and expand the script. I'm currently writing a GUI front end for the script in Xojo (see below) to customise and generate the script for any environment. If it works, I hope to expand it to include the ability to deploy the script remotely as well as generate it.

 

#!/bin/bash
## The following commands increase the number of packages available
##
yum -y update
yum -y install deltarpm
yum -y install epel-release
## Uncomment for REMI
# rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
# yum --enablerepo=remi,remi-test

yum repolist
yum -y update

## RHEL instructions
# wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm
# rpm -ivh epel-release-7-10.noarch.rpm
# rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
# yum --enablerepo=remi,remi-test
# yum repolist
# yum -y update

## Install SELinux utilities
##
yum -y install policycoreutils policycoreutils-python selinux-policy selinux-policy-targeted libselinux-utils setroubleshoot-server setools setools-console mcstrans

## Install PERL and useful modules and utilities via cpan and cpanm
##
yum -y install perl perl-Net-SSLeay perl-IO-Zlib openssl perl-IO-Tty cpan
cpan App::cpanminus
cpanm Net::FTPSSL
cpanm App::cpanoutdated
cpan-outdated -p | cpanm
cpan-outdated -p | cpanm

## Install and start Apache Web server.
##
yum -y install httpd
#<change httpd.conf listen to 0.0.0.0:80>
systemctl enable httpd.service
systemctl start  httpd.service
systemctl status httpd.service
firewall-cmd --zone=public --permanent --add-service={http,https}
systemctl restart firewalld.service
firewall-cmd --list-all
apachectl graceful
apachectl configtest

## Install NGINX
##
# echo -e "[nginx]\nname=nginx repo\nbaseurl=http://nginx.org/packages/centos/$releasever/$basearch/\ngpgcheck=0\nenabled=1" > /etc/yum.repos.d/nginx.repo
# yum -y install nginx php php-common php-fpm
# firewall-cmd --zone=public --permanent --add-service={http,https}
# systemctl stop httpd
# systemctl disable httpd
# systemctl enable nginx
# systemctl enable php-fpm
# systemctl start nginx
# systemctl start php-fpm
# mkdir -p /srv/www/<website>/public_html
# mkdir /srv/www/<website>/logs
# chown -R nginx:nginx /srv/www/<website>

## Install PHP for Apache, mysql and MSSQL
##
yum -y install php php-pear  php-mysql
## Enable the following as required
# yum -y install php-pgsql php-odbc php-mssql php-ldap php-soap
# yum -y install phpmyadmin httpd-tools
# cp /etc/phpMyAdmin/config.inc.php /etc/phpMyAdmin/config.inc.php.orig
# Harden PHP
# cat /etc/phpMyAdmin/config.inc.php.orig | sed -e 's/AllowRoot\'\]\ \=\ TRUE/AllowRoot\'\]\ \=\ FALSE/g' > /etc/phpMyAdmin/config.inc.php

## Test with http://server/test.php
## Test with http://server/phpMyAdmin

## Upgrade to PHP 5.6 (optional)
##
#rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
#yum repolist
#yum -y remove php-common
#yum -y update
#yum -y install php56w php56w-opcache php56w-xml php56w-mcrypt php56w-gd php56w-devel php56w-mysql php56w-intl #php56w-mbstring
#systemctl restart httpd.service
#php -v

systemctl restart httpd.service
echo -e "<?php phpinfo(); ?>" > /var/www/html/info.php
systemctl restart httpd.service

## Install and run Webmin administration (requires PERL)
##
echo -e "[Webmin]\nname=Webmin Distribution Neutral\nbaseurl=http://download.webmin.com/download/yum\nenabled=1" > /etc/yum.repos.d/webmin.repo
rpm --import http://www.webmin.com/jcameron-key.asc
yum -y install webmin
systemctl enable webmin.service
systemctl start  webmin.service
systemctl status webmin.service
firewall-cmd --zone=public --permanent --add-port=10000/tcp
firewall-cmd --reload
firewall-cmd --list-all

## Install and start VSFTPD
##
#yum -y install vsftpd ftp
#systemctl enable vsftpd.service
#systemctl start vsftpd.service
#firewall-cmd --zone=public --permanent --add-service=ftp
#systemctl restart firewalld.service
#firewall-cmd --list-all

## Install, secure and run MySQL
##
#yum -y install mariadb-server mariadb
#systemctl start mariadb
#mysql_secure_installation
#systemctl enable mariadb.service
#systemctl status mariadb.service

## Install tools for vmware
##
yum install open-vm-tools
vmware-toolbox-cmd -v

## Install Misc utilities as desired
##
yum -y install wget telnet bind-utils nmap mlocate mc
yum -y install elinks

## Remove old kernels and set yum to preserve only two. Setup auto updates
##
yum -y install yum-utils yum-cron
package-cleanup -y --oldkernels --count=2
echo -e "\ninstallonly_limit=2\n" >> /etc/yum.conf
#yum -y install yum-cron # edit /etc/yum/yum-cron.conf or do this via webmin instead on yum-cron
#systemctl start yum-cron
#systemctl enable yum-cron

## Setup email forwarding
##
yum -y install mailx
ln -s /bin/mailx /bin/email
echo -e "set smtp=smtp://<mail forwarding ip address> " >> /etc/mail.rc


## Install Wordpress
#yum -y install php-gd
#systemctl restart httpd.service
#wget http://wordpress.org/latest.tar.gz
#tar xzvf latest.tar.gz
#rsync -avP ~/wordpress/ /var/www/html/
#mkdir /var/www/html/wp-content/uploads
#chown -R apache:apache /var/www/html/*


## Setup WordPress Database
#mysql -u root -p <password>
#CREATE DATABASE wordpress;
#CREATE USER wordpressuser@localhost IDENTIFIED BY 'password'
#GRANT ALL PRIVILEGES ON wordpress.* TO wordpressuser@localhost IDENTIFIED BY 'password';
#FLUSH PRIVILEGES;
#exit

## Configure WordPress
#cd /var/www/html
#cat wp-config-sample.php | sed 's/database_name_here/wordpress/g' | sed 's/username_here/wordpressuser/g' | sed 's/password_here/password/g' > wp-config.php

Installing Apache on Windows Server 2016

Occasionally, someone will want something that runs best on Linux to be installed on a Windows server. There are various reasons for this, but usually it comes down to "I'm not comfortable with Linux, I want this to run on Windows". Regardless of the reason, if that's what the customer wants, that's what the customer gets.

Just to be clear this point, I'm referring to a WAMP server (Windows Apache MySQL PHP) as close as possible to a 'standard' LAMP server (Linux Apache MySQL PHP) that you'll get from the repo of your favourite distro.

That being said, the LAMP server you'll get from RHEL/CentOS will be significantly different from the one you get from Ubuntu.

When it comes to running Apache on Windows, for strange reasons, that simple decision makes the world a whole lot more complex. You see in Linux, when you decide to run Apache, pretty much all the decisions of 'how' have been made for you by the distribution you are using. And unless you decide to roll-your-own - which most people don't do AND increasingly this is a bad thing to do - they are the decisions you now deal with.

Under Windows, that's not the case. There are more ways to run Apache under Windows than you can poke a stick it at. (If you want to get pedantic, you can say there are an infinite number of ways, but don't go there). A lot of these methods are excluded by the "why" you are running Apache, we'll look at a few of these.

The default way is to download Apache from Apache.org and go from there. That's a lot of work and leads to a lot of stuff being left out.

However, if you want to run Apache as bog standard as possible, without having to do too much low level work, Apache Lounge is a good way to go and the one I'll be detailing here.

Procedure

Firstly download the binaries from here. Choose the appropriate version (generally 64 bit) and unzip the archive. While you're waiting for it to download, get the Windows c++ runtime redistributable that is appropriate. Apache lounge is written in c++ and requires this in order to run. The Apache Lounge readme and download page will show you which version is necessary - which is currently 2017. Later versions will (may) work, but it's best to get the version the application was written for. Earlier versions will not work.

Apache Lounge is easier to work with than the official Apache distribution. It also 'feels' more flexible. Try both and you'll see what I mean. What I really like is being uncoupled from the "program files" paradigm of Windows. I personally prefer to have both the binaries and the data separate from the C: drive, so that's the assumption we'll be using here.

Next extract the zip files and copy the Apache24 directory structure to D:\apache24. It may be a good idea to change the folder preferences for full visibility including hidden files and showing the filename extension.

As an aside here, it is also a good idea to prep your server to have the following utilities installed and available:

 - Chrome browser (or firefox, safari, opera etc)
 - putty
 - winscp
 - telnet client
 - tftp client

Edit the D:\apache24\conf\httpd.conf file to add/edit lines as follows (in most cases this is done by looking for the near identical line and either editing it or remove the '#':

Define SRVROOT "D:/Apache24"
Listen *:80

LoadModule rewrite_module modules/mod_rewrite.so

ServerAdmin <youremailaddress> 

ServerName <your servername or localhost>:80

DocumentRoot "D:/webroot" 

AllowOverride All  

<Directory "D:/webroot"> 

Check the config file carefully. Work here will pay off later.
Change to the bin directory and run the following:

httpd -t

If the configuration is fine, check that ports 80 and 443 are not in use by running:

netstat -ao | findstr ":80"
netstat -ao | findstr ":443"

Most likely you'll find port 80 in use. The last number is the process ID of the service tying up the port. Identify the service from the process ID using Task Manager and disable that service. If the PID is 4, then this is the system service http.sys and it becomes annoying and potentially a problem....

Under later versions of Windows Server, some processes communicate using http via port 80. The purpose of http.sys is to redirect that data to the appropriate service. You can test this by using the telnet client to telnet to 127.0.0.1 80 and pressing return twice. You should get something like the following as output:

What this means is that one or more native windows services are dependent upon the http.sys service to operate. You have to decide if you can live without these services. If not, then you need to select a different port for Apache to use for http traffic or use it for https only. If this is the case, edit the config file and change the listening port from 80 to something else (for now). We can remove http and make the server https only later on.

If you can live without the services, firstly stop the http.sys service using

net stop http

You will get something like the following:

Answer 'y' and stop the services. You can (and should) script this process by taking not of all the services that need to be stopped and explicitly stopping them. You will likely have to do this frequently.

MySQL/MariaDB

Download MySQL installer and MySQL workbench

(From https://seravo.fi/2015/10-reasons-to-migrate-to-mariadb-if-still-using-mysql) 

The original MySQL was created by a Finnish/Swedish company, MySQL AB, founded by David Axmark, Allan Larsson and Michael ”Monty” Widenius. The first version of MySQL appeared in 1995. It was initially created for personal usage but in a few years evolved into a enterprise grade database and it became the worlds most popular open source relational database software – and it still is. In January 2008, Sun Microsystems bought MySQL for $1 billion. Soon after, Oracle acquired all of Sun Microsystems after getting approval from the European Commission in late 2009, which initially stopped the transaction due to concerns that such a merger would harm the database markets as MySQL was the main competitor of Oracle’s database product.
 

Out of distrust in Oracle stewardship of MySQL, the original developers of MySQL forked it and created MariaDB in 2009. As time passed, MariaDB replaced MySQL in many places

 https://downloads.mariadb.org/

Current is 10.3.15 stable. Download x64 msi package.

Installation of MySQL

Download MySQL Workbench. Trust me: You need this!

 https://dev.mysql.com/downloads/workbench/
 

Current version 8.0.16 Requires

Microsoft .NET Framework 4.5
Visual C++ Redistributable for Visual Studio 2015

MySQL Community Server

https://dev.mysql.com/downloads/windows/

The full installer includes all windows packages and is a 373MB download. The installer is 32 bit but it includes both 32 and 64 bit binaries.

The installer does not permit upgrades between major and minor version numbers - these are installed as separate server instances - but it will perform upgrades within a release series eg: 5.7.18 to 5.7.19.

The installer also requires .NET Framework 4.5.2 or later.

Post installation testing of MySQL/MariaDB

The service should start immediately after installation. Check the MySQL service is running.

Check you can login from the command line using the root user and password generated before. 

Installing PHP

(From https://www.php.net/manual/en/install.windows.php)

 

PHP 5.5+ require at least Windows 2008/Vista, or 2008r2, 2012, 2012r2, 2016 or 7, 8, 8.1, 10. Either 32-Bit or 64-bit (aka X86 or X64. PHP does not run on Windows RT/WOA/ARM). As of PHP 7.2.0 Windows 2008 and Vista are no longer supported.

 

PHP requires the Visual C runtime(CRT). Many applications require that so it may already be installed. 

 

PHP 5.5 and 5.6 require VC CRT 11 (Visual Studio 2012). See: » https://www.microsoft.com/en-us/download/details.aspx?id=30679

 

PHP 7.0 and 7.1 require VC CRT 14 (Visual Studio 2015). PHP 7.2 and 7.3 require VC CRT 15 (Visual Studio 2017). PHP 7.4+ requires VC CRT 16 (Visual Studio 2019). 

 

The Microsoft Visual C++ Redistributable for Visual Studio 2019 is suitable for all these PHP versions, see » https://visualstudio.microsoft.com/downloads/. 

So we will be installing 7.3 from https://windows.php.net/download

Which version do I choose?

IIS

If you are using PHP as FastCGI with IIS you should use the Non-Thread Safe (NTS) versions of PHP.

Apache

Please use the Apache builds provided by Apache Lounge. They provide VC14 and VC15 builds of Apache for x86 and x64. We use their binaries to build the Apache SAPIs.
With Apache you have to use the Thread Safe (TS) versions of PHP.

VC14 & VC15

More recent versions of PHP are built with VC14 or VC15 (Visual Studio 2015 or 2017 compiler respectively) and include improvements in performance and stability.
- The VC14 builds require to have the Visual C++ Redistributable for Visual Studio 2015 x86 or x64 installed
- The VC15 builds require to have the Visual C++ Redistributable for Visual Studio 2017 x64 or x86 installed

TS and NTS

TS refers to multithread capable builds. NTS refers to single thread only builds. Use case for TS binaries involves interaction with a multithreaded SAPI and PHP loaded as a module into a web server. For NTS binaries the widespread use case is interaction with a web server through the FastCGI protocol, utilizing no multithreading (but also for example CLI).

What is PGO?

Profile Guided Optimization is an optimization feature available in Microsoft's Visual C++ compiler that allows you to optimize an output file based on profiling data collected during test runs of the application or module.

Links:

• PGO on MSDN
• Visual C++ Team Blog - PGO with PHP
• PHP PGO build for maximum performance (old)

amd64 (x86_64) Builds

PHP 7 provides full 64-bit support. The x64 builds of PHP 7 support native 64-bit integers, LFS, 64-bit memory_limit and much more.

  copy to D:\php7

  copy php.ini

  edit php.ini

Add C:\php to the path environment variable

To ensure Windows can find PHP, you need to change the path environment variable. Open Settings, type ‘environment variables’ into the search field and open the result. Select the “Advanced” tab, and click the “Environment Variables” button.
Scroll down the System variables list and click on “Path” followed by the “Edit” button. Click “Edit text” and add ;C:\php to the end of the Variable value line (remember the semicolon).

 edit httpd.conf to add index.php to the DirectoryIndex variable

At the bottom of the file, add the following lines (change the PHP file locations if necessary):

# PHP5 module
LoadModule php5_module "c:/php/php5apache2_2.dll"
AddType application/x-httpd-php .php
PHPIniDir "C:/php" 

Save the configuration file and test it from the command line (Start > Run > cmd):

cd Apache2bin
httpd -t

PECL extensions are pre-built for Windows and available from: » http://windows.php.net/downloads/pecl/releases/

 

Some extensions use features specific to some Unix systems and so are not available on Windows. Otherwise, all extensions are available for Windows. 

PEAR
Open a command prompt with administrator privileges. Use curl to download the pear install file into the php7 directory

curl -OL http://pear.php.net/go-pear.phar

If you use a browser to download the file, it renders as text but with binary characters embedded. This will stop the script from working, hence the need to use curl.

set PHP_PEAR_SYSCONF_DIR=D:\php7

php -d phar.require_hash=0 go-pear.phar

You will then be prompted with a few questions. If the defaults don't work, you probably haven't elevated the privileges of the command prompt. If option 12 lists the Windows path, you haven't set the PHP_PEAR_SYSCONF_DIR variable. Editing here doesn't work, which is why you need to set the variable. You will need to enter the path to cli.exe- in our case this is d:\php7.

Note: If any of the directories already exists, you may get a 'permission denied' error for that directory. In which case, delete the directory and run the script again.

The rest of the defaults should be okay, but check them anyway. Once finished, a reg file called PEAR_ENV.reg will be created. Execute that to add the necessary keys to the registry.

Finally, check that PEAR is working using pear version:

Conclusion

Now, hopefully you read all this through before attempting it and are utterly confused by the process. If you, that's great! No go and install Linux and deploy Apache that way. Trust me! It's much easier and safer. Plus when you get into trouble and need to troubleshooting, pretty much any article you'll find will assume you are using Linux.


The Sony Vaio VGN and Linux

For quite some time now, my preferred desktop OS has been Ubuntu. I made the switch from Fedora after a bad experience with graphics drivers between Core 5 & 6 and decided that Fedora was just a little too cutting edge for me. This was around when I had been playing around with Ubuntu for client desktops and warming to with each new version. For me, the transition to a Debian base system was uncomfortable and I avoided it for a while. The crunch came in early 2007 when I received a Sony Vaio VGN series "hip-top" computer. At the time it was pretty stunning and Sony were pushing them hard - which is probably why they were giving them away to guys like me: so that we would show them off to our clients. In the end, tablets killed off this technology avenue.

Sony Vaio VGN series

I was pretty impressed with the VGN, mainly because it was loaded with peripherals and came with a great docking station. Of course, the first task was to give Windows the flick and install Linux.

With Fedora, almost nothing worked, so I blew that away and installed SuSe. This was much better and most things "worked". The problem was the peculiar UI - which for a touch screen was difficult to work with. Changing the UI on SLED was a problem and it felt a little too "locked down" for my liking. It became painful, so it had to go as well.

Ubuntu seemed to be about halfway there. Some of the special buttons would not work, but I could live without them. It fitted with my purposes at the time, so I was off and running with Ubuntu. It didn't take too long before I preferred it to Fedora.

One of the things I used to dislike about Ubuntu was the default Gnome interface - I preferred KDE. Yes, I know it's also the default for Fedora, but switching interfaces with Fedora was simple. For the Vaio, I chose to install Kubuntu; which at the time was the specific distro for Ubuntu users with KDE. Later I decided to switch back to Gnome (when Gnome updated)and that proved to be a superior experience. The ever present issue was always screen resolution - which never really "fit" the screen. 

I persevered for a few years with the Vaio - mainly for it's 'wow' factor - until I bought a Asus Transformer T300 tablet with keyboard and after that I almost immediately stopped using the Vaio. Despite the tablet being far less powerful, in had a 10" screen and a real keyboard. To make the Vaio usable, I had to become "walking tech" by carrying around a folding keyboard, port replicator, external HDD and a wireless mouse - plus the charger. I ended up selling the Vaio to a colleague for $50.

Tutorial: CentOS 7 Installation guide for vmware ESX environments (Part 1)

Although this guide is written specifically for ESX environments, it translates easily to other virtual environments. The same general rules apply even if he steps are slightly different. There is an exception for MS HyperV where its easier just to leave the FDD emulation rather than removing it.

If you're installing RHEL, Fedora, OL or Scientific Linux rather than CentOS, most of the installation will apply although OL has very different recommendations. The instructions here detail a minimal install (Micro-Instance) which will give you an very basic server. Instructions for different footprints will follow in subsequent blog entries. These instructions are based on the 7.4 release of CentOS.

Best practice is to install CentOS from either the NetInstall ISO or the Minimal ISO. Currently, the NetInstall (rev 1708) is 422MB. The Minimal is 792MB. The minimal install contains everything needed to setup a basic server. Additional services must be installed from repositories. The NetInstall requires an active Internet connection (and hence a working NIC) in order to complete even a basic installation. By contrast, the standard ISO is 4.2GiB in size.

It must be assumed that everything is out-of-date as of the initial release. So installing from the standard ISO achieves little as most packages will need to be updated from the repositories. The procedure shown will be using the NetInstall ISO.

FirstCopy the ISO to the vmware ESX server’s ISO folder in the datastore or some location that is easy for you to install from.

Minimal Server Installation (Micro-Instance)

1) Create new server in ESX using ‘typical’ configuration and give it a name.

2) Select the Datastore for the vmdk files and press next.

3) Select Operating System (Linux, CentOS 64bit)

4) Select appropriate NICs. You will need at least one that has access to the Internet. You can add others now (preferably) or later. Make sure you use the VMXNET3 adapter with connect at power on. Use of the e1000 adapter has been shown to cause problems, particularly during ESX upgrades.

5) Setup virtual disks on the datastore. For a micro-instance, we will need to have it thick provisioned. For larger servers, we can create virtual disks for other partitions – these can be thin provisioned, but any partition containing a boot or swap partition needs to be thick. For larger servers, you will likely have the data-containing partitions on a separate SAN connected via multipath.

The disk allocation for a micro-instance should be between 12GB and 32GB. Larger than that and you should allocate a second(or more) disks. If the specific calls for a specific disk size, definitely create another disk – either thick or thin provisioned. For large footprints, eager zero is preferred, but lazy zeroed is fine for a micro-instance.

6) Edit the VM settings (tick box) and click continue. Modify the settings as follows:

1. Memory: 2048MB is the default. For a micro-instance, this can be reduced to as low as 1024MB – particularly if you don’t plan on a GUI (which you shouldn’t anyway).
2. CPUs: 2 virtual sockets with 1 core per socket for micro-instance or medium. Increase only as required by application. By choosing 2 cores, SMP will be installed in the kernel. You can reduce it to one later, but make sure there are at least two at installation time.
3. Video Card: Leave at 4MB with 1 display unless you need a GUI. Then increase to 12MB.
4. CD/DVD: Set to the CentOS NetInstall or MinimalInstall ISO you added earlier. Check ‘Connect at power on’.
5. Floppy: Remove

7) Click ‘Finish’

8) Select newly created server in the client, connect to the console and power on.

There should be no need to test the media, so just select ‘Install Centos’. The CentOS installer will then boot:

CentOS Installation (GUI)

1) Language: Select English and your location in the world. In my case, this is Australia.

2) Installation Summary: You will need to modify ‘Date/Time’, ‘Security Policy’, ‘Installation Source’, ‘Installation Destination’ and ‘Network and Host Name’. Since this is a Net Install, we need to modify the Network and Hostname first.

3) Network and Hostname. Enable the network adapter. This will perform an automatic DHCP. Unless you plan on using DHCP for your servers, you should change this to a static address. Change the hostname to the FQDN of the server.

Use the configure button to change the name of the adapter to something easier to remember. You can also change the MAC address here. This is useful if you are creating an immutable server. Under the ‘General’ tab, select “Automatically connect to this network when it is available”. Set the IPv4 address and DNS settings. Add IPv6 if utilised. Click ‘Save’ and then ‘Done’ when finished.

3) Date/Time: Change the timezone if required. Check that NTP is enabled and working.

4) Installation Source: Enter the Centos mirror address and uncheck the mirror list checkbox. Click done and CentOS will begin updating its repository list.

5) Software Selection: I know this is tempting to select what you want here, but stick with the minimal install for now. The only time you really should select anything is if you are installing a compute node or virtualization host.

6) Installation Destination: For a micro-instance, you can settle for the defaults. This will create two physical partitions. The first (sda1) will be a 1GiB boot partition formatted for xfs (the default now under CentOS7). You can change this to ext4 if you like. The second physical partition will use LVM (Logical Volume Management) with two LVM partitions made up of a 1.5GiB swap partition and the rest allocated to root. For simple servers, this will be adequate. Otherwise, modify it here according to the size/space/capability recommendations. Everything here is completely customisable.

7) Once you click ‘done’. You will be given a list of changes the installer is about to do based upon your selections. Check them carefully before accepting.

8) Lastly, we will need to select a security policy. This is an area often overlooked and most installations will choose the default XCCDF profile which contains no rules, although rule lists can be downloaded and applied here. Unless mandated otherwise, use the Standard System Security Profile. The security profiles are administered by Red Hat and not even vetted by CentOS. You will need to consult RHEL documentation for details. In brief, it contains thirteen rules to ensure a basic level of security compliance.

There is a HUGE caveat with the use of security policies: They are something of a black art and emphasise security above all else. This can result in unpredictable changes to your system without notification.

If you want to harden the CentOS setup, I’ll deal with that later. For now, just select the Standard Profile.

Note: There is a bug with the 7.3.1611 ISO with all four STIG security policies that has been fixed with 7.4.1708. Security profiles "Standard System Security Profile" and "C2S for CentOS Linux 7" can't be used in the CentOS 7.5.1804 installer. A bug causes the installer to require a separate partition for /dev/shm, which is not possible.

We could spend hours here. We won’t. That will come with server hardening - when we get to it.

9) Next, click ‘Begin installation’ and installation will start. The next page will work concurrently with the installation.

10) Create a root user and a normal (administrative) user. Ideally, you will only ever use the root login once. After that we will disable root logins completely except from the console. To gain root privileges, you will need to elevate them using sudo by placing administrative users in the ‘wheel’ group (more on that later).

Set the root password to something long a difficult but easy to remember. Make it at least 15 characters by using a phrase with upper/lowercase and numbers eg: 1veGot4LovelyBunch0fCoconuts. Store this password in a secure place (eg keysoft secured by two factor authentication). You will only need this password once and in case of emergencies.

The second user should be your account or a GP admin user. Make sure the password used is strong.

Once installed, select ‘Reboot’. You should disconnect the ISO at some point. Once the server has booted, you will be greeted with the CLI login screen:

At this point, the installation is complete! Login with your admin user and proceed to configuration.

Tomorrows blog entry will deal with configuration.