A response to "James Hansen's many and varied furphies" by John Happs

The following is a critique of the article "James Hansen's many and varied furphies" by John Happs as published in Quadrant on 26 August 2013 from a Critical Thinking perspective. I will state at the outset that I regard Quadrant to be a reich-wing rag that barely deserves the title "magazine". It is produced as propaganda for extreme right wing political viewpoints and openly states its position as "sceptical of 'unthinking Leftism, or political correctness, and its "smelly little orthodoxies"'. The original article may be found here.

Rather than tackle the issues of climate change, Happs has chosen a time-honored practice of avoiding a discussion of facts and engaging in a smear campaign against a respected scientist. 

"Hansen has given numerous public talks about what he sees as an urgent need to reduce greenhouse gas emissions by stopping the mining and burning of coal. He also wants oil exploration stopped, as well as the further exploitation of tar sands." 

Without discussing the merits here, the statement is factually incorrect. The impression is given that Hansen is a radical that wants to completely ban oil and gas exploration. Hansen's opposition has very focused at environmentally sensitive areas such as the Amazonian rainforests and Arctic exploration. His similar concerns about the tar sands in Canada are based on the readily available evidence of the damage to the local environment. 

Happs is attempting to create a straw-man argument: painting a picture that is simply not realistic, unless of course Happs thinks that anyone who doesn't approve of the obliteration of Amazonian rainforests is a loony radical. 

"He unashamedly promotes alarmism about the trivial levels of atmospheric carbon dioxide" 

This is the fallacy of "Conclusion entered as evidence".  

The way you identify propaganda from a well-argued position is the former starts with the conclusion. The latter reports facts, makes inferences based upon syllogistic arguments which lead to a conclusion. From there it is simply a matter of establishing if the facts are true and if the argument is sound. A sound argument is such that if the facts or true then the conclusion must also be true. A strong (or cogent) argument is such that if the facts are true then the conclusion is probably true. 

In this article you see none of these things, as I will demonstrate. 

The adjective "unashamedly" is judgemental. It assumes the person has something to be ashamed of. 

"Promotes" denigrates the fact that Hansen is an expert (and conversely that Hall isn't). 

"Alarmism" is another judgement. Used together we have a fallacy known as a "thought stopping cliche". It means nothing and serves only to create a negative impression. 

Also in the same sentence "trivial levels of carbon dioxide". This is "facts not entered into evidence". This hasn't been demonstrated, and in fact, cannot be as this is one of the most provable evidences demonstrating the reality of climate change. 

"In 2012 Robert Bryce quoted Hansen in the Dallas News: “The trains carrying coal to power plants are death trains. Coal fired power plants are factories of death.” " 

This one is truly brilliant! Notice that he doesn't say that Hansen said this, he says that "Robert Bryce quoted Hansen". Interesting! Why did he do this rather than simply quote Hansen and provide a reference? 

The original words were written in this article from the Guardian. Of course, if Happs had referred to the original article - and someone checked it out - they might actually read him in context: 

"The trains carrying coal to power plants are death trains. Coal-fired power plants are factories of death. When I testified against the proposed Kingsnorth power plant, I estimated that in its lifetime it would be responsible for the extermination of about 400 species - its proportionate contribution to the number that would be committed to extinction if carbon dioxide rose another 100 ppm."  

Puts that a little more in perspective here, doesn't it? Read the full article here to see how it all fits in contextually.

Again, the purpose here is to smear Hansen and make him look like an activist first and a scientist second - leading us up to the fallacious sucker punch of them all: 

"Hansen sees himself as an activist and a scientist, but he can’t be both and remain credible." 

This of course, is a ridiculous statement. History is full of activists that were professionals in their field. In fact, most of them are activists precisely because they know they are talking about. According to Happs, a "good" scientist is not allowed to have an opinion. "Good" scientists stay in their boxes and report their findings to their vastly more intelligent "masters" who are permitted to have opinions.  

"Any scientist with a pet hypothesis who selects only data which support that hypothesis, whilst ignoring conflicting data, must lose the respect of their scientific colleagues." 

Happs is assuming here that any scientist with an opinion cannot be objective and will of course commit scientific fraud. Happs offers no evidence that Hansen has EVER done this, he is simply saying it is ipso facto. In other words, Hansen's scientific results are a direct result of his prejudiced opinions rather than the other way around. Presumably as well, Happs places all climate scientists in this box, as almost to a person, they agree with Hansen's results. 

"Real science actually looks for refutation whereas pseudoscience is intolerant of dissent." 

Well, yes, this true. However the pseudoscience is with the deniers. That's because the "scientists" say that climate change is real. The deniers are not scientists and therefore - by definition - "pseudoscientists", and Happs is one of them.  

The article is quite long and continues to cherry-pick results and apply statements that were never intended to be applied to those statements. Happs is not looking at the holistic science, he is picking and choosing and committing every single sin he subscribes to Hansen: He isn't looking for facts, he is starting from his prejudiced conclusion and working backwards from there. This is called "conclusion shopping". 

This article is an A1 demonstration of yellow journalism in action. Happs' dishonest portrayal of Hansen and the science of Climate Change is staggering in its tenacious mendacity. Godwin's Law precludes me from making the obvious comparisons that scream from the pages of Quadrant. I'd say that the article was a discredit to the journal it was published in, but that would only serve to elevate Quadrant to a level it doesn't deserve.

CentOS 7 post installation script

I like automation. Anything I can do to speed up repetitious tasks is worth the effort. The following shell script is a work-in-progress. Each time I run up a server, I use this script to automate the post installation tasks. I just uncomment the sections I need and run the script. Parts of it take some time and require some human intervention, but usually that just involves pressing 'enter' a few times. The script doesn't do much in the way of hardening the server - that's the stuff of another blog entry.

To use the script, cut and paste it into an editor, make it executable and run it as root. Unfortunately sudo doesn't cut it for some steps. Make sure you uncomment the sections you need first. There are modifications in there for RHEL if you're using that instead of CentOS. I have created a version for CentOS 8, but the it now EOL at the end of this year, I figure it's a little pointless. If anyone wants it, just ask and I'll make it available.

Briefly, the script does the following things:

1. Runs a yum update using deltas
2. Adds the epel repository and optionally the remi repository
3. Installs selinux utilities
4. Installs PERL, CPAN and useful utilities for PERL.
5. Installs Apache or NGINX
6. Installs php for apache
7. Optional upgrade for php 5.6 
8. Installs webmin
9. Installs vsftpd
10. Installs and secures mysqld
11. Installs vmtools
12. Installs miscellaneous utilities
13. Configure to remove old kernels automatically
14. Setup email forwarding
15. Setup and configure WordPress

As I runup more servers with different services, I will update and expand the script. I'm currently writing a GUI front end for the script in Xojo (see below) to customise and generate the script for any environment. If it works, I hope to expand it to include the ability to deploy the script remotely as well as generate it.

 

#!/bin/bash
## The following commands increase the number of packages available
##
yum -y update
yum -y install deltarpm
yum -y install epel-release
## Uncomment for REMI
# rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
# yum --enablerepo=remi,remi-test

yum repolist
yum -y update

## RHEL instructions
# wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm
# rpm -ivh epel-release-7-10.noarch.rpm
# rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
# yum --enablerepo=remi,remi-test
# yum repolist
# yum -y update

## Install SELinux utilities
##
yum -y install policycoreutils policycoreutils-python selinux-policy selinux-policy-targeted libselinux-utils setroubleshoot-server setools setools-console mcstrans

## Install PERL and useful modules and utilities via cpan and cpanm
##
yum -y install perl perl-Net-SSLeay perl-IO-Zlib openssl perl-IO-Tty cpan
cpan App::cpanminus
cpanm Net::FTPSSL
cpanm App::cpanoutdated
cpan-outdated -p | cpanm
cpan-outdated -p | cpanm

## Install and start Apache Web server.
##
yum -y install httpd
#<change httpd.conf listen to 0.0.0.0:80>
systemctl enable httpd.service
systemctl start  httpd.service
systemctl status httpd.service
firewall-cmd --zone=public --permanent --add-service={http,https}
systemctl restart firewalld.service
firewall-cmd --list-all
apachectl graceful
apachectl configtest

## Install NGINX
##
# echo -e "[nginx]\nname=nginx repo\nbaseurl=http://nginx.org/packages/centos/$releasever/$basearch/\ngpgcheck=0\nenabled=1" > /etc/yum.repos.d/nginx.repo
# yum -y install nginx php php-common php-fpm
# firewall-cmd --zone=public --permanent --add-service={http,https}
# systemctl stop httpd
# systemctl disable httpd
# systemctl enable nginx
# systemctl enable php-fpm
# systemctl start nginx
# systemctl start php-fpm
# mkdir -p /srv/www/<website>/public_html
# mkdir /srv/www/<website>/logs
# chown -R nginx:nginx /srv/www/<website>

## Install PHP for Apache, mysql and MSSQL
##
yum -y install php php-pear  php-mysql
## Enable the following as required
# yum -y install php-pgsql php-odbc php-mssql php-ldap php-soap
# yum -y install phpmyadmin httpd-tools
# cp /etc/phpMyAdmin/config.inc.php /etc/phpMyAdmin/config.inc.php.orig
# Harden PHP
# cat /etc/phpMyAdmin/config.inc.php.orig | sed -e 's/AllowRoot\'\]\ \=\ TRUE/AllowRoot\'\]\ \=\ FALSE/g' > /etc/phpMyAdmin/config.inc.php

## Test with http://server/test.php
## Test with http://server/phpMyAdmin

## Upgrade to PHP 5.6 (optional)
##
#rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
#yum repolist
#yum -y remove php-common
#yum -y update
#yum -y install php56w php56w-opcache php56w-xml php56w-mcrypt php56w-gd php56w-devel php56w-mysql php56w-intl #php56w-mbstring
#systemctl restart httpd.service
#php -v

systemctl restart httpd.service
echo -e "<?php phpinfo(); ?>" > /var/www/html/info.php
systemctl restart httpd.service

## Install and run Webmin administration (requires PERL)
##
echo -e "[Webmin]\nname=Webmin Distribution Neutral\nbaseurl=http://download.webmin.com/download/yum\nenabled=1" > /etc/yum.repos.d/webmin.repo
rpm --import http://www.webmin.com/jcameron-key.asc
yum -y install webmin
systemctl enable webmin.service
systemctl start  webmin.service
systemctl status webmin.service
firewall-cmd --zone=public --permanent --add-port=10000/tcp
firewall-cmd --reload
firewall-cmd --list-all

## Install and start VSFTPD
##
#yum -y install vsftpd ftp
#systemctl enable vsftpd.service
#systemctl start vsftpd.service
#firewall-cmd --zone=public --permanent --add-service=ftp
#systemctl restart firewalld.service
#firewall-cmd --list-all

## Install, secure and run MySQL
##
#yum -y install mariadb-server mariadb
#systemctl start mariadb
#mysql_secure_installation
#systemctl enable mariadb.service
#systemctl status mariadb.service

## Install tools for vmware
##
yum install open-vm-tools
vmware-toolbox-cmd -v

## Install Misc utilities as desired
##
yum -y install wget telnet bind-utils nmap mlocate mc
yum -y install elinks

## Remove old kernels and set yum to preserve only two. Setup auto updates
##
yum -y install yum-utils yum-cron
package-cleanup -y --oldkernels --count=2
echo -e "\ninstallonly_limit=2\n" >> /etc/yum.conf
#yum -y install yum-cron # edit /etc/yum/yum-cron.conf or do this via webmin instead on yum-cron
#systemctl start yum-cron
#systemctl enable yum-cron

## Setup email forwarding
##
yum -y install mailx
ln -s /bin/mailx /bin/email
echo -e "set smtp=smtp://<mail forwarding ip address> " >> /etc/mail.rc


## Install Wordpress
#yum -y install php-gd
#systemctl restart httpd.service
#wget http://wordpress.org/latest.tar.gz
#tar xzvf latest.tar.gz
#rsync -avP ~/wordpress/ /var/www/html/
#mkdir /var/www/html/wp-content/uploads
#chown -R apache:apache /var/www/html/*


## Setup WordPress Database
#mysql -u root -p <password>
#CREATE DATABASE wordpress;
#CREATE USER wordpressuser@localhost IDENTIFIED BY 'password'
#GRANT ALL PRIVILEGES ON wordpress.* TO wordpressuser@localhost IDENTIFIED BY 'password';
#FLUSH PRIVILEGES;
#exit

## Configure WordPress
#cd /var/www/html
#cat wp-config-sample.php | sed 's/database_name_here/wordpress/g' | sed 's/username_here/wordpressuser/g' | sed 's/password_here/password/g' > wp-config.php

COVID-19: AstraZeneca Vaccine and Blood Clots

There's a huge amount of misinformation floating around about COVID-19 and in particular, the AstraZeneca vaccine and the issue of blood clots. Media reporting is partly to blame for simultaneously sensationalising and glossing over important information. So I thought it was about time to present the facts of the issue. I will source all statements made, if I miss anything out - let me know.

I'll add a caveat here: I'm not a medical expert. Don't get medical advice from social media or even blogs like mine. Go to your doctor. What I've done is glean what I can from the information that is publicly available. I may be wrong on many points, so apply a good deal of critical thinking to what I've written here as you should to any information that is presented to you: particularly if it claims to be authoritative.

COVID-19 Vaccines in general

There are currently four vaccines that are authourised for use in preventing the SARS-CoV-2 infections. Technically, they are in phase IV global post-licensure surveillance. They are:

• Pzifer-BioNTech (mRNA)
  
• Moderna (mRNA)
  
• J&J/Janssen (viral vector)
  
• Oxford/AstraZeneca (viral vector)
  

Other vaccines are in different stages of development. Most of these will not make it into public release. Now that there are already vaccines in production and distribution, any vaccines further developed will have to demonstrate they are better than those currently available. Not listed is the Chinese based Sinovac vaccine called CoronaVac and the Russian Gamaleya (Sputnik V) vaccine. Both of these vaccines use traditional production techniques making them cheap to produce but they have poor effectiveness profiles. Many third world countries have opted to use them, but not a single western nation has.

The numbers change daily. You can follow the progress of vaccine candidates here. For those who claim the vaccines haven't been tested, download the daily R&D blueprint and check any trial link you link. Thousands of clinical trials for safety & effectiveness have been and are being conducted.

Oxford-AstraZeneca (AZ) Vaccine

The method used to provoke an immune response is called "viral vector'. It uses a modified version of a different virus - called the vector. This vector is injected intramuscularly and enters the cells of the muscle. From there, the vector manufactures a spike protein identical to that found uniquely on the surface of the SARS-CoV-2 virus that causes COVID-19. The cell then displays the protein on its surface. Our immune system recognises it shouldn't be there and triggers an immune response targeted at the spike protein. Since the body is fighting an 'infection', we have the usual symptoms that accompany an immune response. This includes injection site pain, fever, lethargy, feeling 'sick' etc. However these are immune response reactions, not an actual disease. Different people will have different responses. source

At the end of the process, our bodies have learnt how to protect us against this infection. Effectiveness after two vaccines vary according to the variant encountered. Certainly, this vaccine is much less effective than the Pfzer vaccines (>90%). However, best data to date shows at worst it is 61.7% effective (UK B117 variant) and at best it is 81.5%, with the average being 77.3%. source

A recently published paper in The Lancet is a little more critical, estimating the effectiveness at 67%.

Blood Clots

Once the AZ vaccine was being administered globally, reports started emerging of blood clots at an significant uptick over the background rate. In about 1 in 4 of these, death was the result.

It's important to clarify something here as a sidebar. When you administer a medication to millions of people, large numbers will have 'something' medical happen to them over the month following the administration of the medication. People will have heart attacks, strokes, embolisms etc. that have nothing to do with the administration of the medication. However we do have a good idea of the general rate at which these events occur. In fact, your life insurance company can tell you the percentage chance of any medical event happening to you over the next year. It's quite a well studied science. This is why all adverse events that happen following vaccination are reported. This reporting is mandatory and the reports are publicly available and searchable. In Australia, it is the DAEN (Database of Adverse Events Notification). In the United states it is VAERS (Vaccine Adverse Event Reporting System). Researchers compare adverse events with the rate that normally occurs. When there is an 'uptick' in an event, it is investigated. In the case of serious events, they are all investigated thoroughly as a public health measure.

At first, since the number was very low, there was skepticism that the AZ vaccine was the cause as vaccines have never been known to cause blood clots. Vaccines are injected intramuscularly not intravenously and have not direct connection with blood.

From the investigations, however, the AZ vaccine has been identified as the likely cause in over 100 cases. From these numbers, it is likely the incidence of a blood clot from the AZ vaccine is about 1 in 1 million. The phase III clinical trials involved 20,000 people who received the vaccine. Blood clots were not observed in any of the participants.

UPDATE 30 Sep 2021: We have a lot more data on the incidence. With increased surveillance for TTS, including very minor instances (headaches, thigh soreness etc) there have been 125 reported cases in Australia out of 9.6 million vaccinations. This makes the chance of a blood clot 1 in 76,800. Now, that's a lot higher than one in a million, but remember, these include minor cases. Of those 125 people, 8 people died, six of them women. This is a mortality rate of 1 in 1.2 million. Certainly much better than catching COVID, but since there is an alternative, if you are female and under 50, definitely don't get the AZ vaccine, get the Pfizer or Moderna instead. See  ‘Weighing up the potential benefits against risk of harm from COVID-19 Vaccine AstraZeneca’

source

How can the AstraZeneca vaccine cause blood clots?

So far, experts do not know, however there are some conjectures. Foremost amongst these is that in rare cases, the vaccine also induces and immune reaction against a protein found in blood platelets. On the sniff factor, this sounds (at least) like a good explanation. 

Patients with blood clots were found to have antibodies against Platelet Factor 4 (PF4) which is a symptom of heparin-induced thrombocytopenia. So it may be the case that those patients have this disease already, but it is undiagnosed and the vaccine triggers this response. People are only tested for this if they present to hospital with a blood clot. Blanket testing of people would be pointless because of false positives according to Baye's Theorem.

However, everything is still conjecture at the moment. The reality is that we don't know if it is the adenovirus (the vector) or the spike protein is linked to the clots.

What do we know?

In simplistic bullet point form, this is everything we do know:

• It occurs at a rate of approximately 1 in 76,800 1 million people
  
• It mostly affects women under 55
• Heparin-induced thrombocytopenia is more common in women than men and less likely to be diagnosed at a young age.
• Blood clots are a symptom of COVID-19. You are far more likely to get a blood clot from COVID-19 than from the AZ vaccine.
• To put the risk into comparison, the risk of DVT from flying longer than four hours is 0.079% or 1 in 1265. The risk of a blood clot from taking oral contraceptives is around 0.04% or 1 in 2500.

So, is it safe?

Relative to what?

Taking the AZ vaccines is certainly safer than getting COVID-19 by several orders of magnitude. Millions of people have died from COVID-19 compared to the 25 people that have died probably because of the AstraZeneca vaccine. In addition, antivaxers are lying about adverse reactions by posting fake testimonials to social media. People inundated with hundreds of personal stories may find it compelling, but the reality is that this vaccine is still far safer than not only most other medical interventions, but safer than most daily activities such as driving and eating.

It's only natural to be concerned about risks we hear about in the media regularly or on social media, particularly for activities we don't regularly undertake. But in reality, if you are concerned about the risk of taking the AZ vaccine: Don't fly in a plane, don't drive a car and don't take paracetamol. All of these have higher risk profiles by several orders of magnitude.

However, if we are looking at the risk of taking the AZ vaccine compared to the Pfizer vaccine, then yes. Take the Pfizer vaccine anytime. The Pfizer vaccine will knock you around more - many people need a day or two off work after having it. But its effectiveness is MUCH higher and serious adverse event reports do not statistically exceed the background rate (despite what you facebook group might say).

Achieving Herd Immunity

Besides protecting the individual, the purpose of vaccines is to reduce the R(Eff) rate below 1. For COVID-19, R0=2.4. Herd Immunity (s) = 1-1/R0 = 58.33%.

This means that we need to vaccinate 87% of the population with the AZ vaccine, or 62% of the population with the Pfizer vaccine. However, the higher the rate the better. Anything below these numbers will lead to outbreaks. Leaving children out of the equation (no COVID-19 vaccine has been certified for paediatric use) then it is impossible to achieve herd immunity with the AZ vaccine.

Broken YUM on CentOS 7 (or how I learned to love DNF)

It's a simple thing to keep your system updated. A quick yum update every few days - just like brushing your teeth. Completely painless, quick, efficient.

Not today it wasnt.

---> Package screen.x86_64 0:4.1.0-0.26.20120314git3c2946.el7 will be updated
---> Package screen.x86_64 0:4.1.0-0.27.20120314git3c2946.el7_9 will be an update
---> Package skypeforlinux.x86_64 0:8.67.0.96-1 will be updated
---> Package skypeforlinux.x86_64 0:8.71.0.36-1 will be an update
Error: Invalid version flag: or

This was the error I received. Okay, so when yum update fails, there's always the tried and true command sequence that fixes it:

sudo yum clean all
sudo yum update --skip-broken

This time it didn"t work. I still received the same error. Time to google for more info... 

I found a kool command sequence on John S. De Stefano's blog that looked promising:

sudo yum check all                # tells you of any problems
sudo package-cleanup --problems   # lists all known package problems
sudo package-cleanup --dupes      # lists duplicate packages
sudo package-cleanup --cleandupes # actually cleans up duplicates
sudo yum check all                # run again to check for remaining problems
sudo yum-complete-transaction --cleanup-only

However, this command sequence failed to fix the issue too. Looks like I'm going to have to work out what's broken and why. No easy way out with this problem.

Rich Dependencies

Scrolling through bugzilla, I found the following entry:

There was a mistake made in the rpmlib() dep for rich deps. You need
at least rpm 4.13 for the base rich deps, and rpm 4.13.1 for the rest.

yum and related packages are no longer actively developed.
They are being replaced with dnf, dnf-utils, etc.

I'm closing this bug because it's most likely never going to be fixed.
If you still consider your bug report important, reopen it, please.

https://bugzilla.redhat.com/show_bug.cgi?id=1578942

This was actually a bug report for F28+. Since Fedora uses DNF primarily and YUM is deprecated, no one seemed particularly interested in fixing it in Fedora. RHEL/CentOS 8 both use DNF as well. Could this bug have percolated down into CentOS 7? Time to check rpm versions and make sure I have at least 4.13:

$ yum --showduplicate list rpm

Installed Packages
rpm.x86_64                          4.11.3-45.el7                          @base
Available Packages
rpm.x86_64                          4.11.3-45.el7                          base 

Well, that's just peachy!

I'm running an old version of rpm that doesn't support rich dependencies. It also appears that yum may not handle them well either - although the indications are the problem really lies with rpmlib().

I've never really spent a lot of time looking at the inner workings of package managers and their respective update managers. Now that ignorance is coming back to haunt me and it's time for some old fashioned studying the matter.

From rpm.org there's an excellent description of how the boolean operators work and how they help avoid dependency hell. This is what is being referenced in the error I received. The 'or' operator is unknown because my version of rpm is too old. Boolean operators enable what is termed "rich dependencies". Basically providing a logical sequence for resolving dependency issues across multiple versions. A good example is community-mysql and mariadb. Both packages do the same thing - provide a mysql style database. Without rich dependencies, if another package requires mysql, you have to choose which package is required. With rich dependencies you can state:

Package A: Requires: mysql
Package mariadb: Provides: mysql
Package community-mysql: Provides: mysql
Suggests: mariadb to Package A.

Which means that if community-mysql is already installed, that is used as the dependency, otherwise mariadb is installed.

This is really cool, but yum simply ignores it.

Put simply, the root cause of the problem is:

1. RPM supports "rich dependencies"
2. DNF supports resolving packages with "rich dependencies"
3. YUM does not support resolving packages with "rich dependencies"

The solution then is obvious: Since CentOS 7 supports DNF, it's time to switch. Fiddling with yum and manually resolving the dependencies will only delay the inevitable.

YUM vs DNF 

source

"Dandified YUM" or DNF, is the replacement package update manager for RHEL/CentOS. It's been part of Fedora for a long time now. I acknowledge it is clearly superior to YUM in most aspects, plus it doesn't suffer from some of the issues that Debian apt does. The major goal is to eliminate (where possible) dependency hell. But it also has other advantages.

It was also designed to be as drop-in replaceable to yum as possible. It comes very close, but some commands have no equivalent (some of these are deliberate actions). So, for me, if your used to using yum, dnf just represents yet another sequence of commands that must be memorised just to continue doing your job.

I'm not going to bore you with lists here of features, commands, comparisons etc. The links I've provided do that well enough. Plus, if you want a deep dive into dnf, you can go here. Suffice it to say that I decided that installing and using dnf was the simplest and most effect potential solution to the immediate and potentially long term problems.

Install DNF on CentOS7

Pretty simple really, however there are a number of dependencies since it leverages by Py2 and Py3. A total of 11 dependent packages were installed, however YMMV.

$sudo yum install dnf
...
Resolving Dependencies
--> Running transaction check
---> Package dnf.noarch 0:4.0.9.2-2.el7_9 will be installed
--> Processing Dependency: python2-dnf = 4.0.9.2-2.el7_9 for package: dnf-4.0.9.2-2.el7_9.noarch
--> Running transaction check
---> Package python2-dnf.noarch 0:4.0.9.2-2.el7_9 will be installed
--> Processing Dependency: dnf-data = 4.0.9.2-2.el7_9 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-libdnf >= 0.22.5 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-libcomps >= 0.1.8 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-hawkey >= 0.22.5 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: libmodulemd >= 1.4.0 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-libdnf for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python-enum34 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Running transaction check
---> Package dnf-data.noarch 0:4.0.9.2-2.el7_9 will be installed
---> Package libmodulemd.x86_64 0:1.6.3-1.el7 will be installed
---> Package python-enum34.noarch 0:1.0.4-1.el7 will be installed
---> Package python2-hawkey.x86_64 0:0.22.5-2.el7_9 will be installed
--> Processing Dependency: libdnf(x86-64) = 0.22.5-2.el7_9 for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolvext.so.0(SOLV_1.0)(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolv.so.0(SOLV_1.0)(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolvext.so.0()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolv.so.0()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: librepo.so.0()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libdnf.so.2()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
---> Package python2-libcomps.x86_64 0:0.1.8-14.el7 will be installed
--> Processing Dependency: libcomps(x86-64) = 0.1.8-14.el7 for package: python2-libcomps-0.1.8-14.el7.x86_64
--> Processing Dependency: libcomps.so.0.1.6()(64bit) for package: python2-libcomps-0.1.8-14.el7.x86_64
---> Package python2-libdnf.x86_64 0:0.22.5-2.el7_9 will be installed
--> Running transaction check
---> Package libcomps.x86_64 0:0.1.8-14.el7 will be installed
---> Package libdnf.x86_64 0:0.22.5-2.el7_9 will be installed
---> Package librepo.x86_64 0:1.8.1-8.el7_9 will be installed
---> Package libsolv.x86_64 0:0.6.34-4.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

Next I tried dnf update:

$ sudo dnf update
<snip>
Running transaction check
Error: transaction check vs depsolve:
(libatomic or libatomic1) is needed by skypeforlinux-8.71.0.36-1.x86_64
rpmlib(RichDependencies) <= 4.12.0-1 is needed by skypeforlinux-8.71.0.36-1.x86_64
To diagnose the problem, try running: 'rpm -Va --nofiles --nodigest'.
You probably have corrupted RPMDB, running 'rpm --rebuilddb' might fix the issue.
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.

DNF is smart enough to know why the update failed. The package 'skypeforlinux' uses rich dependencies. These dependencies require a version of rpmlib that is greater than the CentOS 7 repositories can provide. Therefore, the dependencies cannot be resolved. There's a tip there to use rpm directly to reconcile the issues, but since I know that the version of rpm is also too low, that won't work.

I decide that I can live without skypeforlinux, so I remove it and both dnf and yum are happy.

At this point in time, since both work, I can use either to keep my system updated. However, having installed dnf and now that I'm comfortable working with it (having learned the syntax and equivalent commands) I think I'll use it from now on.

How SORBS ruined my life (old blog)

When this blog entry was written, Open Relay blockers like SORBS were considered the primary defence mechanism against spam email. They are still often used, but not as much as they used to be - for many of the reasons discussed in this article. SPF, DKIM & DMARC have largely taken their place.

 

 

I've had a helluva week so far, and I've got SORBS to thank for it.

SORBS (Spam and Open Relay Blocking System) is an email server reputation service. It (along with other reputation services) collect data on servers that are are either poorly configured (and open to abuse) or actively send spam. Reputation services are an excellent way of protecting against spam. They have very high hit rates and (generally) very low false positives. They also have a very low performance overhead compared with heuristic and bayesian filtering techniques. The mail servers I admister are configured to quarantine all email that is listed by either SORBS, SpamHaus or SpamCop (two other reputation services).

However, sometime last week (when I was off work sick), SORBS listed pretty much all of the hotmail, windows live, yahoo, bigpond, optusnet and myplace servers. There may have been others listed as well.

Responding to a few complaints that came in on Monday about email not being received, I began checking. Then began the sinking feeling that goes along with knowing that I'd have to change the email filtering rules, do some regression tests and then resubmit a week's work of spam through the email filters. SORBS is a very aggressive filter and I've been quite reliant on it for some time.

Less fun was trying to explain the problem to the CEO.

The first part of the quick fix was to move the SORBS check to the end of the anti-SPAM rules of the DMZ mail filter and set it to monitor only - not block. The DMZ mail server is exposed to the Internet and performs basic/quick checks only and does not look at the content of the email messages. It is highly robust and is meant as the front line of defence. It experiences an average of 90 security attacks per hour. It filters about 75% of the email traffic as inappropriate before passing the "possibly okay" messages to the second filter.

The second part was to add a check for zero day threats on the DMZ server and tag messages accordingly.

The third part was to tighten the screws a bit on the second mail filter. This meant decreasing the tolerance limits a little and added a few more checks which included looking for the tags on the email messages placed there by the DMZ mail filter. Usually, this server intercepts about 8% of messages pass to it. Messages quarantined by this server may result in a quarantine message and even a self-release option depending upon the spam score the email receives.

Now that these changes were made, I resubmitted 2143 spam quarantined messages. With the new rules in place, 1963 messages were blocked and 180 passed through to the second filter which blocked a further 68 messages leaving 112 messages sailing through to the mail server. I don't know how what the spam:ham ratio of the 112 messages was but the count seemed to be manageable so I released the remaining 14,000 quarantined messages and sent an email instructing people to forward any received spam to the spam submit mailbox for heuristic analysis. So far, I have 35 spam messages submitted by users which (if accurate) we now have a spam hit rate of 99.75% with (hopefully) a close to zero false positive rate. The hit rate has dropped to 59.4% for the DMZ server and risen to 12.5% for the second filter. Time and monitoring will determine how successful the changes have been.

Moving forward, I have written replacement rules that will no longer quarantine email based upon failed reputation. The new rules will look at SPF (Send Policy Framework - RFC4088) in addition to reputation. If the SPF check passes, the email will be accepted. If SPF fails, it will be dropped. If SPF is 'softfail' or 'none' then it will be subject to a reputation check. Any failure will drop then connection with a 46x error - Temporary Failure, with details as to why sending failed. The sending MTA will then notify the recipient that the email failed to send. The sending server then has the option of implementing SPF. I'd like at some point to add a DKIM check, but that's a reasonably difficult task.

BTW, I am heavily influenced by Ming Weng Wong's whitepaper on Messaging Anti-Abuse:

http://www.openspf.org/blobs/sender-authentication-whitepaper.pdf

 

 

The Dream (Creative Writing)

The world around me starts to dissolve into nothing. Two men are begging, pleading with a man in a suit "No! Please he'll forget us."

"He won't forget you. He'll see you again on TV. You will be a science fiction show and you.." he turns to the second man "..will be a soap opera"

"No!" they both shout as they are dragged away.

"Excuse me!" I say

The suited man looks surprised, turning both ways to see if there is anyone else I may be speaking to.

"Yes you" I demanded "I'm talking to you!"

"I'm sorry" he replies "but you shouldn't be able to perceive me, let alone speak to me. Something is obviously wrong with the system at the moment. Are you sick, ill or delirious?” he asks.

"I have a head cold at the moment, but that's beside the point."

"Ah! That must be it, you're delirious - seeing things."

"I'm fine, but who are you?"

"Well, since you've asked, I am your id."

"My what?"

"The anthropomorphic personification of your imagination. I am the one who keeps control of your dreams - tries to help you make sense of them." then his demeanor becomes more sinister "And then eliminates them before you awake."

At that moment I recognise my id as a dead ringer for James Spader.

"You’re serious?"

"Oh yes. Right now you're asleep - probably feverish actually, it's the only explanation for why you can perceive me. Usually you are blissfully unaware of my existence. But come now, dream number three is about to start."

I turn to ask him what he means, but as I do so, I realize I'm on a footpath, somewhere, who knows where. I'm dragging a some pieces of cardboard around and a few meager possessions rolled up in a blanket. I select a good piece of footpath, under cover and begin to lay out the cardboard on the ground. Soon four other men come near to do the same, but they see me there and look disappointed.

"Everything okay?" I ask.

"Sorry, it's just that we usually setup here. We'll find somewhere else."

"There's room for all of us." I say, and they look overjoyed.

They begin to setup and soon it looks like there's no place for me. I turn to walk away and one of them says, ‘Hey your spot is over here, in between all of us. We'll keep you safe tonight in return for letting us stay here.’

I thank them and put down my cardboard, rudimentary mattress and pillow, other possessions and lie down, wrapped in a minke blanket to keep me warm. As soon as I think 'I shouldn't be here' I'm instantly outside of myself, observing. Standing right next to my id. Standing next to him is another-me wrapped up tightly in white linen and bound with blinking Christmas tree lights and struggling to get out. It looks like he's going to succeed.

"What? I though I got rid of you?"

"Who is that?” I ask, pointing to homeless-me.

"Isn't it obvious? That's your fear of failure. It's quite a common theme. Frankly, I find it boring. The feckless self-pity, the meekness rewarded with gratitude. Seriously B grade. I wish he would get shanked once in a while just to add some reality."

"What? You want to see me shanked?"

"Oh no. Not you. Just that perception of yourself. I am you."

"I'm nothing like you!"

"Oh please! Spare me the self-loathing. I do enough of it for the both of us. You revel so much in your analytical capabilities and your critical thinking that every night I have to cleanse your soul of the monsters you unleash upon your psyche. If it wasn't for me you'd wake up a gibbering mess that needs to be spoon fed."

"May the shades of Jung and Freud haunt you forever."

"Plagiarist! You got that from Zelazny. Remember: everything you know, I know. But come, it's time for Christmas dinner." he finishes as the other-me successfully escapes from his bonds.

I turn around and realise I'm in a buffet of some kind. I offer my plate to the server and receive a slice of roast ham. Further down the line, there's turkey, cranberry sauce, duck and other foods typically found at Christmas. I realise I am moving slowly and my skin is wrinkled. I am old. I make my way to a large table full of people I know: my kids - all adults now - with their wives and children. I sit down at the table and a small girl climbs onto my lap and kisses my cheek and looks at me with a sparkle in her eye.

"Merry Christmas Grandpa! I love you."

A wave of bittersweet happiness washes over me as I look at the empty seat next to me. As I do so I think again 'I shouldn't be here' and once more I am an observer, standing next to my id.

"I can't seem to keep you in one place right now, can I? It's a pity, I like this one, however you need to do something about that empty chair."

"Why is it empty?" I ask "Did someone die."

"No. Someone never happened. Your future-me will always be like that as long as you keep feeding homeless-me. However, I no longer have any time to indulge you, I have work to do."

In an instant we are on top of a ten-story building. It looks like the outside portion of a luxurious penthouse apartment. I instantly know my id lives there. There are three representations of me standing on the ledge, all pleading for their lives. They notice me.

"Don't let him do it!"

"Who are they? What parts of me do they represent?"

"Destructive influences. You're better off without them."

"Who gets to decide that? You?"

"Why yes, as a matter of fact. But more to the point *you* did, by giving that power to me."

"I don't remember doing that!"

"Of course you don't. You didn't want the responsibility. However I tell you what, I'll let them decide."

My id pulls out a handgun.

"I'll make all of you a one-time deal. Trust me, have faith and jump. You have my word that you will be unharmed. Hesitate, and I will shoot you in the head. You have three seconds. One...two...."

All three of them jump. Seconds later there is a sickening thud. I rush to the side and peer over. There, lying on the ground are the bodies of my three doppelgängers.

"You said they'd be unharmed!"

"I lied. Time to go."

Instantly I find I'm sitting on a recliner watching TV with what feels like a heavy, soft warm blanket over me. I have comfort food at my disposal: cheeses, olives, salmon, chocolates that I am constantly eating. The room I'm in is a large one and there's a constant stream of people entering the room at the rear and muttering their disapproval of the situation. My youngest son is beside me also watching. I'm on TV, telling jokes - all ones I've told before. Comical-me has a TV audience, all laughing at every single joke. I start thinking this is simply a comedy show, but the jokes are intertwined around a complex melodramatic love triangle. I can tell this is going to end badly for comical-me so I change channels. I end up with an explicit porno playing. Since my son is watching beside me I try to change channels but I can't. This is when I notice the blanket on top of me is actually partly a naked woman and what I thought was the remote control was actually one of the blanket-woman's breasts that I'm squeezing.

"Don't stop!" the blanket-woman moans. Instantly I'm an observer again.

Blanket-woman looks at me and says "Great! A threesome!"

I find the remote and quickly change the channel. It's a televangelist channel and I am the preacher.

"Good choice!" says preacher-me to real-me. He then points an accusing finger at lustful-me "You need to repent!"

I change the channel again, this time it's a an episode of Star Trek. I'm one of the engineers. The engines are about to explode, but engineer-me grabs some tools and climbs into the bowels of the engine room saying "Don't worry. I can fix anything!"

I walk to the back of the room where my id is congregating with the disapproving crowd.

"Had enough yet?" he asks smugly.

"I'm still trying to make sense of it all."

"Give up. You'll go crazy." He leads me out of the room into a featureless wasteland. "And this, I'm afraid, is where we part company. You're about to wake up and within thirty minutes, you won't remember a thing. Subconscious memory doesn't translate to long-term memory - as you know."

"No, I don't think so. I've seen enough to know I don't want you in charge."
"That's preposterous! I AM you. Remember? I am simply a representation of your id. I'm the one who keeps the nightmares at bay. Without me, you wouldn't be functional from day to day."

"And yet, strolling through my dreams here, amongst the random collections of my Jungian archetypes, there's one thing I've noticed."

"And what's that? That they all look like you?" he sneers.

"Almost. They all look like me EXCEPT for you."

The color drains from his face as the featureless background starts to dissolve, slowly becoming the Christmas cafeteria scene from earlier.

"You know, I'm starting to think about this, about how it all fits together, and what your place in this is. More importantly, how did you get to be in charge here."

My id retorts uncertainly "I told you, *you* did.”

“Yes, I know. And yet something’s been bothering me. My id is supposed to be the primal, uncontrolled part of myself. That’s not you, that’s why you don’t look like me at all. You’re not my id, you’re my Super-Ego.”

As I say the words, my fake id deflates a little and the Christmas cafeteria fully appears. Seated at the table are all the representations of me. The empty chair beside mine is still empty. I wave my hand at the table.

“These are the various manifestations of my id...including you”

“I though you said I was your super-ego.” He sneered.

“That’s a part you play, as the ruler archetype. You’re both: my super-ego as I imagine you to be. An idealised version of myself that I can never achieve.”

“You’re making a mistake - can’t you see that?”

“The mistake I make is in suppressing my realities. I *need* my fear of failure - it keeps me from making stupid decisions.”

My super-ego is noticeably shorter now - and younger.

“I’m just happy to eat the crumbs from your table.” responds homeless-me. Lustful-me (complete with blanket-woman) is already gorging himself on food.

"Oh shut up!" retorts my super-ego.

"I'm the sum total of all my fears and dreams - positive and negative."

"Well, I know when I'm not wanted. I guess you want to go and shoot myself then!"

"That's where you're wrong. See the empty seat at the table? Well, that's yours. I do need to be reminded from time to time that I have responsibilities and obligations outside of myself. However, I also have an obligation to myself as well."

Almost with tears in his eyes my super-ego sits down. The next time I look at him I see myself - my twelve-year old child self. I realize I am being judged by the expectations that I made of myself before I became an adult. He looks up at me and says "You won't win. I told you, you'll forget all about this within thirty minutes."

I smile back and say "Not if I write this down as soon as I awake."

As the scene fades to wakefulness, I enjoy watching all of my archetypes arguing with each other over Christmas dinner.

"Time to write." I say to myself.

Why is Backup so hard?

Performing a backup never used to be hard, in fact, it used to be the easiest job a sysadmin could perform. So, the ever present question many sysadmins end up asking is: Why is something that used to be so easy, now so difficult? The next equally valid question is:

Why is it also so EXPENSIVE?

As with most things, we are dealing with a history lesson.

In the early days of mainframes, you dealt with a monolithic system. Everything (and I mean everything!) came from a single vendor - and that included a backup system.

How the backup system actually worked was never much of a question. It just worked OOTB (Out Of The Box). Despite this, there was a reasonable degree of busy work involved. In fact, the primary job of the sysadmin was backups.

Consider the situation: You have several rows of IBM 72x TBUs all purring away madly. Now the tape software may know what tape it is writing to, but these are reel-reel tapes. There are no barcode labels to be read and no link between the media header on the tape and what you write on the tape (and it's box). You have to be super organised. A bell (literally a bell!) will go off on the operator console, you read the message to remove tape xyz01 from TBU 23 and replace it with tape qwe12 etc. So you run around like a blue-assed fly to make sure the tapes are properly stored in sequence and you know where all the catalog tapes are just in case the storage decides to die - a not uncommon occurrence.

In fact, in days of very expensive storage, tape was considered the mainstay. Several units were often dedicated for "offline" and "standby" storage. The only difference between the two was that standby storage was on tapes that hadn't been removed yet. Often the TBUs would double up - standby during the day and backup operation at night. Tape was cheap, disk was expensive.

The popularity of UNIX destroyed proprietary backup systems. On a UNIX system, a TBU was just another device to be written to or read from. The 'tar' utility (Tape ARchiver) turned one an entire subdirectory structure into a single file for writing (or reading from) a tape device. For raw block level backups of a disk volume, "dump" and "restore" were popular from early days of UNIX. All of these commands were frequently used in script. Sysadmins developed their own scripts and cronjobs. Usually, very little effort was put into documenting these scripts. The sysadmin 'knew' what was going on - and that was all that mattered.

As the desktop computer began pushing into the business computer market, initially little thought was put towards backups. You made copies of floppies and if you were lucky enough to have a hard disk drive, you just manually copied files onto floppy disk. Some early backup systems began to appear like Fastback which offered full duplex support and compression.

As the size of desktop computer HDDs increased, the ability to 'back up to floppy' dropped off considerably. Once again, proprietary backup systems began to appear. One of these was the 4mm Colorado Jumbo, which was an inexpensive TBU that connected to the FDD interface - meaning a separate controller did not need to be purchased. Colorado bundled some backup software - but it wasn't particularly good.

Around this time, Novell Netware 2 and later Netware 3.12 was increasing in popularity. Vendors would sell Netware 5 user version and install it on a glorified workstation. It was common to install either a Colorado Jumbo TBU or a more expensive QIC TBU. The latter would require a SCSI card. 

However, Netware didn't have a native backup software.

About the only backup software that was available for Netware was ArcServe. 

It was horrible.

Sure, it wasn't too bad at backing up your files. It just a hard time restoring them. It also had a habit of crashing servers. On Netware this was called an ABEND. Pretty soon, people the idea of setting up a dedicated backup server, so if the backup server crashed, it wasn't too bad. People also found that particular SCSI cards and TBUs caused Arcserve to be more stable than others. Particular servers were also kinder. Unfortunately, this "stability" came at a price.

Arcserve wasn't too expensive - which was its only real saving grace. The other was the complete lack of any alternative. When Windows NT came out, Arcserve made a version for it - which managed to port all the same stability problems to the new platform.

Then came Backup Exec. It was a breath of fresh air! Simple. Stable. Affordable.

Okay, it was more expensive, but you didn't need a dedicated backup server. In fact, there was a cut down version that came with a SCSI card and a Colorado Travan TBU. It all worked out of the box for under $500! Overnight, pretty much everyone switched to Backup Exec.

Successive versions added more options: open file, DR, Groupwise, BTree, NDS, Windows, Unix, Linux. However the stability gradually fell away. It was still good, but the dedicated backup server was resurrected. The price increased too. It doubled, then tripled. Soon, the best thing you could really say about Backup Exec was that it was better than ArcServe - which by now had been purchased by Computer Associates and was called ArcserveIT.

Veritas (who owned Backup exec) spawned another backup product very similar to Backup Exec - Netbackup. It became the standard for heterogeneous backup. To celebrate this, they added a zero on the price tag.

Netbackup Management Console

Then came a sequence of enterprise backup products. They were all better than Backup Exec. More stable. Heterogeneous. Great support. Agents for everything you could imagine. Policy based. But by this time, they cost in the tens of thousands of dollars.

They were Syncsort, CommVault, HP Data Protector, Portlock. No sooner did one product come out, but another did that was better with extra features. 

Then the game changer of them all came out: Veeam.

It was like Backup Exec all over again. Relatively cheap. designed primarily for virtual machines, it could do what all the others struggled with: Restore a complete working server in minimal time. 

Pretty soon, Veeam became the dominant backup software. It was a little feature poor at first, but you could do cloud based backup - meaning you didn't need to buy expensive tapes or TBUs. Veeam charged very little for cloud storage.

Slowly, as successive versions of Veeam came out, features were added. However costs began to go up again and reliability and stability began to drop. Veeam also started increasing the cost of cloud restore operations - so whilst backing up to the cloud was cheap, restoring from it cost a fortune!

Now, organisations using Veeam are casting around for alternatives. There's  EMC Networker, Altaro, Nakivo plus there are appliance based systems like Datto that work on a different paradigm. They all have one thing in common:

They are very expensive.

The goal seems to be to create hassle free simple backups. Over time, software companies sem to forget that and overload their software with Netfeatures few people use.

It seems to me the solution is to create a two tier system: the simple software that always works and the heterogenous one that deals with all of the weird and wonderful situations. 

That sounds like a simple solution, unfortunately it's always the low cost product that makes the profits and drives innovation. And nobody wants to run two pieces of software. This means the larger business that pay the most for your product, don't use the cheaper versions. When Backup Exec came out, all the large organisations stuck with ArcServe. The same is true of Veeam. Netbackup continues to hold sway in larger organisations. A quick persual of the Netbackup support forums gives the distinct impression that those who administer NetBackup have only one job to do: Backup support. If that's your only job, you don't really care that much if it's difficult to administer and requires arcane knowledge and sophisticated scripting skills. It also doesn't matter that it's hyper-expensive. In fact, that expense is a good thing as it masks you salary in the TCO.

So, I fear history is doomed to continually repeat itself. Think about that the next time a new piece of backup software appears on the market that seems too good to be true.