Saturday, 31 July 2021

Wayne's Deadly Toblerone Cheesecake

Toblerone Cheesecake

Firstly, make your favourite biscuit base and place it in a shallow, springform cake dish that is well greased and chill. You can cheat and use a prepared crust if you like. As for me, I use crushed Maria biscuits and butter.

Ingredients: (metric - 1 tablespoon = 40ml, 1 cup = 250 ml)
750g Philadelphia cream cheese (3 blocks) - one can be light Philly if you're a wimp
2 eggs
1 tablespoon of cocoa
1 tablespoon of white rice flour (or plain flour)
1 tablespoon of butter
1 x 100g Toblerone (preferably dark Toblerone)
3/4 metric cup castor sugar
condensed milk
liquid pouring cream (to serve)

Method:
1. Beat cream cheese and sugar together, add some condensed milk to help soften and to add to flavour. Generally no more than 1/4 tin of condensed milk should be added.
2. Melt Toblerone in the microwave with the butter, be careful not to boil it. Use small steps and mix with the butter until smooth and not split. The butter is optional but does aid in the melting and binding and reduces the risk of cracking. The less the better as the more you use the higher the chance of splitting the chocolate.
3. Add the melted Toblerone, flour and cocoa to the cream cheese and mix well. Then add the eggs and mix again.
4. Pour into prepared crust and bake at 160 degrees celsius for 60 minutes.
5. When cooked it should still have a slight wobble in the middle. Leave in oven to cool to avoid cracking. Remove from oven and chill overnight.
6. When serving dust with icing sugar or top with grated Toblerone.

tips:
1 Soften Cream Cheese.
2 Be gentle with the eggs - Overbeating after the eggs are added can result in cracked cheesecakes.
3 No peeking! Opening the oven door during baking can cause cheesecakes to crack.
4 Set the timer. Start with the shortest baking time and add more time only if necessary.
5 You can also cook this in ramekins in a steamer for 10-15 minutes. When cooked, cut a hole in the centre, remove the cheesecake 'hole'. Fill with pouring cream, stir and replace the cheesecake hole prior to serving. Steaming will leave the centre slightly gooey. This will mix with the cream to make a liquid ganache centre. When steaming, do not chill, but serve immediately.
6 I plan to try cooking this in an air fryer sometime to see if it works.

NB: This is VERY rich! I serve it on a plate surrounded by liquid pure cream. The cream is necessary as the cheesecake will soak up the cream so as not to make it too rich. I also make a redcurrant sauce that I boil in the microwave and poor over the top immediately prior to serving.

To date, I have only found one person who can eat a whole slice without turning green.

Nutrional information:
I estimate the fat content to be about 400g of which 300g would be saturated and about 5000 calories. This is about 8 times the daily allowance of fat and enough calories to keep a fit, active adult going for two days.

My favourite part of this dish is watching the face of people when they first take a bite!

Friday, 11 June 2021

Men are from Mars etc

The following is a story I've been lugging around in hard copy, always looking for an excuse to throw it out. Well, what better way than to put it into blog entry!

Remember the book "Men are from Mars, Women are from Venus" by Dr John Gray? Well, here's a prime example offered by an English professor at Southern Methodist University, English 44A, SMU, Creative Writing, Prof. Miller.

In-class Assignment for Wednesday

'Today we will experiment with a new form called the tandem story.  The process is simple.  Each person will pair off with the person sitting to his or her immediate right.  One of you will then write the first paragraph of a short story.  The partner will read the first paragraph and then add another paragraph to the story.

The first person will then add a third paragraph, and so on back and forth.  Remember to re-read what has been written each time in order to keep the story coherent.T he story is over when both agree a conclusion has been reached.'

"The following was actually turned in by two of my English students:

Rebecca - last name deleted, and Gary - last name deleted." 

------------------------------------------------------------------------------

Friday, 30 April 2021

A response to "James Hansen's many and varied furphies" by John Happs

The following is a critique of the article "James Hansen's many and varied furphies" by John Happs as published in Quadrant on 26 August 2013 from a Critical Thinking perspective. I will state at the outset that I regard Quadrant to be a reich-wing rag that barely deserves the title "magazine". It is produced as propaganda for extreme right wing political viewpoints and openly states its position as "sceptical of 'unthinking Leftism, or political correctness, and its "smelly little orthodoxies"'. The original article may be found here.

Rather than tackle the issues of climate change, Happs has chosen a time-honored practice of avoiding a discussion of facts and engaging in a smear campaign against a respected scientist. 

"Hansen has given numerous public talks about what he sees as an urgent need to reduce greenhouse gas emissions by stopping the mining and burning of coal. He also wants oil exploration stopped, as well as the further exploitation of tar sands." 

Without discussing the merits here, the statement is factually incorrect. The impression is given that Hansen is a radical that wants to completely ban oil and gas exploration. Hansen's opposition has very focused at environmentally sensitive areas such as the Amazonian rainforests and Arctic exploration. His similar concerns about the tar sands in Canada are based on the readily available evidence of the damage to the local environment. 

Happs is attempting to create a straw-man argument: painting a picture that is simply not realistic, unless of course Happs thinks that anyone who doesn't approve of the obliteration of Amazonian rainforests is a loony radical. 

"He unashamedly promotes alarmism about the trivial levels of atmospheric carbon dioxide" 

This is the fallacy of "Conclusion entered as evidence".  

The way you identify propaganda from a well-argued position is the former starts with the conclusion. The latter reports facts, makes inferences based upon syllogistic arguments which lead to a conclusion. From there it is simply a matter of establishing if the facts are true and if the argument is sound. A sound argument is such that if the facts or true then the conclusion must also be true. A strong (or cogent) argument is such that if the facts are true then the conclusion is probably true. 

In this article you see none of these things, as I will demonstrate. 

The adjective "unashamedly" is judgemental. It assumes the person has something to be ashamed of. 

"Promotes" denigrates the fact that Hansen is an expert (and conversely that Hall isn't). 

"Alarmism" is another judgement. Used together we have a fallacy known as a "thought stopping cliche". It means nothing and serves only to create a negative impression. 

Also in the same sentence "trivial levels of carbon dioxide". This is "facts not entered into evidence". This hasn't been demonstrated, and in fact, cannot be as this is one of the most provable evidences demonstrating the reality of climate change. 

"In 2012 Robert Bryce quoted Hansen in the Dallas News: “The trains carrying coal to power plants are death trains. Coal fired power plants are factories of death.” " 

This one is truly brilliant! Notice that he doesn't say that Hansen said this, he says that "Robert Bryce quoted Hansen". Interesting! Why did he do this rather than simply quote Hansen and provide a reference? 

The original words were written in this article from the Guardian. Of course, if Happs had referred to the original article - and someone checked it out - they might actually read him in context: 

"The trains carrying coal to power plants are death trains. Coal-fired power plants are factories of death. When I testified against the proposed Kingsnorth power plant, I estimated that in its lifetime it would be responsible for the extermination of about 400 species - its proportionate contribution to the number that would be committed to extinction if carbon dioxide rose another 100 ppm."  

Puts that a little more in perspective here, doesn't it? Read the full article here to see how it all fits in contextually.

Again, the purpose here is to smear Hansen and make him look like an activist first and a scientist second - leading us up to the fallacious sucker punch of them all: 

"Hansen sees himself as an activist and a scientist, but he can’t be both and remain credible." 

This of course, is a ridiculous statement. History is full of activists that were professionals in their field. In fact, most of them are activists precisely because they know they are talking about. According to Happs, a "good" scientist is not allowed to have an opinion. "Good" scientists stay in their boxes and report their findings to their vastly more intelligent "masters" who are permitted to have opinions.  

"Any scientist with a pet hypothesis who selects only data which support that hypothesis, whilst ignoring conflicting data, must lose the respect of their scientific colleagues." 

Happs is assuming here that any scientist with an opinion cannot be objective and will of course commit scientific fraud. Happs offers no evidence that Hansen has EVER done this, he is simply saying it is ipso facto. In other words, Hansen's scientific results are a direct result of his prejudiced opinions rather than the other way around. Presumably as well, Happs places all climate scientists in this box, as almost to a person, they agree with Hansen's results. 

"Real science actually looks for refutation whereas pseudoscience is intolerant of dissent." 

Well, yes, this true. However the pseudoscience is with the deniers. That's because the "scientists" say that climate change is real. The deniers are not scientists and therefore - by definition - "pseudoscientists", and Happs is one of them.  

The article is quite long and continues to cherry-pick results and apply statements that were never intended to be applied to those statements. Happs is not looking at the holistic science, he is picking and choosing and committing every single sin he subscribes to Hansen: He isn't looking for facts, he is starting from his prejudiced conclusion and working backwards from there. This is called "conclusion shopping". 

This article is an A1 demonstration of yellow journalism in action. Happs' dishonest portrayal of Hansen and the science of Climate Change is staggering in its tenacious mendacity. Godwin's Law precludes me from making the obvious comparisons that scream from the pages of Quadrant. I'd say that the article was a discredit to the journal it was published in, but that would only serve to elevate Quadrant to a level it doesn't deserve.

Sunday, 25 April 2021

CentOS 7 post installation script

automatic basic server runup tasks

I like automation. Anything I can do to speed up repetitious tasks is worth the effort. The following shell script is a work-in-progress. Each time I run up a server, I use this script to automate the post installation tasks. I just uncomment the sections I need and run the script. Parts of it take some time and require some human intervention, but usually that just involves pressing 'enter' a few times. The script doesn't do much in the way of hardening the server - that's the stuff of another blog entry.

To use the script, cut and paste it into an editor, make it executable and run it as root. Unfortunately sudo doesn't cut it for some steps. Make sure you uncomment the sections you need first. There are modifications in there for RHEL if you're using that instead of CentOS. I have created a version for CentOS 8, but the it now EOL at the end of this year, I figure it's a little pointless. If anyone wants it, just ask and I'll make it available.

Briefly, the script does the following things:
  1. Runs a yum update using deltas
  2. Adds the epel repository and optionally the remi repository
  3. Installs selinux utilities
  4. Installs PERL, CPAN and useful utilities for PERL.
  5. Installs Apache or NGINX
  6. Installs php for apache
  7. Optional upgrade for php 5.6 
  8. Installs webmin
  9. Installs vsftpd
  10. Installs and secures mysqld
  11. Installs vmtools
  12. Installs miscellaneous utilities
  13. Configure to remove old kernels automatically
  14. Setup email forwarding
  15. Setup and configure WordPress
As I runup more servers with different services, I will update and expand the script. I'm currently writing a GUI front end for the script in Xojo (see below) to customise and generate the script for any environment. If it works, I hope to expand it to include the ability to deploy the script remotely as well as generate it.
 


#!/bin/bash
## The following commands increase the number of packages available
##
yum -y update
yum -y install deltarpm
yum -y install epel-release
## Uncomment for REMI
# rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
# yum --enablerepo=remi,remi-test

yum repolist
yum -y update

## RHEL instructions
# wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm
# rpm -ivh epel-release-7-10.noarch.rpm
# rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
# yum --enablerepo=remi,remi-test
# yum repolist
# yum -y update

## Install SELinux utilities
##
yum -y install policycoreutils policycoreutils-python selinux-policy selinux-policy-targeted libselinux-utils setroubleshoot-server setools setools-console mcstrans

## Install PERL and useful modules and utilities via cpan and cpanm
##
yum -y install perl perl-Net-SSLeay perl-IO-Zlib openssl perl-IO-Tty cpan
cpan App::cpanminus
cpanm Net::FTPSSL
cpanm App::cpanoutdated
cpan-outdated -p | cpanm
cpan-outdated -p | cpanm

## Install and start Apache Web server.
##
yum -y install httpd
#<change httpd.conf listen to 0.0.0.0:80>
systemctl enable httpd.service
systemctl start  httpd.service
systemctl status httpd.service
firewall-cmd --zone=public --permanent --add-service={http,https}
systemctl restart firewalld.service
firewall-cmd --list-all
apachectl graceful
apachectl configtest

## Install NGINX
##
# echo -e "[nginx]\nname=nginx repo\nbaseurl=http://nginx.org/packages/centos/$releasever/$basearch/\ngpgcheck=0\nenabled=1" > /etc/yum.repos.d/nginx.repo
# yum -y install nginx php php-common php-fpm
# firewall-cmd --zone=public --permanent --add-service={http,https}
# systemctl stop httpd
# systemctl disable httpd
# systemctl enable nginx
# systemctl enable php-fpm
# systemctl start nginx
# systemctl start php-fpm
# mkdir -p /srv/www/<website>/public_html
# mkdir /srv/www/<website>/logs
# chown -R nginx:nginx /srv/www/<website>

## Install PHP for Apache, mysql and MSSQL
##
yum -y install php php-pear  php-mysql
## Enable the following as required
# yum -y install php-pgsql php-odbc php-mssql php-ldap php-soap
# yum -y install phpmyadmin httpd-tools
# cp /etc/phpMyAdmin/config.inc.php /etc/phpMyAdmin/config.inc.php.orig
# Harden PHP
# cat /etc/phpMyAdmin/config.inc.php.orig | sed -e 's/AllowRoot\'\]\ \=\ TRUE/AllowRoot\'\]\ \=\ FALSE/g' > /etc/phpMyAdmin/config.inc.php

## Test with http://server/test.php
## Test with http://server/phpMyAdmin

## Upgrade to PHP 5.6 (optional)
##
#rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
#yum repolist
#yum -y remove php-common
#yum -y update
#yum -y install php56w php56w-opcache php56w-xml php56w-mcrypt php56w-gd php56w-devel php56w-mysql php56w-intl #php56w-mbstring
#systemctl restart httpd.service
#php -v

systemctl restart httpd.service
echo -e "<?php phpinfo(); ?>" > /var/www/html/info.php
systemctl restart httpd.service

## Install and run Webmin administration (requires PERL)
##
echo -e "[Webmin]\nname=Webmin Distribution Neutral\nbaseurl=http://download.webmin.com/download/yum\nenabled=1" > /etc/yum.repos.d/webmin.repo
rpm --import http://www.webmin.com/jcameron-key.asc
yum -y install webmin
systemctl enable webmin.service
systemctl start  webmin.service
systemctl status webmin.service
firewall-cmd --zone=public --permanent --add-port=10000/tcp
firewall-cmd --reload
firewall-cmd --list-all

## Install and start VSFTPD
##
#yum -y install vsftpd ftp
#systemctl enable vsftpd.service
#systemctl start vsftpd.service
#firewall-cmd --zone=public --permanent --add-service=ftp
#systemctl restart firewalld.service
#firewall-cmd --list-all

## Install, secure and run MySQL
##
#yum -y install mariadb-server mariadb
#systemctl start mariadb
#mysql_secure_installation
#systemctl enable mariadb.service
#systemctl status mariadb.service

## Install tools for vmware
##
yum install open-vm-tools
vmware-toolbox-cmd -v

## Install Misc utilities as desired
##
yum -y install wget telnet bind-utils nmap mlocate mc
yum -y install elinks

## Remove old kernels and set yum to preserve only two. Setup auto updates
##
yum -y install yum-utils yum-cron
package-cleanup -y --oldkernels --count=2
echo -e "\ninstallonly_limit=2\n" >> /etc/yum.conf
#yum -y install yum-cron # edit /etc/yum/yum-cron.conf or do this via webmin instead on yum-cron
#systemctl start yum-cron
#systemctl enable yum-cron

## Setup email forwarding
##
yum -y install mailx
ln -s /bin/mailx /bin/email
echo -e "set smtp=smtp://<mail forwarding ip address> " >> /etc/mail.rc


## Install Wordpress
#yum -y install php-gd
#systemctl restart httpd.service
#wget http://wordpress.org/latest.tar.gz
#tar xzvf latest.tar.gz
#rsync -avP ~/wordpress/ /var/www/html/
#mkdir /var/www/html/wp-content/uploads
#chown -R apache:apache /var/www/html/*


## Setup WordPress Database
#mysql -u root -p <password>
#CREATE DATABASE wordpress;
#CREATE USER wordpressuser@localhost IDENTIFIED BY 'password'
#GRANT ALL PRIVILEGES ON wordpress.* TO wordpressuser@localhost IDENTIFIED BY 'password';
#FLUSH PRIVILEGES;
#exit

## Configure WordPress
#cd /var/www/html
#cat wp-config-sample.php | sed 's/database_name_here/wordpress/g' | sed 's/username_here/wordpressuser/g' | sed 's/password_here/password/g' > wp-config.php

Thursday, 22 April 2021

COVID-19: AstraZeneca Vaccine and Blood Clots

AZ COVID-19 vaccine and blood clots
There's a huge amount of misinformation floating around about COVID-19 and in particular, the AstraZeneca vaccine and the issue of blood clots. Media reporting is partly to blame for simultaneously sensationalising and glossing over important information. So I thought it was about time to present the facts of the issue. I will source all statements made, if I miss anything out - let me know.

I'll add a caveat here: I'm not a medical expert. Don't get medical advice from social media or even blogs like mine. Go to your doctor. What I've done is glean what I can from the information that is publicly available. I may be wrong on many points, so apply a good deal of critical thinking to what I've written here as you should to any information that is presented to you: particularly if it claims to be authoritative.

COVID-19 Vaccines in general

There are currently four vaccines that are authourised for use in preventing the SARS-CoV-2 infections. Technically, they are in phase IV global post-licensure surveillance. They are:

  • Pzifer-BioNTech (mRNA)
  • Moderna (mRNA)
  • J&J/Janssen (viral vector)
  • Oxford/AstraZeneca (viral vector)

Other vaccines are in different stages of development. Most of these will not make it into public release. Now that there are already vaccines in production and distribution, any vaccines further developed will have to demonstrate they are better than those currently available. Not listed is the Chinese based Sinovac vaccine called CoronaVac and the Russian Gamaleya (Sputnik V) vaccine. Both of these vaccines use traditional production techniques making them cheap to produce but they have poor effectiveness profiles. Many third world countries have opted to use them, but not a single western nation has.

COVID-19 vaccine current state of development

The numbers change daily. You can follow the progress of vaccine candidates here. For those who claim the vaccines haven't been tested, download the daily R&D blueprint and check any trial link you link. Thousands of clinical trials for safety & effectiveness have been and are being conducted.

Oxford-AstraZeneca (AZ) Vaccine

The method used to provoke an immune response is called "viral vector'. It uses a modified version of a different virus - called the vector. This vector is injected intramuscularly and enters the cells of the muscle. From there, the vector manufactures a spike protein identical to that found uniquely on the surface of the SARS-CoV-2 virus that causes COVID-19. The cell then displays the protein on its surface. Our immune system recognises it shouldn't be there and triggers an immune response targeted at the spike protein. Since the body is fighting an 'infection', we have the usual symptoms that accompany an immune response. This includes injection site pain, fever, lethargy, feeling 'sick' etc. However these are immune response reactions, not an actual disease. Different people will have different responses. source

At the end of the process, our bodies have learnt how to protect us against this infection. Effectiveness after two vaccines vary according to the variant encountered. Certainly, this vaccine is much less effective than the Pfzer vaccines (>90%). However, best data to date shows at worst it is 61.7% effective (UK B117 variant) and at best it is 81.5%, with the average being 77.3%. source

A recently published paper in The Lancet is a little more critical, estimating the effectiveness at 67%.

Blood Clots

Once the AZ vaccine was being administered globally, reports started emerging of blood clots at an significant uptick over the background rate. In about 1 in 4 of these, death was the result.

It's important to clarify something here as a sidebar. When you administer a medication to millions of people, large numbers will have 'something' medical happen to them over the month following the administration of the medication. People will have heart attacks, strokes, embolisms etc. that have nothing to do with the administration of the medication. However we do have a good idea of the general rate at which these events occur. In fact, your life insurance company can tell you the percentage chance of any medical event happening to you over the next year. It's quite a well studied science. This is why all adverse events that happen following vaccination are reported. This reporting is mandatory and the reports are publicly available and searchable. In Australia, it is the DAEN (Database of Adverse Events Notification). In the United states it is VAERS (Vaccine Adverse Event Reporting System). Researchers compare adverse events with the rate that normally occurs. When there is an 'uptick' in an event, it is investigated. In the case of serious events, they are all investigated thoroughly as a public health measure.

At first, since the number was very low, there was skepticism that the AZ vaccine was the cause as vaccines have never been known to cause blood clots. Vaccines are injected intramuscularly not intravenously and have not direct connection with blood.

From the investigations, however, the AZ vaccine has been identified as the likely cause in over 100 cases. From these numbers, it is likely the incidence of a blood clot from the AZ vaccine is about 1 in 1 million. The phase III clinical trials involved 20,000 people who received the vaccine. Blood clots were not observed in any of the participants.

UPDATE 30 Sep 2021: We have a lot more data on the incidence. With increased surveillance for TTS, including very minor instances (headaches, thigh soreness etc) there have been 125 reported cases in Australia out of 9.6 million vaccinations. This makes the chance of a blood clot 1 in 76,800. Now, that's a lot higher than one in a million, but remember, these include minor cases. Of those 125 people, 8 people died, six of them women. This is a mortality rate of 1 in 1.2 million. Certainly much better than catching COVID, but since there is an alternative, if you are female and under 50, definitely don't get the AZ vaccine, get the Pfizer or Moderna instead. See  ‘Weighing up the potential benefits against risk of harm from COVID-19 Vaccine AstraZeneca’

source

How can the AstraZeneca vaccine cause blood clots?

So far, experts do not know, however there are some conjectures. Foremost amongst these is that in rare cases, the vaccine also induces and immune reaction against a protein found in blood platelets. On the sniff factor, this sounds (at least) like a good explanation. 

Patients with blood clots were found to have antibodies against Platelet Factor 4 (PF4) which is a symptom of heparin-induced thrombocytopenia. So it may be the case that those patients have this disease already, but it is undiagnosed and the vaccine triggers this response. People are only tested for this if they present to hospital with a blood clot. Blanket testing of people would be pointless because of false positives according to Baye's Theorem.

However, everything is still conjecture at the moment. The reality is that we don't know if it is the adenovirus (the vector) or the spike protein is linked to the clots.

What do we know?

In simplistic bullet point form, this is everything we do know:

  • It occurs at a rate of approximately 1 in 76,800 1 million people
  • It mostly affects women under 55
  • Heparin-induced thrombocytopenia is more common in women than men and less likely to be diagnosed at a young age.
  • Blood clots are a symptom of COVID-19. You are far more likely to get a blood clot from COVID-19 than from the AZ vaccine.
  • To put the risk into comparison, the risk of DVT from flying longer than four hours is 0.079% or 1 in 1265. The risk of a blood clot from taking oral contraceptives is around 0.04% or 1 in 2500.

So, is it safe?

Relative to what?

Taking the AZ vaccines is certainly safer than getting COVID-19 by several orders of magnitude. Millions of people have died from COVID-19 compared to the 25 people that have died probably because of the AstraZeneca vaccine. In addition, antivaxers are lying about adverse reactions by posting fake testimonials to social media. People inundated with hundreds of personal stories may find it compelling, but the reality is that this vaccine is still far safer than not only most other medical interventions, but safer than most daily activities such as driving and eating.

It's only natural to be concerned about risks we hear about in the media regularly or on social media, particularly for activities we don't regularly undertake. But in reality, if you are concerned about the risk of taking the AZ vaccine: Don't fly in a plane, don't drive a car and don't take paracetamol. All of these have higher risk profiles by several orders of magnitude.

However, if we are looking at the risk of taking the AZ vaccine compared to the Pfizer vaccine, then yes. Take the Pfizer vaccine anytime. The Pfizer vaccine will knock you around more - many people need a day or two off work after having it. But its effectiveness is MUCH higher and serious adverse event reports do not statistically exceed the background rate (despite what you facebook group might say).

Achieving Herd Immunity

Besides protecting the individual, the purpose of vaccines is to reduce the R(Eff) rate below 1. For COVID-19, R0=2.4. Herd Immunity (s) = 1-1/R0 = 58.33%.

This means that we need to vaccinate 87% of the population with the AZ vaccine, or 62% of the population with the Pfizer vaccine. However, the higher the rate the better. Anything below these numbers will lead to outbreaks. Leaving children out of the equation (no COVID-19 vaccine has been certified for paediatric use) then it is impossible to achieve herd immunity with the AZ vaccine.

Wednesday, 21 April 2021

Broken YUM on CentOS 7 (or how I learned to love DNF)

Yellowdog Update Modified

It's a simple thing to keep your system updated. A quick yum update every few days - just like brushing your teeth. Completely painless, quick, efficient.

Not today it wasnt.

---> Package screen.x86_64 0:4.1.0-0.26.20120314git3c2946.el7 will be updated
---> Package screen.x86_64 0:4.1.0-0.27.20120314git3c2946.el7_9 will be an update
---> Package skypeforlinux.x86_64 0:8.67.0.96-1 will be updated
---> Package skypeforlinux.x86_64 0:8.71.0.36-1 will be an update
Error: Invalid version flag: or

This was the error I received. Okay, so when yum update fails, there's always the tried and true command sequence that fixes it:

sudo yum clean all
sudo yum update --skip-broken

This time it didn"t work. I still received the same error. Time to google for more info... 

I found a kool command sequence on John S. De Stefano's blog that looked promising:

sudo yum check all                # tells you of any problems
sudo package-cleanup --problems   # lists all known package problems
sudo package-cleanup --dupes      # lists duplicate packages
sudo package-cleanup --cleandupes # actually cleans up duplicates
sudo yum check all                # run again to check for remaining problems
sudo yum-complete-transaction --cleanup-only

However, this command sequence failed to fix the issue too. Looks like I'm going to have to work out what's broken and why. No easy way out with this problem.

Rich Dependencies

Scrolling through bugzilla, I found the following entry:

There was a mistake made in the rpmlib() dep for rich deps. You need
at least rpm 4.13 for the base rich deps, and rpm 4.13.1 for the rest.

yum and related packages are no longer actively developed.
They are being replaced with dnf, dnf-utils, etc.

I'm closing this bug because it's most likely never going to be fixed.
If you still consider your bug report important, reopen it, please.
https://bugzilla.redhat.com/show_bug.cgi?id=1578942

This was actually a bug report for F28+. Since Fedora uses DNF primarily and YUM is deprecated, no one seemed particularly interested in fixing it in Fedora. RHEL/CentOS 8 both use DNF as well. Could this bug have percolated down into CentOS 7? Time to check rpm versions and make sure I have at least 4.13:

$ yum --showduplicate list rpm

Installed Packages
rpm.x86_64                          4.11.3-45.el7                          @base
Available Packages
rpm.x86_64                          4.11.3-45.el7                          base 

Well, that's just peachy!

I'm running an old version of rpm that doesn't support rich dependencies. It also appears that yum may not handle them well either - although the indications are the problem really lies with rpmlib().

I've never really spent a lot of time looking at the inner workings of package managers and their respective update managers. Now that ignorance is coming back to haunt me and it's time for some old fashioned studying the matter.

From rpm.org there's an excellent description of how the boolean operators work and how they help avoid dependency hell. This is what is being referenced in the error I received. The 'or' operator is unknown because my version of rpm is too old. Boolean operators enable what is termed "rich dependencies". Basically providing a logical sequence for resolving dependency issues across multiple versions. A good example is community-mysql and mariadb. Both packages do the same thing - provide a mysql style database. Without rich dependencies, if another package requires mysql, you have to choose which package is required. With rich dependencies you can state:

Package A: Requires: mysql
Package mariadb: Provides: mysql
Package community-mysql: Provides: mysql
Suggests: mariadb to Package A.

Which means that if community-mysql is already installed, that is used as the dependency, otherwise mariadb is installed.

This is really cool, but yum simply ignores it.

Put simply, the root cause of the problem is:

1. RPM supports "rich dependencies"
2. DNF supports resolving packages with "rich dependencies"
3. YUM does not support resolving packages with "rich dependencies"

The solution then is obvious: Since CentOS 7 supports DNF, it's time to switch. Fiddling with yum and manually resolving the dependencies will only delay the inevitable.

YUM vs DNF 

source

"Dandified YUM" or DNF, is the replacement package update manager for RHEL/CentOS. It's been part of Fedora for a long time now. I acknowledge it is clearly superior to YUM in most aspects, plus it doesn't suffer from some of the issues that Debian apt does. The major goal is to eliminate (where possible) dependency hell. But it also has other advantages.

It was also designed to be as drop-in replaceable to yum as possible. It comes very close, but some commands have no equivalent (some of these are deliberate actions). So, for me, if your used to using yum, dnf just represents yet another sequence of commands that must be memorised just to continue doing your job.

I'm not going to bore you with lists here of features, commands, comparisons etc. The links I've provided do that well enough. Plus, if you want a deep dive into dnf, you can go here. Suffice it to say that I decided that installing and using dnf was the simplest and most effect potential solution to the immediate and potentially long term problems.

Install DNF on CentOS7

Pretty simple really, however there are a number of dependencies since it leverages by Py2 and Py3. A total of 11 dependent packages were installed, however YMMV.

$sudo yum install dnf
...
Resolving Dependencies
--> Running transaction check
---> Package dnf.noarch 0:4.0.9.2-2.el7_9 will be installed
--> Processing Dependency: python2-dnf = 4.0.9.2-2.el7_9 for package: dnf-4.0.9.2-2.el7_9.noarch
--> Running transaction check
---> Package python2-dnf.noarch 0:4.0.9.2-2.el7_9 will be installed
--> Processing Dependency: dnf-data = 4.0.9.2-2.el7_9 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-libdnf >= 0.22.5 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-libcomps >= 0.1.8 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-hawkey >= 0.22.5 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: libmodulemd >= 1.4.0 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python2-libdnf for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Processing Dependency: python-enum34 for package: python2-dnf-4.0.9.2-2.el7_9.noarch
--> Running transaction check
---> Package dnf-data.noarch 0:4.0.9.2-2.el7_9 will be installed
---> Package libmodulemd.x86_64 0:1.6.3-1.el7 will be installed
---> Package python-enum34.noarch 0:1.0.4-1.el7 will be installed
---> Package python2-hawkey.x86_64 0:0.22.5-2.el7_9 will be installed
--> Processing Dependency: libdnf(x86-64) = 0.22.5-2.el7_9 for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolvext.so.0(SOLV_1.0)(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolv.so.0(SOLV_1.0)(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolvext.so.0()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libsolv.so.0()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: librepo.so.0()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
--> Processing Dependency: libdnf.so.2()(64bit) for package: python2-hawkey-0.22.5-2.el7_9.x86_64
---> Package python2-libcomps.x86_64 0:0.1.8-14.el7 will be installed
--> Processing Dependency: libcomps(x86-64) = 0.1.8-14.el7 for package: python2-libcomps-0.1.8-14.el7.x86_64
--> Processing Dependency: libcomps.so.0.1.6()(64bit) for package: python2-libcomps-0.1.8-14.el7.x86_64
---> Package python2-libdnf.x86_64 0:0.22.5-2.el7_9 will be installed
--> Running transaction check
---> Package libcomps.x86_64 0:0.1.8-14.el7 will be installed
---> Package libdnf.x86_64 0:0.22.5-2.el7_9 will be installed
---> Package librepo.x86_64 0:1.8.1-8.el7_9 will be installed
---> Package libsolv.x86_64 0:0.6.34-4.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

Next I tried dnf update:

$ sudo dnf update
<snip>
Running transaction check
Error: transaction check vs depsolve:
(libatomic or libatomic1) is needed by skypeforlinux-8.71.0.36-1.x86_64
rpmlib(RichDependencies) <= 4.12.0-1 is needed by skypeforlinux-8.71.0.36-1.x86_64
To diagnose the problem, try running: 'rpm -Va --nofiles --nodigest'.
You probably have corrupted RPMDB, running 'rpm --rebuilddb' might fix the issue.
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.

DNF is smart enough to know why the update failed. The package 'skypeforlinux' uses rich dependencies. These dependencies require a version of rpmlib that is greater than the CentOS 7 repositories can provide. Therefore, the dependencies cannot be resolved. There's a tip there to use rpm directly to reconcile the issues, but since I know that the version of rpm is also too low, that won't work.

I decide that I can live without skypeforlinux, so I remove it and both dnf and yum are happy.

At this point in time, since both work, I can use either to keep my system updated. However, having installed dnf and now that I'm comfortable working with it (having learned the syntax and equivalent commands) I think I'll use it from now on.

Tuesday, 20 April 2021

How SORBS ruined my life (old blog)

When this blog entry was written, Open Relay blockers like SORBS were considered the primary defence mechanism against spam email. They are still often used, but not as much as they used to be - for many of the reasons discussed in this article. SPF, DKIM & DMARC have largely taken their place.
 
SORBS (SPAM and Open Relay Blocking System)
 
I've had a helluva week so far, and I've got SORBS to thank for it.

SORBS (Spam and Open Relay Blocking System) is an email server reputation service. It (along with other reputation services) collect data on servers that are are either poorly configured (and open to abuse) or actively send spam. Reputation services are an excellent way of protecting against spam. They have very high hit rates and (generally) very low false positives. They also have a very low performance overhead compared with heuristic and bayesian filtering techniques. The mail servers I admister are configured to quarantine all email that is listed by either SORBS, SpamHaus or SpamCop (two other reputation services).

However, sometime last week (when I was off work sick), SORBS listed pretty much all of the hotmail, windows live, yahoo, bigpond, optusnet and myplace servers. There may have been others listed as well.

Responding to a few complaints that came in on Monday about email not being received, I began checking. Then began the sinking feeling that goes along with knowing that I'd have to change the email filtering rules, do some regression tests and then resubmit a week's work of spam through the email filters. SORBS is a very aggressive filter and I've been quite reliant on it for some time.

Less fun was trying to explain the problem to the CEO.

The first part of the quick fix was to move the SORBS check to the end of the anti-SPAM rules of the DMZ mail filter and set it to monitor only - not block. The DMZ mail server is exposed to the Internet and performs basic/quick checks only and does not look at the content of the email messages. It is highly robust and is meant as the front line of defence. It experiences an average of 90 security attacks per hour. It filters about 75% of the email traffic as inappropriate before passing the "possibly okay" messages to the second filter.

The second part was to add a check for zero day threats on the DMZ server and tag messages accordingly.

The third part was to tighten the screws a bit on the second mail filter. This meant decreasing the tolerance limits a little and added a few more checks which included looking for the tags on the email messages placed there by the DMZ mail filter. Usually, this server intercepts about 8% of messages pass to it. Messages quarantined by this server may result in a quarantine message and even a self-release option depending upon the spam score the email receives.

Now that these changes were made, I resubmitted 2143 spam quarantined messages. With the new rules in place, 1963 messages were blocked and 180 passed through to the second filter which blocked a further 68 messages leaving 112 messages sailing through to the mail server. I don't know how what the spam:ham ratio of the 112 messages was but the count seemed to be manageable so I released the remaining 14,000 quarantined messages and sent an email instructing people to forward any received spam to the spam submit mailbox for heuristic analysis. So far, I have 35 spam messages submitted by users which (if accurate) we now have a spam hit rate of 99.75% with (hopefully) a close to zero false positive rate. The hit rate has dropped to 59.4% for the DMZ server and risen to 12.5% for the second filter. Time and monitoring will determine how successful the changes have been.

Moving forward, I have written replacement rules that will no longer quarantine email based upon failed reputation. The new rules will look at SPF (Send Policy Framework - RFC4088) in addition to reputation. If the SPF check passes, the email will be accepted. If SPF fails, it will be dropped. If SPF is 'softfail' or 'none' then it will be subject to a reputation check. Any failure will drop then connection with a 46x error - Temporary Failure, with details as to why sending failed. The sending MTA will then notify the recipient that the email failed to send. The sending server then has the option of implementing SPF. I'd like at some point to add a DKIM check, but that's a reasonably difficult task.

BTW, I am heavily influenced by Ming Weng Wong's whitepaper on Messaging Anti-Abuse: