IdPv4 upgrade guide

Last Update: 2023-11-07 (minor edit, last major edit 2020-10-02)

Introduction

This page collects our guidance for operators of Shibboleth Identity Providers (IdPs) within the UK federation. Specifically, it is aimed at operators of an existing IdP running the Shibboleth IdP v3 software, that will need to be upgraded to the v4 idP.

This upgrade should be completed before 31st December 2020, when the Shibboleth IdP v3 software will reach end of support with the Shibboleth project, and v4 will be the only supported version of the Identity Provider software.

Whilst the UK federation support desk will provide support for operators upgrading their IdP beyond 31st December 2020, we would advise using the following guide, in addition to documentation on the official Shibboleth IdP v4 Wiki, and upgrading as soon as possible, to ensure that you can receive necessary security updates to the software.

For organisations requiring in-depth and hands-on support we offer a Trust and Identity consultancy service

Using the guide

Linux-specific
Windows-specific
- Windows platform support
- Java upgrade - Windows
General

Linux-specific

Linux platform support

You should be running a current, supported distribution and release of Linux. It can be quite hard to determine when support will end for a given platform. Check https://endoflife.software/operating-systems/ for your platform and release.

You will need to know the Linux distribution and release you are running. Most distributions provide a standard command:

 lsb_release -a

This will provide output similar to:

 LSB Version:	 core-4.1-amd64:core-4.1-noarch
 Distributor ID: CentOS
 Description:	 CentOS Linux release 7.6.1810 (Core)
 Release:	 7.6.1810
 Codename:	 Core

If that doesn't provide the information then you may find it by running:

 cat /etc/os-release

Does my Linux distribution supply a recent enough Java and web container?

This guide covers both the Tomcat and Jetty web containers. Different Linux distributions and releases have different versions of Java and Tomcat or Jetty available.

To run IdP4 You will need Java 11 (or later) and either Tomcat 9 (or later) or Jetty 9.4 (or later) so you may need to upgrade various components.

The tables below will allow you to check if your distribution makes suitable versions of the web containers available.

If you're running a generally supported distribution release which doesn't provide the relevant supporting packages then you will need to either upgrade/migrate to one that does or manage the software dependencies yourself.

Tomcat

Distibution	Release	Java available	Java suitable	Tomcat available	Suitable	End of life
CentOS / RedHat	7	11	Yes	7	No	June, 2024
CentOS / RedHat	8	11	Yes	None	No	May, 2029
Debian	10 buster	11	Yes	9	Yes	January, 2022
Debian	9 stretch	8	No	8	No	January, 2020
Ubuntu	16.04	9	No	8	No	April, 2021
Ubuntu	18.04	11	Yes	9	Yes	April, 2023
Ubuntu	20.04	14*	Yes	9	Yes	April, 2025

Note

Ubuntu 20.04 ships various versions of OpenJDK (currently 8, 11, 13 & 14). We have only tested the deployment on OpenJDK 11.

If you already have a suitable version of Java installed, you can skip ahead to Tomcat upgrade for Linux deployers.

Jetty

It can be tricky obtaining a suitable version of Jetty, the following table describes the situation for major distributions.

Distibution	Release	Java available	Java suitable	Jetty available	Suitable	End of life
CentOS / RedHat	7	11	Yes	9.0.3-8.el7	No	June, 2024
CentOS / RedHat	8	11	Yes	None	No	May, 2029
Debian	10 buster	11	Yes	9.4.31-1~bpo10+1	Yes	January, 2022
Debian	9 stretch	8	No	9.2.26-1~bpo9+1	No	January, 2020
Ubuntu	16.04	9	No	9.2.14-1	Yes	April, 2021
Ubuntu	18.04	11	Yes	9.4.15-1	Yes	April, 2023
Ubuntu	20.04	14*	Yes	9.4.26-1	Yes	April, 2025

If you already have a suitable version of Java installed, you can skip ahead to Jetty upgrade for Linux deployers.

Java upgrade - Linux

In general, you should select a Linux distribution and release which provides a supported Java runtime.

Support matrices for popular combinations of platform, Java and web container are given above.

Using the distributions package manager (yum, dnf, apt etc) is the most convenient way to manage the available Java.

First check which version is currently installed:

 $ java -version
 openjdk version "1.8.0_262"
 OpenJDK Runtime Environment (build 1.8.0_262-b10)
 OpenJDK 64-Bit Server VM (build 25.262-b10, mixed mode)

If you need to upgrade then find the appropriate package name using the package manager's search function:

CentOS / RedHat: yum search openjdk
Debian / Ubuntu: apt-cache search openjdk

When you've found the package, install it using the package manager. Below are some examples but check first:

 # CentOS / RedHat 
 yum install -y java-11-openjdk

 # Debian / Ubuntu
 apt-get install -y openjdk-11-jdk

Verify the new version is being used by default:

 $ java -version
 openjdk version "11.0.8" 2020-07-14
 OpenJDK Runtime Environment (build 11.0.8+10-post-Ubuntu-0ubuntu118.04.1)
 OpenJDK 64-Bit Server VM (build 11.0.8+10-post-Ubuntu-0ubuntu118.04.1, mixed mode, sharing)

If the install process has not enabled by default then you will need to use the update-alternatives system to select it:

 update-alternatives --config java

Tomcat upgrade for Linux deployers

Tomcat upgrade/install

If your distribution has the required version available then your first choice should be that package. Please see your distribution's documentation on how to install or upgrade packages.

When doing an upgrade, if you are using AJP and you traverse version 9.0.31, you may need to update your server.xml to include an extra attribute on the existing <Connector> element. It may look like this:

 <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

Update it to add secretRequired="false" if you're running Apache httpd and Tomcat on the same server. If you're not, please seek further guidance.

 <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" secretRequired="false" />

More information on this change is available at https://tomcat.apache.org/tomcat-9.0-doc/changelog.html

Manual installation of Tomcat

Should your distribution not provide an appropriate Tomcat package then you will need to install and maintain this yourself. The procedure below is just one way of doing this and may need altering to suit your environment.

Note

Before doing, this, ensure you have a supported version of Java available.

Overview

Download the latest version from Tomcat and verify
Decompress the archive
Create a tomcat user if required
Prepare the new directory tree (move bundled directories and symlink to system locations)
Create master symlink
Create service
Configure Tomcat
Start Tomcat service

Details

Download the latest version

Check the latest version of Tomcat available. Download the package and verify the signature to ensure it's not been tampered with:

 cd /usr/local/src
 curl -O https://downloads.apache.org/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.tar.gz
 curl -O https://downloads.apache.org/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.tar.gz.sha512

 sha512sum -c apache-tomcat-9.0.37.tar.gz.sha512

Decompress the archive

 cd /opt
 tar xfz /usr/local/src/apache-tomcat-9.0.37.tar.gz

If required, create a tomcat user

Check if a tomcat user is already available:

 getent passwd tomcat

If nothing is returned then you will need to create a user to run tomcat as. This can vary between distributions but generally looks like:

 useradd --system --no-create-home --home-dir /opt/tomcat tomcat

Prepare the directory tree

To make upgrades easier, move the components that need to persist across Tomcat upgrades out to their traditional system locations and replace them with symlinks. Lastly, ensure the required directories are writeable by the tomcat user.

 mkdir -p /var/cache/tomcat /var/lib/tomcat

 cd /opt/apache-tomcat-9.0.37

 mv conf /etc/tomcat
 mv logs /var/log/tomcat
 mv temp /var/cache/tomcat/temp
 mv webapps /var/lib/tomcat/webapps
 mv work /var/cache/tomcat/work

 ln -s /etc/tomcat conf
 ln -s /var/log/tomcat logs
 ln -s /var/cache/tomcat/temp temp
 ln -s /var/lib/tomcat/webapps webapps
 ln -s /var/cache/tomcat/work work

 chown -R tomcat . /var/log/tomcat /var/cache/tomcat /var/lib/tomcat

Create the master symlink

To make upgrades easier, create a symlink to the current version:

 cd /opt
 ln -s apache-tomcat-9.0.37 tomcat

Create the Tomcat service

Most modern Linux distributions use systemd to manage services. Some distributions use "RC" scripts instead.

To see if your system uses systemd, run:

 systemctl status

If this returns information about running services then you can add a new service by creating a new service unit file.

 nano /etc/systemd/system/tomcat.service

Make the contents of the file:

 [Unit]
 Description=Tomcat
 After=network.target

 [Service]
 Type=forking
 ExecStart=/opt/tomcat/bin/catalina.sh start
 ExecStop=/opt/tomcat/bin/catalina.sh stop
 User=tomcat

 [Install]
 WantedBy=multi-user.target

Save and quit (Ctrl-X and follow the prompts).

Lastly tell systemd to reload its configuration:

 systemctl daemon-reload

Configure Tomcat

The Tomcat configuration requires a few tweaks to support the IdP.

 mkdir -p /etc/tomcat/Catalina/localhost
 nano /etc/tomcat/Catalina/localhost/idp.xml

Make the contents of this file:

 <Context docBase="/opt/shibboleth-idp/war/idp.war"
     privileged="true"
     antiResourceLocking="false"
     swallowOutput="true">

     <!-- Work around lack of Max-Age support in IE/Edge for Tomcat 8.0.x  -->
     <CookieProcessor alwaysAddExpires="true" />

    <Parameter name="idp.home" value="/opt/shibboleth-idp" override="true" />
 </Context>

Save and quit.

Start the Tomcat service

 systemctl start tomcat

Check the startup logs:

journalctl -f
cat /var/log/tomcat/catalina.out

Upgrade of manually installed Tomcat

Once you're managing Tomcat as described above, the upgrade process is:

Download the latest version (check Shibboleth IdP requirements) from Tomcat and verify
Decompress the archive
Prepare the new directory tree (remove bundled directories and symlink system locations)
Replace master symlink
Restart Tomcat

This may look like:

 # Download the latest version
 cd /usr/local/src
 curl -O https://downloads.apache.org/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.tar.gz
 curl -O https://downloads.apache.org/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.tar.gz.sha512

 sha512sum -c apache-tomcat-9.0.37.tar.gz.sha512

 # Decompress the archive
 cd /opt
 tar xfz /usr/local/src/apache-tomcat-9.0.37.tar.gz

 # Prepare the new directory tree
 cd apache-tomcat-9.0.37
 rm -fr conf logs temp webapps work
 ln -s /etc/tomcat conf
 ln -s /var/log/tomcat logs
 ln -s /var/cache/tomcat/temp temp
 ln -s /var/lib/tomcat/webapps webapps
 ln -s /var/cache/tomcat/work work
 chown -R tomcat .

 # Replace master symlink
 cd /opt
 ls -l tomcat # note current version
 lrwxrwxrwx. 1 root root 20 Jul 16 10:34 tomcat -> apache-tomcat-9.0.35

 rm -f tomcat
 ln -s apache-tomcat-9.0.37 tomcat

 # Restart Tomcat
 systemctl restart tomcat

Jetty upgrade for Linux deployers

Manual installation of Jetty

Should your distribution not provide an appropriate Jetty package then you will need to install and maintain this yourself. The procedure below is just one way of doing this and may need altering to suit your environment.

Note

Before doing, this, ensure you have a supported version of Java available.

Overview

Download the latest version of Jetty and Verify
Decompress the archive for Jetty
Download jetty-idp-base files from the Shibboleth build server
Decompress the archive for jetty-idp-base
Configure idp.xml
Configure certificates into Jetty
Add an init or systemd service for Jetty

Details

Download the latest version

Check the latest version of Jetty available. Download the package and verify the signature to ensure it's not been tampered with:

 cd /usr/local/src
 curl -O https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.4.31.v20200723/jetty-distribution-9.4.31.v20200723.zip
 curl -O https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.4.31.v20200723/jetty-distribution-9.4.31.v20200723.zip.sha1

 sha1sum jetty-distribution-9.4.31.v20200723.zip
 cat jetty-distribution-9.4.31.v20200723.zip.sha1

Decompress the archive

We recommend you unzip the archive into /opt as this is a binary distribution rather than a source package. You may need to install unzip within your linux distribution, create a symlink so that you can easily re-point to a new version.

 cd /opt
 unzip /usr/local/src/jetty-distribution-9.4.31.v20200723.zip
 ln -s jetty-distribution-9.4.31.v20200723.zip jetty-distribution

Download jetty-idp-base files from the Shibboleth build server

 cd /usr/local/src
 curl -O https://build.shibboleth.net/nexus/service/local/repositories/releases/content/net/shibboleth/idp/idp-jetty-base/9.4.0/idp-jetty-base-9.4.0.tar.gz

 curl -O https://build.shibboleth.net/nexus/service/local/repositories/releases/content/net/shibboleth/idp/idp-jetty-base/9.4.0/idp-jetty-base-9.4.0.tar.gz.sha1

 sha1sum idp-jetty-base-9.4.0.tar.gz
 cat idp-jetty-base-9.4.0.tar.gz.sha1

Decompress the archive of jetty-idp-base

 cd /opt
 tar -xzvf /usr/local/src/idp-jetty-base-9.4.0.tar.gz

Configure certificates into Jetty

You will need to configure your browser facing and backchannel certificates and their passwords into Jetty in the following files;

/opt/jetty-base/start.d/idp.ini
/opt/jetty-base/start.d/idp-backchannel.ini

Configure idp.xml

You need to update /opt/jetty-base/war/webapps/idp.xml to point to the war file (typical path is /opt/shibboleth-idp/war/idp.war)

 <Set name="war">
     <SystemProperty name="idp.war.path" default="/opt/shibboleth-idp/war/idp.war" />
 </Set>

Add an init or systemd service for Jetty

The following a is a basic example of a systemd service file for starting and stopping the jetty service. You'll need to ensure it's enabled at start-up and that any previous version of Jetty doesn't start.

 # Add to /etc/systemd/system/jetty.service
 [Unit]
 Description=Jetty provides a Web server and javax.servlet container, plus support for HTTP/2, WebSocket, OSGi, JMX, JNDI, JAAS and many other integrations.
 After=network.target

 [Service]
 WorkingDirectory=/opt/jetty-base
 ExecStart=/usr/bin/java -Djdk.tls.ephemeralDHKeySize=2048 -jar /opt/jetty-distribution/start.jar lib=/opt/jetty-distribution/lib

 [Install]
 WantedBy=multi-user.target

To start the service:

 systemctl start jetty

To enable the service at boot:

 systemctl enable jetty

Windows-specific

Windows platform support

OS Support

Shibboleth V4 only runs a 64 bit installer so your environment will need to support this. It�s strongly advised to only use the 64 bit software (java, tomcat, shib) and not to mix 32 bit software with 64 bit. You should be running a current supported windows environment, the current supported windows environment can be found here:

https://support.microsoft.com/en-gb/help/13853/windows-lifecycle-fact-sheet

Or search for your environment from here to see life cycle of the product

https://support.microsoft.com/en-gb/lifecycle/search?alpha=Windows

Visual C++ redistributable Package

Check you have a valid version of Visual C++ redistributable package for Visual Studio 2015, 2017 and 2019. The latest package can be downloaded for your 64 bit Windows architecture here:

https://support.microsoft.com/en-gb/help/2977003/the-latest-supported-visual-c-downloads

Rehosting

If you want to preserve your current implementation to test out the latest version of Shibboleth then you can use Disk2vhd to create a virtual Machine disk format of your current environment. Then you can safely use this virtual disk on another system and then you can upgrade your environment in a test environment. More information can be found at the following link:

https://docs.microsoft.com/en-us/sysinternals/downloads/disk2vhd

Java upgrade - Windows

Migrating from JAVA 8 or OpenJDK to Corretto

Amazon Corretto 11 for Windows is the officially supported Java Distribution for shibboleth 4 Identity Provider. Installation instructions are available from Amazon Corretto.

Once installed remember to update your JAVA_HOME path from your previous Java distribution to point to your Corretto 11 installation. A reboot of your server will force the path to be updated.

Corretto 11 is supported on Shibboleth 3.4.x so you can install this prior to upgrading your Shibboleth installation.

General

This section contains those steps of the upgrade process that are not platform-dependent: which is to say, those steps that relate to the IdP itself.

Upgrade from earlier 3.x.x to 3.4.7 steps

Part of the process of upgrading to version 4 of the Shibboleth Identity Provider (IdP) should involve updating to the latest version 3 release.

Version 3 includes support for legacy v2 configuration elements which are not present in v4. Warnings were added to the startup process in v3.4.4 and with some more appearing in v3.4.7. Once you've upgraded to the latest v3 release (currently 3.4.7), review the warnings logged on startup and resolve them (see https://www.ukfederation.org.uk/content/Documents/DeprecationIdPv4 and https://wiki.shibboleth.net/confluence/display/IDP30/DeprecatedIdPV4 for more information).

Further background on the Upgrade process is available at https://wiki.shibboleth.net/confluence/display/IDP30/Upgrading

This would also be a good point to consider hardening your IdP by ensuring that the relevant security related settings are set correctly (see, for example,the section on IdP cookies below).

Check release notes

The IdP can usually be upgraded in place by running the desired version's installer script but you should review the Release Notes for all intermediate versions to ensure you understand the changes that are taking place.

To find which version you're currently running, run this (assumes default installation location):

 /opt/shibboleth-idp/bin/version.sh

or, on Windows:

 "C:\Program Files (x86)\Shibboleth\IdP\bin\version.bat"

Get the latest version 3 installer

UNIX (including Linux)

1. Visit https://shibboleth.net/downloads/identity-provider/latest3/ and get the URL for the .tar.gz file (currently https://shibboleth.net/downloads/identity-provider/latest3/shibboleth-identity-provider-3.4.7.tar.gz)

2. Download the new release to the server:

 cd /usr/local/src
 curl -O https://shibboleth.net/downloads/identity-provider/latest3/shibboleth-identity-provider-3.4.7.tar.gz

3. Extract the new release:

    tar xf shibboleth-identity-provider-3.4.7.tar.gz

4. Run the installer for the new release:

 cd shibboleth-identity-provider-3.4.7
   ./bin/install.sh

5. Ensure the correct default responses have been generated for the release location and deployment location (either press ENTER to accept or enter the correct location)

6. Restart the Java Container (usually Tomcat/Jetty)

7. Check the log for startup issues

8. Review the warning log for deprecation issues and resolve (see above)

Windows MSI installer

The Windows install and upgrade process is well documented on the Shibboleth Wiki (https://wiki.shibboleth.net/confluence/display/IDP30/WindowsInstallation#WindowsInstallation-Upgrade).

1. Check if your deployment includes Jetty (does a Jetty directory exist under C:\Program Files (x86)\Shibboleth?)

2. Get latest MSI installer from https://shibboleth.net/downloads/identity-provider/latest3

3. Run the downloaded MSI and follow the wizard taking care to upgrade Jetty if required

4. Restart the "Shibboleth 3 IdP Daemon" service using the standard Windows tools

Worked example (Linux with Tomcat)

This run through assumes Ubuntu 18 running Tomcat 9.

 # cd /opt/shibboleth-idp
 # ./bin/version.sh
 3.4.1

 # cd /usr/local/src
 # curl -O https://shibboleth.net/downloads/identity-provider/latest3/shibboleth-identity-provider-3.4.7.tar.gz
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
 100 45.4M  100 45.4M    0     0  3114k      0  0:00:14  0:00:14 --:--:-- 3650k

 # tar xf shibboleth-identity-provider-3.4.7.tar.gz

 # cd shibboleth-identity-provider-3.4.7
 # ./bin/install.sh
 Source (Distribution) Directory (press <enter> to accept default): [/usr/local/src/shibboleth-identity-provider-3.4.7]

 Installation Directory: [/opt/shibboleth-idp]

 Rebuilding /opt/shibboleth-idp/war/idp.war ...
 ...done

 BUILD SUCCESSFUL
 Total time: 13 seconds

 # systemctl restart tomcat9
 # ./bin/version.sh
 3.4.7
 # tail -f /opt/shibboleth-idp/logs/idp-process.log

When the IDP has started, you can exit the tail using Control-C.

Review deprecation warnings in 3.4.xx

Please read carefully our documentation on deprecated features in IdP v3.

Not doing so may result in values of released attributes changing or the IdP failing to operate

Upgrade from 3.4.7 to 4.x.x Steps

Prerequisites

Currently running a recent version of IdP v3 (v3.4.7 or later, see below)
A service restart and general use shows no deprecation warnings (see deprecation warnings above)
A suitable platform such as Operating System, Java and Container (see guidance for Linux or Windows above, as appropriate)
Review the Release Notes

For detailed instructions the official upgrade notes are available at https://wiki.shibboleth.net/confluence/display/IDP4/Upgrading.

Release Notes main points

A number of previously deprecated features, settings, APIs, etc. have been removed. The use of these deprecated bits should have produced a standard warning either at startup or during use in the most recent releases of the software, and most of them are now non-functional. Eliminating warnings before upgrading is strongly advised.
The IdP includes a new Cross-Site Request Forgery (CSRF) Protection feature that is enabled by default for new installs. The idp.csrf.enabled property is a general on/off switch if problems are encountered or upgraders wish to enable it. The feature can also be easily incorporated in custom webflow views with minimal effort.
Simplification of basic attributes directly generated from a DataConnector is possible using the exportAttributes attribute.

Upgrade process

Record some exemplar user assertions for comparison later
Verify your current IdP and Java versions and IdP file ownership
Get latest and extract IdP release
Backup existing deployment
Run installer
Fix permissions and deploy WAR file if required
Restart J2EE container (Tomcat/Jetty)
Test

Record some exemplar user assertions for comparison later

One thing that break in a non-obvious way is the way attributes are generated for users. To test this, use the aacli tool to generate an assertion for some normal users (eg. a member of staff and a student):

 /opt/shibboleth-idp/bin/aacli.sh -r https://test.ukfederation.org.uk/entity -n testusername
 - or -
 "C:\Program Files (x86)\Shibboleth\IdP\bin\aacli.bat" -r https://test.ukfederation.org.uk/entity -n testusername

This should output a data block which you can copy/paste somewhere for later reference, at a minimum you should expect to see values for eduPersonScopedAffiliation and eduPersonTargetedID. Repeat the command as required.

On some Linux based installations, you may need to override the endpoint used to generate the data:

 /opt/shibboleth-idp/bin/aacli.sh -u http://localhost:8080/idp -r https://test.ukfederation.org.uk/entity -n testusername

Please note that there are various ways that this functionality may be broken.

Verify your current IdP and Java versions and IdP file ownership

Assuming the IdP is installed in the default location, run the following:

 /opt/shibboleth-idp/bin/version.sh
 - or -
 "C:\Program Files (x86)\Shibboleth\IdP\bin\version.bat"

Both should output a version number which should be v3.4.7 or greater. If this is not the case then perform that upgrade first (see: Upgrade from earlier 3.xx to 3.4.7 steps)

 # /opt/shibboleth-idp/bin/version.sh
 3.4.7

On Linux, check the owner of the files in the IdP deployment directory for comparison later:

 ls -ld /opt/shibboleth-idp/logs
 drwxr-x--- 2 tomcat tomcat 200704 Sep 25 00:25 /opt/shibboleth-idp/logs
              ^^^^^^ ^^^^^^
               user   group

If this is not listed as root then you may need to fix the permissions after installation.

Get latest and extract IdP release

Visit https://shibboleth.net/downloads/identity-provider/latest/ and get the URL for the .tar.gz file or, if on Windows, the .msi file
Download the new release to the server (on Windows just download the file)

 cd /usr/local/src
 curl -O https://shibboleth.net/downloads/identity-provider/latest/shibboleth-identity-provider-4.0.1.tar.gz
 tar xf shibboleth-identity-provider-4.0.1.tar.gz

Backup existing deployment

Before performing the upgrade, take a backup of the existing deployment in case you need to revert.

Linux

 cd /opt
 tar cfz shibboleth-idp-v3-backup.tar.gz shibboleth-idp

Windows
1. In Windows Explorer, navigate to C:\Program Files (x86)
2. Right click on `Shibboleth` and select Send to > Compressed (zipped) folder

Run installer

Running the installer will detect that an upgrade is needed (rather than installing a new IdP) and perform the necessary changes:

Linux - you will be prompted for the installer location and the target installation directory (normally /opt/shibboleth-idp)

 cd /usr/local/src/shibboleth-identity-provider-4.0.1
 ./bin/install.sh

 Buildfile: /opt/src/shibboleth-identity-provider-4.0.1/bin/build.xml

 install:
 Source (Distribution) Directory (press <enter> to accept default): [/opt/src/shibboleth-identity-provider-4.0.1] ?

 Installation Directory: [/opt/shibboleth-idp] ?

 INFO [net.shibboleth.idp.installer.V4Install:155] - Update from version 3 to version 4.0.1
 INFO [net.shibboleth.idp.installer.BuildWar:72] - Rebuilding /opt/shibboleth-idp/war/idp.war, Version 4.0.1
 INFO [net.shibboleth.idp.installer.BuildWar:81] - Initial populate from /opt/shibboleth-idp/dist/webapp to /opt/shibboleth-idp/webpapp.tmp
 INFO [net.shibboleth.idp.installer.BuildWar:90] - Overlay from /opt/shibboleth-idp/edit-webapp to /opt/shibboleth-idp/webpapp.tmp
 INFO [net.shibboleth.idp.installer.BuildWar:99] - Creating war file /opt/shibboleth-idp/war/idp.war

 BUILD SUCCESSFUL
 Total time: 5 seconds

Windows - run the MSI installer file and follow the wizard taking care to upgrade Jetty if that container is in use

Fix permissions and deploy WAR file if required

If the directory ownership above did not show root then you may need to fix the permissions on the IdP tree by using the user and group reported above and running:

 chown -R user:group /opt/shibboleth-idp
 - for example -
 chown -R tomcat:tomcat /opt/shibboleth-idp

Some Linux environments need the rebuild idp.war file to be copied to the J2EE container before it will be deployed. This will vary by platform but may be something like:

 cp /opt/shibboleth-idp/war/idp.war /var/lib/tomcat/webapps

Restart J2EE container

This varies by Linux environment but is usually one of:

 systemctl restart tomcat
 - or -
 systemctl restart tomcat9

On Windows, use the Services MMC snapin to restart the Shibboleth service.

Test

Carry out the same user tests using the aacli tool as above and compare the output with the pre-upgrade values to ensure, for example, that the eduPersonTargetedId attribute has not changed.

IdP deployers should use MDQ rather than metadata aggregates

The UK federation publishes metadata via its Metadata Publication Service describing all participating entities. This metadata file (ukfederation-metadata.xml) provides the information required for entities to know how to communicate with each other, and establishes a trust fabric permitting entities to verify each other�s identities. The IdP relies on the regular downloading of this metadata aggregate which is currently (Septermber 2020) over 58MB and will continue to grow with new entities added to the federation. This download is then processed locally within Java, including signature verification, which overall requires a large amount of RAM and CPU resources.

We have a new publication which use a newer protocol called Metadata Query (MDQ) to retrieve metadata. Rather than downloading the whole aggregate XML file, the metadata is looked up on demand by the IdP by querying a UK federation MDQ publication service. Thus, the required CPU and memory footprint is massively reduced.

Further details on MDQ and specifically on configuring metadata providers in the IdP

Note: that deployers who continue to use the federation metadata aggregate will eventually hit either the limits of either CPU and Memory available on the host or limits within the configuration such as Java memory assignment, resulting in a service outage or needing to size-up their environment to cope, and incurring further costs.

IdP cookies

Check your idp.properties file for idp.cookie.secure to ensure cookies are not sent over insecure connections:

 idp.cookie.secure=true

If the line has a # at the start then it's commented out so takes the default. Until version 4, the default was false.

You should only do this if your IdP is using the HTTPS transport or it will stop working. If you're not using HTTPS (ie. you're using HTTP) then you should strongly consider moving to the HTTPS transport as soon as possible and then enable this setting.