Upgrade of Shibboleth IdP from 1.3 to 2.x

Introduction

As the Shibboleth 1.3 identity provider (IdP) software approaches end-of-life, sites using it will need to address the issue of upgrading to Shibboleth 2.x. The two products are completely different and there is no software upgrade path from one to the other – you have to install the new IdP and then see about moving users from one to another.

Important note 1: the "rolling upgrade" method described below (approach c) is only recommended for large organisations with heavily-used IdPs that cannot tolerate any loss of service, who have sufficient experience and resources in their IT department to be able to support the complexity involved. Other similarly complex approaches should be approached in the same way. All such complex methods should be thoroughly tested with non-production installations before upgrading the production IdP. Note also that the system should be carefully monitored for at least a week or two after the upgrade has taken place.

Important note 2: the UK federation strongly recommends that the IdP be configured in the traditional manner of having a separate port, normally port 8443, for "back channel" communications with service providers. This is because a recent vulnerability in SSL has resulted in a patch to some dependent libraries being issued. An IdP which relies on single host/port operation will cease to work when it encounters this patch, whether the patch is installed on the IdP or on the SP it is communicating with. Anyone currently running a Shibboleth 1.3 IdP with the single host/port configuration is advised to move to the traditional two port configuration when upgrading to Shibboleth 2.x. Information about changing a Shibboleth 1.3 or 2.x IdP to the two port configuration can be found here.

There are several approaches which trade off implementation ease against user-friendliness and availability. Ranging from easy to complex, these include:

  1. Install a Shibboleth 2.x IdP. Give it a new entityID. When it is running correctly change the visibility in the WAYF from the old IdP to the new one.

    This has the disadvantage that all personalisation will be lost (since it is associated with the entityID value). Any SPs which rely on the entityID value for authorisation will also stop working.
  2. Install a Shibboleth 2.x IdP. Give it the same entityID as the old one. Change the federation metadata to refer to the new IdP. Fix anything about the new IdP that doesn't work.

    This conserves the entityID but has this disadvantage: until the metadata has propagated to a given SP, that SP will not be able to communicate with the new IdP.
  3. Extend the above by devising a mechanism which allows a rolling upgrade of federation metadata such that when the service moves from the old IdP to the new, SPs need not immediately have access to the new metadata.

    This has the minor disadvantage that users may see momentary downtime as the IdPs are swapped and they may see an extra login screen if the IdPs are swapped while they have an active session.
  4. Extend the above by running multiple IdPs (for availability) and using SSO techniques to remove the extra login screens.

In the remainder of this document we describe how EDINA carried out option c. on a local IdP. Details of how to achieve option d. (which is an extension of the work described here) can be found here.

You are assumed to be an experienced Shibboleth IdP manager and comfortable with editing and configuring Shibboleth IdP installations, both 1.3 and 2.x as well as their containers (e.g. Tomcat) and any fronting web server (e.g. Apache). If you are a less confident Shibboleth manager you might prefer to go for one of the less technically challenging upgrades, namely a or b as described above.

The information here is provided for guidance only. You will need to adjust the details to fit your local setup.

The IdP described here runs on a single server. A clustered set-up would be more challenging to roll forward, and if the mechanism below was followed the old cluster would need to be fully stopped before the new one was started. The SSO-preserving mechanisms described here could be used to relax this.

Overview

The upgrade took place in five stages:

  1. The 2.x IdP was installed, configured and registered in the UK federation.
  2. The 1.3 IdP was configured to listen on Shibboleth 2.x endpoints as well as on its existing 1.3 endpoints.
  3. The 1.3 IdP metadata was changed to reflect this and allowed to propagate around the federation.
    During this period the 1.3 IdP responded to SPs which had either of the entity descriptions loaded.
  4. Once a suitable period had elapsed, the 1.3 IdP was turned off and the 2.x IdP started.
  5. Once the new IdP was running, the final Shib 2 metadata could be rolled into the federation.
    No configuration was needed at this stage – as SPs spotted the new metadata they could take advantage of the new facilities described.

It is appealing to consider configuring a Shibboleth 2.x IdP to listen on Shibboleth 1.3 endpoints. This would have allowed stages 1 and 2 to be done as one. Unfortunately it is very hard, if not impossible, to achieve.

Example system

This document describes the mechanism by which we rolled forward an EDINA IdP with entityID https://idp.edina.ac.uk/shibboleth. EDINA uses this IdP for training and test purposes; as such, 100% availability was not required. It was, however, a suitable test-bed for proving the mechanisms described below.

The 1.3 EDINA IdP was initially installed by the Shibboleth QuickInstaller. This gave it a few specific characteristics which affected our choices as we applied the upgrade:

  • The IdP was free-standing in Tomcat (with no Apache front end)
  • Authentication was performed in the Tomcat container by a special purpose JAAS plugin.
  • The IdP used ports 8442 and 8443 for Browser and SOAP access respectively.
  • The IdP ran on a Tomcat 5 container using a Java 5 VM.

If your IdP configuration is different, you will need to adapt these instructions appropriately.

Details

1) Install, configure and register the Shibboleth 2.x IdP

We decided to use this upgrade opportunity also to upgrade the Java VM (to V6) and the Tomcat container (to V6). This had the additional benefit that it allowed a “side-by-side” installation with the existing IdP. The two containers were configured to listen on different ports, but both were resident on the same (virtual) hardware.

The new IdP was therefore installed on the same host, but on ports 443 and 8441.

1.1) Credentials

The 1.3 IdP credentials were used to configure the Shibboleth 2.x IdP. The locations of the 1.3 .key and .pem files are to be found in the <Credentials> section of the 1.3 idp.xml file: 

    <FileResolver Id="example_cred">
        <Key>
            <Path>file:/Shib1InstallPath/IdP/etc/keys/1.3Key.key</Path>
        </Key>
        <Certificate>
            <Path>file:/Shib1InstallPath/IdP/etc/keys/1.3cert.pem</Path>
        </Certificate>
    </FileResolver>

The .key and .pem files from the 1.3 installation were copied to the 2.x credentials directory and the appropriate change made in relying-party.xml: 

    <security:Credential id="IdPCredential" xsi:type="security:X509Filesystem">
        <security:PrivateKey>Shib2InstallPath/credentials/1.3Key.key</security:PrivateKey>
        <security:Certificate>Shib2InstallPath/credentials/1.3cert.pem</security:Certificate>
    </security:Credential>

Since both old and new IdPs were fronted by Tomcat (with no Apache), the .jks file was also copied and the appropriate modification made to Tomcat 6’s server.xml file.

1.2) Authentication

The custom JAAS module required minor changes to work with Shib 2, but was configured as expected in login.conf.

1.3) entityID

The new IdP was given (as a temporary measure) a similar entityID to that of the 1.3 IdP. The 2.x entityID was "https://idp.edina.ac.uk/shibboleth-XFR" and the 1.3 one was "https://idp.edina.ac.uk/shibboleth". This allowed easy editing when changing the entityID later on.

The entityID is specified in relying-party.xml in at least two places: in the <DefaultRelyingParty> and in the <AnonymousRelyingParty> elements. If your 2.x installation has explicit <RelyingParty> elements they may need to be changed as well.

1.4) Servlet path

The 2.x Servlet path was made to be the same as the 1.3 servlet path. The path for the 1.3 IdP was shibboleth-idp/ - the name of the war file. Since the deployment was in Tomcat via a context deployment fragment (described here), all that was needed was to name the file containing the context to be shibboleth-idp.xml.

1.5) Attributes

The old IdP took advantage of the smart scoping in Shibboleth 1.3. Hence the attribute store might contain "member" or "staff@trial.edina.ac.uk". The IdP would automatically append the smartscope where one was missing. The Shibboleth 2.x pre-scoped connector does not have this facility and so work was needed in the connector to deal with this restriction.

1.6) Registration

Once this was all done the new Shibboleth 2.x IdP was tested against TestShib and then registered in the UK federation metadata with entityID "https://idp.edina.ac.uk/shibboleth-XFR" and organisation display name "EDINA Shibboleth 2 (test)" and the IdP kept hidden from the federation WAYF. This allowed easy editing when changing the entityID later on. At this stage we ran tests against other federation SPs to ensure correct functioning. Of particular importance was to ensure that the eduPersonTargetedID was the same (and that both versions were released in SAML1 mode).

We did not widely announce the availability of this IdP to users of the EDINA IdP. We asked interested parties to test the IdP, but it was understood that the IdP (or more accurately the "...-XFR" entityID) would disappear shortly.

2) Add Shibboleth 2.x endpoints to the Shibboleth 1.3 installation

Three endpoints need to be added: SingleSignOn, AttributeAuthority and ArtifactResolution.

For a default Shibboleth 2.x IdP Installation, these are: 

    ...profile/Shibboleth/SSO
    ...profile/SAML1/SOAP/AttributeQuery
    ...profile/SAML1/SOAP/ArtifactResolution

2.1) Changes to the IdP configuration

We needed to add these endpoints to the idp.xml file. As mentioned above, the EDINA 1.3 IdP is configured on ports 8442 (Browser) and 8443 (SOAP), and so after editing, the relevant section of idp.xml was: 

    <ProtocolHandler implementation=
        "edu.internet2.middleware.shibboleth.idp.provider.ShibbolethV1SSOHandler">
        <Location>https://[^:/]+:8442/shibboleth-idp/SSO</Location> 
    </ProtocolHandler>

    <ProtocolHandler implementation=
        "edu.internet2.middleware.shibboleth.idp.provider.ShibbolethV1SSOHandler">
        <Location>https://[^:/]+:8442/shibboleth-idp/profile/Shibboleth/SSO</Location>
    </ProtocolHandler>

    <ProtocolHandler implementation=
        "edu.internet2.middleware.shibboleth.idp.provider.SAMLv1_AttributeQueryHandler">
        <Location>.+:8443/shibboleth-idp/AA</Location>
    </ProtocolHandler>

    <ProtocolHandler implementation=
        "edu.internet2.middleware.shibboleth.idp.provider.SAMLv1_AttributeQueryHandler">
        <Location>.+:8443/shibboleth-idp/profile/SAML1/SOAP/AttributeQuery</Location>
    </ProtocolHandler>

    <ProtocolHandler implementation=
        "edu.internet2.middleware.shibboleth.idp.provider.SAMLv1_1ArtifactQueryHandler">
        <Location>.+:8443/shibboleth-idp/Artifact</Location>
    </ProtocolHandler>

    <ProtocolHandler implementation=
        "edu.internet2.middleware.shibboleth.idp.provider.SAMLv1_1ArtifactQueryHandler">
        <Location>.+:8443/shibboleth-idp/profile/SAML1/SOAP/ArtifactResolution</Location>
    </ProtocolHandler>

    <ProtocolHandler implementation=
        "edu.internet2.middleware.shibboleth.idp.provider.Shibboleth_StatusHandler">
        <Location>https://[^:/]+:8442/shibboleth-idp/Status</Location>
    </ProtocolHandler>

2.2) Changes to <servlet-mapping>

The container also needs to be made aware of the new endpoints. This is done via the WEB-INF\web.xml in the war file. Hence we had to rebuild the war file. For a 1.3 IdP the source of this file is in the WebAppConfig subdirectory of the installation directory and is usually called dist.idp.xml. After editing, the relevant portion of the file looked like this: 

    <servlet-mapping>
        <servlet-name>IdP</servlet-name>
        <url-pattern>/SSO</url-pattern>
    </servlet-mapping>

    <servlet-mapping>
        <servlet-name>IdP</servlet-name>
        <url-pattern>/profile/Shibboleth/SSO</url-pattern>
    </servlet-mapping>

    <servlet-mapping>
        <servlet-name>IdP</servlet-name>
        <url-pattern>/AA</url-pattern>
    </servlet-mapping>

    <servlet-mapping>
        <servlet-name>IdP</servlet-name>
        <url-pattern>/profile/SAML1/SOAP/AttributeQuery</url-pattern>
    </servlet-mapping>

    <servlet-mapping>
        <servlet-name>IdP</servlet-name>
        <url-pattern>/profile/SAML1/SOAP/ArtifactResolution</url-pattern>
    </servlet-mapping>

    <servlet-mapping>
        <servlet-name>IdP</servlet-name>
        <url-pattern>/Artifact</url-pattern>
    </servlet-mapping>

    <servlet-mapping>
        <servlet-name>IdP</servlet-name>
        <url-pattern>/Status</url-pattern>
    </servlet-mapping>

2.3) Changes to <security-constraint>

As mentioned above, the EDINA 1.3 IdP relied on container authentication and so the new Shibboleth SSO endpoint needed to be protected. This required changes to the same file as the servlet-mapping changes. After editing, the appropriate section looked like this: 

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Shibboleth SSO Service</web-resource-name>
            <url-pattern>/SSO</url-pattern>
            <url-pattern>/profile/Shibboleth/SSO</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>idpuser</role-name>
        </auth-constraint>
    </security-constraint>

After the changes noted in sections 2.2 and 2.3 were made, we rebuilt the war file. This is done by running the 'ant' script from within in the installation folder and accepting the defaults. The war file was then redeployed; as it did so the new configuration was loaded.

At this stage the 1.3 IdP was listening on both the Shibboleth 1.3 endpoints and appropriate (SAML 1) Shibboleth 2.x endpoints.

We tested the new endpoints by putting appropriate metadata into TestShib. We extracted the appropriate <Entity> element from the UK federation metadata, embedded the certificates in the metadata (since TestShib does not rely on PKIX) and changed the endpoints to be the new ones. We then inserted it into the TestShib metadata and tested it.

This test gave us confidence that everything would continue to work when the UK federation metadata was changed in the next stage.

2.4) Swap the new endpoints into the UK federation metadata

We sent an email to UK federation support requesting that the 1.3 IdP's three endpoints be changed to their Shibboleth 2 equivalents. This change mirrors the change made in the TestShib metadata as part of testing the previous stage. For each of the three endpoints the hostname and port remains unchanged.

The SingleSignOnService location changed from: 

 <SingleSignOnService Binding="urn:mace:shibboleth:1.0:profiles:AuthnRequest"
     Location="https://hostname:port/shibboleth-idp/SSO" />

to: 

 <SingleSignOnService Binding="urn:mace:shibboleth:1.0:profiles:AuthnRequest"
     Location="https://hostname:port/shibboleth-idp/profile/Shibboleth/SSO"/>

The AttributeService changed from: 

 <AttributeService Binding="urn:oasis:names:tc:SAML:1.0:bindings:SOAP-binding" 
     Location="https://hostname:port/shibboleth-idp/AA" />

to: 

 <AttributeService Binding="urn:oasis:names:tc:SAML:1.0:bindings:SOAP-binding"
    Location="https://hostname:port/shibboleth-idp/profile/SAML1/SOAP/AttributeQuery"/>

And the ArtifactResolutionService location changed from: 

 <ArtifactResolutionService index="1"
     Binding="urn:oasis:names:tc:SAML:1.0:bindings:SOAP-binding"
     Location="https://hostname:port/shibboleth-idp/Artifact"

to: 

 <ArtifactResolutionService index="1"
     Binding="urn:oasis:names:tc:SAML:1.0:bindings:SOAP-binding"
     Location="https://hostname:port/shibboleth-idp/profile/SAML1/SOAP/ArtifactResolution"/>

3) Finish configuring the 2.x IdP

At this stage the Shibboleth 1.3 IdP was running and servicing requests on the old and new endpoints. In theory all SPs would move over within 2 days since SP managers are asked to update their metadata at least daily. However, we left this setup for a week to allow for failed updates and for less frequently updating SPs to get the new metadata.

As the metadata propagated we made the final changes to the new IdP. Two changes were required:

3.1) Convert the entityID to its final form

The mechanism to do this is described above. After this change the new IdP would not work in the UK Federation (because its entityID did not match its metadata).

At this stage we tested the new IdP against TestShib (not the UK federation). We used the same metadata which we used to test the 1.3 IdP above, but with adjusted port numbers. This test was important since it gave us confidence that the IdP would serve the UK federation correctly when it was started.

3.2) Configure the container to listen on the Shibboleth 1.3 IdP ports as well as on the default ports

In order to allow the new IdP to be swapped in, the container needs to be configured to listen on the same ports as the old IdP. Recall that these are 8442 (Browser) and 8443 (SOAP). We also wanted to move the configuration to listen on the more usual ports of 443 (Browser) and 8443 (SOAP).

This meant configuring the container to listen to:

  • 443 – as per the initial configuration
  • 8442 – as a Browser port
  • 8443 – as per the initial configuration.

Once this change was made the new Shibboleth IdP could not be started (because there would be a port clash) and the IdP remained quiescent.

At this stage we also asked for a change to the metadata for the new IdP, to reflect the final ports. Although this was not strictly required (since the IdP was down and would remain so), it prepared the way to make the final stage below significantly easier.

4) Switch over the IdPs

Once a suitable time had elapsed to allow the metadata to propagate, the IdPs were switched over. We chose a known quiet period during the weekend when there was no training and little, if any, other activity. The old IdP was stopped and the new IdP started. We did a quick test to ensure that the new IdP was running as expected. Had it failed we could have stopped the new IdP and restarted the old one. The worst case total downtime would therefore have been less than a minute.

5) Roll Forward the metadata

We checked the two <Entity> elements by hand to ensure that the SAML1 endpoints were the same, apart from the port numbers: the port numbers could differ because the Shibboleth 2.x IdP is not sensitive to port numbers in the way that the 1.3 IdP is.

The new IdP was left to run for a few days so as to allow any misconfiguration bugs to appear. In all cases the response to a loss of service would have been to restart the old IdP. After it became apparent that the new IdP had bedded down successfully, the new metadata was rolled in. This was achieved by asking UK federation Support to:

  1. Delete the old IdP metadata
  2. Change the entityID for the new IdP to that of the old IdP
  3. Change the organisation display name for the new IdP to that of the old IdP
  4. Make the new IdP visible to the UK federation WAYF.
 Rod Widdowson
 SDSS, EDINA