Microsoft Updates and RSA Secure ID:

Today I had the pleasure of working with a customer who uses RSA Secure ID for VPN and MS Terminal Services access.  Without getting into too much detail, let’s just say anyone currently running RSA, or evaluating an RSA deployment, should pay special attention to Microsoft security updates and service packs.  Please realize that while RSA offers enhanced security for Windows networks, it comes at a price – mainly additional planning, testing, and the occasional troubleshooting exercise.  Read on for more information.

Issue #1: RSA Secure ID Agent 6.0 and Windows Server 2003 SP1 Domain Controllers don’t play well together.

If you try to update any Windows 2003 Domain Controller running RSA Secure ID Agent 6.0 to SP1 (either on purpose, or as the result of Automatic Updates) you will find yourself locked out after a reboot.  This is a known issue, and is resolved by RSA 6.0 Agent Patch 2.  Unfortunately, most people will discover this incompatibility after running into the problem.  If you find yourself in this situation, simply boot your DC using the Windows install CD and run a repair operation.  This replaces the GINA and some other stuff that RSA and 2003 fight over.  Fun times!  Don’t forget to apply Agent Patch 2 before attempting the SP1 update again ;)

Issue #2: Recommended lock down procedures for the RSA ACE Server prevent Windows Automatic Updates from functioning.

The RSA ACE Server (recently renamed RSA Authentication Manager) is the heart of an RSA deployment.  It processes requests from RSA Agents for dual-factor authentication.  Anyway – the primary ACE server, and any replica ACE servers, are a critical security risk if compromised.  Therefore, RSA recommends very strict lock down procedures for these servers.  While I’m all for the concepts of least privilege and a reduced attack surface, some of the RSA recommendations will cause you problems if you want to use Automatic Updates or install Windows 2003 SP1.  RSA recommends disabling multiple services – including the Cryptographic Service and the Distributed Transaction Coordinator.  Neither AU or SP1 will work correctly with these two services in this state.  No problem – simply configure them for manual or automatic startup type and you are good to go.  You can either leave these two services in an operational state at all times, or only change their startup type when you need to run updates.  The choice is up to you.  And by the way – make sure you apply RSA ACE Patch 2 while you’re at it.  This is recommended if you are also updating your RSA Agent installs to Patch 2.