I was amazed that in a roughly twelve month period, the largest open systems database players, namely Oracle and Microsoft, have once again introduced major changes to their patching methodologies. So much for anyone that has attempted to automate this process on their own as most of that work will now need to be thrown away. It seems the days of leaving critical systems unpatched, the most common behavior of database administrators, are long gone unless your business doesn't care about protecting its most valuable asset. How does an IT organization cope?

Let's briefly look at what the changes are.

The folks at Oracle have once again outdone themselves as it appears they have reversed direction with the introduction of the patch set update (PSU). If you're an Oracle DBA, I suspect you are aware of Oracle's critical patch updates (CPUs) that are also delivered quarterly. The introduction of CPUs meant a new OPatch methodology ('napply' vs. 'apply') and the wondrous management of molecules or individual patch 'components' that could be applied at the DBA's discretion. CPUs are security focused and often co-exist with a number of individual hot fixes. PSUs are a new beast all together. As opposed to security fixes they also focus on database stability. They use the old 'apply' methodology and seem to be more inline with a database upgrade than a patch (theoretically you get a new version number too, although that didn't work with the original PSU).

Well that doesn't seem like such a big deal until the concept of patch conflicts come into play. Oracle tells us that they will identify and roll back patch conflicts within the context of a patch event. As a user you will be responsible for finding (which may involve working with Oracle support on a case by case basis) and applying the individual hot fixes that are 'certified' with whatever PSU is being applied. If you don’t think this is a complete cluster f***, you're out of your mind. Think about it. A patch event goes from apply a patch to apply a patch, rollback 'n' number of hot fixes and reapply those hot fixes with the replacement patches (which may or may not be readily available).

While we can applaud Microsoft for being more consistent with methodologies (they do have a broader management ecosystem to deal with that people actually use, unlike Oracle) they have also introduced a fundamental change to patching in SQL Server 2008. In hope of minimizing downtime, passive nodes in a SQL Server cluster can now be patched while the active node remains operational. While this is useful, especially for shops striving for minimal downtime, the patching workflow has become orders of magnitude more complex. SQL Server DBAs now must patch the passive node(s) of a cluster, initiate a failover to a designated node, patch the former active node and then presumably failback. Sounds easy but for shops with hundreds if not thousands of instances this is a complete nightmare.

If you've invested in automation (home grown or via some DCA/workflow solution) be prepared to spend significant time reworking your solution. Much of which you have created will be useless and expect lots of errors in applying your new automation. Part of the problem here is that the user experience needs to have enough intelligence to reduce the chances of human error. It’s not just the scripts but the ability for the operational DBA to easily and reliably patch databases independent of version, patch type, etc.

GridApp Clarity is designed to handle these cases along with all the other patch complexities your organization is facing. The entire process is built from the perspective of the operator who must contend with human error, failed pre-requisites and ongoing updates to run book processes. GridApp handles all of these areas to deliver a practical and dynamic solution to address the increasingly complex challenge of patching your database.