Media Coverages
So Many Patches - So Little Time!
When's the right time to patch - as patching for the sake of patching must end.
By Eric Gross, Guest Columnist
June 2007 -- Every business wants the same thing: a database that services clients with the utmost performance and is always available. In the war against downtime, patches can be a rogue force. When something goes wrong, patches may let instability - the enemy - in through the front door. Patches always promise to fix a problem. If not for that promise, there would be no reason to let such an untested piece of code into your environment. Before you let any patch into your camp, you need to make sure it won't have a malicious impact or any other negative effects. It makes sense to initially let it only into a bastion where you can analyze its every impact. Patches should always be held to the standard of providing a positive result - not creating any negative effects which will outweigh the intended result. If possible, businesses should always wait for the subsequent patchset, which normally contains all recommended patches made public since the last release. If it's not feasible to wait that long, a patch bundle may be an alternative since bundles can be more widely deployed and tested.
Whenever a patch presents itself, you should vet it just as you would challenge any other change to a working system. Sometimes patches turn into candidates for installation based on a clear- cut symptom a specific patch has been issued to address. Other times, the confidence that a particular patch will fix a symptom is lower, and sometimes, it can be a shot in the dark. It is always prudent to search the Web for patch- related issues specifically related to that patch. Beyond that, take advantage of the limitless number of service requests available to you to confirm all implications of using a particular patch. You may find during your interaction with product support that you are missing a prerequisite for patch installation. This approach makes even more sense, because critical patch updates clutter the landscape in different ways based on the releases you are running. No matter how much confidence you have that a patch will remedy an issue, you should always test it in an environment that is as similar as possible to the production case. A point patch can break seemingly unrelated things, so testing needs to include a full regression suite (in addition to the obvious and critical confirmation that the patch fixes its intended symptom).
Patching is one of the endless reasons that you should minimize the number of configurations in play at any one time. Since you'll need to test each candidate environment for each patch, you should - if possible - reduce the number of unique configurations. Once you have decided to apply a patch, the next step is creating a procedure that you'll use to deploy and rollback this patch. Ideally this script will be completely automated so that one command takes care of the entire apply/ rollback use case: stop dependent processes, backup, apply patch, and restart processes. All of this should be performed in addition to your custom requirements - those related to change control or regulatory requirements. Each part of the procedure that is manually executed increases the probability of failure and inconsistent results. Equally important to the application process is the rollback process. This can be required without notice and in critical situations where unattended work is far preferable for its speed and reliability. Backups are a fundamental part of most rollback procedures, so you should be confident in your ability to restore to a working state.
Remember - once a patch is let loose, it is important that it be monitored. This crucial final step ensures that the intended results are realized and any negative effects are tracked and potentially resolved.
GridApp Systems' Eric Gross, has 12 years of experience working with large database infrastructures. Since 1994, he has provided smart and innovative ways for companies such as Register.com and eXtraStructure to manage their database environments more efficiently and securely. His realm of expertise includes feature development, performance- tuning, and the automation of database administration.