Patch Test Procedures?

[Brian Rodeck]

I couldn't find this somewhat obvious question in my searches so I'll just
ask:

I'm looking for a patch test methodology to follow for my Microsoft patches.
Everyone says to test patches before deploying them to my 100+ Microsoft
servers - how do I do that?

How do I make sure (for example) that every part of my branch Win2K server's
DHCP service works correctly? Is there a list of functions to test?

TIA for your help!

 

[Bill Sanderson]

I'm not your best respondent, I'm afraid, because I really don't have a lot
of experience in an objective test methodology, but I can give a little
practical experience.

I've been patching Microsoft networking OS's since they were based on
OS/2--but in very small shops--the largest of which involved perhaps 15
servers and a nationwide WAN. In all that time, the worst patch nightmare I
ran into was an incompatibility between an NT Service Pack and the drivers
for a multi-port serial board used to control modem banks for dial-in
users--I got in early in the morning after managing to download the SP,
installed it on the server, rebooted, and the thing blue-screened
immediately. It took some trial-and-error effort, as I recall to figure out
the solution, which was removing the hardware from the server until and
updated driver could be put in place.

Other issues I've seen others mention are application-program
incompatibilities.

I'd not worry, myself, about problems with the basic Microsoft services and
OS pieces--I would trust Microsoft's internal testing to do a good job with
those. I would worry about third-party hardware and drivers, and about
crucial application programs that are (relatively) unique to your usage.
Having a test server which you can load with the patch and some clear
methodology for testing a range of functionality that you know is crucial to
your network is what's needed.

So--I hope others will contribute some more realistic experience with
this--I suspect some shops use scripts to do some application level testing,
for example.

 

[Brian Rodeck]

Your point about trusting Microsoft to test against their own services
(DHCP, etc) is well-made, though I remember reading an article this month
about a Microsoft patch breaking another Microsoft service. I wasn't able
to find the URL to post here.

My dream world includes a written test script for my branch file/print
servers with screen shots showing how to test each service. I would like
that to include Microsoft services and wonder who else has already been
there, done that.

 

[Brian Rodeck]

Here is an example of why I want to test Microsoft services before
distributing a Microsoft patch. The Microsoft patch broke the Microsoft RAS
service on NT4 servers.

http://www.microsoft.com/security/security_bulletins/ms03-029.asp.

In the bulletin you'll see the following history:

7/23: Patch created/distributed

7/29: Notification that patch breaks RAS service

8/13: Redistribution of "updated patch" that doesn't break RAS service

 

[Bill Sanderson]

Here is an example of why I want to test Microsoft services before
distributing a Microsoft patch. The Microsoft patch broke the Microsoft RAS
service on NT4 servers.

http://www.microsoft.com/security/security_bulletins/ms03-029.asp.

In the bulletin you'll see the following history:

7/23: Patch created/distributed

7/29: Notification that patch breaks RAS service

8/13: Redistribution of "updated patch" that doesn't break RAS service

That's a valid example. I was lucky on that one. My clients servers were
Windows 2000, and the NT4 workstations I installed that one on could do
without the RAS services.

Other SBS admins with older version servers were bitten.

The groups in which I've found the best shared expertise about network admin
scripting have included microsoft.public.security.hfnetchk, and others
related to command-line patching--maybe
microsoft.public.security.baseline_analyzer.

The group best suited to the topic might well be:

microsoft.public.windows.server.scripting

 

[Brian Rodeck]

Thanks Bill, I'll post my question over at those forums as well.