Low Availability

Side-by-Side !Solution

Geoff N. Hiten — Thu, 14 Aug 2008 17:41:29 GMT

Solutions and Projects were one of the really cool features introduced in SQL Server 2005, judging from the responses I got when I showed how it works. Personally, I use them a lot. SQL 2008 has the same feature in SQL Server Management Studio. However, SSMS 2008 breaks this feature in SQL 2005 SSMS when installed side-by-side.

See all the ugly details here:

https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=361706

This also affects 64-bit systems, I just happened to find it on a 32-bit box first.

I am guessing that this failure is inherited from SSMS's Visual Studio ancester, which may make it difficult for the SQL team to fix.. No matter how hard I try, I cannot come up with a good reason that it should work this way. I have thought of several bad reasons, but I will hold off on the rock throwing.

Wait for it...

Geoff N. Hiten — Thu, 07 Aug 2008 17:09:19 GMT

It looks like SQL 2008 may have a slight dependency issue. If you have already installed Visual Studio 2008, you will be blocked from installing SQL 2008 until you install Visual Studio SP1. The problem is that Visual Studio SP1 is not released yet. Our guys came in ahead of schedule and they still get no respect.

Not to worry, Visual Studio 2008 SP1 should be out very soon (think days, not weeks) and this problem goes away.

Microsoft actually documented this issue here:

Visual Studio 2008 SP1 may be required for SQL 2008 Installations

http://support.microsoft.com/kb/956139

--GNH

UPDATE: It's Here

http://www.microsoft.com/downloads/details.aspx?FamilyId=FBEE1648-7106-44A7-9649-6D9F6D58056E&displaylang=en

The worst IT job (at least for today)

Geoff N. Hiten — Fri, 28 Dec 2007 20:24:28 GMT

There is an old saying in politics and entertainment that there is no such thing as BAD publicity. Unfortunately, the opposite is true for IT, especially in the Database realm. If your name gets mentioned outside of a technology publication, you can bet it is something bad. Data losses, system outages, privacy disclosures, and other public failures are how DBAs get famous. Unfortunately, that is the world we operate in. Today we have another poor soul to add to the roll call of failure. The anonymous person in charge of the Webkinz web site database is now one of the most reviled people in IT, not to mention one of the most overworked. He has managed to single-handedly kill Christmas joy for untold numbers of small children (one of mine amongst them).

For those who do not have kids in the 5-12 year old range, here is the quick version of Webkinz. Take a Beanie-Baby like plush toy that costs pennies to make in China (slight upcharge for the unleaded version). Put a "secret code" on a toe-tag. The "Secret Code" gets you access to an on-line "world" where a virtual avatar of your pet can play. More activity on the site earns you points towards virtual stuff for your "adopted" pet. Sell plush toys for $8-$15 each to parents of demanding children. Expire accounts that do not register a new code within one year to drive ongoing demand. When you get "critical mass" sell ads in the virtual world. Of course, don't mention your plans for the last step until you actually do have critical mass. J

Sheer genius. Moneymaking gold mine, right? Except what happens when you sell more plush toys than your web site can handle? What happens when you end up with one of the most popular toys of the Christmas season amongst the tween and pre-tween age set? You would think that scalability would be easy to plan for here. You write a code, you add a user slot into your infrastructure. It ain't like a Super Bowl ad where you can only guess at the response. Plus, you plan that ALL of your customers will want to play with their new toys during Christmas. Free clue to site administrator: Kids are out of school during this time of year.

As of this writing, the Webkinz World web site has been effectively unusable since sometime early on Christmas day. The site operators threw in the towel late yesterday and took the entire thing offline for "maintenance". So, either we have a hopelessly incompetent IT Staff and DBA that cannot do basic math, or a company that willfully refused to put enough resources to fulfill their promises. Promises made to children. I doubt this is the image or publicity that Ganz (the parent company) wanted. Of course, they can always make it worse with a bad or nonexistent public communications response, but that remains to be seen.

So, what are you doing to make sure your employer never gets this kind of publicity? Capacity planning is not optional in our profession.

Update: SQL 2005 Build 3186 Cluster problem

Geoff N. Hiten — Mon, 12 Nov 2007 21:09:24 GMT

Microsoft does not have a complete resolution for this problem yet, but they have found some more details. Evidently the problem with SQL Agent failure only occurs on systems using a domain admin account for the SQL Agent Service account. Microsoft is not 100% sure yet, so this is just a preliminary finding. However, it does match my own personal experiences. Worst Practices always has a cost. Several DBAs just found that out the hard way. Just as a reminder, this problem only occurs on x64 clusters using SQL Server 2005 build 3186 and higher.

One step forward, two steps back.

Geoff N. Hiten — Thu, 18 Oct 2007 19:24:49 GMT

We sometimes do things as DBAs that are self defeating, especially regarding high availability. We can get so focused on the One True Thing™ that will solve all of our problems that we don't realize that the way we implement something can end up costing us all the benefit. Clustering is often seen as the complete solution to availability. Unfortunately, clustering adds complexity to the system which can then impact stability. The way we remove that uncertainty is to use high quality hardware that is tested and approved for clustering. We have the Windows Catalog for Clustering (formerly the Hardware Compatibility List) that tells us that our proposed solution will work. When we stray from this list, we are courting disaster. Most of the failures I have seen in the past can be attributed to NOT following the guidelines on recommended hardware. I have two items in particular that stick out as Low Availability solutions.

The first Low Availability technique I want to talk about is clustering blade servers. Let's look at what clustering does for us. Clustering's primary benefit is an immediate hot-standby server to protect us from hardware failure. Clustering will do nothing to stop or recover from a "DROP DATABASE Payroll" command. Blades, on the other hand, exist to reduce data center cost by consolidating hardware. Blades share various components depending on the manufacturer. Power supplies are almost always shared in a blade chassis. Some chassis share network switches, KVM connections, or even centrally stored boot images. These common connections are common points of failure. Some failures will take out all blades in a chassis. Some central configuration changes can drop a chassis offline. Common failure points reduce the benefit of a cluster, sometimes below the availability a stand-alone server can offer.

The second Low Availability hardware component for SQL Server is iSCSI. Right now, there is limited support for iSCSI and SQL, but even Microsoft cautions that iSCSI is not typically a high-performance solution (http://support.microsoft.com/kb/833770). Every single cluster I have been involved with that used iSCSI has had performance and stability issues. Every. Single. One. The stability issues are a result of the low-performance. Sometimes, I/O latency on the device means the Quorum drive becomes unresponsive. Nothing good happens after that. I know iSCSI is a cheap way to buy a multi-connected storage device, but "cheap" and "Highly Available" just don't go together. Sometimes the best you can do is to get a good stand-alone server and leave clustering until you can get a system that solves more availability issues than it creates.