Jeff Smith Blog

Random Thoughts & Cartesian Products with Microsoft SQL Server

You Know, There's a Much Easier Way…

Let's say you are struggling on a programming project.  Your code is growing exponentially and becoming more convoluted by the day, and it is clearly out of control.  You're getting run-time errors, compile-time errors, wrong output, no output, endless loops, your machine is overheating, and perhaps you are starting to feel like you might be a little over your head.  Maybe this is the first time you are using a particular programming tool or language, or it's the first time you've worked with a particular database, or maybe it's your first complicated project overall in any technology and you're getting hopelessly lost.

So, it's come to this:  it's time to swallow your pride, and ask for help. 

Maybe you decide to visit a programming forum online, or maybe you have coworkers or colleagues or an old college professor that you can contact for help.  Either way, it is time to get to the bottom of this.  You have deadlines that you are getting dangerously close to missing.  People are starting to doubt your abilities; heck, you are even starting to question them yourself.

However, you are not a "clueless newbee".  You know how to break down the problem and explain the specifics in detail, so that makes it much easier for others to help you.  You have specific questions that you need answered, not just general "how to write a query?" questions.   Asking for help may not be ideal, but if you can get the right person to assist you, it should produce results and really get you back on track.

So, you post your question.   You've explained it well, and two pieces of advice come back.

Advice #1:

"Ah, a challenge!  Here's a pretty hacky idea that should work; cut and paste and try it.  It's about 85 lines long and you'll need to create 4 auxiliary temp tables, and you'll have to manually handle a few other situations that I didn't account for but this should serve as a guideline that you can expand upon.  It will probably be slow and inefficient and maybe a little buggy, but overall, it should fix the error you are getting and give you the result you need, at least in the short term."

Advice #2:

"You have a poor design that is really making what should be really simple nearly impossible.  Just break out these two columns into a new table to normalize your design, use the correct data types,  do the formatting at the presentation layer, and all you need is this one simple, single line of code which will do exactly what you want."

What would you do when presented with these options?  How would you respond?

The Common Response

Let's look at the most common reaction to this scenario.

First, you consider advice #1.  You've looked at his code and it is ingenious – at first glance you can't understand a single line!  It take about 2 hours to really process what he is doing, and a lot of it still is way over your head, but some of the hacks and tricks look pretty darn cool.   It isn't perfect, but it should do the trick. A little tweaking and maybe some cutting and pasting to handle a few other cases you didn't specify and you should be back in business in no time.  As for efficiency, that is not a factor here – you just need the darn thing to work! You can always optimize later.

As for advice #2 – WTF?  "Normalize?"  What a bunch of crap – theory is great, but you have a project to complete.   Who does this guy think he is preaching to you like that?   The first response was helpful and solved your problem – this one is just a waste of time, someone trying to stroke his ego online and demonstrate how smart he is.  And he is really trivializing your problem! To suggest that you have a "poor design" and that a single line of code that anyone could write will solve the issues you've been struggling with for weeks is an insult to you as a programmer. Thanks, but no thanks, jerk!

A "Radical" Alternative

Let's take a look at an alternate way to respond to the two pieces of advice offered.

Regarding advice #1: Yikes … if that's what it takes to solve this issue, there must be a better way… This approach may work, I suppose, but now my already impossibly messy code will get even worse and this will be a nightmare to maintain.  I must be doing something wrong if this is the best solution anyone can offer.

As for advice #2:  Wait a minute – there's an easier way to do this?  This is the best news I've heard all week!   "Normalizing" – I've heard of that, I should look that up to make sure I get it.  And using the right data type instead of strings everywhere – kind of new to me, I never thought of the importance of it, but if it helps and I can solve this thing in only one line of code, it is probably something I should learn about and try.  It would be really sweet if I could simplify this whole system by simply making some changes to the design – which I know is a mess.  It might be a bit of work, but it should be much less work in the long run if I can learn a few things about database/application design, and eliminate most of this crappy code I've been working on for weeks and replace it all with boring, simple, easy to read and maintain code! I knew there had to be an easier way!

Which Approach Do You Take?

So, how do you respond when someone suggests that you learn a new technique? Do you listen and consider the advice, or do you immediately reject it?  If someone recommends a "best practice" for you to follow, do you immediately assume that you are being preached to and spoken to condescendingly, or do you research the idea to see if it will help you?  If someone comments that you've made poor design decisions and recommends an alternative, do you get defensive and refuse to even ponder the possibility?

If I personally am struggling with anything, and someone offers to teach me an "easier way" – well, for some crazy reason, and unlike a lot of people – I listen!  And I listen with an open mind and really try to consider the approach described to me.  It may not ultimately be any better than what I am doing now, and in fact it may be very poor advice, but I at least listen to it and consider it.  The news that there is a "better way" to some people is an insult and upsetting; to me that's the best possible news I could hope to receive!

I think there's an odd mentality that programmers have when asking for help:  "Even though I am asking you for help, I am still smarter than you!"   I find the opposite approach works better: Assume you have something to learn from everybody, whether it is programmers on an online forum or people you meet face to face at a cocktail party.  You might be surprised at what you can learn if you have an open mind.

At my very first job years ago as a dishwasher, there was a sign hanging in the restaurant's kitchen that read: "Why is there never enough time to do a job right, but always enough time to do it over?"  And it's true in programming much as it is in the food service industry.  You might argue that altering a key design decision this late in the game is not possible.  Well, in some cases, it may not be – you've got to get it done and live with your decisions.  However, it is always worth 5-10 minutes to at least consider a new approach to your problem, and maybe even give it a try.  In fact, you might even learn a little something if you listen to advice that aims to improve your knowledge and skills as a programmer with a nice side effect of also simplifying your code, instead of complicating it.

In short, if you are struggling with your code, there usually is an easier way. The hardest part is often simply acknowledging it.

see also:

Legacy Comments

Jon H.
re: You Know, There's a Much Easier Way...
Great post, I actually get a lot of help understanding business rules from several functional groups in the company I work for. People will say, well why in the heck did you ask him / response is those are my customers. Programmers tend to assume a lot of things, then when its time to release they run around like a chicken with it's head cut off.

We're smart people but sometimes we don't give our customers credit..I used to fall prey to this when I first started out in the industry. Now I bug the hell out of my end users :-).

Excellent post though.


jeff m
re: You Know, There's a Much Easier Way...
Never worked as a dishwasher, but in the building trade, but learned the same lessons there. My dad told me a similar story about his growing up. He was pushing cement in a wheelbarrow along muddy ground, when a man of lower social class, who was "supposed" to be ( in those times ) of lesser intelligence clued him into an easier path involving a sidewalk. It was a real shocker for dad to find out he was a dumbass and didn't know everything. If we're lucky, we all have that experience, learn from it, and benefit. Thanks for the article.

re: You Know, There's a Much Easier Way...
Wisdom starts when you stop saying, Gee that's dumb, and start saying, What don't I understand?

re: You Know, There's a Much Easier Way...
That's interesting. I was just struggling with a dilemma, and off the sources I asked, all gave me one of the two responses you mentioned. Not much help. I was simple looking for a "Yeah, I would do that " or "No, here's what I would do." I think the hardest thing for people to do is put themselves into your shoes, which is why they give such roundabout responses.

Personally, I was just wondering if a TEXT data type will slow down a query even if it is empty or null.I have a table that has a varchar(7900) field for efficiency sake, and I need another few hundred chars that will usually be empty anyway. Would you add a text field, or an additional table?


re: You Know, There's a Much Easier Way...
Yes, there's a better way! Unfortunately you will see it only when you make something working and useful. Something that may look ugly and crude, but will contain a granule of truth, will give you something to work with, something to optimize, something for your brain to chew on, and to start recognizing grains of gold from plain sand, in a mudpool of your prototype code. Yes, you will HAVE to get your hands dirty, get on your knees, and up to your shoulders in mud. This is the only way you'll get your answers. It's the only silver bullet of the software design and architecture.

The only problem is that after all that trouble, when you will finally start seeing the light, and understand how to make your code simpler, will you actually do that? Will you want to delete and externalize extra lines of clutter code from a complex solution with no perfect answer? Will you want to wash away less perfect code-lines, when what you see in the code is not some uniformely spread out dirt, but clues and techniques that you will need to constantly keep in mind while searching for more gold pieces - making yet another additions to the working feature-set.

Liked the article also. Thanks.

re: You Know, There's a Much Easier Way...
The previous comment was meant to start with a sentence:

"> Regarding advice #1: Yikes ... if that's what it takes to solve this issue, there must be a better way"

Forgot to include it, sorry. Hope somebody with administrator priviledges will fix it. :)

Derek W
re: You Know, There's a Much Easier Way...
"Personally, I was just wondering if a TEXT data type will slow down a query even if it is empty or null.I have a table that has a varchar(7900) field for efficiency sake, and I need another few hundred chars that will usually be empty anyway. Would you add a text field, or an additional table?"

The maximum size of a row in ms sql is 8060 bytes. Having a varchar(7900) will put you near that max row size pretty quickly. SQL likes to store rows in a single page of the database. A page is 8192 bytes, 8060 of that useable. Large row sizes can make sql server slow. A TEXT data type is a pointer to a page (or pages) in the database. The page stores the content of the text field. If you are performing a query based on the value of the text data type, it will be slow if you don't use full text indexing on the table.

So, your question "would you add a text field, or an additional table" doesn't make much sense if the problem is caused by the "varchar(7900)".

re: You Know, There's a Much Easier Way...
Surely I can't be the only person who, given those two pieces of advice, would instinctively give more credence to the second one?

re: You Know, There's a Much Easier Way...
gwenhwyfaer -- You would not be the only one, but you'd be one of the few. Ever participate in a programming forum? People don't want to hear about things they should read up on or learn or techniques they *should* be using, they just want cut and paste code that solves their most immediate problem at hand. Or, at least 90% of them do. A big problem in our industry these days, and one that seems to be getting worse.

Martin Jiminez
re: You Know, There's a Much Easier Way...
Yes, A text datatype would certainly slow down this query, but only if it is null. That is my understanding. Thoughts?


re: You Know, There's a Much Easier Way...
I've been a developer for 20+ years and I occassionally catch myself having that same "wrong" reaction as you described. I've bookmarked your entry to remind myself of the "right" way to react when facing such delimas. Thank you for the article - its refreshing and inspirational.