Best & Worst Practices

How to Return Record #n

AjarnMark — Fri, 25 Jan 2008 07:52:40 GMT

I got this question in email a while ago, and it seems to come up quite a bit, which frankly surprises me. I guess it's because I almost never* write an application where I need to retrieve the 10th (or whatever number) record. I am always trying to retrieve the record for Joe Smith, or Order Number 1256487, or the last record entered on 12/31/2007, or... well, you get the point. I am frequently searching for records based on a data value. But I rarely, if ever, have had to go looking for the nth record.

I see this question often enough that it makes me wonder if it isn't a school assignment somewhere. It sounds like the type of question you would get in school (or maybe in an interview)... "Write a SQL statement that will return the 17th record in the table." Or maybe it comes from people using Access where they had the FIRST() statement...UGH!

Of course the problem with this is that there is no such thing as the first, or 17th or nth record by itself. There is no guarantee in a SQL Server database (nor probably any relational database) that records will be retrieved in a particular order without the use of an ORDER BY clause**. And if you cannot guarantee the order that records will be retrieved, then the ordinal position of a record is completely useless.

Now, after all of that, if you still think that you have a need to get the nth record, here is my suggestion. First add your ORDER BY clause. And second, read Graz's article entitled What's After TOP?

* The one exception is where I want the SQL Server to handle paging for me instead of my front-end system. If you are in that position, then I suggest you do a search on the term paging on the SQLTeam web site. There are numerous articles about how to accomplish this.

** OK, someone is likely going to point out that if you put a Clustered Index on the field that you want it to sort by, that it will retrieve the data in that order. For a single table retrieval, you may be right, but I wouldn't bet my career on that always working out, especially when joining other tables with clustered indexes. Also, you will end up in a world of hurt if you rely on that mechanism and down the road your Production DBA decides to redefine the clustered index onto different columns for overall system performance.

Even More SQLPrompt Goodness

AjarnMark — Fri, 28 Jul 2006 15:07:00 GMT

If you want to read more about the product and the upcoming changes, check out Red-Gate's forum for SQLPrompt. It includes a special price offer: $99 introductory offer for pre-orders.

SQLPrompt Revisited

AjarnMark — Thu, 27 Jul 2006 15:42:00 GMT

A couple of months ago, I wrote a quick post about SQLPrompt which had been purchased by Red-Gate Software. I have been using it since then and it really has proven to be a handy tool. Of course, like any free tool, there are a couple of things I'd like to see changed, and I fully expected taht Red-Gate already had plans for making those changes when they re-release it as a paid tool (called SPLPrompt 3). I have now had the chance to communicate via email with a couple of representatives from Red-Gate and can tell you a little more about the upcoming release.

Probably the most useful feature of SQLPrompt is when writing JOIN statements, it pops up a list of ON clauses for the tables in your query. If your table names are aliased, it uses the alias. If you have Foreign Keys defined between the tables, it recommends those. There is an option where you can also have it recommend fields that are named the same, whether there is a Foreign Key relationship or not. This makes writing JOINs pretty darn quick. Sometimes it would recommend several more options than you were looking for, but hey, better to have too many than not enough in this case. The free tool also stumbled occasionally in handling joins across databases on the same server. Among other things, I have been told that they have improved the JOIN support, so I look forward to checking out the new version.

I believe it is safe to post the summary list of improvements that Red-Gate communicated to me, which is:

Significantly increased speed and accuracy
Keystrokes much closer to Visual Studio
Greater ease of use
Fewer prompts for authentication
New tables recognized as you create them
Improved JOIN support

They currently expect the price of SQLPrompt 3 to be $195 per user when it is released in September. I do not yet have any information on group or site license discounts.

So, you still have a month to check out the free edition and budget for the new release. Happy coding!

SQLPrompt - Intellisense in Query Analyzer - Initial Review

AjarnMark — Wed, 24 May 2006 20:53:00 GMT

I just downloaded and installed the free tool SQLPrompt, now from Red-Gate Software, and I'm really liking it. It provides Intellisense or statement-completion features in SQL Query Analyzer, SQL Server Management Studio, Visual Studio, and other development tools. For example, it not only pops up a list of fields after you type a table name followed by period, but it will also lookup relationships and give you options for your JOIN ON statement (and of course a whole lot more). So, even if you are working in a small database where you already know all the fields and relationships, it will save you time typing.

It also provides snippet shortcuts, for example one that it comes with already setup is that you can type in ssf and hit TAB and it replaces it with the syntax SELECT * FROM. You can build your own snippets. Oh sure, I know that SQL Management Studio already has that ability, but if you still do any work in SQL 7 or 2000 in QA, it's a nice addition.

For now, until September 1, 2006, this tool is free. If I understand the scenario, Red-Gate bought this tool and plans to enhance it, so I would expect there to be a charge for it after September 1, or perhaps they will just roll it into one of their bundles. So, if you're interested, get it now!

Top 10 Blog Design Mistakes

AjarnMark — Mon, 09 Jan 2006 06:53:00 GMT

Jakob Nielsen, a renowned web design and usability expert, has come up with the Top 10 Design Mistakes for Weblog Usability. You might think, from the title, that there is nothing you can do because you did not design your weblog, you are using someone else's software. But in reality, they are all focused on things that the blog writer does or does not do, and are great things to keep in mind. I especially like this one:

9. Forgetting That You Write for Your Future Boss

How NOT to Select Records

AjarnMark — Wed, 01 Sep 2004 20:19:00 GMT

From The Daily WTF... How NOT to SELECT records in your database.

Apparently in his version of SQL Server, WHERE clauses only work on Temp tables.

Separate Your Singular From Your Plurals

AjarnMark — Sat, 14 Aug 2004 06:03:00 GMT

This is a real pet peeve of mine, and I know some of you consider what I'm about to rant about to be some form of “best practices” or that “it just makes sense”, but I don't like it and I wish you'd stop. Here's the issue: Singular vs. Plural names of objects and variables. Everyone will probably agree that have a naming convention for your database objects is important. And most developers agree that having meaningful variable names is important too. Now, do everyone a favor and stop mixing singulars and plurals together!

For database objects, just decide whether they are going to be singular or plural and stick with it. It is really annoying to have to remember that the table names are Employer and Customers. Or was that Employers and Customer? And Microsoft is guilty of this too! Have you looked at the Northwind and Pubs databases? Northwind: shippers and suppliers but region; Pubs: authors, discounts, jobs, and publishers, but employee, roysched, and titleauthor. My position is that not every tablename will make sense as a plural, but everything can make sense as singular, so I only use singular names.

I know some of you will argue that a table is a collection of records and so the table would be properly known as Employees and one record is an Employee, but I just don't buy it. It's easier to adapt to knowing that everything will be singular. If you want plurals, then talk about the Employee records that are found in the Employee table. Or, heck, talk about Employees all you want, but everyone knows that when they have to code, the real object name is Employee.

And programmers are just as bad with their variable names. It drives me crazy to see something like For Each Employee in Employees... or some such nonsense. First, it makes it hard to read all the other lines where you have to distinquish between whether the single or plural is being acted upon. And if you're editing, it's really easy to overlook whether the “s” was there or not or should have been. Second it makes all communication difficult. It is too easy to say or type the singular when you meant the plural or vice-versa. Try using something like EmployeeList for the plural, or ThisEmployee for the singular. Just don't make them identical except for one letter!

I actually ran into this problem today talking about two different tables in the same database. They were named almost identically, except to really irritate us, not only was there a mix of singular and plural, but somebody used an abbreviation for one of them. So, for example, assume that the table names were Letter and Ltrs. They are related but contain very different information. Now, imagine having a long-distance phone conversation with someone explaing that they need to make a change to the “letters” table. Which one am I talking about? Did I mistakenly add an “s” when I am talking about the Letter table? Or am I trying to pronounce Ltrs without saying L-T-R-S?

Save yourself the headache and money. Remember I bill by the hour, and if it takes an extra 10 minutes per day for us to go through the “Do you mean the X or Y table?” or “Oh! I thought you meant X not Y.” then that's nearly an extra hour per week that you are paying me just to clarify our communication because somebody used a bad naming convention. Over the course of a year-long contract, that's 6 full days' worth of work that you paid for.

What Language do you THINK in?

AjarnMark — Fri, 25 Jun 2004 06:39:00 GMT

A question on SQLTeam a few weeks ago got me thinking about the idea of what language you THINK in. Do you remember the movie Firefox? (Okay, that came out in 1982, before some of you were even born, but at the time I was a Clint Eastwood fan and into military aircraft, so back off!) The key to the movie is that the jet (codenamed Firefox) has a neural connection and the pilot controls things like firing armament by thinking the command. Of course, it being the 1980s, the bad guys we were stealing the plane from were the Russians (or probably more accurately Soviet Union) so Clint had to THINK in Russian.

Okay, maybe you don't like that example, so let's talk about learning any foreign language. You know you're truly fluent when you can think in that language instead of thinking in your native language and translating. You may be able to translate on the fly pretty fast, but you're nowhere near as efficient as someone who thinks in that language.

So when you're building systems, what language do you think in? Thinking in Access SQL is different from thinking in SQL Server T-SQL and both of those are very different from thinking in Oracle. And the differences are even greater when you're talking about stored procedures instead of ad-hoc query syntax. Another contrast we see on SQLTeam a lot is people who think in procedural languages like VB or C# versus those who think in set-based languages like SQL. You can tell what your thinking language is by looking at your initial thoughts on how to tackle a problem. A good friend and client of mine, who is very smart technically, tends to think procedurally. I can think procedurally, but I've also learned to think set-based. The difference often comes out when we are talking about various approaches to a problem. For example, his idea might be to extract a set of data and iterate through it categorizing and perhaps performing math; whereas my approach would be to see if we can craft a single SELECT statement with the right grouping and aggregate functions to have the database engine do all the hard work for us more efficiently. My approach works better when you can use a set-based language like SQL. His works better on some of the legacy systems we deal with that just don't support the syntax I would need. But it's interesting to see what causes us to come at the same problem with two very different solutions, and that being whichever language we are thinking in.

As for the question on SQLTeam...you'll notice that the person is essentially asking for a translation from Oracle, the language he's thinking in, to SQL Server syntax, but the recommended answers are to scrap that approach and instead learn to think in SQL Server.

So, what language do you think in? Is it the same one that you're currently being paid to use? If not, you are not as efficient as you could be. And your code may be horribly inefficient, too.

How Many Test/QA Instances do you Need?

AjarnMark — Thu, 03 Jun 2004 06:56:00 GMT

A while back, I blogged about having separate dev, test/QA, and production environments. Recently, I was discussing with one of my clients the fact that they really needed more than one test/QA environment because they needed specific scenarios in their data to test the functionality of a broad spectrum of code paths. But they also needed a large volume of data in order to do some performance testing. And you could argue that these two different raison d'etre of the test systems would be better served with separate instances.

So this got me thinking about just how many different test or QA instances do you need or would you want in an ideal world. Here are some thoughts about different types of testing:

Unit Tests: In many shops this is done by the developer to make sure that their code is doing what's expected, at least to pass the 80/20 rule (i.e. it works on the common or “normal“ data sets). Small data set is fine here.
Functionality QA Tests: These verify that the Unit Tests passed, and expand to include the “edge cases“ (the 20% of the scenarios that take up 80% of your time). Small data sets are fine, with the focus being on a broad spectrum of data scenarios.
Integration Testing: Do all the parts (units) which are often written by different people fit together properly?
Performance / Volume Tests: Ideally, an exact copy of your production environment to really know how long something will take. In practice, there is probably some smaller percentage that would allow you to give a statistically significant forecast.
Bug replication, testing, and analysis: Again, ideally and exact copy of your production environment so you can ultimately test your theory on a copy of the real data that had the original problem. Probably want this on a delayed-update such as refreshed nightly so you have time to catch the original data condition before the bug gets replicated into your test system. Also could be built from backups on an as-needed basis.
Staging: Test your build and deployment process. Needs to match production in terms of code and schema versions. Also needs broad enough variety of data to touch all components at least once.
Bug-fix / Minor Release Testing: May be same as Staging, but the emphasis is on making sure no new major version release code is included.
Multiple Supported Versions Testing: If you support multiple versions of your product for clients, then you'll probably want to test bugs reported to find out which versions are affected. If your latest version is 5.0, and a client running version 3.7 reports a bug, you want to know if it can be fixed by uprading the client to the latest version, or if everyone is facing the same problem.

Any others that come to mind for you? Add them in the comments section.

Scripts and Source Control - 102

AjarnMark — Wed, 31 Mar 2004 16:42:00 GMT

In the comments section of my SCRIPT IT post, Lisa asked a couple of questions for clarification. These were a good reminder to me that some of the things that I just take for granted these days are actually very new concepts to others who haven't had the same experience. So rather than just post a reply in comments, I thought it'd be worth a post unto its own. Lisa's questions are in summarized italics, with my responses following.

Q1) You stated: "Often in EM there is an option to 'Script Changes'”... I don't see any "Script Changes" in the BOL when I did a search. However, I did a search in the index of BOL and I find this: "script generating"...GREAT. However, when I open up the sql file, I was surprised by the amount of the code that is generated. I mean, there are about 60 lines of code!?!??!??! So far, I only created 1 db called, "TestingDB". I thought for a simple db, it should only a few line of commands.

A1) You found the option to generate scripts based on the current database settings. This is a good way to get an intial snapshot in script form of your database. As you have seen, there are many different options that will get scripted in order to have a script that would rebuild your database exactly the way it is now. Many of those options are default settings and many people don't bother to keep them in their scripts, but the script generator produces everything it can.

Where I was talking about scripting the changes, I meant, for example, changing the code in a stored procedure, or changing a table definition. Here's an example: Open Enterprise Manager, open the pubs database, click on tables, right-click authors and choose Design Table. You are now in the table definition grid. Change the length of column au_lname from 40 to 50. Now, BEFORE YOU SAVE the change you just made, on the toolbar there is a button that looks like a small disk with a scroll behind it (should be the third button from the left). That is the "Save Change Script" button. If you click it, it will generate a script in a window that represents the change you just made. Then you can save that to a file. But remember that here, too, the generator will provide a lot of standard default settings in the script and it also may not produce the most efficient way of making the change. Because it's an automated process, it follows certain rules in order to generate a script that definitely will produce the change you have just made, but there may be other more efficient ways of achieving the same result. As you gain more experience with different SQL statements, and since you can analyze the table and your needs, you may come up with shorter methods than what the generator produces.

Q2) You stated: "You can (and should) save your script files (usually ending with a .sql extension) into some form of source control (Visual Source Safe, Concurrent Versioning System, etc.). "
- What is the benefit of saving .sql script into VSS or CVS?
- Do I need a special softwares to open these files?

You also stated: "This allows you to know exactly what changes have been made, and if you ever need to “roll back“ you have a history of changes." I understand what you are saying but it is hard for me to visualize without real example.

A2) The .sql files are just plain text so they can be opened by any editor, including Notepad. Some people like to associate the .sql extension with Query Analyzer so if they double-click the file in Windows Explorer it opens with that. Personally, I don't want the overhead of opening QA when all I want is to do a quick view of the text file, so I use a simple editor like Notepad for quick reading. But I use QA for real editing because it has color formatting and I can execute tests while I'm working.

As far as the benefits of saving changes into source control, there are many. One is that it's a central storage area, so you always know where to find your scripts. Another is that it's a nice way to organize a history of copies of the file. Source control will be able to show you not only the current version of your script, but also previous versions. Most source control programs also let you do a "diff" which is a way of showing the differences between two versions. This is very handy for answering questions like "Just exactly what changes were made the last time?" I find this most useful when I'm working on scripts for stored procedures. And you can check out an old copy of the script and execute it in order to "rollback". For example, let's say you made changes to a stored procedure script, executed it in your database and saved (checked-in) the changes to your source control. Later you discover that there is something wrong with the change you made. Assuming that it is not significant enough to require restoring the database from backup, you could check-out the previous version of the script and execute it, thus effectively rolling back (undoing) the change you made. Of course this all only works if you've been checking-in the changes to your script and thus producing a history of changes.

Some source control systems automatically check in any changes you made every time you save or execute. I don't like this approach. I don't want the previous version listed in history to be my previous typo or some unit-test run with part of the code commented out for testing purposes. I want it to be truly the last version that was believed by the developer to be a working version, so I want it to be a conscious act to check-in updated scripts.

Hope that helps you Lisa, and all others who are coming in to this arena anew.