GROUP BY

GROUP BY ALL

Jeff Smith — Mon, 05 May 2008 16:25:24 GMT

Here's an obscure piece of SQL you may not be aware of: The "ALL" option when using a GROUP BY.

Consider the following table:

Create table Sales
(
    SaleID int identity not null primary key,
    CustomerID int,
    ProductID int,
    SaleDate datetime,
    Qty int,
    Amount money
)

insert into Sales (CustomerID, ProductID, SaleDate, Qty, Amount)
select 1,1,'2008-01-01',12,400 union all
select 1,2,'2008-02-25',6,2300 union all
select 1,1,'2008-03-02',23,610 union all
select 2,4,'2008-01-04',1,75 union all
select 2,2,'2008-02-18',52,5200 union all
select 3,2,'2008-03-09',99,2300 union all
select 3,1,'2008-04-19',3,4890 union all
select 3,1,'2008-04-21',74,2840

SaleID      CustomerID ProductID   SaleDate                Qty         Amount
----------- ----------- ----------- ----------------------- ----------- ---------------------
9           1           1           2008-01-01 00:00:00.000 12          400.00
10          1           2           2008-02-25 00:00:00.000 6           2300.00
11          1           1           2008-03-02 00:00:00.000 23          610.00
12          2           4           2008-01-04 00:00:00.000 1           75.00
13          2           2           2008-02-18 00:00:00.000 52          5200.00
14          3           2           2008-03-09 00:00:00.000 99          2300.00
15          3           1           2008-04-19 00:00:00.000 3           4890.00
16          3           1           2008-04-21 00:00:00.000 74          2840.00

(8 row(s) affected)

Suppose we'd like to see the customers that were sold Product #1 along with the total amount that they spent.

We would basically write a simple SELECT with a GROUP BY like this:

select CustomerID, sum(Amount) as TotalAmount
from Sales
where ProductID = 1
group by CustomerID

And sure enough, we'd get our answer:

CustomerID TotalAmount
----------- ---------------------
1 1010.00
3 7730.00

(2 row(s) affected)

Now, let's say that we'd like to see all customers that have been sold any products, but we still just want to see the "TotalAmount" for ProductID #1. For customers that have never ordered ProductID #1, it should output a "TotalAmount" value of $0. One way to do this is with a CASE expression; instead of filtering so that only ProductID #1 is returned, we can conditionally SUM() the Amount only for orders for ProductID #1. Like this:

select CustomerID, sum(case when ProductID=1 then Amount else 0 end) as TotalAmount
from Sales
group by CustomerID

CustomerID TotalAmount
----------- ---------------------
1           1010.00
2           0.00
3           7730.00

(3 row(s) affected)

That gives us the results we want. Because we are not using a WHERE clause to filter the data, we see an entry for CustomerID #2 in the output.

However, in situations where you have written the above SQL, you could actually replace the SUM(CASE...) expression by using GROUP BY ALL, instead of just a standard GROUP BY, like this:

select CustomerID, sum(Amount) as TotalAmount
from Sales
where ProductID = 1
group by all CustomerID

CustomerID TotalAmount
----------- ---------------------
1           1010.00
2           NULL
3           7730.00
Warning: Null value is eliminated by an aggregate or other SET operation.

(3 row(s) affected)

Notice that now all Customers are now returned, and a NULL is shown as the TotalAmount for Customer #2, who has no orders for ProductID #1 ... Even though though the WHERE clause seems to indicate that we should not be seeing customer #2 in the results!

The ALL option basically says "ignore the WHERE clause when doing the GROUPING, but still apply it for any aggregate functions". So, in this case, the WHERE clause is not considered when generating the population of CustomerID values, but it is applied when calculating the SUM. This is very much like our first solution, where we removed the WHERE clause completely, and used a SUM(CASE...) expression to conditionally calculate the aggregate.

Values that are excluded from the aggregation according to the WHERE clause have NULL values returned, as you can see in the result. A simple ISNULL() or COALESCE() will allow us to return 0 instead of NULL:

select CustomerID, isnull(sum(Amount),0) as TotalAmount
from Sales
where ProductID = 1
group by all CustomerID

CustomerID TotalAmount
----------- ---------------------
1           1010.00
2           0.00
3           7730.00
Warning: Null value is eliminated by an aggregate or other SET operation.

(3 row(s) affected)

Notice that the warning about NULL being aggregated still displays, since that is the standard behavior in SQL Server when you calculate an aggregate on a NULL value. You can turn these warnings off if you like for the during of the batch by issuing a set ANSI_WARNINGS off command before your SELECT.

GROUP BY ALL is kind of obscure and neat to know, but not really useful in most situations since there are usually easier or better ways to get this result. For one thing, this won't work if we want all Customers to be displayed, since a customer must have at least one order to show up in the result. If we want to see all customers, even those that have never ordered, we would need to do a LEFT OUTER JOIN from the Customers table to our Orders aggregate SELECT:

create table Customers (CustomerID int primary key)
insert into Customers
select 1 union all
select 2 union all
select 3 union all
select 4

-- Notice that we have 4 customers, but our Sales data has sales for only 3.

select c.customerID, isnull(s.TotalAmount,0) as TotalAmount
from Customers c
left outer join
    (select customerID, sum(Amount) as TotalAmount
    from Sales
    where ProductID = 1
    group by customerID) s on c.customerID = s.customerID

customerID TotalAmount
----------- ---------------------
1           1010.00
2           0.00
3           7730.00
4           0.00

(4 row(s) affected)

That is typically the standard way to return data for an entire population, regardless of existing transactions. GROUP BY ALL gets us close, but if a new customer has never made an Order, they will never show up in the results. Of course, depending on your needs, that may be what you want.

Another limitation is we can not use GROUP BY ALL if we want to return a grand total for all orders, along with the total just for ProductID #1. For example, using the SUM(CASE...) expression along with a regular SUM(), we can do this:

select CustomerID, sum(case when ProductID=1 then Amount else 0 end) as Product1Amount,
    sum(Amount) as TotalAmount
from Sales
group by CustomerID

CustomerID Product1Amount        TotalAmount
----------- --------------------- ---------------------
1           1010.00               3310.00
2           0.00                  5275.00
3           7730.00               10030.00

(3 row(s) affected)

That lets us calculate two different totals all in one pass through the table. However, we cannot translate that using GROUP BY ALL, because while we will be able to return the Product1Amount, there would be no easy way to also get the TotalAmount for all products without an additional join or sub-query.

. . .

So, that's the story with GROUP BY ALL. It is interesting, and not widely well-known, and may even make for a good interview question if you really want to see how much SQL a candidate knows. But for practical purposes, it is pretty rarely used and there are generally better ways to get the same results more easily or more efficiently.

Anyone have a good situation or an example of where GROUP BY ALL really worked well for you? Be sure to share your experiences in the comments.

Simplify Your SQL with Variables and Derived Tables (or Common Table Expressions)

Jeff Smith — Thu, 20 Dec 2007 17:20:25 GMT

As with any programming language, it is important in SQL to keep your code short, clear and concise. Here are two quick tips that I find are very helpful in obtaining this goal.

Tip 1: For any relatively complicated constant expression, always declare a variable

Consider the following:

select *
from Transactions
where TranDate >= {some long, complicated expression to determine the start date} and
TranDate < {some long, complicated expression to determine the end date}

This is one of the most common things that I see in the SQLTeam forums, from both the people asking questions and the people giving answers. If the starting date and ending date range are constants for the entire SELECT (i.e., they don't vary by row, they are based on the current date or some parameters passed to the stored procedure), then simply declare them as variables, set the values once, and reference those variables in your WHERE clause:

declare @start datetime, @end datetime
set @start = {some long, complicated expression to determine the start date}
set @end = {some long, complicated expression to determine the end date}

select * from Transactions
where TranDate >= @start and TranDate < @end

Now, you could argue that we just technically made the code longer. But, we have made our code clearer and much easier to read and debug! We can now print and/or analyze the @start and @end dates to make sure that our formulas are working, without repeatedly running the entire SELECT over and over and glancing at the results and trying to determine if they "look right".

We can also potentially use these variables to make these formulas (or others) much simpler. For example, if the end date is always one day later than the start date, we can write:

declare @start datetime, @end datetime
set @start = {some long, complicated expression to determine the start date}
set @end = dateadd(day, 1, @start)

select * from Transactions
where TranDate >= @start and TranDate < @end

That is much, much easier to read, write and debug then if we repeated the long complicated expression twice, and it makes it very clear what the relationship is between the start date and the ending date. If we need to tweak our starting date formula, we can do it in one place and again it is easier to debug since we can just print out the value of the variable. We also have the ability to name our variables intelligently to help with the readability of our code.

Tip 2: To avoid repeating non-constant expressions, use a Derived Table or a Common Table Expression (CTE)

Another common thing found in lots of SQL code is to repeat the non-constant expressions over and over. For example, if we have a DateTime column called TranDate, this expression will round that date to the first day of the month:

dateadd(m, datediff(m,0, TranDate),0)

Using that knowledge, if we need to summarize transactions by month, the result often ends up looking something like this:

select
dateadd(m, datediff(m,0, TranDate),0) as [Month], sum(value)
from
table
where
dateadd(m, datediff(m,0, TranDate),0) >= @min and
dateadd(m, datediff(m,0, TranDate),0) <= @max
group by
dateadd(m, datediff(m,0, TranDate),0)
order by
dateadd(m, datediff(m,0, TranDate),0)

Notice that the date expression is repeated five times in the SQL statement! Looks kind of silly, but I see it over and over again from both beginner and experienced programmers alike. The solution to simplifying this is to calculate the formula once and assign it a meaningful alias using a derived table:

select
[Month], sum(value)
from
(
   select
     dateadd(m, datediff(m,0, TranDate),0) as [Month], value
   from
     table
) x
where
    x.[Month] >= @min and x.[Month] <= @max
group by
    x.[Month]
order by
    x.[Month]

Now the SELECT is much easier to read and maintain, and we can access the result of the calculation in the outer SELECT as many times as we need by referencing the "month" column returned by the derived table. Also, if our "month" formula is wrong, or needs tweaking or testing, it can all be done in one place and it doesn't need to be repeated over and over.

If you are using SQL Server version 2005 or greater, you can also use a Common Table Expression (CTE) in the same way. In fact, in many ways CTEs are even more readable and easier to work with than derived tables since the code doesn't require as much nesting and is more linear. Here's the previous example as a CTE:

with MonthSummary as
(
   select
     dateadd(m, datediff(m,0, TranDate),0) as [Month], value
   from
     table
)
select
[Month], sum(value)
from
MonthSummary
where
[Month] >= @min and [Month] <= @max
group by
[Month]
order by
[Month]

Repeating expressions is common for mathematical formulas as well, such as using CASE to avoid dreaded "divide by zero" errors when the denominator is an expression:

select
c as Numerator, a+b as Denominator,
case when a+b = 0 then null else c / (a+b) end as Result
from
tbl

Notice that a+b is repeated three times in the previous SELECT statement. Again, using a derived table, we can easily change this to:

select
Numerator,
Denominator,
case when Denominator = 0 the null else Numerator / Denominator end as Result
from
(
select
c as Numerator, a+b as Denominator
from
tbl
) x

... and once again, we can now clearly label our expression and see exactly what it is and our code is much clearer and easier to maintain. This example is quite simple and the difference doesn't seem like much, but if that expression is long and complicated and used in other places as well, the improvement in the code can be tremendous.

Conclusion

It is not just readability that we are concerned with, it is also eliminating and finding bugs and optimizing code. Repeating expressions in more than one place, or embedding them deep into SQL statements where we can never really be sure they are working, can result in bugs that are tough to track down. It is also can force the optimizer to calculate these expressions over and over as well, when they should optimally only be calculated once.

So, remember: for constant expressions that don't change, just declare and set variables, and examine the variables' contents before using them anywhere to ensure that your expressions are working as you intend. For repeated expressions that are based on values in the tables you are querying, consider calculating them once within a Derived Table or a Common Table Expression. In addition, be sure to use meaningful variable names and aliases which will help to greatly improve the clarity and intentions of your code.

It all comes back to the "golden rule" of programming: If you need to cut and paste, you're probably doing it wrong!

SELECT DISTINCT and ORDER BY

Jeff Smith — Thu, 13 Dec 2007 19:58:57 GMT

Let's take a look at another one of those stupid, arbitrary SQL Server error messages that Bill Gates clearly only created because Micro$oft is evil and incompetent and they want to annoy us (and probably kill baby squirrels, too):

Msg 145, Level 15, State 1, Line 4
ORDER BY items must appear in the select list if SELECT DISTINCT is specified.

This message pops up when you ask for DISTINCT rows for one set of columns, but you'd like to have the results ordered by one or more columns not specified in your distinct set. For some reason, SQL Server will not allow this! Why not?

Let's look at an example. Suppose we have the following data:

Letter Value
----    ------
A       1
A       1
B       3
B       3
C       2

From that, let's say we'd like see only DISTINCT letters, ordered by value. Clearly, this means we want to return:

A
C
B

... right? "A" has the lowest values of 1, followed "C" with a value of 2, and finally "B" with values of 3. Yet, if we write:

select distinct letter
from x
order by Value

we receive back the above error message. Why?

Remember, the "rules" of what is allowed and not allowed in SQL is not determined by how your data looks at a particular point in time; it is a constant. Just because one set of data seems to be perfectly orderable in this scenario does not mean that all data is always going to be perfectly orderable as well. What if we had this set of data instead:

Letter Value
----    -----
A       1
A       1
B       3
B       0
C       2

Look carefully at the above, and think about what should be returned when asking for distinct Letters ordered by Value.

The answer is .... there is no correct answer! B contains a value of both 0 and 3, so should it appear first, or should it appear last? The answer is -- who knows?! It is not specified when simply asking for distinct Letters ordered by Value. It is not a clear, complete, deterministic question that you are asking SQL Server to answer; it is like asking someone "what color is it when you add 4 and 3?" -- the question makes no sense and cannot be conclusively answered as stated.

Now, one person might say, "well, clearly, we want to B to be returned first, since it has the value of 0 and that is lower than all the other values", and another might say "well, clearly, we want B to be returned last, since it has the value of 3 and that is clearly higher than all the other values." And both people would technically be right, and both are entitled to their logic and their needs -- but that logic and those requirements need to be specified; they cannot be implied, since there is more than one way to interpret the question. Computers are funny that way, right? They are pretty darn smart, but they always insist on being told exactly what it is we want. How annoying!

(I often use this analogy regarding users or programmers who refuse to express their requirements or needs logically and completely: It is like a magic Genie saying "you can have anything in the world, all you need to do is tell me specifically what you want and make it perfectly clear and you will get it!", and the users thinking about it, and eventually concluding "nahh... not worth it, sounds like too much work; can't you just figure out what I want?")

So, what is the solution here? To make the error "just go away", we can try just adding the Value column to the SELECT clause like this:

select distinct Letter, Value
from x
order by Value

In our original set of data, this works fine -- it returns the results we want and expect, only with an additional column. If your data has a 1-1 relation between the ORDER BY columns and the SELECT columns, then simply adding the ORDER BY columns to your SELECT does the trick.

However, running that SELECT on the second set of data (in which "B" has values of 0 and 3) results in "B" being repeated twice:

B    0
A    1
C    2
B    3

To solve this, like usual, we simply must decide what we want, and explicitly tell SQL Server what that is. If we want to order by the letter using the lowest associated value, then we simply ask for that ... not by using DISTINCT, but by using GROUP BY:

select Letter
from x
group by Letter
order by min(Value)

This results in:

B
A
C

... And if we want to do the opposite, and order by the Letter with the greatest associated value, then we write:

select Letter
from x
group by Letter
order by max(Value)

A
C
B

We may even want to add up all of the values for each letter and order by that, or maybe there is some other rule. The key is that SQL Server cannot help you until you specifically express what you are looking for. And that is what this error message is all about -- "Hey, what you are asking for can be interpreted many different ways; can you be more specific?"

SQL (and any other programming language) really is like the magic Genie; it can give you whatever you want, but there is a catch -- you need to ask for it!

see also:

Some SELECTs will never return 0 rows -- regardless of the criteria

Jeff Smith — Tue, 13 Nov 2007 16:11:19 GMT

In SQL, the general rule of thumb is that the number of rows returned from a SELECT will be zero if your criteria did not match any data. However, there is an important exception to this rule: it does not apply when asking for aggregate calculations such as SUM(), MIN() or MAX(), without any grouping. This is rather interesting and important to know and look out for, as it can cause some confusion and recently some of my ASP.NET code failed due to this. Let's take a look.

We'll be using the following simple table for our examples:

create table tmp (f1 int, f2 int)

insert into tmp
select 1,1 union all
select 1,2 union all
select 2,3

Now, if we SELECT from this table without any criteria, we will get 3 rows returned:

select f1,f2 from tmp

f1          f2
----------- -----------
1           1
1           2
2           3

(3 row(s) affected)

If we put criteria on our SELECT that doesn't match any rows -- for example, where f2 > 3 -- we will get back zero rows:

select f1,f2 from tmp where f2 > 3

f1 f2
----------- -----------

(0 row(s) affected)

So far, so good. Let's try adding a GROUP BY clause to this SELECT. If we want just the the MAX(f2) per f1, we get back two rows:

select f1, max(f2) as MaxF2 from tmp group by f1

f1          MaxF2
----------- -----------
1           2
2           3

(2 row(s) affected)

Once again, let's constrain our results so that no data matches:

select f1, max(f2) as MaxF2 from tmp where f2 > 3 group by f1

f1 MaxF2
----------- -----------

(0 row(s) affected)

... and once again, no rows are returned. This should all be expected, and when writing our client code, we can do simple checks to verify if any data matched our SELECT statement's criteria simply by checking the row count. In the case of a SqlDataReader, the Read() method will return False immediately and we know that no rows were returned, so no data matched our criteria.

So, it seems that we have simply verified the following: In a SELECT statement, if the WHERE clause results in no matching rows in the underlying table(s), zero rows are returned in the result set.

However, there is an exception to this rule! Let's get the SUM() of the f1 column from the entire table, without any criteria or grouping. As expected, this will result in a single row returned with the total:

select sum(f1) as TotalF1 from tmp

TotalF1
-----------
4

(1 row(s) affected)

Next, let's filter the result, just as before, so that no rows match our criteria, and again ask for the SUM(f1). Since no rows match our criteria, we expect to once again get 0 results returned. However, notice what happens:

select sum(f1) as TotalF1 from tmp where f1 > 4

TotalF1
-----------
NULL

(1 row(s) affected)

A row is returned, with a Null value! Yet, there are *no* rows that match our criteria! This is very interesting, and rather important to know. Please note that there is a big difference between getting a single row back with a NULL value as opposed to getting no rows back at all, even though we might logically interpret the results the same way. In fact, if you just are selecting only aggregate calculations from a table, without any grouping, one row will always be returned -- never more, never less (note that this does not apply if you add a HAVING clause -- see below).

As an example, say that you have a simple stored procedure like this:

create procedure GetPreviousWeek @CurrentWeek DateTime
as
select Max(Week) as PreviousWeek
from Calendar
where Week < @CurrentWeek

Written as is, it is important to remember that you will always get a single row back, even if there is no previous week before the @CurrentWeek parameter! I had a similar stored procedure, and incorrectly wrote my client logic like this:

SqlDataReader r = cmd.ExecuteReader()

if (r.Read())
{
// there is a previous week
}
else
{
// there is not a previous week
}

That was based on the assumption that the WHERE clause would result in no rows being returned if there was no previous week and therefore no matching data. However, since the stored procedure was only calculating and returning an aggregate total, a single row is always returned, which meant that my logic was flawed. What I really needed was something like this:

SqlDataReader r = cmd.ExecuteReader()

if (r.Read())
{
if (r["PreviousWeek"] == dbNull.Value)
{
// there is no previous week
}
else
{
// there IS a previous week value
}
}
else
{
// this should never occur!
}

Notice the difference in the logic.

In short, I think it makes sense: If I ask for only an aggregate value for some data, even if that data has no rows, I should always get a value back -- though, of course, it may be NULL. However, if I specifically ask for a set of rows (by not aggregating or by providing a GROUP BY clause) then I should expect that I might not receive any rows back at all if no data matches my criteria.

Consider a "real world" example: If I asked you for the total population of all U.S. states that start with "Z", you would be able to reply with a single value (NULL, or perhaps 0). But, if I ask you to list all of the states that start with "Z" along with their individual population, the answer would be an empty set; you would not be able to give me a value or set of values, you would reply that "no states start with Z so there is no data to return."

Please note that adding a HAVING clause to filter your SELECT may indeed result in zero rows being returned, since the HAVING is applied after the aggregate is calculated. For example:

select sum(f1) as TotalF1 from tmp having sum(f1) > 10

TotalF1
-----------

(0 row(s) affected)

In that case, the single value is calculated, but it is not returned in the final results because the HAVING clause prevented it.

So, keep this in mind when returning single, aggregate values in a SELECT: You will always get a single row returned, and never an "empty set", unless you also provide a HAVING clause filter. If you have any columns in your GROUP clause, however, this does not apply and you may indeed get an empty set returned if your criteria does not match any data.

see also:

The Mailbag: Referencing Assemblies in Reporting Services; some SQL help

Jeff Smith — Thu, 18 Oct 2007 16:19:54 GMT

As David Letterman would say, wake the kids, call the neighbors, it's time for The Mailbag! Just some quickies today.

Christopher writes:

Greetings Jeff,

First and foremost, great job with all of the blogs. I have a questions
that I cannot seem to get a straight answer for. I am working with SQL
Server Reporting Services (SSRS) and have the need to create VB
functions to customize the reports generated. For example, a setter/getter to
display information that would not be readily available from the
query. SSRS allows this type of custom Visual Basic code to reside in the
report itself, but since most of my code is across multiple reports, it
makes 0 sense to place the same code in each report. Do you know of a
way that I can create a "code library" so that there is one location for
all of the code and all of the reports can access it?

Thank you very much for your time and effort.

Sincerely,
Christopher McGraw

Chris, Here's all the information that you need: Using Custom Assemblies with Reports (from SQL Server 2005 Books On Line). I can't really add much to it -- it explains everything in detail.

Traner writes:

I have two tables, table1's primary key is table2's foreign key.
Table2 has multiple dates that relate to table1.

ie.

Table1 : Table2
JOE : 10/18/07
DAVE : 11/14/07
JOE : 11/27/08

I am trying to get highest date from table2 related to table1. Do you
know how this can be accomplished. Any assistance will be greatly
appreciated.

That really isn't the clearest way to convey your schema, but I think I know what you mean. Let's assume that Table1 has two columns, ID and Name, and that table2 has 3 columns, ID, Date and Value.

First, to get the highest date per ID, you simply use the MAX() aggregate function along with GROUP BY when selecting from Table2:

select ID, Max(Date) as MaxDate
from Table2
group by ID

Be sure to read up and practice with GROUP BY and aggregate functions if you are not familiar with them -- they are very powerful and very fundamental to SQL.

Now, you can select from Table1 and join to the above SQL statement in a derived table to return all rows from Table1 and the maximum date from table 2:

select Table1.ID, Table1.Name, t2.MaxDate
from Table1
inner join
( select ID, Max(Date) as MaxDate
from Table2
group by ID
) t2 on Table1.ID = Table2.ID

As you can see, we have enclosed our previous SQL statement in parenthesis, and assigned it an alias ("t2" in this case), and we can join to it and select from it just like any other table. So, we have simply selected from Table1 and joined to a summarized version of Table2 that only returns the maximum date per ID.

Finally, notice that all we are returning from Table2 is the date; if there are other columns from Table2 that we wish to return (for example, the "Value" column), we must add another join to Table2, that returns only the row with the maximum date per ID:

select Table1.ID, Table1.Name, t2.MaxDate, Table2.Value
from Table1
inner join
( select ID, Max(Date) as MaxDate
from Table2
group by ID
) t2 on Table1.ID = Table2.ID
inner join Table2 on t2.ID = Table2.ID and t2.MaxDate = t2.Date

Notice that the "on" clause indicates that we are joining on ID as you might expect, but also on MaxDate = Date, so that we only return the row in Table2 that has the maximum date per ID.

If you are using SQL Server 2005, this can be done a little easier, with only one join, using the new Rank() function:

select x.ID, x.Name, x.Date, x.Value
from
(
select Table1.ID, Table1.Name, Table2.Date, Table2.Value,
rank() over (partion by Table1.ID order by Table2.Date DESC) as Rank
from Table1
inner join Table2 on Table1.ID = Table2.ID
) x
where x.Rank = 1

More on Rank() and partitions here. Hope this helps you out.

Group by Month (and other time periods)

Jeff Smith — Mon, 10 Sep 2007 15:28:56 GMT

When you need to summarize transactional data by Month, there are several ways to do it, some better than others. What to ultimately choose depends on your needs, but remember: Keep it short and simple in T-SQL, and always do all of your formatting at your presentation layer where it belongs. read more...

More on GROUP BY; Examining SUM(Distinct)

Jeff Smith — Tue, 31 Jul 2007 16:44:49 GMT

I've written a two part article on using SQL GROUP BY clauses over at SQLTeam.com. It's always a common topic of discussion and confusion amongst beginner and intermediate SQL programmers alike, so I thought I'd write a fairly long and hopefully comprehensive piece that takes a common summary report request and works towards the solution step-by-step.

Part I: Intro to GROUP BY; Duplicates caused by JOINS; Identifying "Virtual Primary Keys"; Using COUNT(Distinct)

Part II: Examining SUM(Distinct); GROUPING before JOINING; Using Derived Tables

I tried to focus on what happens when you join two tables and then try to group and aggregate the results, and where and how to do this to avoid aggregating duplicate values. As always, I attempt to demonstrate breaking down the larger problem into smaller, simpler parts and then putting those pieces together to produce the final result.

Perhaps most importantly: if you've ever used SUM(Distinct) in your code in an attempt to handle duplicates, be sure to read Part II. As Inigo Montoya would say: I do not think it means what you think it means!

But WHY Must That Column Be Contained in an Aggregate Function or the GROUP BY clause?

Jeff Smith — Fri, 20 Jul 2007 14:33:29 GMT

Column 'xyz' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.

Arggh!! There it is, yet again .. that annoying error message. Why is SQL so picky about this? What's the deal!? read more...

Using GROUP BY to avoid self-joins

Jeff Smith — Tue, 12 Jun 2007 15:35:33 GMT

Sometimes, it appears that a necessary solution to common SQL problems is to join a table to itself. While self-joins do indeed have their place, and can be very powerful and useful, often times there is a much easier and more efficient way to get the results you need when querying a single table.

read more...

SQL Server 2005: Specifying Partitions for Aggregate Functions

Jeff Smith — Mon, 21 May 2007 18:52:05 GMT

Did you know that a new feature in SQL Server 2005 allows you to specify an OVER partition for aggregate functions in your SELECT statements?

read more...