Forced Parameterization: A Turbo Button?

I never had the need to turn on the PARAMETERIZATION FORCED database option until this week.  We pretty much use only stored procedures for our internal applications so the execution plans are almost always in cache and reused.  This practice of using parameterized stored procedure calls, together with attention to detail in query and index tuning, allows us to comfortably handle several thousand requests per second on commodity hardware without taking special measures.

The Perfect Storm

We acquired a third-party application which had to sustain thousands of batch requests per second in order to keep up with our peak demand.  Our first attempt to use the application out-of-the box failed miserably when the 16-core database server quickly hit 100% CPU and stayed there.  An examination of the most frequently run query soon revealed why CPU was so high.  Not only was the moderately complex query not parameterized, each invocation required a full table scan.  The schema (EAV model, missing primary keys and indexes), application code (ad-hoc, non-parameterized queries) and inattention to indexing seemed the perfect storm to guarantee failure. 

Our hands were tied in what the vendor could/would do to address our performance concerns.  We worked with the vendor to optimize indexes and this brought the CPU down to about 65% but the batch requests/sec rate and slow response time was still unacceptable.   We needed to increase performance by at least an order of magnitude to meet SLAs.

The Perfect Fix

I then recalled an experience that SQL Server MVP Adam Machanic shared not long ago:

CPU was 95%+ at peak time (several thousand batch requests/second, via an ASP (classic) front end), and the peak time lasted 8+ hours every day.  The server was one of the big HP boxes -- not sure if it was a Superdome or some other model -- with something like 56 cores and 384 GB of RAM.  The database itself was only 40 or 50 GB, as I recall, so the entire thing was cached.  Long story short, I logged in during peak load, did a quick trace and noticed right away that none of the queries were parameterized.  I decided to throw caution to the wind and just go for it.  Flipped the thing into Forced Parameterization mode and held my breath as I watched the CPU counters *instantly* drop to 7% and stay there. I thought I'd broken the thing, but after checking my trace queries were running through the system same as before, and with the same number of errors (another story entirely <g>). Luckily the head IT guy happened to be watching his dashboard right as I made the change, and after seeing such an extreme result thought I was a god...

 

I knew of PARAMETERIZATION FORCED but never realized how big a difference the option could make until I learned of Adam's experience.  I'm not quite as adventuresome as he is so I restored the production database to a separate environment for some cursory testing.  To my amazement, I watched the rate of my single-threaded test jump from a few dozen batch requests/sec to several hundred immediately after I executed "ALTER DATABASE...SET PARAMETERIZATION FORCED".  CPU dropped by half even with the tenfold increase in throughput. 

The production improvement was even more impressive - the 16 core Dell R900 hasn't exceeded 8% CPU since the change.  Response time is excellent, we have happy users and plenty of CPU headroom to spare.

A Turbo Button?

Despite anecdotal success with PARAMETERIZATION FORCED, I wouldn't turn it on indiscriminately.  When the PARAMETERIZATION FORCED database option is on, all queries are parameterized, including complex ones.  This is good in that compilation costs are avoided due to cache hits.  The bad news is that a single plan might not be appropriate for all possible values of a given query.  Worse overall performance will result when higher execution costs (due to sub-optimal plans) exceed compilation savings so you should understand the query mix before considering the option.

In contrast, SQL Server parameterizes only relatively simple "no brainer" queries in the default PARAMETERIZATION SIMPLE mode.  This behavior promotes reuse of plans for queries that will yield the same plan anyway regardless of the literal values in the query.  Complex queries are not parameterized automatically so that the optimizer can generate the optimal plan for the values of the current query in the event of a cache miss.  The downside with simple parameterization, as Adam and I observed, is that complex queries not already in cache will incur costly compilation costs that are a CPU hog in a high-volume OLTP workload.

There is also middle ground between PARAMETERIZATION SIMPLE and PARAMETERIZATION FORCED.  One can use plans guides with PARAMETERIZATION SIMPLE to avoid compilation for selected queries while other complex queries are compiled as normal.  In my case, a plan guide may have been a better option because the culprit was a single query rather than many different unpredictable ones.

In my opinion, the best solution is to use stored procedures and/or parameterized queries in the first place.  These methods provide the performance benefits of PARAMETERIZATION FORCED and add other security and application development benefits.  Unfortunately, third-party vendors are notorious for not following parameterization Best Practices so DBAs need to keep PARAMETERIZATION FORCED and plan guides in their tool belt.

 

Restore Database Stored Procedure

A user in the SQL Server public newsgroups asked about how to restore a database with many files and rename during the process:

I am restoring a database onto another server with different drive
sizes and mappings.
The thing is, I have over 100 catalogs to restore. I don't want to
have to define each catalog name and its new location Like below:

RESTORE DATABASE Northwinds
FROM DISK = 'C:\db.bak'
WITH MOVE 'Catalog1' TO 'D:\Catalog1'
WITH MOVE 'Catalog2' TO 'D:\Catalog2
WITH MOVE 'Catalog3' TO 'D:\Catalog3'
WITH MOVE 'Catalog4' TO 'D:\Catalog4
WITH MOVE 'Catalog5' TO 'D:\Catalog5'
WITH MOVE 'Catalog6' TO 'D:\Catalog6'
...WITH MOVE 'Catalog100' TO 'D:\Catalog100'

Is it possible to restore the catalgos using a wilcard as such?

RESTORE DATABASE Northwinds
FROM DISK = 'C:\db.bak'
WITH MOVE 'Catalog%' TO 'D:\Catalog%'

 

This reminded me of a stored procedure I wrote several years ago for SQL Server 2000 that would be perfect for such a task.  The proc generates and optionally executes the necessary RESTORE and ALTER commands to make quick work of what is otherwise a long and tedious process if you have many files and databases.  I updated my old proc for SQL Server 2008 and thought I'd share it here.    Below is the proc with documentation and samples in the comments.  I hope you find this useful.

IF OBJECT_ID(N'tempdb..#RestoreDatabase_SQL2008') IS NOT NULL

      DROP PROCEDURE #RestoreDatabase_SQL2008

GO

 

CREATE PROCEDURE #RestoreDatabase_SQL2008

      @BackupFile nvarchar(260),

      @NewDatabaseName sysname = NULL,

      @FileNumber int = 1,

      @DataFolder nvarchar(260) = NULL,

      @LogFolder nvarchar(260) = NULL,

      @ExecuteRestoreImmediately char(1) = 'N',

      @ChangePhysicalFileNames char(1) = 'Y',

      @ChangeLogicalNames char(1) = 'Y',

      @DatabaseOwner sysname = NULL,

      @AdditionalOptions nvarchar(500) = NULL

AS

 

/*

 

This procedure will generate and optionally execute a RESTORE DATABASE

script from the specified disk database backup file.

 

Parameters:

 

      @BackupFile: Required. Specifies fully-qualified path to the disk

            backup file. For remote (network) files, UNC path should

            be specified.  The SQL Server service account will need

            permissions to the file.

 

      @NewDatabaseName: Optional. Specifies the target database name

            for the restore.  If not specified, the database is

            restored using the original database name.

 

      @FileNumber: Optional. Specifies the file number of the desired

            backup set. This is needed only when when the backup file

            contains multiple backup sets. If not specified, a

            default of 1 is used.

 

      @DataFolder: Optional. Specifies the folder for all database data

            files. If not specified, data files are restored using the

            original file names and locations.

 

      @LogFolder: Optional. Specifies the folder for all database log

            files. If not specified, log files are restored to the

            original log file locations.

 

      @ExecuteRestoreImmediately: Optional. Specifies whether or not to

            execute the restore. When, 'Y' is specified, then restore is

            executed immediately.  When 'Y' is specified, the restore script

            is printed but not executed. If not specified, a default of 'N'

            is used.

           

      @ChangePhysicalFileNames: Optional. Indicates that physical file

            names are to be renamed during the restore to match the

            new database name. When 'Y' is specified, the leftmost

            part of the original file name matching the original

            database name is replaced with the new database name. The

            file name is not changed when 'N' is specified or if the

            leftmost part of the file name doesn't match the original

            database name. If not specified, a default of 'Y' is used.

 

      @ChangeLogicalNames: Optional. Indicates that logical file names

            are to be renamed following the restore to match the new

            database name. When 'Y' is specified, the leftmost part

            of the original file name matching the original database

            name is replaced with the new database name. The file name

            is not changed when 'N' is specified or if the leftmost

            part of the file name doesn't match the original database

            name. If not specified, a default of 'Y' is used.

           

      @DatabaseOwner: Optional. Specifies the new database owner

            (authorization) of the restored database.  If not specified, the

            database will be owned by the accunt used to restore the database.

           

      @AdditionalOptions:  Optional.  Specifies options to be added the the

            RESTORE statement WITH clause (e.g. STATS=5, REPLACE).  If not

            specified, only the FILE and MOVE are included.

 

Sample usages:

 

      --restore database with same name and file locations

      EXEC #RestoreDatabase_SQL2008

            @BackupFile = N'C:\Backups\Foo.bak',

            @AdditionalOptions=N'STATS=5, REPLACE';

           

      Results:

      --Backup source: ServerName=MYSERVER, DatabaseName=Foo, BackupFinishDate=2009-06-13 11:20:52.000

      RESTORE DATABASE [MyDatabase]

            FROM DISK=N'C:\Backups\Foo.bak'

            WITH

                  FILE=1, STATS=5, REPLACE

 

      --restore database with new name and change logical and physical names

      EXEC #RestoreDatabase_SQL2008

            @BackupFile = N'C:\Backups\Foo.bak',

            @NewDatabaseName = 'Foo2';

           

      Results:

      --Backup source: ServerName=MYSERVER, DatabaseName=Foo, BackupFinishDate=2009-06-13 11:20:52.000

      RESTORE DATABASE [Foo2]

            FROM DISK=N'C:\Backups\Foo.bak'

            WITH

                  FILE=1,

                        MOVE 'Foo' TO 'C:\DataFolder\Foo2.mdf',

                        MOVE 'Foo_log' TO 'D:\LogFolder\Foo2_log.LDF'

      ALTER DATABASE [Foo2]

                        MODIFY FILE (NAME='Foo', NEWNAME='Foo2');

      ALTER DATABASE [Foo2]

                        MODIFY FILE (NAME='Foo_log', NEWNAME='Foo2_log');

                       

      --restore database to different file folders and change owner after restore:

      EXEC #RestoreDatabase_SQL2008

            @BackupFile = N'C:\Backups\Foo.bak',

            @DataFolder = N'E:\DataFiles',

            @LogFolder = N'F:\LogFiles',

            @DatabaseOwner = 'sa',

            @AdditionalOptions=N'STATS=5;

           

      Results:

      --Backup source: ServerName=MYSERVER, DatabaseName=Foo, BackupFinishDate=2009-06-13 11:20:52.000

      RESTORE DATABASE [Foo]

            FROM DISK=N'C:\Backups\Foo.bak'

            WITH

                  FILE=1,

                        MOVE 'Foo' TO 'E:\DataFiles\Foo.mdf',

                        MOVE 'Foo_log' TO 'F:\LogFiles\Foo_log.LDF'

      ALTER AUTHORIZATION ON DATABASE::[Foo] TO [sa]

*/

 

SET NOCOUNT ON;

 

DECLARE @LogicalName nvarchar(128),

      @PhysicalName nvarchar(260),

      @PhysicalFolderName nvarchar(260),

      @PhysicalFileName nvarchar(260),

      @NewPhysicalName nvarchar(260),

      @NewLogicalName nvarchar(128),

      @OldDatabaseName nvarchar(128),

      @RestoreStatement nvarchar(MAX),

      @Command nvarchar(MAX),

      @ReturnCode int,

      @FileType char(1),

      @ServerName nvarchar(128),

      @BackupFinishDate datetime,

      @Message nvarchar(4000),

      @ChangeLogicalNamesSql nvarchar(MAX),

      @AlterAuthorizationSql nvarchar(MAX),

      @Error int;

 

DECLARE @BackupHeader TABLE

      (

      BackupName nvarchar(128) NULL,

      BackupDescription  nvarchar(255) NULL,

      BackupType smallint NULL,

      ExpirationDate datetime NULL,

      Compressed tinyint NULL,

      Position smallint NULL,

      DeviceType tinyint NULL,

      UserName nvarchar(128) NULL,

      ServerName nvarchar(128) NULL,

      DatabaseName nvarchar(128) NULL,

      DatabaseVersion int NULL,

      DatabaseCreationDate  datetime NULL,

      BackupSize numeric(20,0) NULL,

      FirstLSN numeric(25,0) NULL,

      LastLSN numeric(25,0) NULL,

      CheckpointLSN  numeric(25,0) NULL,

      DatabaseBackupLSN  numeric(25,0) NULL,

      BackupStartDate  datetime NULL,

      BackupFinishDate  datetime NULL,

      SortOrder smallint NULL,

      CodePage smallint NULL,

      UnicodeLocaleId int NULL,

      UnicodeComparisonStyle int NULL,

      CompatibilityLevel  tinyint NULL,

      SoftwareVendorId int NULL,

      SoftwareVersionMajor int NULL,

      SoftwareVersionMinor int NULL,

      SoftwareVersionBuild int NULL,

      MachineName nvarchar(128) NULL,

      Flags int NULL,

      BindingID uniqueidentifier NULL,

      RecoveryForkID uniqueidentifier NULL,

      Collation nvarchar(128) NULL,

      FamilyGUID uniqueidentifier NULL,

      HasBulkLoggedData bit NULL,

      IsSnapshot bit NULL,

      IsReadOnly bit NULL,

      IsSingleUser bit NULL,

      HasBackupChecksums bit NULL,

      IsDamaged bit NULL,

      BeginsLogChain bit NULL,

      HasIncompleteMetaData bit NULL,

      IsForceOffline bit NULL,

      IsCopyOnly bit NULL,

      FirstRecoveryForkID uniqueidentifier NULL,

      ForkPointLSN decimal(25, 0) NULL,

      RecoveryModel nvarchar(60) NULL,

      DifferentialBaseLSN decimal(25, 0) NULL,

      DifferentialBaseGUID uniqueidentifier NULL,

      BackupTypeDescription  nvarchar(60) NULL,

      BackupSetGUID uniqueidentifier NULL,

      CompressedBackupSize binary(8) NULL

);

 

DECLARE @FileList TABLE

      (

      LogicalName nvarchar(128) NOT NULL,

      PhysicalName nvarchar(260) NOT NULL,

      Type char(1) NOT NULL,

      FileGroupName nvarchar(120) NULL,

      Size numeric(20, 0) NOT NULL,

      MaxSize numeric(20, 0) NOT NULL,

      FileID bigint NULL,

      CreateLSN numeric(25,0) NULL,

      DropLSN numeric(25,0) NULL,

      UniqueID uniqueidentifier NULL,

      ReadOnlyLSN numeric(25,0) NULL ,

      ReadWriteLSN numeric(25,0) NULL,

      BackupSizeInBytes bigint NULL,

      SourceBlockSize int NULL,

      FileGroupID int NULL,

      LogGroupGUID uniqueidentifier NULL,

      DifferentialBaseLSN numeric(25,0)NULL,

      DifferentialBaseGUID uniqueidentifier NULL,

      IsReadOnly bit NULL,

      IsPresent bit NULL,

      TDEThumbprint varbinary(32) NULL

 );

 

SET @Error = 0;

 

--add trailing backslash to folder names if not already specified

IF LEFT(REVERSE(@DataFolder), 1) <> '\' SET @DataFolder = @DataFolder + '\';

IF LEFT(REVERSE(@LogFolder), 1) <> '\' SET @LogFolder = @LogFolder + '\';

 

-- get backup header info and display

SET @RestoreStatement = N'RESTORE HEADERONLY

      FROM DISK=N''' + @BackupFile + ''' WITH FILE=' + CAST(@FileNumber as nvarchar(10));

INSERT INTO @BackupHeader

      EXEC('RESTORE HEADERONLY FROM DISK=N''' + @BackupFile + ''' WITH FILE = 1');

SET @Error = @@ERROR;

IF @Error <> 0 GOTO Done;

IF NOT EXISTS(SELECT * FROM @BackupHeader) GOTO Done;

SELECT

      @OldDatabaseName = DatabaseName,

      @ServerName = ServerName,

      @BackupFinishDate = BackupFinishDate

FROM @BackupHeader;

IF @NewDatabaseName IS NULL SET @NewDatabaseName = @OldDatabaseName;

SET @Message = N'--Backup source: ServerName=%s, DatabaseName=%s, BackupFinishDate=' +

      CONVERT(nvarchar(23), @BackupFinishDate, 121);

RAISERROR(@Message, 0, 1, @ServerName, @OldDatabaseName) WITH NOWAIT;

 

-- get filelist info

SET @RestoreStatement = N'RESTORE FILELISTONLY

      FROM DISK=N''' + @BackupFile + ''' WITH FILE=' + CAST(@FileNumber as nvarchar(10));

INSERT INTO @FileList

      EXEC(@RestoreStatement);

SET @Error = @@ERROR;

IF @Error <> 0 GOTO Done;

IF NOT EXISTS(SELECT * FROM @FileList) GOTO Done;

 

-- generate RESTORE DATABASE statement and ALTER DATABASE statements

SET @ChangeLogicalNamesSql = '';

SET @RestoreStatement =

      N'RESTORE DATABASE ' +

      QUOTENAME(@NewDatabaseName) +

      N'

      FROM DISK=N''' +

      @BackupFile + '''' +

      N'

      WITH

            FILE=' +

      CAST(@FileNumber as nvarchar(10))

DECLARE FileList CURSOR LOCAL STATIC READ_ONLY FOR

      SELECT

            Type AS FileTyoe,

            LogicalName,

            --extract folder name from full path

            LEFT(PhysicalName,

                  LEN(LTRIM(RTRIM(PhysicalName))) -

                  CHARINDEX('\',

                  REVERSE(LTRIM(RTRIM(PhysicalName)))) + 1)

                  AS PhysicalFolderName,

            --extract file name from full path

            LTRIM(RTRIM(RIGHT(PhysicalName,

                  CHARINDEX('\',

                  REVERSE(PhysicalName)) - 1))) AS PhysicalFileName

FROM @FileList;

 

OPEN FileList;

 

WHILE 1 = 1

BEGIN

      FETCH NEXT FROM FileList INTO

            @FileType, @LogicalName, @PhysicalFolderName, @PhysicalFileName;

      IF @@FETCH_STATUS = -1 BREAK;

 

      -- build new physical name

      SET @NewPhysicalName =

            CASE @FileType

                  WHEN 'D' THEN

                        COALESCE(@DataFolder, @PhysicalFolderName) +

                        CASE

                              WHEN UPPER(@ChangePhysicalFileNames) IN ('Y', '1') AND

                                    LEFT(@PhysicalFileName, LEN(@OldDatabaseName)) = @OldDatabaseName

                              THEN

                                    @NewDatabaseName + RIGHT(@PhysicalFileName, LEN(@PhysicalFileName) - LEN(@OldDatabaseName))

                              ELSE

                                    @PhysicalFileName

                        END

                  WHEN 'L' THEN

                        COALESCE(@LogFolder, @PhysicalFolderName) +

                        CASE

                              WHEN UPPER(@ChangePhysicalFileNames) IN ('Y', '1') AND

                                    LEFT(@PhysicalFileName, LEN(@OldDatabaseName)) = @OldDatabaseName

                              THEN

                                    @NewDatabaseName + RIGHT(@PhysicalFileName, LEN(@PhysicalFileName) - LEN(@OldDatabaseName))

                              ELSE

                                    @PhysicalFileName

                        END

            END;

 

      -- build new logical name

      SET @NewLogicalName =

            CASE

                  WHEN UPPER(@ChangeLogicalNames) IN ('Y', '1') AND

                        LEFT(@LogicalName, LEN(@OldDatabaseName)) = @OldDatabaseName

                        THEN

                              @NewDatabaseName + RIGHT(@LogicalName, LEN(@LogicalName) - LEN(@OldDatabaseName))

                        ELSE

                              @LogicalName

            END;

           

      -- generate ALTER DATABASE...MODIFY FILE statement if logical file name is different

      IF @NewLogicalName <> @LogicalName

            SET @ChangeLogicalNamesSql = @ChangeLogicalNamesSql + N'ALTER DATABASE ' + QUOTENAME(@NewDatabaseName) + N'

                  MODIFY FILE (NAME=''' + @LogicalName + N''', NEWNAME=''' + @NewLogicalName + N''');

'

 

      -- add MOVE option as needed if folder and/or file names are changed

      IF @PhysicalFolderName + @PhysicalFileName <> @NewPhysicalName

      BEGIN

            SET @RestoreStatement = @RestoreStatement +

                  N',

                  MOVE ''' +

                  @LogicalName +

                  N''' TO ''' +

                  @NewPhysicalName +

                  N'''';

      END;

 

END;

CLOSE FileList;

DEALLOCATE FileList;

 

IF @AdditionalOptions IS NOT NULL

      SET @RestoreStatement =

            @RestoreStatement + N', ' + @AdditionalOptions

           

IF @DatabaseOwner IS NOT NULL

      SET @AlterAuthorizationSql = N'ALTER AUTHORIZATION ON DATABASE::' +

            QUOTENAME(@NewDatabaseName) + N' TO ' + QUOTENAME(@DatabaseOwner)

ELSE

      SET @AlterAuthorizationSql = N''

--execute RESTORE statement

IF UPPER(@ExecuteRestoreImmediately) IN ('Y', '1')

BEGIN

 

      RAISERROR(N'Executing:

%s', 0, 1, @RestoreStatement) WITH NOWAIT

      EXEC (@RestoreStatement);

      SET @Error = @@ERROR;

      IF @Error <> 0 GOTO Done;

 

      --execute ALTER DATABASE statement(s)

      IF @ChangeLogicalNamesSql <> ''

      BEGIN

            RAISERROR(N'Executing:

%s', 0, 1, @ChangeLogicalNamesSql) WITH NOWAIT

            EXEC (@ChangeLogicalNamesSql);

            SET @Error = @@ERROR;

            IF @Error <> 0 GOTO Done;

      END

     

      IF @AlterAuthorizationSql <> ''

      BEGIN

            RAISERROR(N'Executing:

%s', 0, 1, @AlterAuthorizationSql) WITH NOWAIT

            EXEC (@AlterAuthorizationSql);

            SET @Error = @@ERROR;

            IF @Error <> 0 GOTO Done;

      END

     

END

ELSE

BEGIN

      RAISERROR(N'%s', 0, 1, @RestoreStatement) WITH NOWAIT

      IF @ChangeLogicalNamesSql <> ''

      BEGIN

            RAISERROR(N'%s', 0, 1, @ChangeLogicalNamesSql) WITH NOWAIT;

      END

      IF @AlterAuthorizationSql <> ''

      BEGIN

            RAISERROR(N'%s', 0, 1, @AlterAuthorizationSql) WITH NOWAIT;

      END

END;

 

Done:

 

RETURN @Error;

GO

 

 

Database Mail Configuration

I recently had to setup Database Mail on dozens of SQL Server instances.   Rather than perform this tedious task using the SSMS GUI, I developed a script that saved me a lot of time which I'm sharing here.   

My needs were simple so I only needed a single SMTP account and profile.  I decided to make the profile the default public one so that all msdb users would use this profile unless a different sp_send_dbmail @profile value was explicitly specified.  You might want to extend this script if you need other accounts/profiles, such as separate ones for administrative alerts or user reports.

Setup Script

Below is the template script I used for my task.  The sysmail_add_account_sp @username and @password parameters might be required depending on your SMTP server authentication and you will of course need to customize the mail server name and addresses for your environment.

-- Enable Database Mail for this instance

EXECUTE sp_configure 'show advanced', 1;

RECONFIGURE;

EXECUTE sp_configure 'Database Mail XPs',1;

RECONFIGURE;

GO

 

-- Create a Database Mail account

EXECUTE msdb.dbo.sysmail_add_account_sp

    @account_name = 'Primary Account',

    @description = 'Account used by all mail profiles.',

    @email_address = 'myaddress@mydomain.com',

    @replyto_address = 'myaddress@mydomain.com',

    @display_name = 'Database Mail',

    @mailserver_name = 'mail.mydomain.com';

 

-- Create a Database Mail profile

EXECUTE msdb.dbo.sysmail_add_profile_sp

    @profile_name = 'Default Public Profile',

    @description = 'Default public profile for all users';

 

-- Add the account to the profile

EXECUTE msdb.dbo.sysmail_add_profileaccount_sp

    @profile_name = 'Default Public Profile',

    @account_name = ' Primary Account',

    @sequence_number = 1;

 

-- Grant access to the profile to all msdb database users

EXECUTE msdb.dbo.sysmail_add_principalprofile_sp

    @profile_name = 'Default Public Profile',

    @principal_name = 'public',

    @is_default = 1;

GO

 

--send a test email

EXECUTE msdb.dbo.sp_send_dbmail

    @subject = 'Test Database Mail Message',

    @recipients = 'testaddress@mydomain.com',

    @query = 'SELECT @@SERVERNAME';

GO

Not Before Service Pack 1

In case you haven't yet heard, Microsoft SQL Server 2008 service pack 1 was released on April 7.  This milestone is especially significant for those of you who could not previously deploy the latest SQL Server release because your organization has a "not before the first service pack" policy.  I want to go on record as one who believes that such a policy is flawed and has needlessly delayed many organizations from using the new SQL Server 2008 features.

There is nothing magical about the first service pack compared to the initial RTM release with regards to production readiness.  SQL Server releases nowadays are scheduled based quality rather than just hitting a date.  Buggy features will be dropped from a release rather than included and in need of a service pack.  I'm not saying that every SQL Server release is flawless but the number of serious bugs (e.g. corruption or wrong results) are few, thanks to internal testing by Microsoft as well as those in the community that kick the tires with the pre-release CTP bits.

It's understandable that those who are risk-adverse might wait until after the first service pack with the belief that other adopters may have smoothed out the bumps in the road a bit.  I can see how postponing installation in this way might mitigate some of the risk but SP1 is a completely arbitrary milestone that is a hold-over from before SQL 7 was released over a decade ago.  I think a better approach is to adopt new releases based on quality as determined in one's own environment.  Whether the target is a new SQL Server installation or an upgrade of an existing instance, one still needs to perform testing before installing any new version, service pack or patch in production.  It is those test results that should determine production readiness, not the results of SELECT SERVERPROPERTY('ProductLevel').

QUOTED_IDENTIFIERS and ANSI_NULLS ON

I suggest that one always turn on both the QUOTED_IDENTIFIERS and ANSI_NULLS session settings.  Not only do these settings provide ANSI-standard behavior, these must be turned on in order to use features like indexed views, indexes on computed columns and query notifications.  It is tricky to ensure the settings are as desired, though, because the default session settings are different depending on the tools you use.

DDL Script Considerations

It is especially important to ensure the QUOTED_IDENTIFIERS and ANSI_NULLS session settings are correct with DDL scripts because both QUOTED_IDENTIFIERS and ANSI_NULL are "sticky".  The settings in effect when a stored procedure, view, function or trigger are created are also used at execution time.  The create time settings override run-time session settings. 

SQLCMD and OSQL Turn Settings Off

QUOTED_IDENTIFIERS and ANSI_NULLS are on by default when you connect using modern client APIs like ODBC, SQLOLEDB, SQL Native Client and SqlClient.  The SQL Server Management Studio and Query Analyzer tools keep those settings on unless you override the connection behavior under the tool connection options or run SET QUOTED_IDENTIFIERS ON or SET ANSI_NULLS ON commands in the query window.

The SQLCMD and OSQL command prompt utilities are different, tough.  These tools explicitly turn off QUOTED_IDENTIFIERS after connecting, presumably to provide backwards compatibility.  One must either specify the “-I” (upper-case “eye”) command-line argument to turn on QUOTED_IDENTIFIERS or include a SET QUOTED_IDENTIFIERS ON command in all the SQL scripts run from those utilities.  I personally like avoid SET commands in my DDL scripts so I make it a habit to specify the -I command line option.

“UPSERT” Race Condition With MERGE

I mentioned in Conditional INSERT/UPDATE Race Condition that most “UPSERT” code is defective and can lead to constraint violations and data integrity issues in a multi-user environment .  In this post, I’ll show how to prevent duplicate key errors and data problems with the MERGE statement too.  You might want to peruse Conditional INSERT/UPDATE Race Condition before reading this for a background on these concurrency concerns.

Background on MERGE

Microsoft introduced the ANSI-standard MERGE statement in SQL Server 2008.  MERGE is very powerful in that it can perform multiple actions in a single statement that previously required separate INSERT/UPDATE/DELETE statements.  MERGE is also a good alternative to the proprietary UPDATE…FROM syntax allowed in the T-SQL dialect.

MERGE can (and in my opinion should) be used to address the requirement to either INSERT or UPDATE depending on whether the source data already exists.  One need only include the MERGE statement clauses WHEN MATCHED THEN UPDATE and WHEN NOT MATCHED THEN INSERT in order to take the proper action, all within a single statement.

 “UPSERT” MERGE Concurrency Test

Even though MERGE provides the means to perform multiple actions within a single statement, developers still need to consider concurrency with MERGE to prevent errors and data issues.  Let me illustrate using the table and stored procedure that I originally posted in Conditional INSERT/UPDATE Race Condition:

CREATE TABLE dbo.Foo

(

      ID int NOT NULL

            CONSTRAINT PK_Foo PRIMARY KEY,

      Bar int NOT NULL

);

GO

 

CREATE PROCEDURE dbo.Merge_Foo

      @ID int,

      @Bar int

AS

 

SET NOCOUNT, XACT_ABORT ON;

 

MERGE dbo.Foo AS f

USING (SELECT @ID AS ID, @Bar AS Bar) AS new_foo

ON f.ID = new_foo.ID

WHEN MATCHED THEN

    UPDATE SET f.Bar = new_foo.Bar

WHEN NOT MATCHED THEN

    INSERT (ID, Bar)

        VALUES (new_foo.ID, new_foo.Bar);

       

RETURN @@ERROR;

GO

I ran the script below from 2 different SSMS windows after changing the time to the near future so that both executed at the same time.  My test box had a single quad-core processor with SQL Server 2008 Developer Edition installed, which I expected to have enough multi-processing power to create the error.

WAITFOR TIME '08:00:00'

 

EXEC dbo.Merge_Foo

      @ID = 1,

      @Bar = 1

I got a primary key violation error, showing that MERGE is vulnerable to concurrency problems like a multi-statement conditional INSERT/UPDATE technique. However, I couldn’t reproduce the error with MERGE nearly as consistently as I could with the conditional INSERT/UPDATE in Conditional INSERT/UPDATE Race Condition.  This could be due to a number of reasons (e.g. faster processor, different SQL Server version, MERGE locking behavior) but I wanted to make sure I could reproduce the error reliably.  I created a more robust test to exercise MERGE on a loop:

CREATE TABLE dbo.Foo2

(

      ID int NOT NULL

            CONSTRAINT PK_Foo2 PRIMARY KEY,

      InsertSpid int NOT NULL,

      InsertTime datetime2 NOT NULL,

      UpdateSpid int NULL,

      UpdateTime datetime2 NULL

);

 

CREATE PROCEDURE dbo.Merge_Foo2

      @ID int

AS

 

SET NOCOUNT, XACT_ABORT ON;

 

MERGE dbo.Foo2 AS f

USING (SELECT @ID AS ID) AS new_foo

      ON f.ID = new_foo.ID

WHEN MATCHED THEN

    UPDATE

            SET f.UpdateSpid = @@SPID,

            UpdateTime = SYSDATETIME()

WHEN NOT MATCHED THEN

    INSERT

      (

            ID,

            InsertSpid,

            InsertTime

      )

    VALUES

      (

            new_foo.ID,

            @@SPID,

            SYSDATETIME()

      );

       

RETURN @@ERROR;

I ran the script below from 4 different SSMS windows after changing the time to the near future so that all executed at the same time.   

DECLARE

    @NextTime datetime,

    @ID int,

    @MillisecondDelay int;

SELECT

    @NextTime = '08:10:00',

    @ID = 1,

    @MillisecondDelay = 100;

--execute 10 times per second for 1 minute

WHILE @ID <= 600

BEGIN

    --pause and sync with other sessions

    WAITFOR TIME @NextTime;

    EXEC dbo.Merge_Foo2

        @ID = @ID;

    SELECT

        @ID = @ID + 1,

        --assume no more that 100ms per execution

        @NextTime = DATEADD(MILLISECOND, @MillisecondDelay, @NextTime);

END;

I was able to reproduce the primary key violation every time with this test script.

Addressing the MERGE Race Condition

The underlying issue with any conditional insert technique is that data must be read before the determination can be made whether to INSERT or UPDATE.  To prevent concurrent sessions from inserting data with the same key, an incompatible lock must be acquired to ensure only one session can read the key and that lock must be held until the transaction completes.

I showed how one might address the problem using both UPDLOCK and HOLDLOCK locking hints in Conditional INSERT/UPDATE Race Condition.  MERGE is slightly different, though.  I repeated the test with only the HOLDLOCK hint added:

ALTER PROCEDURE dbo.Merge_Foo2

      @ID int

AS

 

SET NOCOUNT, XACT_ABORT ON;

 

MERGE dbo.Foo2 WITH (HOLDLOCK) AS f

USING (SELECT @ID AS ID) AS new_foo

      ON f.ID = new_foo.ID

WHEN MATCHED THEN

    UPDATE

            SET f.UpdateSpid = @@SPID,

            UpdateTime = SYSDATETIME()

WHEN NOT MATCHED THEN

    INSERT

      (

            ID,

            InsertSpid,

            InsertTime

      )

    VALUES

      (

            new_foo.ID,

            @@SPID,

            SYSDATETIME()

      );

       

RETURN @@ERROR;

This test showed that simply adding the HOLDLOCK hint prevented the primary key violation error.  Unlike the conditional INSERT/UPDATE in Conditional INSERT/UPDATE Race Condition, MERGE acquired a key update lock by default so UPDLOCK was not needed.  Also, in contrast the multi-statement conditional INSERT/UPDATE technique, no explicit transaction is required because MERGE is an atomic DML statement.  The HOLDLOCK hint was still needed, though, because MERGE otherwise releases the update key lock before the insert.  I gleaned this by examining the locks from a Profiler trace of the MERGE without the HOLDLOCK:

EventClass

TextData

Mode

ObjectID

Type

SP:Starting

EXEC dbo.Merge_Foo2 @ID = 1

1314103722

Lock:Acquired

8 - IX

1330103779

5 - OBJECT

Lock:Acquired

1:173

7 - IU

0

6 - PAGE

Lock:Acquired

(10086470766)

4 - U

0

7 - KEY

Lock:Released

(10086470766)

4 - U

0

7 - KEY

Lock:Released

1:173

7 - IU

0

6 - PAGE

Lock:Acquired

1:173

8 - IX

0

6 - PAGE

Lock:Acquired

(10086470766)

15 - RangeI-N

0

7 - KEY

Lock:Acquired

(10086470766)

5 - X

0

7 - KEY

Lock:Released

(10086470766)

5 - X

0

7 - KEY

Lock:Released

1:173

8 - IX

0

6 - PAGE

Lock:Released

8 - IX

1330103779

5 - OBJECT

SP:Completed

EXEC dbo.Merge_Foo2 @ID = 1

1314103722

If another concurrent MERGE of the same key occurs after the update lock is released and before the exclusive key lock is acquired, a duplicate key error will result.

The trace below of the MERGE with the HOLDLOCK hint shows that locks aren’t released until the insert (and statement) completes, this avoiding the concurrency problem with MERGE.

EventClass

TextData

Mode

ObjectID

Type

SP:Starting

EXEC dbo.Merge_Foo2 @ID = 1

 

1314103722

 

Lock:Acquired

 

8 - IX

1330103779

5 - OBJECT

Lock:Acquired

1:173

7 - IU

0

6 - PAGE

Lock:Acquired

(10086470766)

4 - U

0

7 - KEY

Lock:Acquired

1:173

8 - IX

0

6 - PAGE

Lock:Acquired

(10086470766)

15 - RangeI-N

0

7 - KEY

Lock:Acquired

(10086470766)

5 - X

0

7 - KEY

Lock:Released

(10086470766)

5 - X

0

7 - KEY

Lock:Released

1:173

8 - IX

0

6 - PAGE

Lock:Released

 

8 - IX

1330103779

5 - OBJECT

SP:Completed

EXEC dbo.Merge_Foo2 @ID = 1

 

1314103722

 

 

Don’t Use sp_attach_db

I’ve used sp_detach_db and sp_attach_db to relocate database files for many years.  I know that sp_attach_db was deprecated in SQL 2005 but, like most DBAs, I’ve continued to use sp_attach_db mostly out of habit.  I want to share with you why I’ve decided to change my ways.

Planned File Relocation

Let’s say you want to move the log file from to a separate drive.  The following script shows how to accomplish in SQL Server 2000 using sp_attach_db.  The only sp_attach_db parameters required are the database name, primary data file path and the log file that was moved from the original location.

EXEC sp_detach_db

      @dbname = N'MyDatabase';

--move log file to E drive manually and attach from new location

EXEC sp_attach_db

      @dbname = N'MyDatabase',

      @filename1 = N'D:\DataFiles\MyDatabase_Data.mdf',

      @filename2 = N'E:\LogFiles\MyDatabase_Log.ldf';

 

The deprecated sp_attach_db procedure still works in SQL Server 2005 and SQL Server 2008 but is not recommended.  Instead, the proper method to relocate files in these later versions is with ALTER DATABASE…MODIFY FILE.  Simply execute an ALTER DATABASE…MODIFY FILE for each moved file and toggle the ONLINE/OFFLINE database state.  The script example below shows how the log file would be moved to a different drive with this method.  This method is described in detail in the Books Online.

 

ALTER DATABASE MyDatabase SET OFFLINE;

--move log file to E drive manually and attach from new location

ALTER DATABASE MyDatabase

      MODIFY FILE (

            NAME='MyDatabase_Log',

            FILENAME='E:\LogFiles\MyDatabase_Log.ldf');

ALTER DATABASE MyDatabase SET ONLINE;

 

Unfortunately, the Books Online doesn’t provide much info as to why ALTER DATABASE…MODIFY FILE and ONLINE/OFFLINE is preferred over detach/attach for planned file relocations.  One explanation is illustrated by an issue I ran into recently that motivated this post.  After using the detach/attach method, we ended up with Service Broker disabled.  This is documented behavior that we simply overlooked and didn’t catch until subsequent application problems were reported.  Since exclusive database access was needed to re-enable Service Broker, we had to close all user database connections and before altering the database ENABLE_BROKER setting and this was a real pain. 

 

This problem wouldn’t have happened had we used the recommended method and toggled the OFFLINE/ONLINE database state because the database settings would have remained unchanged.  I wouldn’t be surprised if there were other gotchas with the detach/attach method.  The bottom line is that there is no reason not to use the ALTER DATABASE…MODIFY FILE and OFFLINE/ONLINE method to move files.

Attaching a Database to Another Server or Instance

Note that the deprecated sp_attach_db stored procedure is basically just a wrapper for CREATE DATABASE…FOR ATTACH.  You can use CREATE DATABASE…FOR ATTACH much like you would sp_attach_db: specify the database name, primary file path (mdf file path) along with file paths that differ from the original locations.  For example:

EXEC sp_detach_db

      @dbname = N’MyDatabase’;

--move database files manually to new server

CREATE DATABASE MyDatabase

      ON(NAME=’MyDatabase_Data’,

            FILENAME='C:\DataFiles\MyDatabase_Data.mdf')

      LOG ON(NAME='MyDatabase_Log',

            FILENAME='C:\LogFiles\MyDatabase_Log.ldf')

      FOR ATTACH

      WITH ENABLE_BROKER;

 

The ENABLE_BROKER option is appropriate if the purpose of the attach is to completely move a SB-enabled database to another instance or in a DR scenario.  When attaching to create a database replica (e.g. copy for testing), the NEW_BROKER option is appropriate. 

Summary

I suggest ALTER DATABASE…MODIFY FILE and OFFLINE/ONLINE for planned file relocation and sp_detach_db/CREATE DATABASE…FOR ATTACH for other scenarios.  In any case, sp_attach_db should be avoided going forward.

SQL Server Partition Details Custom Report

I developed a custom report for SQL Server Management Studio that wraps the partition details and row counts query I previously posted.  Visit the Codeplex project site for details.  You can download just the released RDL file or, if you want to customize the report to your liking, download the full project source code.  Alternatively you can create a new report project of your own and add the RDL file as an existing project item.

I welcome any feedback you might have, either here or via the Codeplex project discussion page.  It’s been a while since I’ve done any Reporting Services development so there is certainly room for improvement.  Additional project team members are also welcome.

Partition Details and Row Counts

You will likely find the following query useful if you work with partitioned objects.  I developed this when I first started using table partitioning in order to verify proper partition boundaries, filegroups and row counts.  Not only does this provide much more information that can be obtained by querying the underlying table with the partition function to get partition numbers, it runs much faster because only catalog views are used.

As-is, the query includes both partitioned and non-partitioned user objects in the context database but you can customize the WHERE clauses as desired.  I think this query would make a perfect source for a SSMS custom report so that it can be easily invoked from the SSMS Object explorer.  That’s on my to-do list.

Query for Partition Details Using Catalog Views

--paritioned table and index details

SELECT

      OBJECT_NAME(p.object_id) AS ObjectName,

      i.name                   AS IndexName,

      p.index_id               AS IndexID,

      ds.name                  AS PartitionScheme,   

      p.partition_number       AS PartitionNumber,

      fg.name                  AS FileGroupName,

      prv_left.value           AS LowerBoundaryValue,

      prv_right.value          AS UpperBoundaryValue,

      CASE pf.boundary_value_on_right

            WHEN 1 THEN 'RIGHT'

            ELSE 'LEFT' END    AS Range,

      p.rows AS Rows

FROM sys.partitions                  AS p

JOIN sys.indexes                     AS i

      ON i.object_id = p.object_id

      AND i.index_id = p.index_id

JOIN sys.data_spaces                 AS ds

      ON ds.data_space_id = i.data_space_id

JOIN sys.partition_schemes           AS ps

      ON ps.data_space_id = ds.data_space_id

JOIN sys.partition_functions         AS pf

      ON pf.function_id = ps.function_id

JOIN sys.destination_data_spaces     AS dds2

      ON dds2.partition_scheme_id = ps.data_space_id 

      AND dds2.destination_id = p.partition_number

JOIN sys.filegroups                  AS fg

      ON fg.data_space_id = dds2.data_space_id

LEFT JOIN sys.partition_range_values AS prv_left

      ON ps.function_id = prv_left.function_id

      AND prv_left.boundary_id = p.partition_number - 1

LEFT JOIN sys.partition_range_values AS prv_right

      ON ps.function_id = prv_right.function_id

      AND prv_right.boundary_id = p.partition_number 

WHERE

      OBJECTPROPERTY(p.object_id, 'ISMSShipped') = 0

UNION ALL

--non-partitioned table/indexes

SELECT

      OBJECT_NAME(p.object_id)    AS ObjectName,

      i.name                      AS IndexName,

      p.index_id                  AS IndexID,

      NULL                        AS PartitionScheme,

      p.partition_number          AS PartitionNumber,

      fg.name                     AS FileGroupName,  

      NULL                        AS LowerBoundaryValue,

      NULL                        AS UpperBoundaryValue,

      NULL                        AS Boundary, 

      p.rows                      AS Rows

FROM sys.partitions     AS p

JOIN sys.indexes        AS i

      ON i.object_id = p.object_id

      AND i.index_id = p.index_id

JOIN sys.data_spaces    AS ds

      ON ds.data_space_id = i.data_space_id

JOIN sys.filegroups           AS fg

      ON fg.data_space_id = i.data_space_id

WHERE

      OBJECTPROPERTY(p.object_id, 'ISMSShipped') = 0

ORDER BY

      ObjectName,

      IndexID,

      PartitionNumber;

 

Here is some sample output.

ObjectName

IndexName

IndexID

PartitionScheme

PartitionNumber

FileGroupName

LowerBoundaryValue

UpperBoundaryValue

Boundary

Rows

SalesTransactions_NonPartitioned

cdx_SalesTransactions_SalesTransactionTime

1

NULL

1

PartitioningDemo_Data

NULL

NULL

NULL

6576000

SalesTransactions_NonPartitioned

idx_SalesTransactions_StoreID_ProductID

2

NULL

1

PartitioningDemo_Index

NULL

NULL

NULL

6576000

SalesTransactions_NonPartitioned

idx_SalesTransactions_CustomerID

3

NULL

1

PartitioningDemo_Index

NULL

NULL

NULL

6576000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

1

PartitioningDemo_Data

NULL

2006-01-01 0:00:00

RIGHT

0

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

2

PartitioningDemo_Data

2006-01-01 0:00:00

2006-02-01 0:00:00

RIGHT

0

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

3

PartitioningDemo_Data

2006-02-01 0:00:00

2006-03-01 0:00:00

RIGHT

0

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

4

PartitioningDemo_Data

2006-03-01 0:00:00

2006-04-01 0:00:00

RIGHT

0

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

5

PartitioningDemo_Data

2006-04-01 0:00:00

2006-05-01 0:00:00

RIGHT

180000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

6

PartitioningDemo_Data

2006-05-01 0:00:00

2006-06-01 0:00:00

RIGHT

186000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

7

PartitioningDemo_Data

2006-06-01 0:00:00

2006-07-01 0:00:00

RIGHT

180000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

8

PartitioningDemo_Data

2006-07-01 0:00:00

2006-08-01 0:00:00

RIGHT

186000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

9

PartitioningDemo_Data

2006-08-01 0:00:00

2006-09-01 0:00:00

RIGHT

186000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

10

PartitioningDemo_Data

2006-09-01 0:00:00

2006-10-01 0:00:00

RIGHT

180000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

11

PartitioningDemo_Data

2006-10-01 0:00:00

2006-11-01 0:00:00

RIGHT

186000

SalesTransactions_Partitioned

cdx_SalesTransactions_SalesTransactionTime

1

PS_SalesTransactions_Data

12

PartitioningDemo_Data

2006-11-01 0:00:00

2006-12-01 0:00:00

RIGHT

180000

 

Automating RANGE RIGHT Sliding Window Maintenance

I posted example scripts to automate RANGE LEFT sliding window maintenance in my last post.  As promised, I am sharing a RANGE RIGHT version in this post.

I personally prefer a RANGE RIGHT partition function when partitioning on a data type that includes time.  RANGE RIGHT allows specification of an exact date boundary instead of the maximum date/time value needed for RANGE LEFT to keep all data for a given date in the same partition.  Another nicety with RANGE RIGHT is that same boundaries can be used in a RANGE RIGHT partition function of any date/time data type.  In contrast, the time component of RANGE LEFT boundary values must be customized for the specific data type as I described in Sliding Window Table Partitioning.

The downside with RANGE RIGHT is that maintaining the sliding window isn’t quite as intuitive as with RANGE LEFT.  Instead of switching out and merging the first partition during purge/archive, one needs to switch out and merge the second partition.  This practice avoids the costly movement of retained data from the removed second partition into the retained first partition.  Both the first and second partitions are empty during the merge so no data need to be move moved.  The first partition is normally empty at all times.

The stored procedure below shows how you can automate a RANGE RIGHT daily sliding window.  The main differences between this version and the RANGE LEFT version I posted in Automating Sliding Window Maintenance are

1)      Removed boundary calculation of maximum time (DATEADD(millisecond, -3, @RunDate)).

2)      Added a conditional create of upper boundary of second partition (oldest period retained in third partition)

3)      Added $PARTITION and subtracted 1 for SWITCH instead of hard-coding the partition number

Example of RANGE RIGHT Sliding Window Automation

Below are the demo objects used by the range RIGHT sliding window procedure.

--no boundaries initially - proc will create as needed

CREATE PARTITION FUNCTION PF_MyPartitionFunction(datetime)

AS RANGE RIGHT FOR VALUES();

GO

 

CREATE PARTITION SCHEME PS_MyPartitionScheme

AS PARTITION PF_MyPartitionFunction ALL TO ([PRIMARY]);

GO

 

CREATE TABLE dbo.MyPartitionedTable

(

      PartitionColumnDateTime datetime

) ON PS_MyPartitionScheme(PartitionColumnDateTime);

GO

 

--note staging table uses same partition scheme as primary table

CREATE TABLE dbo.MyPartitionedTable_Staging

(

      PartitionColumnDateTime datetime

) ON PS_MyPartitionScheme(PartitionColumnDateTime);

GO

 

The sliding window proc:

CREATE PROC dbo.SlideRangeRightWindow_datetime

            @RetentionDays int,

        @RunDate datetime = NULL

/*

      This proc maintains a RANGE RIGHT daily sliding window

      based on the specified @RetentionDays.  In addition to

      purging old data, the partition function is adjusted to

      account for scheduling issues or changes in @RetentionDays.

 

      Partitions are split and merged so that the first partition

      contains data older than the specifed retention period and

      the last two partitions (future data) are empty.  The other

      petitions will contain historical and current data for the

      specified number of @RetentionDays. 

 

      After successful execution, (at least) the following

      partitions will exist:

      - partition 1 = data older than retained date (empty)

      - third from last partition = current data (@RunDate)

      - second from last partition = @RunDate + 1 (normally empty)

      - last partition = @RunDate + 2  (normally empty)

*/

 

AS

 

SET NOCOUNT, XACT_ABORT ON;

 

DECLARE

        @Error int,

        @RowCount bigint,

        @ErrorLine int,

        @Message varchar(255),

            @ExpirationDate datetime,

        @PartitionBoundaryDate datetime;

 

SET @Error = 0;

 

BEGIN TRY

 

      IF @RunDate IS NULL

      BEGIN

            --use current date (midnight) if no date specified

            SET @RunDate = DATEADD(day, 0, DATEDIFF(day, '', GETDATE()));