SQL Server: Best way to Update row if exists, Insert if not
This question pops up a lot everywhere and it's a common business requirement and until SQL Server 2008 doesn't come out with its MERGE statement that will do that in one go we're stuck with 2 ways of achieving this. The biggest problem with every update/insert (upsert for those who haven't heard the term yet) is locking.
Each DML statement basicaly contains 2 locks. IX Lock (Intent Exclusive Lock) and X Lock(Exclusive Lock).
When the SQL Server searches for the data to modify it takes IX lock on the whole page or table and then it takes an X Lock on the rows to be modified.
Our goal here is to minimize anykind of locks. As i said earlier there are two methods of doing this:
1. Update ... if rowcount = 0 insert
2. If row exists update else insert
First let's create our test table:
USE tempdbIF OBJECT_ID('t1') IS NOT NULL DROP TABLE t1 GO CREATE TABLE t1 (id INT PRIMARY KEY, name1 VARCHAR(10)) INSERT INTO t1 SELECT 1, 'name 1' UNION ALL SELECT 2, 'name 2' UNION ALL SELECT 3, 'name 3' UNION ALL SELECT 4, 'name 4' UNION ALL SELECT 5, 'name 5' GO
Both methods when Inserting a row:
--------------------------- -- I N S E R T R O W --- --------------------------- DECLARE @rc INT– NO EXISTS ———— BEGIN TRAN – update the row with id = 6. UPDATE t1 SET name1 = 'name 6' WHERE id = 6 – get rowcount SELECT @rc = @@ROWCOUNT – see what locks are being held SELECT resource_type, request_mode, resource_description, 'ROW NOT IN TABLE - No Exists - after update' FROM sys.dm_tran_locks – if update rowcount = 0 means that there is no such row so we insert it IF @rc = 0 BEGIN INSERT INTO t1 SELECT 6, 'name 6' – see what locks are being held SELECT resource_type, request_mode, resource_description, 'ROW NOT IN TABLE - No Exists - after insert' FROM sys.dm_tran_locks END ROLLBACK
– EXISTS ————— BEGIN TRAN – if row already exists in a table update it else insert new row IF EXISTS (SELECT * FROM t1 WHERE id = 6) BEGIN – see what locks are being held from select SELECT resource_type, request_mode, resource_description, 'ROW NOT IN TABLE - Exists - after select' FROM sys.dm_tran_locks
<span class="kwrd">UPDATE</span> t1 <span class="kwrd">SET</span> name1 = <span class="str">'name 6'</span> <span class="kwrd">WHERE</span> id = 6 <span class="kwrd">SELECT</span> resource_type, request_mode, resource_description, <span class="str">'ROW NOT IN TABLE - Exists - after update'</span> <span class="kwrd">FROM</span> sys.dm_tran_locks
END ELSE BEGIN – see what locks are being held from select SELECT resource_type, request_mode, resource_description, 'ROW NOT IN TABLE - Exists - after select' FROM sys.dm_tran_locks
<span class="kwrd">INSERT</span> <span class="kwrd">INTO</span> t1 <span class="kwrd">SELECT</span> 6, <span class="str">'name 6'</span> <span class="kwrd">SELECT</span> resource_type, request_mode, resource_description, <span class="str">'ROW NOT IN TABLE - Exists - after insert'</span> <span class="kwrd">FROM</span> sys.dm_tran_locks
END ROLLBACK
GO
When a row doesn't exist in a table we have to insert it.
The NO EXISTS method is more expensive due to the IX page lock by update and insert followed by X Lock by insert.
The EXISTS method uses a select * to check if the row exists using only an IS (Intent Shared) lock which is inexpensive compared to the IX lock made by the Update. This IS lock is then followed by an IX and X Lock made by the Insert.
Both methods when Updating a row:
--------------------------- -- U P D A T E R O W --- --------------------------- DECLARE @rc INT– NO EXISTS ———— BEGIN TRAN
– update the row with id = 5 UPDATE t1 SET name1 = 'name 6' WHERE id = 5 – get rowcount SELECT @rc = @@ROWCOUNT – see what locks are being held SELECT resource_type, request_mode, resource_description, 'ROW IN TABLE - No Exists - after update' FROM sys.dm_tran_locks – update rowcount = 1 means that there is a row so we updated it IF @rc = 0 BEGIN INSERT INTO t1 SELECT 6, 'name 6'
<span class="rem">-- see what locks are being held</span> <span class="kwrd">SELECT</span> resource_type, request_mode, resource_description, <span class="str">'ROW IN TABLE - No Exists - after insert'</span> <span class="kwrd">FROM</span> sys.dm_tran_locks
END ROLLBACK
– EXISTS ————— BEGIN TRAN – if row already exists in a table update it else insert new row IF EXISTS (SELECT * FROM t1 WHERE id = 5) BEGIN – see what locks are being held from select SELECT resource_type, request_mode, resource_description, 'ROW IN TABLE - Exists - after select' FROM sys.dm_tran_locks
<span class="kwrd">UPDATE</span> t1 <span class="kwrd">SET</span> name1 = <span class="str">'name 6'</span> <span class="kwrd">WHERE</span> id = 5 <span class="kwrd">SELECT</span> resource_type, request_mode, resource_description, <span class="str">'ROW IN TABLE - Exists - after update'</span> <span class="kwrd">FROM</span> sys.dm_tran_locks
END ELSE BEGIN – see what locks are being held from select SELECT resource_type, request_mode, resource_description, 'ROW IN TABLE - Exists - after select' FROM sys.dm_tran_locks
<span class="kwrd">INSERT</span> <span class="kwrd">INTO</span> t1 <span class="kwrd">SELECT</span> 6, <span class="str">'name 6'</span> <span class="kwrd">SELECT</span> resource_type, request_mode, resource_description, <span class="str">'ROW IN TABLE - Exists - after insert'</span> <span class="kwrd">FROM</span> sys.dm_tran_locks
END ROLLBACK
GO
When a row exist in a table we have to update it.
The NO EXISTS method is less expensive due to only one IX and X Lock made by the Update.
The EXISTS method again uses a select * to check if the row exists using an IS (Intent Shared) lock followed by an IX and X Lock made by the Update.
Here the NO EXISTS is better because it doesn't need an IS (Intent Shared) lock.
However on overall lock use i much rather go with EXISTS method than NO EXISTS.
|
Legacy Comments
Jason Stangroome
2007-08-02 |
re: SQL Server: Best way to Update row if exists, Insert if not How do you suppose the locking situation changes with the new MERGE statement in SQL Server 2008? http://blog.benhall.me.uk/2007/06/sql-server-2008-sql-merge-statement.html |
Mladen
2007-08-02 |
re: SQL Server: Best way to Update row if exists, Insert if not i have no idea. hadn't had a chance to play with sql 2008 much. try it and let me know. |
noeld
2007-08-06 |
re: SQL Server: Best way to Update row if exists, Insert if not In my opinion using EXISTS the way you are, is totally flawed on highly concurrent scenarios. Assume two connections; One is sending that code at the "same time" than the other is sending a "delete or that same row". Because of the IS lock your "EXISTS" "passes" that check and the delete could wipe that record out ... can you guess what happens to the update statement ? Cheers! |
Mladen
2007-08-06 |
re: SQL Server: Best way to Update row if exists, Insert if not well... do you have any better suggestion? |
Saad Ahmad
2007-09-07 |
re: SQL Server: Best way to Update row if exists, Insert if not First, both of these methods could theoratically give you a wrong error. 1. Update ... if rowcount = 0 insert - Lets assume two processes P1 and P2 - Lets assume table TAB1 has row R1 Case 1.a (Row R1 does not exist) a. P1 executes it at time T1 (executed update found that row does not exist) b. P2 executes it at time T1 (executed update found that row does not exist) c. Both will try insert and one will fail. Case 1.b (Row R1 exists ) No issue in this case. 2. if row exists update else insert will have the same issue as above. Solution --------- 1. Table TAB1 should have a PK. 2. Try insert, then on exception of duplicate row, update. Lets try our scenario: Case 1.a (Row R1 does not exist) a. P1 executes it at time T1 (insert will create the row) b. P2 executes it at time T1 (insert is attempted, but because of P1 this will wait until P1 comits) c. If P1 commits, P2 will detect duplicate and do the update. If P1 rolls back, P2's insert will succeed. Case 1.b (Row R1 exists) a. P1 executes it at time T1 (insert will detect a duplicate) b. P2 executes it at time T1 (insert will detect a duplicate) c. Both can do the update. Don't think this can't happen; I actually noticied the issue in a high volume environment Saad Ahmad saad.ahmad@gmail.com |
Mladen
2007-09-07 |
re: SQL Server: Best way to Update row if exists, Insert if not i know it can happen. as i said to the previous person who pointed this out: shown me a better way. |
Mladen
2007-09-07 |
re: SQL Server: Best way to Update row if exists, Insert if not well exception handling is pretty heavy duty operation... i do see you point though |
Mladen
2007-09-07 |
re: SQL Server: Best way to Update row if exists, Insert if not i'd try to solve this with isolation levels... |
Saad Ahmad
2007-09-07 |
re: SQL Server: Best way to Update row if exists, Insert if not Isolation levels are not the right solution since the problem is in a high volume scenario where you do not want more isolation. The following code may work. create table saad_test ( pk int not null primary key, data varchar(100) ) insert into saad_test values ( 1, 'saad' ) set nocount on declare @v_pk int, @v_data varchar(100) begin set @v_pk = 1 set @v_data = 'ahmad2' insert into saad_test values ( @v_pk, @v_data ) -- error for duplicate is 2627 if ( @@ERROR = 2627 ) update saad_test set data = @v_data where pk = @v_pk end And I will prefer to use true exceptions if available. But even those do not cause any unnecessary overhead. More importantly the code needs to work ... |
Mladen
2007-09-07 |
re: SQL Server: Best way to Update row if exists, Insert if not excellent! thanx! i love when someone shows me a better way of doing things. |
Jeff Moden
2007-09-08 |
re: SQL Server: Best way to Update row if exists, Insert if not Ran into this because of another forum with a link... appologize if this comes out too negative, but... Works great for a single row... what about batch merges? And, as clever as it is, updating by exception error just doesn't sit right with me... takes too long for large batches... insert has to be tried and failure detected and only then does the update occur. Probably alright for the RBAR necessary for a GUI, though. |
Peter Larsson
2007-09-19 |
re: SQL Server: Best way to Update row if exists, Insert if not Batch merges are easy in SQL Server 2008 See http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=87088 |
Roshni
2007-10-05 |
re: SQL Server: Best way to Update row if exists, Insert if not I want to know how can I prevent inserting a row in a table if it already exists, and insert it when it does not exist. I am running a loop in PLSQL for Insert. |
Mladen
2007-10-05 |
re: SQL Server: Best way to Update row if exists, Insert if not have you tried the method described here? |
ifo
2007-10-15 |
SQL Server: Best way to duplicate record Hello, Please I need advice. SQL Server 2005. I have a table consisting of 200 fields with a primary key(auto-increment). I need to be able to duplicate via a stored procedure a specific row in the table identified by primary key value i.e. insert a record form that table into the SAME table .... the primary key of the inserted record so be incremented and I would need to change to of the field values in the inserted record (i.e. duplicating a row in a table with a primary key(auto increment). What is the best way to do this... one suggested writing to temp table then re-insert to original table. I have had some issues with this is there another method? Please not that other users may try and access the orginal record during this insert process. Can you give me example syntax and comments. Thank You! |
Mladen
2007-10-15 |
re: SQL Server: Best way to Update row if exists, Insert if not insert into yourTable (columnsWithoutIdentity) select columnsWithoutIdentity from yourTable where someCondition |
Sam Saffron
2008-02-07 |
re: SQL Server: Best way to Update row if exists, Insert if not Hi there, Just a note that the method described here has some concurrency issues. I have described some ways around it on my blog at: http://www.samsaffron.com/blog/archive/2007/04/04/14.aspx Cheers Sam |
Mladen
2008-02-07 |
re: SQL Server: Best way to Update row if exists, Insert if not this method can come in handy too: http://www.sqlteam.com/article/application-locks-or-mutexes-in-sql-server-2005 |
Sinister China Penguin
2008-06-30 |
re: SQL Server: Best way to Update row if exists, Insert if not Hi - thanks for this I found it useful for a small App I am working on at the moment. It's a small point but it may be slightly more efficient to replace: IF EXISTS (SELECT * FROM t1 WHERE id = 6) With IF EXISTS (SELECT 1 FROM t1 WHERE id = 6) just so the SQL engine doesn't have to go & fetch any data. Thanks! |
Mladen
2008-06-30 |
re: SQL Server: Best way to Update row if exists, Insert if not hi, acctually it doesn't matter if it's * or 1. sql server doesn't get any extra data than it needs in either case. |
Tom
2008-08-19 |
re: SQL Server: Best way to Update row if exists, Insert if not Throw an error??? Woa I never see something so ugly! Is anyone has a good solution, simple? Why its bad to test before if data already exists? |
Miroslav Vracevic
2008-08-21 |
re: SQL Server: Best way to Update row if exists, Insert if not So how about modifying Saad's example to look like this - no error handling here. With few more modifications this can work for batch inserts/updates too --prepare create table mitch_test ( pk int not null primary key, data varchar(100) ) insert into mitch_test values ( 1, 'mitch' ) --insert/update block set nocount on declare @v_pk int, @v_data varchar(100) begin set @v_pk = 1 set @v_data = 'mitch2' insert into mitch_test select dt.* from (select @v_pk pk, @v_data data) dt left outer join mitch_test t on t.pk = dt.pk where t.pk is null if ( @@rowcount = 0 ) update mitch_test set data = @v_data where pk = @v_pk end |
aneef
2008-09-05 |
re: how to update every 10000 rows of a record... hi, Actually I want a sql statement to update evary 10000 rows of a record.... for eg...if a record consists of 30000 rows.....we created a new column in that record.....in that column we want to place the value 1 for first 10000 rows and the value 2 for next 10000 rows and the value 3 for next rows....and so on.....i.e., the value should be incremented by 1 for every 10000 rows.... please help in this as quickly as possible.... |
GAFEROZ
2008-09-23 |
re: SQL Server: Best way to Update row if exists, Insert if not This reminds me about the good old times of COBOL. I'm glad technology has advanced this much since then... |
darkdusky
2008-12-01 |
re: SQL Server: Best way to Update row if exists, Insert if not How about trying update first then check if row exists and then do insert if needed. The initial update would not lock any rows if the row didn't exist in the first place. e.g. Update ..... set .....=...... where id =@X set @found=0 set @found = Select count(*) from ...... where id=@X If @found =0 --do INSERT --else exit |
Mladen
2008-12-01 |
re: SQL Server: Best way to Update row if exists, Insert if not that is also covered in the post. |
sam
2009-02-01 |
re: SQL Server: Best way to Update row if exists, Insert if not both of these methods are not save for concurrent updates see: http://www.samsaffron.com/blog/archive/2007/04/04/14.aspx |
MAHESH
2009-02-05 |
re: SQL Server: Best way to Update row if exists, Insert if not i SOLVED my PROBLEM BY THIS ARTICLE |
George
2009-03-14 |
re: SQL Server: Best way to Update row if exists, Insert if not The best way of solving this probably depends on your situation. If you are going to do a lot of updating and it will be rare that the record does not exist, it is going to be best to try updating first and if no records are updated, insert one. This is faster in this scenario because the vast majority of the time it is just a single simple update. If you do this from your calling code rather than from SQL (though it is generally better to do everything in SQL), the insert statement would not even be compiled if the update worked. |
escalera
2010-01-06 |
re: SQL Server: Best way to Update row if exists, Insert if not does this procedure only works with primary keys columns? Because I'm using a condition with 2 columns (receiptBal, receiptDate) in a table called RECEIPTS. But it seems like it's not doing what I want as it's inserting a new row even if it's already there. DECLARE @rc INT BEGIN TRAN declare @receiptBal money, @receiptdate datetime, @receiptID int set @receiptBal = '2.23' set @receiptdate = '20091229' insert receipts (receiptBal, StoreID, receiptDate) select @receiptBal, 8, @receiptdate SELECT @rc = @@ROWCOUNT IF @rc = 0 BEGIN update receipts set receiptdate = @receiptdate where receiptBal = @receiptBal and receiptdate is null end COMMIT |
Mladen
2010-01-06 |
re: SQL Server: Best way to Update row if exists, Insert if not i think you have your conditions switched. you shoul dupdate forst then if @RC=0 insert |
brs
2010-03-04 |
re: SQL Server: Best way to Update row if exists, Insert if not How do we modify this when we are working with multiple threads and facing deadlock issue. Also I have a procedure which contains several such statements and I m facing the deadlock . Please let me know how this can be resolved |
Mladen
2010-03-04 |
re: SQL Server: Best way to Update row if exists, Insert if not either by using a serializable tran isolation level or by using sql server application locks http://www.sqlteam.com/article/application-locks-or-mutexes-in-sql-server-2005 |
Priyavadan joshi
2010-08-16 |
re: SQL Server: Best way to Update row if exists, Insert if not Best way to Update row if exists, Insert if not Use This QUERY := > use Condition IF Exise EXAM : IF Exists (select 1 From tblXyz With(nolock) Where ID =2 ) BEGIN Update tblXyz END Else Insert INTo tblXyz |