Fast importing of text files with format files and bulk insert
When creating SQL Server 2000 ETL solutions like importing of text files, it is common for developers to employ DTS packages to accomplish results quickly. The downside of this approach is that DTS packages have certain programmability limitations and don't always offer the best performance.
If you are importing very large raw text files that don't include formatting like column headers, a high-performance solution is to use BULK INSERT with format files.
For example, the format file may be named major.fmt (note the file extension .fmt) and would be placed in the same file system directory where the import source text files are located.
Here is a sample of what a format file looks like:
8.0 4 1 SQLCHAR 0 0 "\"" 0 x SQL_Latin1_General_CP1_CI_AS 2 SQLCHAR 0 0 "\",\"" 1 WTN SQL_Latin1_General_CP1_CI_AS 3 SQLCHAR 0 0 "\",\"" 2 Major SQL_Latin1_General_CP1_CI_AS 4 SQLCHAR 0 0 "\"\r\n" 3 MktSeg SQL_Latin1_General_CP1_CI_AS
Look in SQL Server Books On Line to find more details about how to use format files.
Now you can perform BULK INSERTS from the source text file like this:
BULK INSERT Database_name..MajorAccount FROM '\\ServerName\Textfiles$\major.txt' WITH (FORMATFILE = '\\ServerName\Textfiles$\major.fmt')
Your data loads will now complete way faster than trying to accomplish the same results with DTS packages in SQL Server 2000.
Hope this helps!
Legacy Comments
Emmanouil Karaiskakis
2008-01-11 |
re: Fast importing of text files with format files and bulk insert Hi, The above doesn't work. It moves everything 1 column to the left... Plus Column order should not start with 0. |
Emmanouil Karaiskakis
2008-01-11 |
re: Fast importing of text files with format files and bulk insert Sorry I meant 1 column to the right!! |
alice
2008-01-18 |
re: Fast importing of text files with format files and bulk insert I have tried this but I am getting this error: "Column is too long in the data file for row 1, column 2. Make sure the field terminator and row terminator are specified correctly." Why? |
Ashutosh
2011-01-26 |
re: Fast importing of text files with format files and bulk insert k245 29/02/2010 2341 dm332nenre 4 44kqw2q22w 243 k246 29/02/2010 2345 dm332nenre 4 44kqw2q22w 243 k247 29/02/2010 2343 dm332nenre 4 44kqw2q22w 243 k248 29/02/2010 2344 dm332nenre 4 44kqw2q22w 243 bulk insert in sql 2003 server. i am faceing a problem to create format file. pls tell any software that convert in format file . |