December 23, 2008 at 12:26 am
Comments posted to this topic are about the item Auditing Changes in ETL with SSIS
Tim Mitchell, Microsoft Data Platform MVP
Data Warehouse and ETL Consultant
TimMitchell.net | @Tim_Mitchell | Tyleris.com
ETL Best Practices
December 25, 2008 at 4:35 am
Nice one...:)
December 30, 2008 at 5:40 pm
Very good example of auditing in SSIS. Something to keep in mind is if the percentage of "bad" records is significant, your Audit table will rival in size the actual destination table, so this should be considered at design time. Thank you!
September 5, 2009 at 9:34 pm
mishaluba, you're right - if you're doing a lot of transformation, storage will be a key consideration. I worked recently with a very large ETL process in which tens of millions of records were converted from a legacy system, and every single record had some sort of transformation on it. The auditing tables more than doubled the size of our staging database, but my staff has had to return to that auditing data several different times for forensic purposes.
Tim Mitchell, Microsoft Data Platform MVP
Data Warehouse and ETL Consultant
TimMitchell.net | @Tim_Mitchell | Tyleris.com
ETL Best Practices
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply