Extract, Transform, Load (ETL)

SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

  • Article

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

You rated this post out of 5. Change rating

2007-11-08

6,299 reads

Technical Article

RE: shrink logs

  • Reply

Ritch, I use SQL7 and SQL2000 and i find the SQL7 to be a little buggy in this instance, it could be the setup i have here but i find quitting EM and starting again / or a refresh of the taskpad will give a more accurate result, plus it depends on the type of […]

You rated this post out of 5. Change rating

2003-06-18

1 reads

Blogs

Small Data SF 2024

By

I can’t remember how I heard about Small Data SF 2024, but it caught...

A New Word: Moledro

By

moledro – n. a feeling of resonant connection with an author or artist you’ll...

Snowflake + Azure blob

By

Let’s go back to data platforms today and I want to talk about a...

Read the latest Blogs

Forums

7 sept, scheduled book

By philip.scott

Comments posted to this topic are about the item 7 sept, scheduled book

7 sept, schedlued article

By philip.scott

Comments posted to this topic are about the item 7 sept, schedlued article

6 sept, published book

By philip.scott

Comments posted to this topic are about the item 6 sept, published book

Visit the forum

Question of the Day

Azure Data Lake Storage Gen 2

Azure Data Lake Storage Gen 2 is built on ...?

See possible answers