SSIS SCENARIO

  • hi all,

    My source is Flat file it contains nearly 3 lac records including the duplicate records , now i want send the Distinct records to first target and

    duplicate records to second target

    Using SSIS

    Can any one help on this.....:-)

  • What I'd recommend is you send *all* the records to one table first, then once they're in that table, you can run a stored proc from within SQL to send them to their respective tables. You can do that easily with T-SQL queries.

    If you really want to do the whole thing through SSIS without going to a temp table first, then what you can do is have an aggregate task connected to your source, grouping by all the fields which you need in order to find out the duplicate-ness. Put a COUNT(*) in your aggregate transform.

    Then put a conditional split. On the COUNT(*) being greater than 1, output the records into your duplicates table, and on COUNT(*) = 1, put them in the distinct table. Note that this will not put a row in the distinct table if that row has duplicates - EG:

    ID Name

    1 Kiran

    2 John

    3 John

    4 Kate

    Your distinct table will contain only Kiran and Kate, while your duplicate table will contain John once.

Viewing 2 posts - 1 through 1 (of 1 total)

You must be logged in to reply to this topic. Login to reply