I am trying to bulk insert the client's very large csv file into a table with designated PK fields. It appears to have duplicates in it, so I'd get PK issues and it stops. Thie import is done on a regular basis and their format often changes each time as well.
I don't wish to remove the PK designation but I do anyway. After I do bulk inserts including the dups I then basically make a holding table containing only the dups in distinct mode. Then I'd remove the dups in original table, then reinsert the distinct dups. I wrote a stored procedure to do the removal of dups. Then I'd turn the PK association back on.
Seems like a ha-ha comedy and that there ought to be a better way. Folks, is there a better way? If so, what is it?