File Watcher Task - Multiple File Masks

Question

Post reply

File Watcher Task - Multiple File Masks

lucky-80472

SSCertifiable

Points: 7793
More actions
January 22, 2009 at 10:40 am

#217685

Hello,

I am given a task to convert the existing DTS packages to SSIS. I need some help in dealing with a situation which is a mandatory. This is the setup I have

1. We have an FTP location where .txt files are dropped with different naming conventions like datetime_images.txt, datetime_site.txt, datetimeProd.txt, datetime_jl.txt, datetime_ab.txt etc.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

4. For the next package which looks for remaining files other than the above (point 3), I was not able to successfully implement.

5. I'm using the File Watcher Task from Konesan's. In the editor I've mentioned

Filter : *_*.txt

Find Existing Files : True

Ouput Variable Name : User::MyFile

Path Input Type : Variable

Path Variable : User::SourceDir

The remaining options are not changed in the file watcher task editor.

This issue I'm facing

As the FTP location for all files is the same, this filewatcher task is picking up the other set of files as mentioned in point 3 and I know it will b'coz I've mentioned the filter as *_*.txt.

My question to all experts

1. Is there any way that I can make my logic to look only for the specific files?

Thanks in advance.

Lucky

Viewing 9 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic. Login to reply

Ray M SSC-Insane Points: 21093 More actions · Answer 1

Ray M

SSC-Insane

Points: 21093

January 22, 2009 at 1:25 pm

#931428

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

lucky-80472 SSCertifiable Points: 7793 More actions · Answer 2

Ray M (1/22/2009)

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

Thanks Ray for replying back.

As mentioned in my post (point 2 & 3) makes them unique.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

Hope I'm cleare this time.

Thanks in advance.

Lucky

COZYROC One Orange Chip Points: 28499 More actions · Answer 3

lucky (1/22/2009)

Ray M (1/22/2009)

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

Thanks Ray for replying back.

As mentioned in my post (point 2 & 3) makes them unique.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

Hope I'm cleare this time.

Thanks in advance.

How about this: datetime_??.txt

---
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/

lucky-80472 SSCertifiable Points: 7793 More actions · Answer 4

CozyRoc (1/23/2009)

lucky (1/22/2009)

Ray M (1/22/2009)

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

Thanks Ray for replying back.

As mentioned in my post (point 2 & 3) makes them unique.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

Hope I'm cleare this time.

Thanks in advance.

How about this: datetime_??.txt

Thanks Cozyroc..

In fact I happened to visit your site through a google search.

However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

Lucky

COZYROC One Orange Chip Points: 28499 More actions · Answer 5

lucky (1/23/2009)

CozyRoc (1/23/2009)

lucky (1/22/2009)

Ray M (1/22/2009)

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

Thanks Ray for replying back.

As mentioned in my post (point 2 & 3) makes them unique.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

Hope I'm cleare this time.

Thanks in advance.

How about this: datetime_??.txt

Thanks Cozyroc..

In fact I happened to visit your site through a google search.

However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

Lucky,

The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

---
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/

lucky-80472 SSCertifiable Points: 7793 More actions · Answer 6

CozyRoc (1/23/2009)

lucky (1/23/2009)

CozyRoc (1/23/2009)

lucky (1/22/2009)

Ray M (1/22/2009)

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

Thanks Ray for replying back.

As mentioned in my post (point 2 & 3) makes them unique.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

Hope I'm cleare this time.

Thanks in advance.

How about this: datetime_??.txt

Thanks Cozyroc..

In fact I happened to visit your site through a google search.

However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

Lucky,

The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

Thanks CozyRoc.. that was an absolute answer for my problem.

Thanks so much...

Lucky

lucky-80472 SSCertifiable Points: 7793 More actions · Answer 7

lucky (1/23/2009)

CozyRoc (1/23/2009)

lucky (1/23/2009)

CozyRoc (1/23/2009)

lucky (1/22/2009)

Ray M (1/22/2009)

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

Thanks Ray for replying back.

As mentioned in my post (point 2 & 3) makes them unique.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

Hope I'm cleare this time.

Thanks in advance.

How about this: datetime_??.txt

Thanks Cozyroc..

In fact I happened to visit your site through a google search.

However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

Lucky,

The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

Thanks CozyRoc.. that was an absolute answer for my problem.

Thanks so much...

Hello CozyRoc,

I need your expertize once again...

I could resolve the issue above. But now when I drop all the different pattern files at the same time it is working for the following

1. datetimeProd.txt

2. datetime_jl.txt, datetime_av.txt

but not working for the following

3. datetime_images.txt

4. datetime_site_images.txt

I'm using the filter as *_images.txt but the package is picking up the other one (datetime_site_images.txt). I tried different patterns like *_??????.txt to only pick the one but it is not working.

Thanks in advance.

Lucky

COZYROC One Orange Chip Points: 28499 More actions · Answer 8

lucky (1/23/2009)

CozyRoc (1/23/2009)

lucky (1/23/2009)

CozyRoc (1/23/2009)

lucky (1/22/2009)

Ray M (1/22/2009)

Thats fine that they are using the same location. But makes them unique?

How do you know what site file belongs to?

Thanks Ray for replying back.

As mentioned in my post (point 2 & 3) makes them unique.

2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

Hope I'm cleare this time.

Thanks in advance.

How about this: datetime_??.txt

Thanks Cozyroc..

In fact I happened to visit your site through a google search.

However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

Lucky,

The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

Thanks CozyRoc.. that was an absolute answer for my problem.

Thanks so much...

Lucky,

*_images.txt is not going to work because it means "grab everything which ends with _images.txt" . And datetime_site_images.txt ends just like that. What you have to do is handle explicitly [datetime]_images.txt . So for example if your files are like this:

01-03-05_images.txt

you have to handle it like this: ??-??-??_images.txt

This wildcard will ignore datetime_site_images.txt files.

---
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/