Need to zip and move C2 level audit files

Question

Post reply

Need to zip and move C2 level audit files

Viewing 15 posts - 16 through 30 (of 47 total)

You must be logged in to reply to this topic. Login to reply

ps. One Orange Chip Points: 29255 More actions · Answer 1

ps.

One Orange Chip

Points: 29255

September 19, 2010 at 11:33 pm

#1223816

let the script zip the files in asynchronous mode, a seperate zip process will be created for each file EXCEPT the current file. just dont touch the current file. this way you are only touching the files which are *not active*. will this not serve the purpose or you have something else in your mind?

Pradeep Singh

Ananda-292708 Ten Centuries Points: 1339 More actions · Answer 2

Ananda-292708

Ten Centuries

Points: 1339

September 20, 2010 at 5:32 am

#1223935

You can use 7Za a nice standalone tool from 7Zip family.

the synatx is simple

7za.exe a myzip.7 *.bcp >>compress.log

if you find things are asynchronus try calling the batchfile from mater..xp_cmdshell .

shew SSChampion Points: 11387 More actions · Answer 3

Ananda-292708 (9/20/2010)

You can use 7Za a nice standalone tool from 7Zip family.

the synatx is simple

7za.exe a myzip.7 *.bcp >>compress.log

if you find things are asynchronus try calling the batchfile from mater..xp_cmdshell .

Thanks for the input, but xp_cmdshell is "outlawed" here. Also, up above, I mentioned that we are trying to find a Windows-only solution. Freeware and other third party tools just cause too much of a hassle with our data security folks.

shew SSChampion Points: 11387 More actions · Answer 4

ps. (9/19/2010)

let the script zip the files in asynchronous mode, a seperate zip process will be created for each file EXCEPT the current file. just dont touch the current file. this way you are only touching the files which are *not active*. will this not serve the purpose or you have something else in your mind?

This was the first thing I tried, and it didn't work at all. What I was doing was zipping and deleting in the same loop. Sometimes the zip didn't work because of the asynchronous nature, but the delete worked every time. Consequently, I deleted files that were not zipped yet.

shew SSChampion Points: 11387 More actions · Answer 5

Jeff Moden (9/19/2010)Use to work for a "little" DOD shop known as Raytheon. 😛

Little? LOL

The only way that I can think of to make this synchronus is based on what the Windows native zip thing actually does. Do you need to precreate the zip file or will windows do that when it's done compressing?

Here is the link that I found on google to use as a model. http://blogs.msdn.com/b/daiken/archive/2007/02/12/compress-files-with-windows-powershell-then-package-a-windows-vista-sidebar-gadget.aspx

The way the code works is that it pre-creates the .zip file and adds files to it. So, the .zip file ultimately contains several files when the loop finishes.

Jeff Moden SSC Guru Points: 1000539 More actions · Answer 6

shew (9/20/2010)

Jeff Moden (9/19/2010)Use to work for a "little" DOD shop known as Raytheon. 😛

Little? LOL

The only way that I can think of to make this synchronus is based on what the Windows native zip thing actually does. Do you need to precreate the zip file or will windows do that when it's done compressing?

Here is the link that I found on google to use as a model. http://blogs.msdn.com/b/daiken/archive/2007/02/12/compress-files-with-windows-powershell-then-package-a-windows-vista-sidebar-gadget.aspx

The way the code works is that it pre-creates the .zip file and adds files to it. So, the .zip file ultimately contains several files when the loop finishes.

Sorry for the delay... I'll take a look now.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.
"Change is inevitable... change for the better is not".

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)
Intro to Tally Tables and Functions

Jeff Moden SSC Guru Points: 1000539 More actions · Answer 7

Ah... that may be the key. Is there any problem with you just zipping one file at a time instead of putting a bunch of files in a single zip file?

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.
"Change is inevitable... change for the better is not".

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)
Intro to Tally Tables and Functions

ps. One Orange Chip Points: 29255 More actions · Answer 8

Jeff Moden (9/20/2010)

Ah... that may be the key. Is there any problem with you just zipping one file at a time instead of putting a bunch of files in a single zip file?

Hey Jeff, the problem will arise when there are multiple files on the folder and he tries to zip them in a loop. there will be seperate wipzip processes for each file and they will zip asynchronously. OP wants the loop to run in a way that 2nd file doesnt get zipped unless 1st one completes so that he can move the zipped file.

Pradeep Singh

EdVassie SSC Guru Points: 60312 More actions · Answer 9

Why are you looking at C2 auditing? This is an obsolete standard that does not match current compliance requirements. IMHO you should be looking at Common Criteria, which is the current standard.

Original author: https://github.com/SQL-FineBuild/Common/wiki/ 1-click install and best practice configuration of SQL Server 2019, 2017 2016, 2014, 2012, 2008 R2, 2008 and 2005.

When I give food to the poor they call me a saint. When I ask why they are poor they call me a communist - Archbishop Hélder Câmara

shew SSChampion Points: 11387 More actions · Answer 10

EdVassie (9/21/2010)

Why are you looking at C2 auditing? This is an obsolete standard that does not match current compliance requirements. IMHO you should be looking at Common Criteria, which is the current standard.

I'm a contractor. It is my understanding that C2 Level Auditing is a specific requirement at this site. Just trying to keep the customer happy (and operational, without full disks). 🙂

I suspect larget .trc files will result regardless of the type of auditing being used. We are a hosting facility, and some of the databases monitor virtually every device in the shop. Consequently, heavy database writes take place 24/7. During a sample period, I observed at least 45 GB of .trc files being generated a day. I don't know if this is a peak period or not. It is to our advantage to zip the individual 200 GB .trc files since they normally shrink to 5 GB a piece.

It would be nice to generate the files in zipped format without having to zip manually, but Microsoft tells me that is not an option.

shew SSChampion Points: 11387 More actions · Answer 11

ps. (9/21/2010)

Hey Jeff, the problem will arise when there are multiple files on the folder and he tries to zip them in a loop. there will be seperate wipzip processes for each file and they will zip asynchronously. OP wants the loop to run in a way that 2nd file doesnt get zipped unless 1st one completes so that he can move the zipped file.

Yes, I believe this is correct. I will have the issue regardless of the number of files I try to compress because the script does not pause while the source .trc file is zipping. And, since I am pre-creating the .zip file (I don't know of any other way to make zip work), the destination .zip file will always exist--meaning that I will not be able to use the presence of the .zip file as a "switch" to indicate that the zip operation is finished.

Jeff Moden SSC Guru Points: 1000539 More actions · Answer 12

ps. (9/21/2010)

Jeff Moden (9/20/2010)

Ah... that may be the key. Is there any problem with you just zipping one file at a time instead of putting a bunch of files in a single zip file?

Hey Jeff, the problem will arise when there are multiple files on the folder and he tries to zip them in a loop. there will be seperate wipzip processes for each file and they will zip asynchronously. OP wants the loop to run in a way that 2nd file doesnt get zipped unless 1st one completes so that he can move the zipped file.

Yep... I pretty much got that from the OP, Pradeep. I'm just trying to figure out a work around without using any 3rd party software so that shew doesn't have to jump through any security hoops. You just don't know what a PITA following the computer security for a DOD related company can be. 🙂

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.
"Change is inevitable... change for the better is not".

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)
Intro to Tally Tables and Functions

Jeff Moden SSC Guru Points: 1000539 More actions · Answer 13

shew (9/21/2010)

ps. (9/21/2010)

Hey Jeff, the problem will arise when there are multiple files on the folder and he tries to zip them in a loop. there will be seperate wipzip processes for each file and they will zip asynchronously. OP wants the loop to run in a way that 2nd file doesnt get zipped unless 1st one completes so that he can move the zipped file.

Yes, I believe this is correct. I will have the issue regardless of the number of files I try to compress because the script does not pause while the source .trc file is zipping. And, since I am pre-creating the .zip file (I don't know of any other way to make zip work), the destination .zip file will always exist--meaning that I will not be able to use the presence of the .zip file as a "switch" to indicate that the zip operation is finished.

I guess the only work around is to do the zips in one job and the deletes in another. Other than that, I'm out of tricks because I believe even a DOS Batch Loop is going to do this part asynchronusly.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.
"Change is inevitable... change for the better is not".

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)
Intro to Tally Tables and Functions

ps. One Orange Chip Points: 29255 More actions · Answer 14

ps. (9/17/2010)

may be you can write the move part in a seperate job that moves all zipped files. and run this "before" the the zipping job runs.

i would have done something like this:

1. Job zip - that finds all trace files except current one and zip them asynchrounously in seperate zip files. ( the script i posted earlier does that)

2. job move - this job would move the zipped files one by one to target location. This will run syncronous.

what i have seen is the file which is getting zipped wont get the extension .zip unless it is complete; which means your move job will move only those files which are zipped completely. I hope you will use wildcard something like move files *.zip

Pradeep Singh

ps. One Orange Chip Points: 29255 More actions · Answer 15

Jeff Moden (9/21/2010)[hr

Yep... I pretty much got that from the OP, Pradeep. I'm just trying to figure out a work around without using any 3rd party software so that shew doesn't have to jump through any security hoops. You just don't know what a PITA following the computer security for a DOD related company can be. 🙂

You were too tied up with your test for past couple of days:w00t: so i mentioned that. pls dont mind 🙂

Pradeep Singh