This is a procedure useful not only for DataStage environment. Often you need to delete old logs periodically in order to save disk space. But especially in DataStage, which lacks automatic datasets cleanup is handy to delete old datasets. These may originate from your DataStage Dataset stages, or as a temporary resource from Lookup stage and others.
On our production system there was over 20 GB of old Datasets. But be careful, in case you are using Datasets for some long-term saving of data, don’t use the following commands. Progress on your own discretion!
Solution is simple – build a batch file, which executes a command over all files in specified directory, where file modification date is more than X days ago.
Geniue syntax:
|
1 |
forfiles /P <PATH TO DIRECTORY> /D -<NUMBER OF DAYS> /C "<EXECUTED COMMAND>" |
In case you want to test it and move data to temporary directory first, just change the executed command:
|
1 |
forfiles /P <PATH TO DIRECTORY> /D -<NUMBER OF DAYS> /C "cmd /c move @file <PATH TO ARCHIVE DIRECTORY>" |
An example with deletion (be careful, there is no warning after issuing this command):
|
1 |
forfiles /P <PATH TO DIRECTORY> /D -<NUMBER OF DAYS> /C "cmd /C del @file /q" |
With an actual data:
|
1 2 |
forfiles /P C:\IBM\InformationServer\Server\Datasets /D -7 /C "cmd /C del @file /q" exit |
Save these commands as a BAT file, run Task Scheduler and make a new task running this file on regular basis. Make sure the task is working without a problem and that’s it, problem solved.
Source: http://www.windows-commandline.com/find-files-based-on-modified-time/