Skip to main content

Zip log files via script or cron job

Zipping log files to save disk space is quite a common task for system admin.

Logs can be kept for a certain period of time depending on the company policy.

To keep logs files and not delete them is quite a good strategy.

If something goes wrong, you will be able to trace and go back in time, and check what went wrong.

Zipping log files can be done manually, or via Bash or Shell script.

If the folder has a standard pattern, which is quite ideal then creating a shell script to automate the zipping of log files will be easier.

For example, if  the log files is on this path /mnt/log folder.

And it has this pattern, YYYY-MM-DD. Creating a script to avoid manual task can be done easily.

Depending on how busy the application is, log files can grow easily and may fill-up the disk quickly.


If the log folder structure is based on Year, Month and Date. Then the shell script knows what pattern to look for and the task can be automated.

Example Screenshot of the log folder structure:



Below is a script that follows the above folder structure.

The script below can be executed safely, since the commands are enclosed via an echo command.

Technically, the echo output is the command that will be executed against the system once the echo command is remove.

Echo command is good, to see how the script will work. If ever the output of the echo command is not as expected, then change the script as needed.

The /mnt/logs <<< this can be changed depending on the path of your log files or the path where the files resides that needs to be zipped

The  YYYY-MM-DD <<< this can be changed also depending the folder or pattern set on the system, change it accordingly as needed


if (( i < 10 )); then  

      i="0$i" <<< this part just append "0" to numbers 1 to 9

               without this, the date will be like 2023-04-1 

               2023-04-1 <<< will not match the folder 2023-04-01


for i in {1..10}; <<< Dates of the month from 1 to 10, change 1 to 30 or as desired

2023-04-<<< the year and month is hard coded but the script can be changed; to automatically get the date and month, modify the script if there's a need to

             

The script will automatically change folder, and zip the log files. Depending on how big are the files, it may take some time. But once all the folder has been traverse then the script will exit.


The script will execute properly, provided the system has some space. Since the xz command will create some temp files while zipping is being done.


Here's the script:

#==========================

#!/bin/sh

for i in {1..10}; do

   if (( i < 10 )); then  #<<only for dates 1 - 9 to make it 01, 02...09

      i="0$i" #<< append number "zero" to numbers 1 to 9 

      echo "$i"

echo "cd /mnt/logs/2023-04-$i; xz -z *.log;cd /mnt/logs/;"

    else

     printf "%d " "$i"

     i="$i"

     echo "$i"

echo "cd /mnt/logs/2023-04-$i; xz -z *.log;cd /mnt/logs/;"

    fi

done

#==========================


Echo command is there, to see if the script will execute properly.

Once the output of the echo command is confirmed and will work fine, remove the echo command and the script will do its job.

If unsure on how the script will work, create a test folder with some dummy files and run the script against it and see the output if that's what is expected.


echo "cd /mnt/logs/2023-04-$i; xz -z *.log;cd /mnt/logs/;"

Remove echo " and the " at the end.

After removing echo " and ", the command should look like the command below:

cd /mnt/logs/2023-04-$i; xz -z *.log;cd /mnt/logs/;


; <<< do not remove the semi-colon, the reason why its there is it will execute the commands in a sequential order. Or the commands will execute in-order, one after the other.


What it does change to the directory>>> cd /mnt/logs/2023-04-$i

Zip all the: xz -z *.log  log files

Go back to the logs folder: cd /mnt/logs/  

And try again the next one: cd /mnt/logs/2023-04-$i until the for loop is done.

$i <<< will change its value depending on the count in for loop



Cheers! Take care. Till next Time. Enjoy Bash'ing. Explore Linux and enjoy the journey..

Stay safe! and Keep things up! 

Do ASAP, Always Say A Prayer...

Practice O.T.G. = Obedience To God

Make time for Prayer and Meditation.

================================

Free Android Apps:

Click  links below to find out more:

Free Android Apps:

Click  links below to find out more:

Excel Keyboard guide:

https://play.google.com/store/apps/details?id=chrisjoms.myexcelapplicationguide

Heaven's Dew Fall  Prayer app for Android :

https://play.google.com/store/apps/details?id=soulrefresh.beautiful.prayer

Catholic Rosary Guide  for Android:

Pray the Rosary every day, countless blessings will be showered upon your life if you recite the Rosary faithfully. 

https://play.google.com/store/apps/details?id=com.myrosaryapp

Divine Mercy Chaplet Guide (A Powerful prayer) BFF = Be Filled Faith:

https://play.google.com/store/apps/details?id=com.dmercyapp   

Labels:

Comments

Post a Comment

Popular posts from this blog

WMIC get computer name

WMIC get computer model, manufacturer, computer name and  username. WMIC is a command-line tool and that can generate information about computer model, its manufacturer, its username and other informations depending on the parameters provided. Why would you need a command line tool if there’s a GUI to check? If you have 20 or 100 computers, or even more. It’s quite a big task just checking the GUI to check the computer model and username. If you have remote computers, you need to delegate someone in the remote office or location to check. Or you can just write a batch file or script to automate the task. Here’s the code below on how get computer model, manufacturer and the username. Open an elevated command prompt and type:     wmic computersystem get "Model","Manufacturer", "Name", "UserName" Just copy and paste the code above, the word “computersystem” does not need to be change to a computer name. A...
Post a Comment
Read more

Print error 016-799 - Fuji Film Xerox

016-799 Fuji Xerox or Fuji Film print error code. That shows a description error as “Print instruction Fail detected in decomposer.” The error code and error description are alien languages for users and even system administrators who are not familiar with Fuji Xerox error code. The error code is quite simple and easy to fix, if the job print goes to the printer but print out doesn’t come out. So, basically the print job was received by the printer, but the printer just doesn’t know what type of paper or what size to use or which tray to utilize for the print out. In some instances, this is just a paper mismatch but the error description; if using Windows 10 to print does not exactly points to what is the issue. First thing to check, is the paper size selected by the user to print. Example, if the printer configuration is A3 and A4 sizes only. But then the person printing the file accidentally chooses “A4 Cover” then this error 016-799 will occur. ...
6 comments
Read more

Notepad++ convert multiple lines to a single line and vice versa

Notepad++ is an awesome text editing tool, it can accept regex to process the text data. If the data is in a “.csv” format or comma separated values which is basically just a text file that can either be opened using a text editor, excel or even word. Notepad++ can process the contents of the file using regex. Example if the data has multiple rows or lines, and what is needed is to convert the whole lines of data into a single line. Notepad++ can easily do it using regex. However, if the data is on a single line and it needs to be converted into multiple lines or rows then regex can also be used for this case. Here’s an example on how to convert multiple rows or lines into a single line. Example data: Multiple rows, just a sample data. Press Ctrl+H, and  on "Find what" type: [\r\n]+ and on "Replace with" type with: , (white space) --white space is needed if need to have a space in between the data. See image below, "Regular Expression" must be se...
1 comment
Read more