How can I split large log files to upload them?

Linux systems have a built in set of utilities that make it simple to work with large logs.

The steps are as follows:

  1. Compress the file or files.

  2. Split the files into pieces that are smaller than 20MB.

  3. Upload the split pieces.

For example, given a set of log files with the names AMPS-xxxx.log, you could run the following commands:

$ zip AMPS.logs.zip AMPS-*.log

$ split -b 19M -d AMPS.logs.zip AMPS-Log-Chunk-

At this point, you will have a numbered set of files:

$ ls *Chunk*
AMPS-Log-Chunk-00 AMPS-Log-Chunk-02 AMPS-Log-Chunk-04
AMPS-Log-Chunk-01 AMPS-Log-Chunk-03

Each of the files will be 19MB or less. You can upload the files into ZenDesk, and let the engineer handling the ticket know that these are split pieces of a compressed ZIP file.

Last updated