This is a simple script and container image for regularly copying data from one place to another. The intended use case is log file preservation from limited, rotating storage such as systemd’s journal onto long-term storage.

Usage: copier INTERVAL [MODE:]SOURCEDIR:SOURCEPATTERN:TARGETDIR [...]

• Every INTERVAL seconds, for each argument, files in SOURCEDIR that match SOURCEPATTERN (extglob, globstar) are copied to TARGETDIR.
• Both SOURCEDIR and TARGETDIR must be absolute paths; if either does not exist, the argument is skipped in that iteration.
• If SOURCEPATTERN matches files in subdirectories, the corresponding directories are created in TARGETDIR if required.
• If SOURCEPATTERN matches subdirectories, they are copied recursively.
• By default, existing files in TARGETDIR are never touched.
• Colons are not supported in any of the argument components.
• Valid MODE: clobber (overwrite existing matching file in TARGETDIR)
• Remember to quote the SOURCEPATTERN so it isn’t expanded by the calling shell.

Requirements: Bash, rsync

A pre-built container image is available at atdr.meo.ws/archiveteam/copier. It takes the same arguments as the copier script, which acts as the entrypoint.

For example, to copy systemd logs every hour, you likely want to copy past journals without clobbering and the active journal with clobbering (cf. systemd-journald.service(8)). In a shell, you might start a container like this:

podman run -v /var/log/journal:/journal:ro -v /longterm-storage:/target atdr.meo.ws/archiveteam/copier 3600 /journal:'**/*@*.journal?(\~)':/target/journal clobber:/journal:'**/!(*@*).journal?(\~)':/target/journal

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.