github.com/creativeprojects/resticprofile

Following the release of the long awaited restic 0.14.0, here’s a new resticprofile!

A few big features were introduced in this version:

• HTTP hooks using a similar syntax to command hooks but sending HTTP messages to monitoring platforms
• Better support of the new-ish copy command
• Addition of mixins on configuration file v2 (in preview)
• Ability to choose your preferred shell on Windows (cmd, powershell or bash)
• Send resticprofile logs to a syslog server
• Add armv8 (arm64) CPU target to docker images
• Upgrade to restic 0.14.0 in docker image
• Add rclone to docker image
• Also search for a configuration file in the resticprofile program folder on Windows (to be used in portable mode)

Changelog

• Add http hooks (#114)
• Add support for syslog (#127)
• Allow to choose shell in global config (#112)
• ConfigV2: Mixins (#115)
• ConfigV2: Replace list params - fix #108 (#113)
• Initialise copy repository using --copy-chunker-params (#117)
• Mixins: List merging & inheritance update (#121)
• add binary dir in path and .BinaryDir template var (#134)
• add goreleaser config to also build arm64 images
• add rclone to docker image #131
• upgrade packages (CVE-2022-28948)

Here it is!

It's been a while since we released a new version 😞 The resticprofile team has been busy preparing some really cool new features:

• simplify the command line by allowing the use of [profile].[command] - Thanks @Syphdias for the PR #89
• shell completion (for bash, can also be used by zsh with bash compatibility) - Thanks @jkellerer for the PR #90
• run a shell command to use as a stdin input (like mysqldump) - Thanks @jkellerer for the PR #98
• run a shell command in the background when non fatal errors are detected from restic - Thanks @jkellerer for the PR #99

and a lot of bug fixes 👍🏻

Changelog

• Add run flag to to be able to have profile and job name as one argument (#89)
• Added "generate" command to create resources (#110)
• Added missing cmds that filter by host, tag & path
• Feature: Take command output as stdin for restic (#98)
• Fix and unit tests for #91
• Fix config includes when any pattern has no match (#95)
• Implemented stream-error callbacks (#99)
• Merge pull request #101 from jkellerer/fix-common-args
• Merge pull request #94 from jkellerer/ft-add-missing-commands
• Only pass common CLI args to command hooks [#100]
• Shell completion (#90)
• display a neat stack trace on panic
• fix for profiles command not showing inherited commands fixes second part of #97
• remove "includes" section from profiles in "profiles" command

New maintenance version, with bug fixes:

• fix multiplication of arguments when commands are retried
• fix status file telling the backup was successful when a warning happened (file/dir not found)

Changelog

• Fix args are multiplied when commands are retried (#84)
• Fix for #88 with unit tests
• don't inherit profile description
• prepare for future versions of the configuration
• show schedules separately
• simple implementation of a config file v2
• update packages

This release adds a few new features:

• support for splitting configuration into multiple files
• support for run-finally that runs shell commands every time after restic
• ability to define your own systemd unit and timer files (from go templates)
• support for the restic copy command
• fix for some cosmetic issues with crond scheduler

Changelog

• Add support for copy command (#73)
• Change codecov uploader to use GitHub Action v2 (#79)
• Optional: Allow disabling path in retention with 'false' (#67)
• Scheduler refactoring (#76)
• Support "run-finally" in backup-section & profile (#70)
• Supporting config includes (e.g. profiles.d) (#65)
• Systemd template (#75)
• add tests on crontab
• crontab RemoveJob returns error if the entry was not found
• upgrade dependencies

Docker images

• docker pull creativeprojects/resticprofile:0.16.0

⚠️ Important

Version 0.15 fixed some issues with escaping parameters to the restic command line. If you've used any of these characters in file or directory names in your configuration, please make sure your backup is still working as expected: space, *, ?, \, [, ', ".

If for some reason the fix broke your configuration, there's a new flag legacy-arguments that you can activate in the global section to revert back to the broken implementation:

global:
    legacy-arguments: true

New features in 0.15:

• add .Hostname in configuration template (#55)
• add description field in profile section
• add support for prometheus file export and push gateway
• hide confidential values in output (#58)
• add .TmpDir variable to configuration template (#62)

This version also includes fixes for:

• resolving special paths starting with a ~ (unixes only)
• warn when the restic binary was not found at the specified location (but still found at a different location)
• resolve glob expressions in backup sources (#63)

Changelog

• Add support for prometheus export and push (#57)
• Added "{{.TmpDir}}" to TemplateData (#62)
• Feature: Hide confidential values in output (#58)
• Resolve glob expressions in backup sources (#63)
• Shell escape (#60)
• Update non-confidential values to support shell.Args (#68)
• add Hostname pre-defined variable to template resolver (#55)
• add description field in profile section
• build with go 1.17
• don't send status summary in dry-run
• show description in output of profiles command

Docker images

• docker pull creativeprojects/resticprofile:0.15.0

Bug fix release

This release changes the way the restic binary is searched:

• restic binary path in global configuration can now contain the ~ character like ~restic/bin/restic
• if you specified a path in the global configuration and it cannot find the file, a warning will be displayed and resticprofile will keep trying to find a suitable binary

Changelog

• use shell to resolve special paths with ~
• warning when the restic-binary was not found

Docker images

• docker pull creativeprojects/resticprofile:0.14.1

Release 0.14.0

• New locking/unlocking features. Thanks jkellerer for the PR

Changelog

• Added resticprofile flags --no-lock & --lock-wait (#33)
• Summary from plain output when not run in terminal (#48)
• Updated contrib script systemd/send-error.sh (#49)
• add macOS arm64 target to install.sh script
• add token as an environment variable

Docker images

• docker pull creativeprojects/resticprofile:latest
• docker pull creativeprojects/resticprofile:0.14.0

This version fixes a defect where extended status wasn't returning the extended information on Windows.

Changelog

• add Homebrew Tap (#45)
• add github token in config
• fix test too slow on build agent
• fix windows bogus prefix (#47)

Docker images

• docker pull creativeprojects/resticprofile:latest
• docker pull creativeprojects/resticprofile:0.13.2

Bug fix:

• regression from v0.13.0: a message was sent to stderr when initialize parameter was set and the repository already exists

Changelog

• Increase test coverage (#40)
• fix #41: a message was sent to stderr when parameter initialize=true and repo exists

Docker images

• docker pull creativeprojects/resticprofile:latest
• docker pull creativeprojects/resticprofile:0.13.1

This version adds two new features:

• parameter no-error-on-warning to consider a backup successful when restic produced a new snapshot but some files were missing (https://github.com/creativeprojects/resticprofile/discussions/38)
• resticprofile now catches the error output (stderr) to be written in the status file, also makes the environment variable RESTIC_STDERR available to the targets run-after-fail.

Changelog

• add RESTIC_STDERR env variable to run-after-fail
• quick implementation of ignore warning
• quick mock to do some testing with a fake restic
• returns stderr output in the status file

Docker images

• docker pull creativeprojects/resticprofile:latest
• docker pull creativeprojects/resticprofile:0.13.0

This release mainly brings 2 new features and a few enhancements:

• add support for --all in status, schedule, unschedule commands
• add backup statistics to the status file (via a new extended-status flag)

Changelog

• Add backup statistics in status file (#36)
• Added fail env variable ERROR_COMMANDLINE (#32)
• Added support for --all to status & (un)schedule (#31)
• Enhanced "unschedule" to remove all possible jobs (#28)
• add profile name when running status --all
• fix an issue where status --all was stopping at the first profile with no schedule
• update goreleaser config to v0.154
• upgrade packages
• upgrade self-update library
• use go 1.16

Docker images

• docker pull creativeprojects/resticprofile:latest
• docker pull creativeprojects/resticprofile:0.12.0

Highlights:

• detect systemd using systemctl: https://github.com/creativeprojects/resticprofile/issues/25
• add experimental support for scheduling tasks with crond
• add support for background tasks on Mac OS via a new schedule-priority parameter
• add support for scheduling forget and prune commands

Deprecation

Scheduling in the retention section is now deprecated, please use the forget section instead (https://github.com/creativeprojects/resticprofile/issues/23)

Changelog

• Add background types and low priorityIO to launchd plist (#19)
• Add support for scheduling forget (#26)
• Added prune as supported, schedulable command (#24)
• Move CI build from Travis CI to GitHub Actions
• Refactoring schedule package (#30)
• Schedule priority (#29)
• add crond support to unix targets (except macOS)
• add deprecation notice for schedule on retention
• add docker build to goreleaser
• add pre-release parameter to self-update
• add schedule-priority parameter
• add working directory to crontab line
• allow verbose flag after the command
• display systemd timer status in a nicer way also display log >= "warning" instead of "err"
• move other-flags before the sub-sections
• refactor show command to remove empty lines
• search for systemctl instead of systemd
• upgrade clog package

Docker images

• docker pull creativeprojects/resticprofile:latest
• docker pull creativeprojects/resticprofile:0.11.0

This update changes the way systemd units are generated: These used to be of type oneshot but it means they can be started more than once. They have been changed to notify which is like simple but resticprofile is notifying systemd that the schedule has started and stopped.

To change the type on your existing schedules, you'll need to run the commands unschedule and schedule again.

Changelog

• change systemd unit to "notify"
• update packages verify self-update binary using checksums.txt file
• update restic to 0.11.0 in docker image

New resticprofile version with bug fixes and a big new feature:

• fix --exclude and --iexclude parameters in unix environment not properly escaping ? and * characters
• don't escape (space) if already escaped
• self-update now works on ARM CPUs (like raspberry pi)
• configuration files can embed Go templates for modularity

I think it is now feature complete for a version 1.0 😄

Changelog

• Escape exclude globs passed to /sh
• add proper character escaping: - count the number of \ in front - do not escape it again if it was already escaped
• change self-updater library to work with ARM cpus
• detect arm version from runtime internal register
• new version of the updater
• put armv7 target back
• squash merge of branch config-single-template: add templating and variable expansion
• upgrade clog package
• upgrade selfupdate package

A few minor features and bug fixes:

• add version command
• add force-inactive-lock flag in profiles

Changelog

• Add "option" in example configurations in README and azure.conf fixes https://github.com/creativeprojects/resticprofile/issues/13
• Add version command
• add tests on set and get PID from lock file
• remove openbsd from goreleaser library github.com/shirou/gopsutil cannot be compiled for openbsd
• squash merge of branch pid: should fix https://github.com/creativeprojects/resticprofile/issues/14
• use go 1.15 for ci builds
• write down child PIDs in lock file

Two new features in this release:

• add a few environment variables when running scripts (run-before, run-after, run-after-fail)
• add a status file generated after running a profile (to send to some monitoring software)

Changelog

• Merge branch run-after-fail
• merge from branch status-file

A few new big features in this release:

• add --dry-run flag
• redirect console to a file with --log (for running in the background)
• generate random keys with generate-random command
• schedule/unschedule profiles automatically with command schedule/unschedule (also status to check a scheduled task)

Changelog

• add SUDO_USER to systemd environment
• add dry-run flag
• add file logger
• add journalctl to status output
• add schedule documentation to README
• add schedule parameter in config
• add windows task scheduler
• create plist file for darwin
• create system job on darwin
• full implementation of systemd user unit
• generate random keys
• generate schedule for darwin
• implement schedule-log
• move logger into external package
• only ask for the user password once
• redirect terminal when elevated mode
• send message from elevated process to parent process via some simple http calls
• send terminal output remotely
• sudo trick in the example configuration
• system daemon with launchd
• systemd user/system

Bug fixing release:

• restic flags were not generated for commands forget and mount https://github.com/creativeprojects/resticprofile/issues/9

Changelog

• add restic flags for forget and mount commands
• add test to verify forget flags are loaded for all configuration types
• version 0.8.3

This is a bug fixing release:

• in the configuration file, some strings containing a comma were split into an array of strings. This is now fixed.

Changelog

• add table of contents
• build 0.8.2
• document new features in README
• remove armv7 packages from goreleaser
• remove default decode hooks (which can do funny things with tags)

This is mostly a bug fixing release:

• allow for spaces in directories and files (these were ignored before and were messing up the command line)
• add a show command to see the profile details (mostly for debugging really)
• add a --format flag to specify the type of configuration file format (if you want to use a different extension)

Changelog

• add --format flag to specify the config format
• add command 'show' to display profile properties
• add show command to help
• allow spaces in directories and files
• change expected structure in unit tests
• display correct XDG directories in error message
• update dependencies

This version is introducing a few new features:

• experimental support for HCL configuration files
• a configuration file with no extension is searched using all supported file extensions: -c profiles would either load profiles.conf, profiles.yaml, profiles.toml, profiles.json or profiles.hcl
• a new global parameter to check available memory before starting a profile (default is 100MB)

Changelog

• Initial support for HCL configuration file
• Refactoring y/n question
• add OS & ARCH to panic data
• add missing build info to docker image
• add note with self-updating on linux/arm
• add safeguard to prevent running restic when memory is running low: "min-memory" in "global"
• add section on where to locate configuration file
• bump version
• example of configuration in HCL
• refactor configuration as a dependency as opposed to a global
• search for configuration file without an extension resticprofile will load the first file with a valid extension (conf, yaml, toml, json, hcl)
• update xdg paths in documentation

This a maintenance release:

• add a new parameter run-after-fail which is running the shell commands after any kind of failure (during other commands or during restic execution)
• a minor breaking change if you use the repository auto-initialization, it will now run after the run-before scripts. It makes more sense this way (in case you mount your backup disks in run-before for example)

Changelog

• add raspberry pi to supported platforms
• add run-after-fail parameter to run shell commands after a restic command failed. the auto initialization of a repository now happens after the "run-before" scripts (in version 0.7.0 it was happening before)
• detect panic and display a nice message asking the user to post details on github
• fix self-update not working on windows
• run self-update command with no configuration file

This is a maintenance release:

• fixing a defect when starting a backup with no command definition for backup
• adding two new options to run scripts before and after a profile (not just backup like before)
• implementation of a new module to create systemd units. More to come on future releases

Changelog

• Bump up version
• Fix update confirmation message
• Refactoring resticprofile commands
• add goreleaser target to build for raspberry pi
• initial support for systemd
• run shell commands before and after a profile. In previous versions, you could only run commands before and after a backup
• stop creating empty configuration file at XDG config location

Changelog

• Activate self-update flag
• Add .tar.gz binary for mac os
• Build tar.gz for windows so we can download it from git bash
• Clean up lock file after pressing CTRL-C
• Fix nil pointer panic when retention not defined

Complete rewrite of resticprofile in go:

• I tried python and I wasn't particularly impressed
• I tried go and I loved it (also that's what all the cool kids do nowadays)

The default configuration profiles.conf is still expected to be in TOML format. But now, resticprofile also supports YAML and JSON file format. Simply feed a .toml, .json or a .yaml file to select the desired format.

The configuration file from the previous versions remains unchanged. There are two additions to it:

• priority flags: acts like nice on unixes and now available on all platforms
• lock flag to avoid running two profiles at the same time (this is a local lock)

Accept a repository from the environment instead of the configuration file

• Add a timestamp before each message
• Use colours more readable on light console
• Add a flag to disable all ansi characters (to redirect the output to a file)

Adds support for stdin file stream

• Adding 'mount' command support
• Allow chained inheritance of environment variables
• Allow inheritance on 'retention' section

• Removing ugly KeyboardInterrupt message when you hit CTRL-C
• Allow the host flag to be a boolean (if set to yes the current hostname will be used)
• Also search for restic binary under Windows (like the chocolatey bin folder)
• Adding two options to check repository before and/or after a backup: 'check-before' and 'check-after'
• Run shell commands before and after a backup: 'run-before' and 'run-after'

• Added "groups": ability to define groups of profiles to run at once

• Fixed compatibility with Windows
• Fixed defect in profile inheritance

First 'usable' version of resticprofile!

Please note the groups are not working yet (groups can run multiple configuration in one command)