github.com/creativeprojects/resticprofile

🌸 Easter release 🥚

This release delivers a preview of a new Client/Server mode, Fish shell completions, Windows scheduling enhancements, and several dependency and toolchain upgrades.

Please note the documentation for the Client/Server mode is not available yet.

New Features

Client/Server Mode PREVIEW

• Remote configuration delivery: resticprofile can now send a configuration profile to a remote server, enabling centralized management of backup profiles across multiple hosts.
• A new remote section in the configuration allows specifying the server address and credentials for the client/server communication.

Fish Shell Completions

• Fish shell support: Added shell completion script generation for the Fish shell (--fish-completion flag in the generate command).

Windows Scheduling: Start When Available

• New schedule-start-when-available option: When a scheduled task is missed (e.g. because the machine was off), Windows Task Scheduler will now start it as soon as the machine is available again.

Bug Fixes

Scheduling Fixes

• Battery power: Fixed a bug where Windows Task Scheduler tasks were incorrectly set to not start when running on battery power, regardless of battery state.
• launchd log paths: When schedule-log is configured in a profile, resticprofile now omits the StandardOutPath/StandardErrorPath entries from the launchd plist, preventing the creation of empty log files.

Maintenance and Toolchain

• Upgraded to Go 1.26
• Upgraded golangci-lint to v2

⚠️ Breaking changes ⚠️

This release introduces two breaking changes in resticprofile exit codes during a failure:

Failure because of low memory

When resticprofile failed to start because the memory remaining is under the threshold in the configuration, the exit code is now 5 and no longer 1

Failure to find any restic binary

When resticprofile failed to start because it couldn't find any restic binary, it now fails with exit code 6 and no longer 1

New contributors

Thanks to @gr1da, @wyattjoh, @hollowhemlock and other contributors for their help in this release 🎉

Changelog

• 4181598 Add Fish Shell Completions (#601)
• 81a47cf Client/Server mode: sending configuration profile to a remote server (#377)
• a64f02c Feature/589 windows start when available (#590)
• 736da3f Fix typo in docker docs (#566)
• e2ffdfb Fix windows scheduler flaky tests (#533)
• ca2ff90 Upgrade golangci lint to v2 (#562)
• 9653541 build without fuse code on unsupported platforms
• 1906875 chore: update GitHub Actions workflows and dependencies, remove SonarCloud integration (#608)
• 7bfce0b chore: update GitHub Actions workflows and improve test coverage (#612)
• cc4cec9 chore: upgrade docker/setup-qemu-action to v4 in workflows
• e03d9cd chore: upgrade goreleaser action to v7 in workflows
• 2c47715 chore: upgrade packages
• cc27dd2 chore: upgrade to go 1.26 (#610)
• fd44cf6 ci: update GitHub Actions workflow for documentation build and deployment (#618)
• 4d2a7b5 code modernisation (#611)
• 3c2965e don't check broken links on stackoverflow, it returns too many 403 errors (anti bot detection)
• dd15110 fix(schedule): allow task scheduling when on battery power (#609)
• b4984af fix(schedule): omit launchd log paths when schedule-log is configured (#600)
• 00f3f2d refactor: remove Node.js dependency and update JSON schema tests to use Go-based validation (#563)
• 70fd44b refactor: replace magic numbers with named exit codes (#615)
• c219cf1 refactor: simplify condition for including description in completions

🍁 Autumn release 🍂

This release brings significant improvements to scheduling, HTTP hooks, configuration handling, and copy operations, along with important bug fixes and dependency updates.

New Features

Enhanced Copy Command Support

• Copy from repository: Added support for specifying source repositories in copy operations with new from-repository, from-password-file, from-password-command, and other from-* configuration options.
• This enables more flexible backup copying workflows between different repositories.

Improved Windows Task Scheduler Integration

• Run Level Control: New schedule-run-level option allows setting task privilege levels (lowest, highest, auto) for Windows Task Scheduler.
• Hide Window Option: New schedule-hide-window option to prevent task windows from interrupting user activity when running with user_logged_on permission.
• Enhanced Task Management: Completely rewritten task information parsing to fix issues with spaces in filenames.

Template Function Enhancements

• Random Number Generation: New randInt template function for generating pseudo-random numbers with seeded repeatability.
• Usage: {{ "seed" | randInt low high }} - useful for creating variable schedules or unique identifiers.

HTTP Hooks Improvements

• URL Encoding: Automatic URL encoding applied to ERROR, ERROR_COMMANDLINE, and ERROR_STDERR environment variables when used in URLs.
• Environment Variable Resolution: Enhanced environment variable resolution in both URL and body of HTTP hooks, with proper separation of URL and body processing.

Bug Fixes and Improvements

Signal Handling

• Group Execution: Fixed interrupt signal handling to properly stop execution of remaining profiles in a group when Ctrl+C is pressed.

Path Resolution

• Executable Handling: Enhanced executable path resolution on Linux to properly handle relative paths, PATH lookups, and working directory resolution.
• Configuration Escaping: Fixed proper escaping of backup source paths in configuration to handle special characters and spaces.

Scheduling Fixes

• Crontab Generation: Fixed crontab file generation to work without user field, improving compatibility with various cron implementations.

New contributors

Thanks to @lil5, @jeinwag, @hansmi and @zumm for their valuable help in making this new release 🎉

Changelog

• c7e87a1 Add "randInt" template function for random number (#540)
• f93ce20 Add copy from repository in configuration (#486)
• de391cb [Docs] Add ~/.config/resticprofile/ to macOS default paths (#509)
• f222ed2 add Makefile help target
• 63dee21 add field bytes_added_packed in status file etc. (#519)
• adf90cb enhance Executable function to handle relative paths (#537)
• f98a2f5 feat(schedule): introduce run-level option (#546)
• 5f1c69d feat(schedule): introduce schtasks-hide-window option (#541)
• 461710b fix(http-hooks): apply url encoding for ERROR_* env vars (#538)
• f9039fa fix(http-hooks): resolve profile env in url and body (#544)
• 1dce428 fix(schedule): generate crontab files without user field (#556)
• 25c2f06 linux: don't resolve symlink on executable to keep a constant name when installed via homebrew (#508)
• 859da21 optimize systemctl commands (#531)
• f748032 read task info from list format instead of CSV (#547)
• 99e6d33 stop running other profiles in group after receiving interrupt signal (#539)
• 720d7ee update Node.js version to 22 and upgrade dependencies in go.mod and go.sum (#543)
• f265fa2 update SonarCloud trigger condition to check for SONAR_TOKEN presence (#542)

🍸 Spring Bank Holiday release 🌞

This release includes several long-awaited improvements.

Improvements:

• Add support for the new Restic flag exclude-cloud-files introduced in version 0.18.0 (Windows only).
• Include missing run-* and send-* targets in all commands (previously missing in check and copy commands).
• Send Restic version as a new Prometheus metric.
• Support .yml files when searching for configuration files (previously only .yaml was considered).

Bug fixes:

• Ignore missing drop-in files to prevent creating an empty systemd schedule.

Changelog

• f7566ac Add missing run-* and send-* to all command sections (#504)
• 88fcedb add support for windows only flag exclude-cloud-files (#500)
• 81a6e45 add support for yml extension when searching for configuration file (#506)
• 43b6c84 add version of restic to the Prometheus metrics (#495)
• 763a2cd fix: remove duplicated field
• c3add93 ignore missing drop-in files to avoid creating an empty configuration (#502)
• 8fd3ddb replace deprecated sonar scan action (#503)

🐛 Bug fix release 🐞

This releases fixes a regression bug when scheduling system tasks with systemd: in some cases the HOME directory was replaced by the home directory of the user who ran the sudo command

Bug report: #493

Changelog

• ceff435 fix root home regression bug in 0.30.0 (#497)

🌱 Spring release 🌸

This release introduces features and fixes for profile scheduling.

⚠️ Breaking change

Until this release, the user scheduling permission was broken. With systemd or the default macOS scheduler, the permission functioned as user_logged_on, running the profile only when the user was logged in.

This issue is now fixed for new schedules.

To update existing schedules, run unschedule and then schedule again.

For systemd, resticprofile requires root privileges (via sudo) to schedule with user permission, as it now uses a system unit running as the user.

Scheduler

Scheduling has been significantly improved with the ability to read existing schedules. The status and unschedule commands now detect any resticprofile schedules from the selected configuration file, even if the profile is no longer present.

Due to recent error reports, Windows Task Scheduler integration has been completely rewritten to use the schtasks CLI tool.

Other

• Fixed issue with non-existent battery reported on recent Mac desktop hardware.
• Added support for Restic 0.18.
• Included pre-built binary for Windows on ARM64.
• Upload Resticprofile container to GitHub Container Registry.

Changelog

• f14094b Correct configuration search paths on Windows (#478)
• 1b05c7a Create FUNDING.yml
• b6b8285 Documentation versioning (#464)
• edc1292 Run tests on read only source dir (#461)
• ae073ae Schedule user logged in (#470)
• 8d8a0da Scheduling improvements (#425)
• e490c42 Set schedule permissions (#476)
• 07df141 Upload to Github Container Registry (#479)
• 4103971 Use schtasks.exe CLI to schedule jobs on Windows (#459)
• ae38db9 add microarchitecture to version command (#466)
• 87cfd5d add other restic exit codes
• ac3b810 chore: add pre-built binary for windows arm64
• edbea57 chore: add support for restic 0.18
• c5a5dfa don't include systemd dependencies on macos & windows
• af9ab7f fix for macOs ghost battery (#468)
• 6c9b3cd remove devcontainer config
• b867b01 replace experimental maps package with standard library (#467)

❄️ Small maintenance release ☃️

Not much going on in this maintenance release while I'm still working on a big refactoring of the scheduling for the next release.

• logrotate package added to the Docker image
• upgrade dependencies to fix security vulnerabilities

Changelog

• e2297a6 add logrotate to docker image #440
• 37c9aef upgrade package - CVE-2024-45337
• 467b82e upgrade packages

🧙🏻‍♀️ Halloween edition 🎃

• Finally a long standing bug was fixed in this version: use proper nice and ionice values on systemd scheduled tasks.
• Also the last chunk of work for the configuration v2: we can now schedule groups!
• Improvement on the JSON schema: a single URL can be used for v1 and v2 configuration
• Another long standing bug on Windows: escape configuration file path on scheduled tasks

Changelog

• fb6c7a2 Escape config file name in schedule parameters (#420)
• 1d3ec32 Fix scheduling arguments (#423)
• 62c5c64 Global json schema (auto versioning) (#412)
• 2a34ea7 Group schedule (#418)
• 830d0fd Setup systemd priority (#409)
• 62f3118 doc: upgrade hugo theme
• da03de7 fix deprecated option in goreleaser config

🍂 Autumn bug fixing 🍁

• fix JSON schema for TOML files that stopped working some time ago (when using Even Better TOML extension on VSCode)
• [configuration v2] fix regression bug where profile groups were stopping after an error when the continue-on-error flag was set

Changelog

• 57d1439 Fix for continue-on-error broken in 0.26.0 (#407)
• ccccfa2 Linters (#397)
• 249fd41 add base url on json schema (#408)
• b254503 add misspell linter
• 7dcb41b fix bullet points from restic man (#398)
• 4181d9e fix overflow integer conversion lint warning from gosec (#399)
• 2231793 generate JSON schema for restic 0.17
• 0ad1b68 prepare next release
• 3612b51 use goreleaser v2

🌞 Sunny Summer Edition 🌻

Two big things in this release:

• support for all the new commands and flags of restic 0.17
• experimental support for environment variables in configuration flags

Example of using environment variables in configuration:

check-repo-profile:
    inherit: default

    run-before:
        - "echo DOW=`date +\"%u\"` >> {{ env }}"

    check:
        read-data-subset: "$DOW/7"

Changelog

• 3b42e4a Add contributed docs for protected configuration on Windows (#394)
• 1e80d02 Restic 0.17 (#396)
• eb8c721 build with go 1.23
• 1df3711 convert priority value to lowercase
• d74c521 fix panic when sending on closed handler
• 49add79 refactoring of arguments generation (#387)
• cff446b upgrade packages

🌦️ Rainy Summer Edition ☔

Fix of a regression bug preventing resticprofile from sending logs to a temporary session log file (prefixed with temp:/t/) Thanks @iluvatyr for the quick bug report 👍🏻

Changelog

• fix creation of mock binaries during unit tests (#375)
• fix documentation for release
• fix regression with temporary log file (#386)
• prepare next release
• refactor unit tests on package lock (#374)
• remove flaky test
• remove goarm linked to the internal variable of runtime go1.23 will apparently forbid the use of linkname to the standard library

🌻 Summer release 🌞

Summer is here in the Northern Hemisphere! A new release is here too! 🎉

A lot of goodies in this release:

new run-schedule command

It's more a behind the scene feature: you no longer need to re-schedule your targets after you changed the configuration. The generated schedule command line is now using this new run-schedule command that reads all the newest bits from the configuration.

You might want to unschedule and schedule all your profiles one last time to replace the existing command line using the new run-schedule command.

More information: https://creativeprojects.github.io/resticprofile/schedules/commands/index.html#run-schedule-command

direct support for crontab files

Previously resticprofile was using the crontab tool to read and write crond schedules. Now you can directly setup a crontab file no matter which tool is going to consume it.

More information: https://creativeprojects.github.io/resticprofile/schedules/index.html

keep content of configuration variables between commands

This was a big issue for some time. The wait is over! You can now set a configuration variable anywhere in a script and use its value later.

More information: https://creativeprojects.github.io/resticprofile/configuration/run_hooks/index.html#passing-environment-variables

"RESTICPROFILE_{FLAGNAME}" env vars

All cli flags can now be set using environment variables.

More information: #334

allow controlling command output redirection

Allow redirection of the console messages to a log file or to syslog.

More information: #343

Add "--stderr" to redirect console to stderr (for "cat" and "dump")

More information: https://github.com/creativeprojects/resticprofile/pull/353

Add option to set working directory for restic backup

More information: https://github.com/creativeprojects/resticprofile/pull/354

many bug fixes!

Changelog

• Add "--stderr" to redirect console to stderr (for "cat" and "dump") (#353)
• Add documentation on how to ingest resticprofile stats into telegraf (#366)
• Add option to set working directory for restic backup (#354)
• Fix group priority (#339)
• Relative source followup: Evaluate symlinks (#355)
• Squashed commit of the following: (#259)
• add "profile" flag as an alias for "name" (#357)
• also search configuration from ~/.config/resticprofile on macOS (#370)
• crond: add support for crontab file only (on any OS) (#289)
• don't replace header value in stringifyHeaders (#327)
• drop-ins: move systemd drop-ins into unified schedule struct (#341)
• drop-ins: transparent timer drop-in support (#340)
• env-file: Implement dotenv and {{env}} support (#323)
• fix remaining unit tests failing in Windows (#360)
• fixed link to config reference / global section (#349)
• flags: added "RESTICPROFILE_{FLAGNAME}" env vars (#335)
• log: allow controlling command output redirection (#343)
• restic: use repository-file if the repo flag contains a password (#336)
• schedule: added unified schedule config struct (#333)
• syslog: local syslog and stdout redirection (#344)
• build with go 1.22 and macOs arm64 (#317)
• update test to pass on Windows 11 (#359)
• upgrade packages (CVE-2024-24786) (#347)
• upgrade packages (CVE-2024-6104)

🦆 Second time lucky February release 🚀

🆕 A lot of pre-built binaries have been added on this release. This is to align with the list of pre-built binaries provided by restic

Otherwise this is mostly a maintenance release with a few fixes:

• fixes multiple backup profiles exporting prometheus files to node_exporter
• fix missing fields in the show command
• weekly docker image build (rebuilt under the same version tag, and as latest)
• restrict the copy command to a list of snapshots in the configuration

Changelog

• Add pre-built binary targets to release pipeline (#324)
• Add prometheus label to build info (#319)
• Change priority warning message (#310)
• Fix missing fields in show command (#315)
• Merge pull request #312 from creativeprojects/nightly
• Restrict copy command to a list of snapshots (#291)
• add separate goreleaser config for rebuilding the docker image only (#309)
• add snapshot build to docker hub
• publish docker images and manifest manually (#313)

💖 February release 💝

⚠️ Potential breaking change

The prometheus library used to send metrics to the proxy using protobuf. By default it's now using the more widely used text format.

If you have any issue sending metrics to your proxy, you can revert to the previous behaviour by adding this option to your profile (it's not a global option)

my_profile:
  prometheus-push-format: protobuf

More information about the different formats: https://prometheus.io/docs/instrumenting/exposition_formats/

New in this version

• fix for CVE-2023-48795
• new option prometheus-push-format with values text or protobuf (#281)
• new option to set log output in global section (#277)
• more control over the default systemd service files (#267)
• and bug fixes!

Thanks to all our contributors for the good work!

Changelog

• Add prometheus-push-format to allow selecting text format (#281)
• Pass context to own commands and profile runner (#280)
• Remove rclone binary in make clean target (#283)
• Trying new configuration for CodeQL (#306)
• Upgrade packages (#307)
• chore: prep next release & allow deployment to fail on PR
• chore: various fixes (#285)
• doc: add information about windows path in variables
• doc: add various missing information (#278)
• logging: allow to setup default log output in global (#277)
• macOS: create LaunchAgents folder if it doesn't exist
• profile: support source with "-" (dash) prefix (#276)
• systemd drop-ins support, option to wait for network-online.target (#267)

🎃 October release 👻

• upgrade dependencies to fix CVE-2023-3978, CVE-2023-39325 and CVE-2023-44487
• fix the broken documentation (some tabs were not accessible)
• can now stop the wait for a lock (restic or resticprofile lock). Before the fix the CTRL-C or other signal was ignored until the lock was acquired or timed out.
• resticprofile is now available on scoop! thanks @hgraeber
• detect if the host is running on battery and cancel an action depending on how much battery is left - see schedule section
• bug fixes (see changelog)

Changelog

• Allow to interrupt the wait for a lock (#249)
• Detect if running on battery (#235)
• Docs for installation from scoop (#268)
• Fix doc shortcodes (#271)
• Template: Reduce log level for Getwd() failures (#251)
• systemd: prevent paging in systemd schedules (#270)
• upgrade packages - CVE-2023-3978 - CVE-2023-39325 - CVE-2023-44487

🌞 New summer edition 🌻

This release mostly fixes a few regression from version 0.21.0 and adds a handful of new features:

• Allow to set base-dir and source-dir in a profile so you can start resticprofile from any current folder
• Allow to set keep-tag, tag & group-by as empty string
• Support for restic v0.16 new flags

Thanks to all the contributors for the great work 👍🏻

Changelog

• Added {{ "data" | base64 }} & {{ "data" | hex }} (#213)
• Allow to set a base-dir inside the profile (#183) (#192)
• Fix 194: Allow to set "keep-tag", “tag” & “group-by” as empty string (#220)
• Fix 218: Args filter must not remove paths (#222)
• Fix 223, 230: Escape args and absolute restic path for pwsh (#224)
• Fix 242: iexclude-file not converted to abs path (#243)
• Fix links for versioned JSON schema files (#244)
• Fix schedule tests (#236)
• Fix: Do not add --tag for tag: true (#221)
• Restic: Add restic v16 release (#238)
• Restic: Fixed unit tests for restic v16 (#239)
• Retention: Align host filter with "backup" (#227)
• Schedule: Capture os.Environ on schedule creation (#212)
• Support lock-wait with --lock-retry in restic 0.16 (#240)
• Support build when GOPATH is unset / fix mockery build warning (#234)
• Variables: Allow to escape "$" with "$$" (#216)

⚠️ Breaking change

The default value of the job tag on prometheus gateway push has changed from command to profile.command. But don't worry: you can easily revert back to the original value by adding this option in your configuration:

prometheus-push-job: "${COMMAND}"

Fixes:

• Complicated scheduling on Windows was sometimes setting up a random delay before starting a job

Changelog

• New option to specify Prometheus Pushgateway job name (#193)
• Templates: Add map/splitR/contains/matches (#197)
• Upgrade task scheduler library (#206)

Bug fixes!

This small release fixes two regression bugs introduced in version 0.21.0

• the ~ was no longer replaced by the user's home directory for some flag (repository and others...)
• environment variables were no longer replaced by their value for some flag (repository and others...)

Thanks @jkellerer for the quick fixes 😉 👍🏻

New features from v0.21.0

This release adds a verification of all the flags permitted by restic. Only the flags compatible with your version of restic will be generated (prior to this version, any flag like unknown-flag would end up on the restic command line as --unknown-flag.

Also this version generates a JSON schema: this is the configuration schema which can auto-complete options, and verify that your configuration is correct. It works with any compatible editor for the JSON, TOML and YAML configuration file format. Thanks @jkellerer for the awesome work on this 🎉 . More information here.

Other nice feature is the introduction of the help command which works for both all the resticprofile and restic commands and flags. Try it out!

Changelog

• Fix #187: homedir/env-vars in repo and other flags (#188)
• Fix #189: completion of own commands (#190)
• Updated clog to 0.13 (#191)

This is a great milestone for resticprofile 🥳

This release adds a verification of all the flags permitted by restic. Only the flags compatible with your version of restic will be generated (prior to this version, any flag like unknown-flag would end up on the restic command line as --unknown-flag.

Also this version generates a JSON schema: this is the configuration schema which can auto-complete options, and verify that your configuration is correct. It works with any compatible editor for the JSON, TOML and YAML configuration file format. Thanks @jkellerer for the awesome work on this 🎉 . More information here.

Other nice feature is the introduction of the help command which works for both all the resticprofile and restic commands and flags. Try it out!

And as usual, a lot of bug fixes 😉

Changelog

• Add variables .OS and .Arch to all templates (#181)
• Allow config encoding in UTF16 and ISO88591
• Enhanced: Catch any --help & --dry-run (http only) (#178)
• Feature: JSON schema (#103) & generated reference
• Fix #164: Failure on uppercase mixin names
• Implement tempDir and log without locking (#168) (#174)
• Make URL and header confidential in HTTP-hook (#175)
• Restic: Add restic v15 release (#180)
• add build tag to disable self-update (#184)
• add suppport for user_logged_on (#160)
• build with go 1.20
• upgrade packages CVE-2022-41721

Not too many new features in this release, but I wanted to build a new docker image with the new restic 0.15.0

Improvements

• Adds shell command hooks to the following additional commands: dump, find, ls, mount, restore, snapshots, stats and tag.
• Docker image now contains ssh (to allow sftp repository), curl, tzdata and ca-certificates

Changelog

• Merge pull request #154 from jkellerer/ft-151
• New docker image (#161)
• Run-hooks for all non-conflicting commands (#151)
• add QEMU for github agent to build an arm64 image
• dry-run should not send web hooks #157