Security #318: Fix world-readable permissions on new .htpasswd files

On startup the rest-server Docker container creates an empty .htpasswd file if none exists yet. This file was world-readable by default, which can be a security risk, even though the file only contains hashed passwords.

This has been fixed such that new .htpasswd files are no longer world-readabble.

The permissions of existing .htpasswd files must be manually changed if relevant in your setup.

#318 #340

Change #322: Update dependencies and require Go 1.23 or newer

All dependencies have been updated. Rest-server now requires Go 1.23 or newer to build.

This also disables support for TLS versions older than TLS 1.2. On Windows, rest-server now requires at least Windows 10 or Windows Server 2016. On macOS, rest-server now requires at least macOS 11 Big Sur.

#322 #338

Enhancement #174: Support proxy-based authentication

Rest-server now supports authentication via HTTP proxy headers. This feature can be enabled by specifying the username header using the --proxy-auth-username option (e.g., --proxy-auth-username=X-Forwarded-User).

When enabled, the server authenticates users based on the specified header and disables Basic Auth. Note that proxy authentication is disabled when --no-auth is set.

#174 #307

Enhancement #189: Support group accessible repositories

Rest-server now supports making repositories accessible to the filesystem group by setting the --group-accessible-repos option. Note that permissions of existing files are not modified. To allow the group to read and write file, use a umask of 007. To only grant read access use 027. To make an existing repository group-accessible, use chmod -R g+rwX /path/to/repo.

#189 #308

Enhancement #295: Output status of append-only mode on startup

Rest-server now displays the status of append-only mode during startup.

#295

Enhancement #315: Hardened tls settings

Rest-server now uses a secure TLS cipher suite set by default. The minimum TLS version is now TLS 1.2 and can be further increased using the new --tls-min-ver option, allowing users to enforce stricter security requirements.

#315

Enhancement #321: Add zip archive format for Windows releases

Windows users can now download rest-server binaries in zip archive format (.zip) in addition to the existing tar.gz archives.

#321 #346

Change #267: Update dependencies and require Go 1.18 or newer

Most dependencies have been updated. Since some libraries require newer language features, support for Go 1.17 has been dropped, which means that rest-server now requires at least Go 1.18 to build.

#267

Change #273: Shut down cleanly on TERM and INT signals

Rest-server now listens for TERM and INT signals and cleanly closes down the http.Server and listener when receiving either of them.

This is particularly useful when listening on a unix socket, as the server will now remove the socket file when it shuts down.

#273

Enhancement #271: Print listening address after start-up

When started with --listen :0, rest-server would print start server on :0

The message now also includes the actual address listened on, for example start server on 0.0.0.0:37333. This is useful when starting a server with an auto-allocated free port number (port 0).

#271

Enhancement #272: Support listening on a unix socket

It is now possible to make rest-server listen on a unix socket by prefixing the socket filename with unix: and passing it to the --listen option, for example --listen unix:/tmp/foo.

This is useful in combination with remote port forwarding to enable a remote server to backup locally, e.g.:

rest-server --listen unix:/tmp/foo &
ssh -R /tmp/foo:/tmp/foo user@host restic -r rest:http+unix:///tmp/foo:/repo backup

#272

Bugfix #230: Fix erroneous warnings about unsupported fsync

Due to a regression in rest-server 0.12.0, it continuously printed WARNING: fsync is not supported by the data storage. This can lead to data loss, if the system crashes or the storage is unexpectedly disconnected. for systems that support fsync. We have fixed the warning.

#230 #231

Bugfix #238: API: Return empty array when listing empty folders

Rest-server returned null when listing an empty folder. This has been changed to returning an empty array in accordance with the REST protocol specification. This change has no impact on restic users.

#238 #239

Enhancement #217: Log to stdout using the --log - option

Logging to stdout was possible using --log /dev/stdout. However, when the rest server is run as a different user, for example, using

sudo -u restic rest-server [...] --log /dev/stdout

This did not work due to permission issues.

For logging to stdout, the --log option now supports the special filename - which also works in these cases.

#217

Bugfix #183: Allow usernames containing underscore and more

The security fix in rest-server 0.11.0 (#131) disallowed usernames containing and underscore "". The list of allowed characters has now been changed to include Unicode characters, numbers, "", "-", "." and "@".

#183 #184

Bugfix #219: Ignore unexpected files in the data/ folder

If the data folder of a repository contained files, this would prevent restic from retrieving a list of file data files. This has been fixed. As a workaround remove the files that are directly contained in the data folder (e.g., .DS_Store files).

#219 #221

Bugfix #1871: Return 500 "Internal server error" if files cannot be read

When files in a repository cannot be read by rest-server, for example after running restic prune directly on the server hosting the repositories in a way that causes filesystem permissions to be wrong, rest-server previously returned 404 "Not Found" as status code. This was causing confusing for users.

The error handling has now been fixed to only return 404 "Not Found" if the file actually does not exist. Otherwise a 500 "Internal server error" is reported to the client and the underlying error is logged at the server side.

#1871 #195

Change #207: Return error if command-line arguments are specified

Command line arguments are ignored by rest-server, but there was previously no indication of this when they were supplied anyway.

To prevent usage errors an error is now printed when command line arguments are supplied, instead of them being silently ignored.

#207

Change #208: Update dependencies and require Go 1.17 or newer

Most dependencies have been updated. Since some libraries require newer language features, support for Go 1.15-1.16 has been dropped, which means that rest-server now requires at least Go 1.17 to build.

#208

Enhancement #133: Cache basic authentication credentials

To speed up the verification of basic auth credentials, rest-server now caches passwords for a minute in memory. That way the expensive verification of basic auth credentials can be skipped for most requests issued by a single restic run. The password is kept in memory in a hashed form and not as plaintext.

#133 #138

Enhancement #187: Allow configurable location for .htpasswd file

It is now possible to specify the location of the .htpasswd file using the --htpasswd-file option.

#187 #188

Security #131: Prevent loading of usernames containing a slash

"/" is valid char in HTTP authorization headers, but is also used in rest-server to map usernames to private repos.

This commit prevents loading maliciously composed usernames like "/foo/config" by restricting the allowed characters to the unicode character class, numbers, "-", "." and "@".

This prevents requests to other users files like:

Curl -v -X DELETE -u foo/config:attack http://localhost:8000/foo/config

#131 #132 #137

Bugfix #119: Fix Docker configuration for DISABLE_AUTHENTICATION

Rest-server 0.10.0 introduced a regression which caused the DISABLE_AUTHENTICATION environment variable to stop working for the Docker container. This has been fixed by automatically setting the option --no-auth to disable authentication.

#119 #124

Bugfix #142: Fix possible data loss due to interrupted network connections

When rest-server was run without --append-only it was possible to lose uploaded files in a specific scenario in which a network connection was interrupted.

For the data loss to occur a file upload by restic would have to be interrupted such that restic notices the interrupted network connection before the rest-server. Then restic would have to retry the file upload and finish it before the rest-server notices that the initial upload has failed. Then the uploaded file would be accidentally removed by rest-server when trying to cleanup the failed upload.

This has been fixed by always uploading to a temporary file first which is moved in position only once it was uploaded completely.

#142

Bugfix #157: Use platform-specific temporary directory as default data directory

If no data directory is specificed, then rest-server now uses the Go standard library functions to retrieve the standard temporary directory path for the current platform.

#157 #158

Bugfix #155: Reply "insufficient storage" on disk full or over-quota

When there was no space left on disk, or any other write-related error occurred, rest-server replied with HTTP status code 400 (Bad request). This is misleading (restic client will dump the status code to the user).

Rest-server now replies with two different status codes in these situations: * HTTP 507 "Insufficient storage" is the status on disk full or repository over-quota * HTTP 500 "Internal server error" is used for other disk-related errors

#155 #160

Change #146: Build rest-server at docker container build time

The Dockerfile now includes a build stage such that the latest rest-server is always built and packaged. This is done in a standard golang container to ensure a clean build environment and only the final binary is shipped rather than the whole build environment.

#146 #145

Change #112: Add subrepo support and refactor server code

Support for multi-level repositories has been added, so now each user can have its own subrepositories. This feature is always enabled.

Authentication for the Prometheus /metrics endpoint can now be disabled with the new --prometheus-no-auth flag.

We have split out all HTTP handling to a separate repo subpackage to cleanly separate the server code from the code that handles a single repository. The new RepoHandler also makes it easier to reuse rest-server as a Go component in any other HTTP server.

The refactoring makes the code significantly easier to follow and understand, which in turn makes it easier to add new features, audit for security and debug issues.

#109 #107 #112

Enhancement #122: Verify uploaded files

The rest-server now by default verifies that the hash of content of uploaded files matches their filename. This ensures that transmission errors are detected and forces restic to retry the upload. On low-power devices it can make sense to disable this check by passing the --no-verify-upload flag.

#122 #130

Enhancement #126: Allow running rest-server via systemd socket activation

We've added the option to have systemd create the listening socket and start the rest-server on demand.

#126 #151 #127

Enhancement #148: Expand use of security features in example systemd unit file

The example systemd unit file now enables additional systemd features to mitigate potential security vulnerabilities in rest-server and the various packages and operating system components which it relies upon.

#148 #149