xrootd

New Features

• [Server] User redirector to find directory to be listed (R5 backport).
• [XrdCl] More effective error recovery policy when the source is a metalink.
• [xrdcp] Implement bandwidth limiting (--xrate option).
• [xrdcp] Implement ability to continue interrupted transfer (--continue).
• [xrdcp/Python] Add an option (--zip-mtln-cksum) to use the checksum from a metalink with files extracted from ZIP archives.
• [xrdcp/Python] Automatically infer checksum type if asked to.
• [xrdcp/Python] Add an option (--rm-bad-cksum) to automatically remove destination file if the checksum check failed.

Major bug fixes

• [Server] Correct sequencing of an async close to prevent a SEGV.
• [Server] Add missing initializer to avoid TPC SEGV on busy systems.
• [Server] Initialize the XrdAccEntityInfo structure.
• [XrdHttp] Fix MKCOL response when we have an EEXIST.
• [XrdHttp] Periodically reload verify cert store.
• [XrdHttp] Disable session cache.
• [XrdCl] Don't set on-connection handler for local files.
• [XrdCl] Don't set the stream error window for auth errors.
• [XrdCl] Fix race condition resulting in dangling pointer to SidMgr.
• [XrdCl] Make Channel operations thread-safe in respect to ForceDisconnect.

Miscellaneous

• [Deb] Refine debian packaging.
• [CMake] Repleace XRootD find module with config module.
• [RPM/CMake] Don't build XrdCns.
• [Packaging] Add xrootd-voms and xrootd-voms-devel packages equivalent
• [Packaging] Add additional private headers for vomsxrd. to the vomsxrd packages.
• [Python] Support PyPy.

Major bug fixes

• [Server] Avoid SEGV when skipping over large if/else/fi blocks.
• [XrdHttp] Fix curl speed limit to be really around 10KB/s.
• [xrootdfs] Make sure xrootdfs_create() checks return code of xrootdfs_do_create().
• [XrdHttp] Change the default ssl cipher.
• [XrdHttp] Enable elliptic-curve support for OpenSSL 1.0.2+.
• [XrdHttp] Use Mozilla cipher list only with OpenSSL 1.0.2+.

Minor bug fixes

• [XrdCl] When xrdcp reports an error refine the message so it is clear whether the erros comes from the source or destination.
• [XrdCl] Make sure on error local redirect is not retried infinitely.
• [XrdXrootdConfig] Fixed wrong segsz parameter.
• [XrdHttp] Give a chance to map the issuer name in the case of a proxy certificate (needed to accommodate systems that do user mapping, e.g. eos)
• [XrdHttp] Fix the logic for determining SecEntity.name.
• [XrdHttp] Don't overwrite SecEntity.name after the gridmap phase.
• [xrootdfs] Make sure xrootdfs_create() does check the return code of xrootdfs_do_create().

Miscellaneous

• [XrdSecgsi] In the case of delegation, give client a chance to use XrdSecGSISRVNAMES to limit where it is being redirected to.
• [Python] Make rpath relative to the module.
• [Debian] Proper Debian pkg naming.
• [XrdCl] Use glibc semaphores for rhel>=7.
• [Server] Export pointers to Acc, Ofs, and Sfs plugin into the XrdEnv.
• [XrdMacaroons] Use env ptrs to get authorize obj.

Major bug fixes

• [Proxy] Remove offending CGI elements before passing URL.
• [Server] Accommodate changes in epoll handling in CentOS 7.10 plus.
• [SSI] Fix double delete due to extraneous code addition.
• [XrdOuc] Prevent seqv in XrdOucString::replace().
• [XrdHttp] Fix escape code for \r.
• [XrdHttp] Fix: the url quoting function was swapping the quoting of CR and LF.
• [XrdHttp] Fix the "100 Continue" response.
• [XrdHttp] Prevent the logic that looks for the /n from getting confused by lines containing zeroes.
• [XrdTpc] Properly forward existing opaque info to the OFS layer.
• [XrdTpc] Properly format redirection URL when OFS layer returns also opaque information.
• [SSI] Avoid SIGABRT when init fails. Fixes 751

Minor bug fixes

• [Python] Use right bdist_wheel.
• [XrdCl] Report last error if metalink runs out of replicas, fixes #1071

Miscellaneous

• [CMake] Add XROOTD_PLUGIN_VERSION to FindXRootD.cmake.
• [XrdCl] TPC: use separate cgi element (tpc.dlgon) to mark user intension to delegate.
• [Server] Use new tpc.dlgon CGI token to drive tpc redirection.
• [CMake] Add option to build only client libraries.
• [XrdHttp] If a gridmap file has been configured, XrdHttp now prefers the already mapped username to the non-mapped one that's coming from a security extractor plugin.

Major bug fixes

• [XrdSys] Avoid deadlock on poller stop, fixes #1095
• [XrdCl] Recreate env r/w lock after forking.
• [XrdCl] Use different bit for mkpath and tpc delegation.

Minor bug fixes

• [Server] Use correct mechanism to determine DNS registration.
• [XrdCl] Reflect server added cgi in tpc.scgi.
• [XrdCl] tpc: don't open non-root/xroot source.
• [XrdCl] Set the tpckey to 'delegate' if delegating.
• [XrdCl] Install priv headers in correct dir.
• [CMake] Link to ${UUID_LIBRARIES} instead of uuid.
• [Python] Change pkg name to xrootd.

Miscellaneous

• [Server] Allow redirect differentiation for delegated and undelegated TPC.
• [TPC] Assume delegation if the tpc.scgi token is present.
• [RPM] drop default 'all.export /tmp' in standalone cfg
• [RPM] Package SsiLib and SsiShMap with client.

New Features [Server] Add support for multi-vo credentials. [Python] Set rpath for bindings to correct xrdcl. [Python] Expose XrdCl::Env in Python. [Python] Add version attribute to xrootd module, closes #1027 [Python] Add cat API to FileSystem, closes #1047 [XrdCl] Add unique uuid to open message. closes #1037 [XrdCl] Support zcrc32 when extracting a file from ZIP archive. [XrdCl] Support decompression (inflate) of files in ZIP archives. [XrdCl] Select the least loaded stream for next data chunk. [XrdCl] xrdcp: adjust the number of parallel chunks in real time proportionally to the number of connected data streams. [XCache] Allow up to 512 MB blocksize. [CMake] Add Find XRootD CMake module. [CMake] Add FindLibUuid module. [XrdCms] Add XrdCmsRedirLocal as ofs.cmslib plugin.

Major bug fixes [Xcache] Make sure direct cache access is actually accessible. [Xcache] Fix missing mode setting in the data space. [GSI] Fix SAN check logic due to cut-paste error. [GSI] Fix faulty SAN wildcard check and make message clearer. [Python] Correctly handle install --user. [XrdCl] Fix dirlist response parsing, closes #1038 [XrdCl] Sign endsess if instructed to do so by server, fixes #1045 [SSI] Make sure the request sessN pointer is always valid.

Minor bug fixes [TPC] Make sure to include source CGI in a delegated transfer. [XrdCl] Respect target checksum vs preset source checksum. [XrdCl] Stat local file without opaqueinfo. [XrdCl] xrdcp -S option should set the number of additional data streams. [XrdCl] Avoid overflow of worker jobs size.

Miscellaneous [Misc] Allow client to indicate it supports local files. [UPS] Disable python bindings. [XCache] Mark File is in sync for the final sync. [Xcache] Enhance security of the pss.dca directive. [RPM] Move xrdmapc to client package. [XrdCl] Adjust xrdcp defaults.

New Features

• [POSIX] Add methods to the cache mngt objecT to get status of a cached file.
• [Server] Add xrd.network dyndns option for dynamic DNS handling.
• [Server] Properly handle dynamic DNS environments.
• [Server] Add evict option to the prepare request.
• [Server] Allow better handling for proxy caching clusters.
• [Server] Allow configurable posc sync level.
• [Server] Implement framework for a kXR_prepare plug-in.
• [XrdCl] Implement streaming dirls, closes #225
• [XrdCl] Retry policy changes: see details in #950.
• [XrdCl] Add switch/flag to enforce zip archive listing.
• [XrdCl] Preserve tried/triedrc cgi info on redirect for kXR_locate request, #closes #944
• [XrdCl] Implement prepare evict and expose in xrdfs.
• [XrdCl] Expose prepare abort and query prepare.
• [XrdCl] Add tpc.scgi if delegation is being used.
• [Python] Expose chunked dirlist API in Python.
• [Python] Support globbing.
• [XrdClHttp] Add XrdClHttp submodule.
• [XCache] Implement write-queue number of blocks / threads config options. (#963)
• [CMake] Add switch for XrdCl builds only.

Major bug fixes

• [XrdCl] Allow to cancel classical copy jobs.
• [XrdCl] Fix race condition in error recovery.
• [XrdCl] Prevent race condition between Request T/O and socket read.
• [XCache] Check for errors during XrdOssFile::FSync - do not write cinfo file if it fails.
• [XCache] Deny write access to a file in initial Prepare() call, fixes #663
• [XCache] Review and correct error handling in POSIX and XCache, implement XCache::Unlink()
• [XrdTpc] Always query dual stack for HTTP TPC.
• [XrdTpc] Do not include authz header in the filename.
• [XrdTpc] Add curl handle cleanup on redirections or errors.
• [XrdHttp] Include provided opaque information in the redirection.
• [XrdHttp] Fix digest names returned to clients.
• [XrdHttp] Fix opaque info handling on redirections.
• [XrdMacaroon] Fix macaroon authorization config.
• [XrdSecgsi] Make proxy cache path aware.
• [XrdSecgsi] XrdSecProtocolgsi::Encrypt - set IV correctly and report correct size.

Minor bug fixes

• [XrdCl] Use DirList with Locate and Merge flag in xrdcp
• [XrdCl] Index substreams with right index.
• [XrdCl] Make sure there is no race condition while loading sec protocol.
• [CMake] Enable by default rpaths on MacOSX.
• [Server] Use correct directory path creation flag in kXR_open (compatible).
• [Server] Require localroot to exist and be a directory for default N2N.
• [Server] Avoid SEGV when given an empty directory continuation.
• [SSI] Close loophole between internal "provision" and Finished() calls.
• [POSIX] Correct pfn to lfn translation for the simple case.
• [FRM] Ignore the space assignment directive.
• [XCache] Cache::LocalFilePath() only try to fill output pfn buffer if it is non-null.
• [Python] Make sure pip install fails if install script fails.
• [Python] Check dependencies on pip install, if some are missing list them and raise an exception.

Miscellaneous

• [XrdCl] Enhance redirect back-trace output.
• [Python] Respect pip python version when installing xrootd, fixes #955.
• [Python] Respect pip --user switch, fixes #952
• [CMSD] Allow redirect when limit exceeded in sched directive.
• [Packaging] Add support for Ubuntu disco.
• [Packaging] Discontinue support for Ubuntu artful.
• [XrdCeph] Replace XrdCeph source code with a submodule.
• [XCache] More reasonable defaults for pfc.writequeue parameters.
• [CMake] Declare CMake options, fixes #995
• [CMake] Set -Werror for debug and CI builds only.

Major bug fixes

• [POSIX] Fix VMP issues with POSIX preload library causing SEGV.
• [POSIX] Additional hardening of POSIX functions for Xcache.
• [XrdCl] Make sure local destination path is not scrambled, fixes #925
• [XrdCl] Fix memory leak in AsyncSocketHandler.
• [XrdCl] Fix recursive copy.
• [XrdCl] Make sure released SID is valid object.
• [XrdCl] Correctly calculate the offset of the last file in ZIP64.
• [Server] Liberalize character set allowed in login name (fixes EOS issue).
• [Server] Save the proper ReqID for async responses on close.
• [Server] Handle waitresp sequencing to avoid early bridge termination.
• [XrdHttp] Only consider running plugins for new requests.
• [XrdHttp] Handle clients that request multiple checksums.
• [XrdHttp] Do not accept more data than specified in the HTTP request.
• [XrdHttp] Fix high memory usage caused by cached XrdHttpProtocol objects.
• [XrdSecgsi] Fine tune message bucket content.
• [XrdSecgsi] Make sure options for proxy saving to file are honored.
• [XrdSecgsi] add missing entries in gsiClientStesp and gsiServerSteps.
• [Python] Ensure Python module doesn't deadlock/segv on exit, fixes #330
• [XrdMacaroon] Create macaroon with proper caveats and path.
• [Proxy] Improve handling of failures during opening of local data and cinfo files.
• [Proxy] Fix memory leak - block was not freed when writing to disk failed.
• [XrdTpc] Only use Curl's low-speed-limit with libcurl v7.38 and later.

Minor bug fixes

• [Server] Fix bridge waitResp issues to allow merge of pull #902.
• [Server] Further simplify tracing, especially for var persistece.
• [Server] Fix strlcpy for Group null-string in XrdOfsTPC.
• [XrdMacaroon] Log user caveats with macaroon generation.
• [XrdMacaroon] Allow admin to customize default macaroons authz.
• [XrdMacaroon] Set the macaroons.onmissing default.
• [XrdMacaroon] Interact with json-c appropriately for request.
• [XrdMacaroon] Ensure we delete macaroon object after use.
• [XrdHttp] XrdHttp does not support chunked encoding; respond appropriately.
• [POSIX] Fix string scoping issues in debug messages.
• [XrdSecgsi] Downgrade error msg from key file issues to DEBUG.
• [XrdSecgsi] Improve buffer content dumps for debugging purposes.
• [XrdCl] Use correct flag to remove handler in inqueue.
• [XrdCl] Log properly kXR_waitresp, closes #852
• [XrdCl] Fail ZIP extraction if file is compressed.
• [XrdHttp] Don't assume that a header "Depth: 1" means that the resource is a directory.
• [Python] Expose parallel copy jobs in python bindings.
• [Python] Expose multi-source download in python.

Miscellaneous

• [XrdHttp] Add support for chunked encoding in uploads.
• [XrdHttp] Implement RFC 3230 digests for GET requests.
• [Packaging] Add xrootd-fuse deb package, closes #831
• [XrdCl] Add more descriptive error message on XrdCl::CopyProcess:run, closes #849

New Features

• [XrdCl] Provide operation pipelining API.
• [XrdCl] Redirect traceback dump.
• [XrdCl] Allow specifying plug-ins on protocol level
• [XrdCl] Pass # streams to TPC destination.
• [XrdCl] Enable proxy delegation.
• [XrdCl] Enable state redirection for local files.
• [XrdCl] Add an API to close connection.
• [XrdCl/Server] Implement TPC-Lite for delegated TPC copy.
• [XrdCl/Server] Implement kXR_writev operation.
• [Server] Allow tpc requests at destination endpoint to be redirected.
• [Server] Add &I=<iptype> to 'u' login monitoring record.
• [Server] Allow spaces in id's via acc.spacechar.
• [Server] Allow space preassignment of file creation paths.
• [Server] Identify type of IP connection in monitoring (&I=4|6 in 'u' record).
• [Server] Enable use of delegated credentials for 3rd party copy.
• [Server] Allow callbacks for close() and opague() requests.
• [Server] Allow the CKS manager to run on the redirector.
• [Server] Allow config continuation to files in a directory.
• [Server] Add 'query confg start' to find server start time.
• [SSI] Add ability to spread requests to increase parallism.
• [SSI] Add monitoring capabilities to SSI.
• [SSI] Properly handle Finished() calls prior to response posting.
• [XrdSsi] Provide summary monitoring information to report stream.
• [TPC] Allow number of streams to use to be passed to the server.
• [TPC] Support Http TPC.
• [XrdHttp] Added XrdHttp cipherlist filter string config option.
• [XrdHttp] Map Xrd error codes to HTTP status codes
• [XrdHttp] Add plumbing necessary for RFC3230 digest requests.
• [XrdSecgsi] Use hostname, not reverse DNS, for address comparison.
• [XrdSecgsi] Allow XrdSecGSITrustDNS setting to disable use of all DNS lookups.
• [XrdSecgsi] Add option to save delegated proxies as credentials.
• [XrdSecgsi] Improved checking of CA expiration.
• [XrdSecgsi] Review proxy delegation mechanism.
• [XrdSecgsi] Support signing of server DH public parameters.
• [XrdCrypto] Allow XRootD client to accept subjectAltNames.
• [XrdMacaroons] Macaroons plugin for XRootD.
• [Proxy] Implement new options in pfc.diskusage for better control of purging.
• [Proxy] Support origins that only talk http.
• [Proxy] Add 'pss.tid' CGI element to all outgoing URL's.
• [Proxy] Implement new options in pfc.diskusage for better control of purging.
• [POSIX] Implement Cache Context Management plugin.
• [XrdApps] Implement xrdqstats command to display summary monitoring.

Major bug fixes

• [XrdCl] Handle properly seerver disconnect.
• [XrdCl] apply TPC timeout to the 2nd sync, fixes #800
• [XrdCl] Parse correctly CDFH ZIP64 extension.
• [XrdCl] Correct ownership of virtual redirect msg.
• [SSI] Do not leak memory when a fatal error occurs. Fixes #775
• [SSI] Avoid race condition between Finish() and SetResponse().
• [Server] Avoid SEGV when the close operation fails.
• [Server] Fix SEGV when using delegation and VOMSFUN with raw creds.
• [POSIX] Avoid SEGV due to race condition between open() and close().
• [POSIX] Correct regression that broke POSIX preload.
• [Proxy] Support multiple IO objects working with the same file.
• [Proxy] Fix incorrect rc check when processing Fstat() (disabled progbar).

Minor bug fixes

• [Server] Follow protocol spec when returning file compression info.
• [Server] Fix checksumming on filesystems that don't support fattr.
• [XrdOfs] Prevent double-parsing of parameters for ofs.ckslib.
• [XrdCl] Correctly set path in case of state redirect.
• [XrdCl] Correct handling of --infiles option. Fixes #779
• [XrdCl] Don't read past buffer when parsing protocol response.
• [XrdHttp] Fix error codes in the case of PUT and MKDIR.
• [XrdHttp] Support mv for files with spaces.
• [XrdHttp] Return correct content-range header.
• [Secsss] Pass correct parameters when registering an ID. Fixes #689

Miscellaneous

• [XrdCl] Delete channel on TTL.
• [Server] Do not allow badly formed CGI to pollute subsequent tokens.
• [TPC] Do not take case into account for hostname matching.
• [TPC] Allow source protocol to be transmitted to server via tpc.spr CGI.
• [TPC] Set envars XRD_TRACEID, XRDTPC_SPROT, and XRDTPC_TPROT
• [Proxy] Avoid auth failure due to URL cgi directives.
• [Proxy] Add method to cache to get local file path of cached file.
• [Proxy] Fix wrong logic in cache purge algorithm.
• [Proxy] Resolve conflicts in URL rewriting.
• [POSIX] Avoid unnecessary cache open requests.
• [XrdCrypto] Replace C-style string manipulation with C++ equivalent.
• [Secgsi] Implement stricter version of RFC2818 (host name check) for client.
• [Secgsi] Enable use of an unique IV in enc/dec cipher operations.
• [XrdHttp] Break XrdHttp into a module and a utility library.
• [XrdHttp] Expose query string in external req handler.
• [XrdThrottle] Update the throttle plugin to propogate underlying errors.
• [XrdThrottle] Improve chaining behavior of throttle plugin.
• [RPM] Use OPENSSL_NO_FILENAMES flag.
• [Docs] Update Doxyfile, closes #743
• [All] Place protocol definition under a modified BSD license.
• [All] FreeBSD compatibility patch.

Major bug fixes

• [XrdCrypto] add protection against missing extension.
• [XrdCl] Ensure DeepLocate counter doesn't overflow, fixes #758
• [XrdCl] Fix recursive copy, fixes #792
• [XrdCl] Correctly classify errno while reading/writing to a socket.
• [XrdCl] Fix local checksum double I/O problem.
• [XrdHttp] Fix callback of protocol plugin after bridge delay.
• [XrdHttp] Obey the Connection request header.
• [XrdHttp] Reset filesize when the XrdHttpReq object is used.
• [XrdHttp] Do not increment reqstate when headers are incomplete.
• [Proxy] Fix incorrect rc check when processing Fstat().

Minor bug fixes

• [XrdCl] xrdfs ls: fix formatting issue,fixes #823
• [XrdCl] If stream is broken delete the in-message.
• [XrdCrypto] fix signatures of Export methods.

Miscellaneous

• [XrdCl] Expose kXR_cancel flag in FileSystem::Prepare, closes #699
• [XrdCl] Specify file for extraction from ZIP archive also through opaque info.
• [XrdCl] Document xrdcp/xrdfs return codes, closes #628
• [XrdCl] Explicitly log request retries, closes #690
• [XrdCl] Remove redundant stat from ZIP open sequence.
• [XrdCl/XrdSec] Add CGI to specify 'sss' keytab.
• [XrdSecsss] Add possiblity to inject an endorsement string into an SSS SecEntity object.

Major bug fixes

• [XrdCrypto] Use consistently time_t.
• [XrdCl] Treat empty redirect response as error.

Minor bug fixes

• [XrdCl] Use NEED2SECURE macro properly.

Miscellaneous

• [XrdSecgsi] Improving checking of CA expiration.
• [XrdCl] Improve TPC key generation, fixes #662