github.com/richardlehane/siegfried

Added

• Travis and Appveyor CI automated deployment to Github releases and Bintray
• PRONOM v85 signatures
• LICENSE.txt, CHANGELOG.md
• Go Report Card

Fixed

• golang.org/x/image/riff bug (reported here)
• misspellings reported by Go Report Card
• ineffectual assignments reported by Go Report Card

Added

• implement Library of Congress FDD signatures (beta)
• implement RIFF matcher
• -multi flag replaces -nopriority; based on report by Ross Spencer

Changed

• change to -z output: use hash as filepath separator (and unix slash for webarchives); requested by Ross Spencer

Fixed

• parsing fmt/837 signature; reported by Sarah Romkey

Added

• implement freedesktop.org MIME-info signatures (and the Apache Tika variant)
• implement XML matcher
• file name matcher now supports glob patterns as well as file extensions

Changed

• default signature file now "default.sig" (was "pronom.sig")
• changes to YAML and JSON output: "ns" (for namespace) replaces "id", and "id" replaces "puid"
• changes to CSV output: multi-identifiers now displayed in extra columns, not extra rows

Added

• summarise os errors; requested by Ross Spencer
• code quality: vendor external packages; implemented by Misty de Meo

Fixed

• big file handling
• file handle leak; reported by Ross Spencer
• mscfb; reported by Ross Spencer

Added

• new -throttle flag; requested by Ross Spencer

Changed

• errors logged to stderr by default (to quieten use -log ""); requested by Ross Spencer
• mscfb update: lazy reading
• webarchive update: decode Transfer-Encoding and Content-Encoding; requested by Dragan Espenschied

Fixed

• long windows paths; reported by Ross Spencer
• 32-bit file size overflow; reported by Ross Spencer

Changed

• -log replaces -debug, -slow, -unknown and -known flags (see usage above)
• highlight empty file/stream with error and warning
• negative text match overrides extension-only plain text match

Added

• new MIME matcher; requested by Dragan Espenschied
• support warc continuations
• add all.json and tiff.json sets

Changed

• minor speed-up
• report less redundant basis information
• report error on empty file/stream

Added

• scan within warc and arc files with -z flag; reqested by Dragan Espenschied
• sf -slow FILE | DIR reports slow signatures
• sf -version describes signature file; requested by Michelle Lindlar

Changed

• quit scanning earlier on known unknowns
• don't include byte signatures where formats have container signatures (unless -doubleup flag is given); fixes a mis-identification reported by Ross Spencer
• sf -debug output simplified
• roy -limit and -exclude now operate on text and default zip matches
• roy -nopriority re-configured to return more results

Fixed

• upgraded versions of sf panic when attempting to read old signature files; reported by Stefan
• panic mmap'ing files over 1GB on Win32; reported by Duncan
• reporting extensions for folders with "."; reported by Ross Spencer

Added

• -noext flag to roy to suppress extension matching; requested by Greg Lepore
• -known and -unknown flags for sf to output lists of recognised and unknown files respectively; requested by Greg Lepore

Added

• support annotation of sets.json files; requested by Greg Lepore
• add warning when use -extendc without -extend

Fixed

• report container extensions in details; reported by Ross Spencer

Added

• text matcher (i.e. sf README will now report a 'Plain Text File' result)
• -notext flag to suppress text matcher (roy build -notext)
• all outputs now include file last modified time
• -hash flag with choice of md5, sha1, sha256, sha512, crc (e.g. sf -hash md5 FILE)
• -droid flag to mimic droid output (sf -droid FILE)

Fixed

• detect encoding of zip filenames reported by Dragan Espenschied
• mscfb reported by Dragan Espenschied

Added

• scan within archive formats (zip, tar, gzip) with -z flag
• format sets (e.g. roy build -exclude @pdfa)
• support bitmask patterns

Changed

• leaner, faster signature format
• mirror bof patterns as eof patterns where both roy -bof and -eof limits set

Fixed

• (mscfb) reported by Pascal Aantz
• race condition in scorer (affected tip golang)

Changed

• user documentation
• bugfixes (mscfb, match/wac and sf)
• QA using comparator

Added

• json output
• server mode

Fixed

• single quote YAML output

Changed

• optimisations (mmap, multithread, etc.)

Added

• csv output

Changed

• periodic priority checking to stop searches earlier

Fixed

• range/distance/choices bugfix

Changed

• change to signature file format

Added

• roy (r2d2 rename) signature customisation
• parse Droid signature (not just PRONOM reports)
• support extension signatures

Added

• container matching

Fixed

• cross-compile was broken (because of use of os/user). Now doing native builds on the three platforms so the download binaries should all work now.

Fixed

• bug in processing code caused really bad matching profile for MP3 sigs. No need to update the tool for this, but please do a sieg -update to get the latest signature file.

Added

• sf command line: descriptive output in YAML, including basis for matches

Changed

• optimisations inc. initial BOF loop before main matching loop