minidump-common

Release Notes

• Improve bit-flip detection reducing the number of false positives
• CFI unwinding on ARM64 has been improved
• Add the "process_uptime" value to the JSON and human output formats
• Parsing procfs files has been made more robust
• no-defaults-features build is working again

Install minidump-stackwalk 0.27.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.27.0/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.27.0

• Improve bit-flip detection reducing the number of false positives
• CFI unwinding on ARM64 has been improved
• Add the "process_uptime" value to the JSON and human output formats
• Parsing procfs files has been made more robust
• no-defaults-features build is working again

Release Notes

• Various improvements during stack walking and when processing malformed
  minidumps.

Install minidump-stackwalk 0.26.1

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.26.1/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.26.1

• Various improvements during stack walking and when processing malformed minidumps.

Release Notes

• Add support for parsing and printing Chromium stability report streams
• Implement frame-pointer scanning in 64-bit Windows minidumps

Install minidump-stackwalk 0.26.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.26.0/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.26.0

• Add support for parsing and printing Chromium stability report streams
• Implement frame-pointer scanning in 64-bit Windows minidumps

Release Notes

• Add "soft_errors" to the JSON, human, and dump output formats
• Updated the wholesym crate to version 0.8.1 to pick up demangling improvements

Install minidump-stackwalk 0.25.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.25.0/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.25.0

• Add "soft_errors" to the JSON, human, and dump output formats
• Updated the wholesym crate to version 0.8.1 to pick up demangling improvements

Release Notes

• Updated the Windows-specific error codes with the ones from
  version 10.0.26100 of the Windows SDK.
• Bumped the wholesym dependency to version 0.8.0 and bumped several other
  dependencies while avoiding major duplications.

Install minidump-stackwalk 0.24.1

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.24.1/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.24.1

• Updated the Windows-specific error codes with the ones from version 10.0.26100 of the Windows SDK.
• Bumped the wholesym dependency to version 0.8.0 and bumped several other dependencies while avoiding major duplications.

Release Notes

• Add MozSoftErrors stream to minidump-common formats

Install minidump-stackwalk 0.24.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.24.0/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.24.0

• Add MozSoftErrors stream to minidump-common formats

Release Notes

Nothing Yet!

• Dependency updates and no functional changes. The new version number is
  required because the previous version accidentally introduced some breaking
  changes to the interface.

Install minidump-stackwalk 0.23.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.23.0/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.23.0

• Dependency updates and no functional changes. The new version number is required because the previous version accidentally introduced some breaking changes to the interface.

Release Notes

• Added inconsisten crash detection. Impossible crashes such as those caused by
  CPU bugs are now flagged with a list describing why they shouldn't represent.
  For example, an access fault caused by an instruction that is not accessing
  memory.

Install minidump-stackwalk 0.22.2

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.22.2/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.22.2

• Added inconsistent crash detection. Impossible crashes such as those caused by CPU bugs are now flagged with a list describing why they shouldn't represent. For example, an access fault caused by an instruction that is not accessing memory.

Release Notes

• Added support for fat Mach-O files in minidump-unwind

Install minidump-stackwalk 0.22.1

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.22.1/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.22.1

• Added support for fat Mach-O files in minidump-unwind

Release Notes

Nothing Yet!

• Added a count of mapped memory regions to the output for Linux minidumps
• Added native thread IDs to the output both in JSON and human-readable form
• Fixed parsing .sym files with several adjacent empty lines
• Updated a number of crucial dependencies (framehop, wholesym) and removed
  duplications where possible

Install minidump-stackwalk 0.22.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/rust-minidump/rust-minidump/releases/download/v0.22.0/minidump-stackwalk-installer.sh | sh

Download minidump-stackwalk 0.22.0

• Added a count of mapped memory regions to the output for Linux minidumps
• Added native thread IDs to the output both in JSON and human-readable form
• Fixed parsing .sym files with several adjacent empty lines
• Updated a number of crucial dependencies (framehop, wholesym) and removed duplications where possible

• Updated the wholesym and framehop crates and reduced their dependencies

• Fixed a small regression in the stack walker when dealing with malformed minidumps

• Native debug information is now read via framehop and wholesym instead of symbolic, this provides significant speedups, reduced memory usage and more flexibility (the stack walker can be built with unwinding support while omitting symbolication for example)
• Added support for the MINIDUMP_THREAD_INFO_LIST stream
• Version numbers are now printed out for ELF files if available
• Symbol files can be passed directly instead of relying on the Breakpad directory structure, assuming they have the correct debug ID and file name
• Removed the dependency on the chrono crate
• Various cleanups and speed-ups to the stack walker

• Fixed several issues when printing error codes on Windows, Linux and macOS
• Fixed the debug id of modules looked up via code info, these now report the correct debug id for the module
• The dump_syms option was removed, support for stack-walking and symbolication using native debug information is mature enough that we don't need it anymore
• The minidump crate doesn't force its dependents to use the same version of memmap2 that it's using
• Updated several dependencies to remove old crates and duplications

• Fixed an issue that prevent the GitHub integration from automatically generating a new release.
• Updated all dependencies to address some mild security issues in some of the dependent crates.

• The stackwalker can now lookup symbols using a module's code identifier if the debug identifier is missing. This is useful with Windows minidumps lacking debug identifiers (a condition that happens rather frequently when Windows is running out of memory at the time of minidump generation).
• Added a custom MozLinuxLimits stream that captures the contents of the /proc/<pid>/limits file of the affected process.
• Add support for the HandleDataStream stream type. This includes all public types that can appear in the stream.
• Remove support for the ThreadIdNameMapping annotation which is unused since we added support for the thread names stream.

• Add support for the MozMacosBootargsStream minidump stream which captures information holding macOS kernel boot arguments.
• Always use the code file for unwinding on Windows when using native debug information. Previously the stack walker could use the PDB file which does not contain unwinding information in 64-bit builds.
• Populate the process ID field on Linux, this displays the PID of the crashed process.
• Expose the MINIDUMP_MEMORY_INFO fields in the public API.
• Disable bit-flip detection on 64-bit ARM minidumps since it's prone to false positives.

• Stack-walking using native debug information was somewhat buggy, after more thorough testing it should be now on-par with breakpad symbol file-based stack-walking.

New minidump-unwind crate

The stack walking machinery has been extracted from the minidump-processor crate and put in a separate one. This crate has significantly less dependencies than the minidump-processor crate which makes it easier to vendor it in projects that only care about stack walking.

Guard-page detection

While analyzing a crash minidump-stackwalk will check if the crashing address hit a potential guard page. Guard pages are usually introduced by the memory allocator around larger allocation and have no permissions set. If the crash address hit one of these pages the memory_accesses filed in the JSON output will contain the is_likely_guard_page: true field.

• Make all minidump-common::format structs writable with scroll.
• Don't fail reading the entire module list if one module has an invalid size.
• All crates now explicitly include the MIT license
• The stack walker will now fetch the CPU microcode value from the evil JSON payload when it's not present in the minidump
• CPU microcode value in the JSON output changed types from u32 to hexstring
• Crashes with jmp/call/ret instruction to non-canonical addresses now show the real address we jumped to instead of 0x0000000000000000 or 0xffffffffffffffff
• Updated several dependencies further decreasing the total number of crates it depends upon.

NULL-pointer crash detection

The minidump-processor crate will use the disassembly of the current instruction to check whether the crash involved a NULL-pointer access. Crashes caused by accessing NULL pointers often exhibit near-NULL address making it less clear what the underlying problem was. When using minidump-stackwalk the JSON output will contain an adjusted_address field holding the reason for the adjustment (null-pointer) as well as the offest from NULL.

Potential bit-flip detection

The minidump-processor crate contains new logic that detects crashes that have been potentially caused by a bit-flip in the user's machine memory. This detection is driven by a heuristic as it is impossible to completely tell apart software crashes from ones induced by flaky hardware. The tests we conducted on real crash data showed this heuristic to be very effective in telling apart such crashes. When using minidump-stackwalk this information will be added to the JSON output under the possible_bit_flips field.

Stack walking using native debug information

The stack walker now supports using native debug information available on the host in addition to Breakpad .sym files. This functionality is enabled by passing the --use-local-debuginfo flag to minidump-stackwalk when processing a crash.

• Updated the num-range crate, further reducing the number of dependencies required when vendoring this crate

• Updated yaxpeax for improved performance when disassembling instructions
• Removed the tracing crate from the dependencies of the minidump-common crate

• More Windows errors are now handled correctly
• Small improvements when handling macOS exceptions
• Hexadecimal values are now printed with consistent width and prefixes
• Several crates were updated
• Fixed some links in the documentation

Support for instruction disassembly

The stackwalker is now capable of disassembling the crashing instruction, printing it out as well as its memory operands. This functionality is used to improve crash analysis: we implemented the ability to extract the real crashing address for global protection fault exceptions on x86-64. These were usually the result of an access to a non-canonical memory location and were reported as either 0x0000000000000000 or 0xffffffffffffffff depending on the platform, irrespective of the actual address.

Commit: f9933c36c5f48bf806a428b06399242e1b170020

• fixed an error in the json schema for the registers field
• we now emit frames based on inlinee info!!! (see below)

Inlinee info

We now can read the new "inlinee" info that dump_syms can add to .sym files. This allows us to report all the inlined functions for a given address in the binary, improving the quality of backtraces. For machine-readable outputs we prefer to emit these as "fake" frames nested under the pre-existing "real" frames. This means you will need to update your infra to make use of the new info (unless using human output), but better reflects the reality that these frames don't really exist at runtime, only in debuginfo.

• minidump_processor::StackFrame now has a new inlines field containing this info
• The JSON schema now has threads.N.frames.N.inlines (see the schema document for details)
• The human output will now include these frames as if they were real, but with "Found by: inlined" and no recovered registers/args

For anyone using breakpad-symbols directly: note that it emits inlines in the reverse order from minidump-processor because the debuginfo is kind of inherently structured that way. minidump-processor immediately reverses them to put them in the same order as the "real" frames.

(Note that this functionality in dump_syms itself is experimental, and needs to be enabled with --inlines, and may not work on all platforms. At this precise moment Windows notably does not have inline info. As such the builtin support for running dump_syms inside breakpad_symbols does not currently attempt to enable it.)

Commit: 9fcb3b5108f4e0583548f163eb7b18dae63d1ebe

• Fixed code_identifier on iOS.
• Stackwalk threads concurrently. Consequently, SymbolProvider now needs to perform request coalescing and take care of internal synchronization.
• minidump-processor's ProcessorOptions now allows you to enable interactive statistics gathering for user interfaces that want immediate feedback

Commit: b5fe86a8f3c1ab8a55f9a6c6e1be72d531af80ee

Lots of fixes and experiments. This change is largely backcompat but we changed a lot of dependencies and have significantly changed ARM/ARM64 stackwalking results (mostly for the better, but in some specific situations you may get worse results, see below).

ARM/ARM64 Stackwalker fixes

All 3 ARM stackwalkers (ARM, ARM64, ARM64_OLD) had completely nonsensical understandings of how lr/fp works. These issues were inherited from breakpad, and still persist in that codebase. We also now ptr_auth_strip in more places and more precisely, improving CFI unwinding.

The old code treated lr as always containing the current return address, and a pushed lr to be a callee-saved version of the caller's lr. This is just completely wrong.

lr is mostly a general purpose register that can contain anything, because it's automatically overwritten whenever you make a call -- the callee simply cannot save it! Most functions will immediately push lr on startup, but thiss is pushing their lr, and therefore their return address. It's more accurate to think of this as saving the caller's pc.

This is a much simpler situation, so the fix was mostly just to rip out a ton of bad code and add/fix tests.

However this fix may cause a regression in some situations for the first frame of the stackwalk. This frame may have a valid lr register value, and if the frame is a proper leaf function (or crashed in the prologue), that lr won't be available on the stack. Without CFI/unwind-tables, we aren't aware of any way to detect that we're in this situation, so we simply assume it's not happening.

If it does happen, then we will effectively end up unwinding our caller instead of ourselves -- but correctly. As a result, the caller will be skipped over, but the rest of the backtrace will behave as normal. Previously we would always assume lr was valid, so we would always handle this situation correctly (but mishandle the much more common non-leaf case).

In the future we may refine our heuristics for the top frame if we determine there's reasonably reliable ways to do that.

Support for MIPS unwinding!

minidump-processor/minidump-stackwalk now have minimal support for unwinding MIPS (32-bit and 64-bit)!

The current implementation only supports Scanning and CFI, because frame pointers seem to be too unreliable (at least for the examples we have).

This should be enabled by default and run automatically, although we don't have a lot of tests or experience, so there's a reasonable chance there's a lot of bugs. symbolic and dump_syms are working on improving the quality of MIPS CFI as well.

Compatibility with inline-ful breakpad .sym files

breakpad-symbols is now compatible with the new INLINE and INLINE_ORIGIN record types in the breakpad .sym format. The current implementation preserves existing rust-minidump behaviour and does not emit any inline frames yet. Instead, as before, symbolicated minidumps will only contain one symbolicated stack frame per "real" stack frame, with the name of the "outer function" at the frame's address, along with the file and line information for the outer function.

We are planning to make use of the new information in a future release.

Experimental native debuginfo support!

breakpad-symbols/minidump-processor/minidump-stackwalk now have new disabled-by-default features

• dump_syms feature

  • enables fetching binaries from --symbols-url assuming it is a Microsoft Symbol Server. the binaries will still be processed into native .sym files, and those will be cached.
  • this will make --symbols-url=https://msdl.microsoft.com/download/symbols/ work
  • because this increases the number of urls we query and we are currently very serial in our http queries, this may significantly increase runtime!
  • this implies the http feature (disabled by default unless you're using minidump-stackwalk)

• mozilla_cab_symbols feature

  • enables the dump_syms feature to also check additional paths that mozilla uses for CABed binaries. If it would check for firefox.exe, it will now also check for firefox.ex_

More generally, breakpad-symbols now has machinery for explicitly requesting binaries. this isn't fully built out but may be useful in the future for implementingthings like bitflip/integrity checks.

Breakpad-symbols has finally migrated from nom 1.2.2 to nom 7.

This came with a nice ~20% reduction in overall runtime, not necessarily because of the migration itself, but because the author took the time to clean up some inefficiencies along the way.

RIP to the funniest old code in the project, thanks mstange!

Migrated to tracing over log

We not emit more structured logs, making it a little harder to read but easier to grep specific tasks

Added more windows-specific error codes from Windows SDK version 10.0.22621.0

You may get some better error pretty-printing

Commit: 4a60e95fd1fceda67aa61cef85461d65457ab046

• Update debugid and uuid dependencies to 0.8.0 and 1.0.0 respectively.
• Make retrieval from http source an optional feature of breakpad-symbols.

Commit: 7811838a0fb6ddac0663fe07026c526bb9012825

• Added trace logging to symbol resolution.
• Make SymbolFile parsing more robust towards enormous symbols.
• Improvements to stack memory pretty printing.
• Add more write implementations for minidump structures.
• Allow unwinding exception contexts that reference different memory regions than the stack memory.

Commit: 7cd02d5824bebf92ba395e3368136ddcff9b9f2c

Just a bugfix, pulling a previous change that was only applied to ARM64 to ARM64_OLD as well (they should have identical behaviour).

Commit: 77b30fc564c8fe23b5ba4dd1663be799d580d290

Some random cleanups and fixes.

Commit: b0af5b4ce2e8b5fb680ef006f415744f4a536d8a

Work on making minidump-common more useful for minidump clients (generators), as well as general improvements.

Commit: a8a4a2228af05b73ee671ae5b8a445b804368ef6 (there was some Cargo.lock messiness, release is smeared between this one and the previous)

This release is a mix of substantial quality improvements, one major breaking change (making some things async), and several smaller changes to APIs. It's a bit of a big release because some major experimentation was going on and we didn't want to release something that we might immediately revert.

Major Breaking Change: Symbolication Is Now async(!)

Making rust-minidump async is in some sense pointless, because it's a single-threaded design that is architected to scale by deploying multiple processes. The primary bottleneck on minidump processing is loading and parsing symbol files, which rust-minidump already maintains a system-global temporary cache for. This cache is designed specifically for the multi-process workflow.

The motivation for introducing async is more of an interoperation concern. For instance, compiling to wasm generally requires I/O to be converted to async. Users of minidump-stackwalk should be unaffected.

Because symbolication is core functionality for minidump-processor, this infects its entire API and means anyone using it will need to run in an async executor. If this proves to be too unpleasant to our users, we may look into making this async-ness configurable with a feature flag (but that would be a lot of work and have a very nasty maintenance burden, so that option isn't to be taken lightly).

If you are building an application, making it work with async may be as simple as adding the following to your Cargo.toml:

tokio =  { version = "*", features = ["full"] }

changing main to the following:

#[tokio::main]
async fn main() {
    ...

and adding .await to the end of your process_minidump call.

If you are building a library, the upgrade story is more complicated: you can either expose the async-ness in your own APIs, or try to hide it with APIs like block_on. Alternatively, you can depend on the minidump-stackwalk binary which behaves the same as it did before.

Major Performance Improvements!

For minidump-stackwalk workloads that make use of large (200MB+) symbol files (e.g. Firefox), peak memory usage has been reduced by about 50%, and runtime decreased by up to 10%! Memory usage wins are consistent whether loading from network or disk. Runtime numbers depend heavily on how much of a bottleneck symbol file I/O is.

Major Reliability Improvements! (Fuzzing!)

Thanks to @5225225, rust-minidump has had a ton of fuzzing infrastructure added. The fuzzers found many subtle bugs in the code and @5225225 fixed most of them too! Thank you so much!! 😭

To the best of our knowledge, none of the bugs found were security issues more serious than:

• denial of service through long loops.
• denial of service through large allocations.
• denial of service through crashes (tripping safe assertions).
• heuristic analyses producing worse results.
• validation steps accidentally discarding valid values.

Thanks to their work, rust-minidump is significantly more robust to "absurd" inputs that could result from either a malicious attacker or random memory corruption (which a crashreporting tool is of course obligated to deal with).

The primary strategy for taming "denial of service" inputs is to realize that although in principle a minidump can specify enormous amounts of work to do or enormous amounts of memory to allocate, a well-formed input will be linearly bounded by the size of the minidump itself. This allows the user of rust-minidump to limit resource usage by setting file-size limits on the inputs they will accept.

For instance, if a minidump reports "I have a list of 10 billion threads" but the minidump itself is only 2MB, we can reject this list length based on our knowledge of how large an entry is in that list (either by rejecting the stream entirely, or by truncating to the maximum possible value for the minidump's size).

Similarly, our stackwalkers have strict "forward progress of the stack pointer" requirements. Although in principle the "language" of stackwalking can tell us to go backwards or loop infinitely in place, we terminate stackwalking whenever this happens. CFI (call frame information) evaluation is similarly bounded by supporting no control flow mechanisms, guaranteeing linear forward progress.

Hardening rust-minidump in this manner is an ongoing project.

(Note however that debuginfo is orders of magnitude larger than a minidump (~2MB vs ~200MB), so it's still quite easy to DOS a rust-minidump instance by just having a stackwalk traverse through a ton of different modules, necessitating an enormous amount of debuginfo to be downloaded, loaded into memory, and parsed -- if a symbol server is made available to rust-minidump. There is no obvious solution to this at the moment.)

Detailed Changes:

Commit: 564ece47dd3b46dd928318fea7ca5f4254dd99c3

Breaking changes to fix integration issues found during deployment.

More docs.

Changes:

minidump-stackwalk/minidump-processor

BREAKING CHANGE: json schema's crashing_thread.thread_index renamed to crashing_thread.threads_index

This was always supposed to be the name, we just typo'd it before publishing and didn't notice.

BREAKING CHANGE: minidump-stackwalk has changed its default output format from --json to --human. Note that the --json flag was added in the previous version, so you can just unconditionally pass --json for both versions to smooth migration.

This change was made to reflect the fact that most users of other flavours of minidump-stackwalk expect the breakpad human-based output more than mozilla's json-based output, minimizing workflow breakage. It's also just the more reasonable output for "casual" usage.

Commit: 445431ce2bfe55fd85b990bb2a5c01867d2a8150

The JSON schema and minidump-stackwalk CLI are now stabilized. They are now reasonable to rely on in production (only reason we would break them is if we ran into a nasty bug).

This release also adds a ton of documentation! (But there can always be more...)

Changes:

rust-minidump

Lots more documentation.

minidump-stackwalk/minidump-processor

Breaking changes:

• Fixed symbols-paths to actually be positional (wasn't supposed to be named)
• Fixed the fact that --symbols-url accepted multiple values per instance
  • You can still pass multiple --symbols-url flags to set multiple http sources, but each one can only have one value
  • This prevents --symbols-url from accidentally greedily parsing the minidump path as one of its arguments
• Legacy truncation fields have been removed from the JSON Schema
  • frames_truncated removed because it was always false
  • total_frames removed because it was always the same as frame_count
  • Both were for a misfeature of a previous incarnation of minidump-stackwalk that we won't implement

New features:

• Cleaned up CLI help messages
• Added "--cyborg=path/to/output/json" output option (producing both --json and --human)
• Added --brief flag for shorter --human output
  • Also introduces ProcessState::print_brief
• Added dummy --json flag to hang docs off of (and to let you be explicit if you want)
• Better feedback for corrupt minidumps
• Added JSON Schema document: https://github.com/rust-minidump/rust-minidump/blob/master/minidump-processor/json-schema.md
  • JSON Schema is now stabilized

Commit: 8308577df997bae72cf952ddbfaeb901a992d950

Removing derelict experiments, and one bugfix.

Changes:

ARM Bugfix

minidump-processor's ARM stackwalker should no longer infinitely loop on misbehaving inputs.

Removed Code

The experimental native DWARF debuginfo symbolizer has been removed from minidump-processor. This code was still technically functional, but it was using very old libraries and not being hooked into new features of minidump-processor. Not worth the maintenance burden until we have a clearer plan for it.

The private minidump-tools subcrate has been completely removed from the project. This has no affect on users using the crates published on crates.io, as it wasn't published. It was a collection of random experiments and tools that are more work to maintain than they're worth now that minidump-processor and minidump-dump work as well as they do. Also it just had some really ancient dependencies -- removing it massively reduces the amount of work needed to compile the workspace.

Commit: 1e7cc1a18399e32b5589d95575447e5f159d275d

New features added to make symbol downloading more reliable.

Changes:

• vendored-openssl feature added to minidump-stackwalk
  • Allows you to statically link openssl (useful for docker)
• --symbol-download-timeout-secs flag added to minidump-stackwalk
  • Sets a timeout for downloading symbol files
  • Forces forward progress for misbehaving http response bodies
  • Default is 1000 seconds for one file

This is a breaking change for the constructor of HttpSymbolSupplier, as it now requires the timeout.

Commit: 4d96a5c49a5e36cf8905cefd5ad8a5041c0d2e72

Tentative parity with mozilla/minidump-stackwalk (and all the breakpad features it uses)! 🎉

All that remains before a potential 1.0 release is testing/documenting/cleanup.

Changes:

minidump

New features:

• GetLastError

  • MinidumpThread now has a method to retrieve the thread's GetLastError value
  • We now parse more Windows error codes

• MemoryInfo:

  • MemoryInfoListStream has been implemented (as MinidumpMemoryInfoList)
    • Provides metadata on the mapped memory regions like "was executable" or "was it freed"
  • LinuxMapsStream has been implemented (as MinidumpLinuxMaps)
    • Linux version of MemoryInfoListStream (using a dump of /proc/self/maps)
  • New UnifiedMemoryInfoList type
    • Takes both MemoryInfoList and LinuxMaps provides a unified memory metadata interface

• Linux Streams:

  • New Linux strings types (LinuxOsString and LinuxOsStr) to represent the fact that some values contain things like raw linux paths (and therefore may not be utf8).
  • Various simple Linux streams have minimal implementations that are exposed as a key-value pair iterator (and also just let you get the raw bytes of the dumped section).
    • LinuxCpuInfoStream (as MinidumpLinuxCpuInfo)
      • A dump of /proc/cpuinfo
    • LinuxProcStatus (as MinidumpLinuxProcStatus)
      • A dump of /proc/self/status
    • LinuxEnviron (as MinidumpLinuxEnviron)
      • A dump of /proc/self/environ
    • LinuxLsbRelease (as MinidumpLinuxLsbRelease)
      • A dump of /etc/lsb-release
  • Because these streams are just giant bags of random info, it's hard to reasonably pick out specific values to expose. The iterator API at least makes it so you can get whatever you want easily.

Improvements:

• Contexts with XSTATE are now properly parsed.
  • (although we still ignore the XSTATE data, but previously we would have returned an error)
• minidump_dump now properly handles bad stack RVAs properly.
• MinidumpSystemInfo::csd_version now works
  • Was reading its value from the wrong array shrug
  • This also improves minidump processor's os_ver string (now at parity with breakpad)
• More docs and tests backfilled (including synth-minidump framework).
• More misbehaving logging removed
• synth-minidump has been pulled out into a separate crate so the other crates can use it for testing.

Breaking changes:

• MinidumpThread and MinidumpException now lazily parse their context value (and stack for MinidumpThread).
  • This is because these values cannot be reliable parsed without access to other streams.
  • These fields have been private, in favour of accessors which require the other streams necessary to properly parse them.
  • print functionality for them (and MinidumpThreadList) now also takes those values.
  • For most users this won't be a big deal since you'll want all the dependent streams anyway.
• Some explicitly typed iterators have been replaced with impl Iterator
  • These were always supposed to be like that, this code just pre-existed the feature
  • Comes with minor efficiency win because they were internally boxed and dynamically dispatched(!) to simulate impl Iterator.
• LinuxLsbRelease has had all its parsed out values removed in favour of the new iterator API. The logic that parsed out specific fields has been moved to minidump-processor.
• LinuxLsbRelease (and some others?) now borrow the Minidump.

minidump-stack/minidump-processor/breakpad-symbols

Thread names:

• Now can retrieve thread names from the evil_json (if this means nothing to you, don't worry about it.)

Symbol cache:

• Now writes (and reads back) an INFO URL line to the symbol file
  • This allows modules[].symbol_url in the json schema to be populated even on cache hit

Json schema:

• Now properly populates the thread.last_error_value field
• Now properly populates the system_info.cpu_microcode field (using LinuxCpuInfoStream)
• system_info.os_ver now includes the contents of MinidumpSystemInfo::csd_version (as intended)

Breaking changes:

• process_minidump_with_evil has been replaced with the more general process_minidump_with_options

minidump-common

• More Windows error type definitions
• CONTEXT_HAS_XSTATE value added
• doc cleanups

Commit: 15d73f888c019517411329213c2671d59335f957

Iterating closer to parity with mozilla's minidump-stackwalk!

Changes:

minidump-stackwalk

json schema:

• "exploitability" is now null instead of "TODO"
• modules now have more debug stats:
  • "missing_symbols"
  • "loaded_symbols"
  • "corrupt_symbols"
  • "symbol_url"
• modules now have "filename" actually be the filename and not full path
• modules now have "cert_subject" indicating the module was code signed
• new top level field "modules_contains_cert_info" (indicating whether we have any known-signed modules.)

cli:

• cli has just been massively cleaned up, now has much more documentation
• --symbols-tmp is now implemented
  • Symbols that are downloaded are now downloaded to this location and atomically swapped into the cache, allowing multiple processes to share the cache safely.
• --symbols-tmp and --symbols-cache now default to using std::env::temp_dir() to improve portability/ergonomics
• new flags for writing output to specific files
  • --output-file
  • --log-file
• --raw-json flag is now implemented
  • feeds into the certificate info in the json schema
  • please don't use this unless you're mozilla
    • if you are mozilla please stop using this too
• logging should be a bit less noisy

breakpad-symbols/minidump-processor

• Symbolizers now have a stats method for getting stats on the symbols
  • See minidump-stackwalk's new "debug stats"
• Symbolizing now has tweaked error types
  • Can now distinguish between
    • "had symbols but address had no entry" and "had no symbols"
    • this is used to refine stack scanning in the unwinder
  • Can now distinguish between "failed to load" and "failed to parse"
    • Surfaced in "corrupt_symbols" statistic
• Symbolizer now truncates PUBLIC entries if there is a FUNC record in the way
  • Reduces the rate of false-positive symbolications
• Unwinding quality has been massively improved
• Unwinders now handle STACK WIN cfi
• Unwinders now more intelligently select how hard they validate output frames
  • "better" techniques like CFI and Frame Pointers get less validation
  • This means we will happily unwind into a frame we don't have symbols for with CFI and Frame Pointers, which makes subsequent Scan and Frame Pointer unwinds more reliable (since they're starting from a more accurate position).
• Unwinders now handle ARM64 pointer auth (high bits masked off)

rust-minidump/minidump-common/minidump-tools

• Should be largely unchanged. Any changes are incidental to refactors.

misc

• removed some excessive logging
• fixed some panics (an overflow and over-permissive parser)