flutter_map_tile_caching

• Added support for flutter_map v8

This update builds on v9 to fully embrace the new many-to-many relationship between tiles and stores, which allows for more flexibility when constructing the FMTCTileProvider.
This allows a new paradigm to be used: stores may now be treated as bulk downloaded regions, and all the regions/stores can be used at once - no more switching between them. This allows huge amounts of flexibility and a better UX in a complex application. Additionally, each store may now have its own BrowseStoreStrategy when browsing, which allows more flexibility: for example, stores may now contain more than one URL template/source, but control is retained.

Additionally, vector tiles are now supported in theory, as the internal caching/retrieval logic of the specialised ImageProvider has been exposed, although it is out of scope to fully implement support for it.

• Major changes to browse caching

  • Added support for using multiple stores simultaneously in the FMTCTileProvider (through the FMTCTileProvider.allStores & FMTCTileProvider.multipleStores constructors)
  • Added FMTCTileProvider.provideTile method to expose internal browse caching mechanisms for external use
  • Added BrowseStoreStrategy for increased control over caching behaviour
  • Added 'tile loading interceptor' feature (FMTCTileProvider.tileLoadingInterceptor) to track (eg. for debugging and logging) the internal tile loading mechanisms
  • Added toggle for hit/miss stat recording, to improve performance where these statistics are never read
  • Replaced FMTCTileProviderSettings.maxStoreLength with a maxLength property on each store individually
  • Replaced CacheBehavior with BrowseLoadingStrategy
  • Replaced FMTCBrowsingErrorHandler with BrowsingExceptionHandler, which may now return bytes to be displayed instead of (re)throwing exception
  • Replaced obscureQueryParams with more flexible urlTransformer (and static FMTCTileProvider.urlTransformerOmitKeyValues utility method to provide old behaviour with more customizability) - also applies to bulk downloading in StoreDownload.startForeground
  • Removed FMTCTileProviderSettings & absorbed properties directly into FMTCTileProvider
  • Performance of the internal tile image provider has been significantly improved when fetching images from the network URL

    There was a significant time loss due to attempting to handle the network request response as a stream of incoming bytes, which allowed for chunkEvents to be reported back to Flutter (allowing it to get progress updates on the state of the tile), but meant the bytes had to be collected and built manually. Removing this functionality allows the network requests to use more streamlined 'package:http' methods, which does not expose a stream of incoming bytes, meaning that bytes no longer have to be treated manually. This can save hundreds of milliseconds on tile loading - a significant time save of potentially up to ~50% in some cases!

• Major changes to bulk downloading

  • Added support for retrying failed tiles (that failed because the request could not be made) once at the end of the download
  • Changed result of StoreDownload.startForeground into two seperate streams returned as a record, one for TileEvents, one for DownloadProgresss
  • Refactored TileEvents into multiple classes and mixins in a sealed inheritance tree to reduce nullability and uncertainty & promote modern Dart features
  • Changed DownloadProgress' metrics to reflect other changes and renamed methods to improve clarity and consistency with Dart recommended style
  • Renamed StoreDownload.check to .countTiles

• Improvements for bulk downloadable BaseRegions

  • Added MultiRegion, which contains multiple other BaseRegions
  • Improved speed (by massive amounts) and accuracy & reduced memory consumption of CircleRegion's tile generation & counting algorithm
  • Fixed multiple bugs with respect to start and end tiles in downloads
  • Deprecated BaseRegion.(maybe)When - this is easy to perform using a standard pattern-matched switch

• Exporting stores is now more stable, and has improved documentation

  The method now works in a dedicated temporary environment and attempts to perform two different strategies to move/copy-and-delete the result to the specified directory at the end before failing. Improved documentation covers the potential pitfalls of permissions and now recommends exporting to an app directory, then using the system share functionality on some devices. It now also returns the number of exported tiles.

• Removed deprecated remnants from v9.*

• Other generic improvements (performance, stability, and documentation)

• Brand new example app to demonstrate the new levels of flexibility and customizability

• Fixed bug in removeTilesOlderThan where actually tiles newer than the specified expiry were being removed (#172)

• Fixed bug where any operation that attempted to delete tiles fatally crashed on some iOS devices
  This appears to be an ObjectBox issue where streaming the results of a database query caused the crash. Instead, FMTC now uses a custom chunking system to avoid streaming and also avoid loading potentially many tiles into memory.

• Fixed compilation on web platforms: FMTC now internally overrides the FMTCObjectBoxBackend and becomes a no-op on non-FFI platforms
• Minor documentation improvements

• Fixed bug where errors within the import functionality would not always be catchable by the original invoker
• Minor other improvements

• Upgraded to flutter_map v7 to support Flutter 3.22 (also upgraded other dependencies)
• Deprecated BaseRegion.toDrawable, and all implementations

• Fixed bug on initialisation, where using multiple/background FlutterEngines would attempt to re-open a single ObjectBox Store (aka. root) multiple times

This update has essentially rewritten FMTC from the ground up, over hundreds of hours. It focuses on:

• improving future maintainability by improving modularity and seperation of concerns
• improving stability & performance across the board
• supporting a many-to-many relationship between tiles and stores to reduce duplication

I would hugely appricate any donations - please see the documentation site, GitHub repo, or pub.dev package.

I would also like to thank all those who have been waiting and contributing their feedback throughout the process: it means a lot to me that FMTC is such a crucial component to your application.

And without further ado, let's get the biggest changes out of the way first:

• Added support for modular storage/root backends through FMTCBackend

  • Removed Isar support
    Isar unfortunately caused too many stability issues, and is not as actively maintained as I would like (I can sympathise :D).
  • Added ObjectBox as the default backend (FMTCObjectBoxBackend)
    ObjectBox uses the same underlying database technology as Isar (MBDX), but is more maintained, and I'm hoping, more stable. Note that ObjectBox declares it only supports 64-bit systems, whereas Isar was just 'mostly unstable' on 32-bit systems until recently (where is also became 64-bit only): it's time for the future!
  • It is expected that backends support a many-to-many relationship between tiles and stores
    This has reduced duplication between stores and tiles massively, and now allows for smaller, fine-grained region control. The default backend supports this with as minimal hit to performance as possible, although of course, database operations are now considerably more complex than in previous versions, and so therefore will take slightly longer. In practise, there is no noticeable performance difference.
  • It is expected that backends cache statistics instead of calculating them at get time
    This has decreased the time spent fetching basic statistics, and allowed for increased efficiency when getting multiple stats at once. Of course, there is some impact on performance at write time: it must all be accurately tracked, else it will be inaccacurate/out-of-sync.

• Restructured top-level access APIs

  • Deprecated StoreDirectory & RootDirectory in favour of FMTCStore and FMTCRoot
    The term 'directory' has been misleading for a couple of years now, as it hasn't been actual filesystem directories storing information since the introduction of v7.
  • Removed the FlutterMapTileCaching/FMTC access object, in favour of FMTCStore and FMTCRoot direct constructors
    Much of the configuration and state management performed by this top-level object and it's close relatives were transferred to the backend, and as such, there is no longer a requirement for these objects.
  • Removed support for synchronous operations (and renamed asynchronous operations to reflect this)
    These were incompatible with the new Isolated FMTCObjectBoxBackend, and to keep scope reasonable, I decided to remove them, in favour of backends implementing their own Isolateion as well.

• Reimplemented bulk downloading

  • Added CustomPolygonRegion, a BaseRegion that is formed of any* outline
  • Added pause and resume functionality
  • Added rate limiting functionality
  • Added support for multiple simultaneous downloads
  • Improved developer experience by refactoring DownloadableRegion and startForeground
  • Improved download speed significantly
  • Fixed instability and bugs when cancelling buffering downloads
  • Fixed generation of LineRegion tiles by reducing number of redundant duplicate tiles
  • Fixed usage of obscuredQueryParams
  • Removed support for bulk download buffering by size capacity
  • Removed support for custom HttpClients

• Deprecated plugins

  • Transfered support for import/export operations to core (RootExternal)
  • Deprecated support for background bulk downloading

• Migrated to Flutter 3.19 and Dart 3.3

• Migrated to flutter_map v6

With those out of the way, we can take a look at the smaller changes:

• Improved recovery system to monitor which tiles can be skipped on re-downloading (via DownloadableRegion.start)
• Improved error handling (especially in backends)
• Added StoreManagement.pruneTilesOlderThan method
• Added shortcut for getting multiple stats: StoreStats.all
• Added secondary check to FMTCImageProvider to ensure responses are valid images
• Replaced public facing RegionType/type with Dart 3 exhaustive switch statements through BaseRegion/DownloadableRegion.when & RecoverableRegion.toRegion
• Removed HTTP/2 support
• Fixed a whole bunch of bugs

In addition, there's been more action in the surrounding enviroment:

• Created a miniature testing tile server
• Created automated tests for tile generation
• Improved & simplified example application
  • Removed update mechanism
  • Added tile-by-tile/live download following