flutter_map_tile_caching

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Fixed bug where import operation fatally crashed on some iOS devices
  This appears to be an ObjectBox issue where streaming the results of a database query caused the crash. Instead, FMTC now uses a custom chunking system to avoid streaming and also avoid loading potentially many tiles into memory.
• Improved performance of import operation

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Added support for flutter_map v8

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

This update builds on v9 to fully embrace the new many-to-many relationship between tiles and stores, which allows for more flexibility when constructing the FMTCTileProvider.
This allows a new paradigm to be used: stores may now be treated as bulk downloaded regions, and all the regions/stores can be used at once - no more switching between them. This allows huge amounts of flexibility and a better UX in a complex application. Additionally, each store may now have its own BrowseStoreStrategy when browsing, which allows more flexibility: for example, stores may now contain more than one URL template/source, but control is retained.

Additionally, vector tiles are now supported in theory, as the internal caching/retrieval logic of the specialised ImageProvider has been exposed, although it is out of scope to fully implement support for it.

• Major changes to browse caching

  • Added support for using multiple stores simultaneously in the FMTCTileProvider (through the FMTCTileProvider.allStores & FMTCTileProvider.multipleStores constructors)
  • Added FMTCTileProvider.provideTile method to expose internal browse caching mechanisms for external use
  • Added BrowseStoreStrategy for increased control over caching behaviour
  • Added 'tile loading interceptor' feature (FMTCTileProvider.tileLoadingInterceptor) to track (eg. for debugging and logging) the internal tile loading mechanisms
  • Added toggle for hit/miss stat recording, to improve performance where these statistics are never read
  • Replaced FMTCTileProviderSettings.maxStoreLength with a maxLength property on each store individually
  • Replaced CacheBehavior with BrowseLoadingStrategy
  • Replaced FMTCBrowsingErrorHandler with BrowsingExceptionHandler, which may now return bytes to be displayed instead of (re)throwing exception
  • Replaced obscureQueryParams with more flexible urlTransformer (and static FMTCTileProvider.urlTransformerOmitKeyValues utility method to provide old behaviour with more customizability) - also applies to bulk downloading in StoreDownload.startForeground
  • Removed FMTCTileProviderSettings & absorbed properties directly into FMTCTileProvider
  • Performance of the internal tile image provider has been significantly improved when fetching images from the network URL

    There was a significant time loss due to attempting to handle the network request response as a stream of incoming bytes, which allowed for chunkEvents to be reported back to Flutter (allowing it to get progress updates on the state of the tile), but meant the bytes had to be collected and built manually. Removing this functionality allows the network requests to use more streamlined 'package:http' methods, which does not expose a stream of incoming bytes, meaning that bytes no longer have to be treated manually. This can save hundreds of milliseconds on tile loading - a significant time save of potentially up to ~50% in some cases!

• Major changes to bulk downloading

  • Added support for retrying failed tiles (that failed because the request could not be made) once at the end of the download
  • Changed result of StoreDownload.startForeground into two seperate streams returned as a record, one for TileEvents, one for DownloadProgresss
  • Refactored TileEvents into multiple classes and mixins in a sealed inheritance tree to reduce nullability and uncertainty & promote modern Dart features
  • Changed DownloadProgress' metrics to reflect other changes and renamed methods to improve clarity and consistency with Dart recommended style
  • Renamed StoreDownload.check to .countTiles

• Improvements for bulk downloadable BaseRegions

  • Added MultiRegion, which contains multiple other BaseRegions
  • Improved speed (by massive amounts) and accuracy & reduced memory consumption of CircleRegion's tile generation & counting algorithm
  • Fixed multiple bugs with respect to start and end tiles in downloads
  • Deprecated BaseRegion.(maybe)When - this is easy to perform using a standard pattern-matched switch

• Exporting stores is now more stable, and has improved documentation

  The method now works in a dedicated temporary environment and attempts to perform two different strategies to move/copy-and-delete the result to the specified directory at the end before failing. Improved documentation covers the potential pitfalls of permissions and now recommends exporting to an app directory, then using the system share functionality on some devices. It now also returns the number of exported tiles.

• Removed deprecated remnants from v9.*

• Other generic improvements (performance, stability, and documentation)

• Brand new example app to demonstrate the new levels of flexibility and customizability

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Fixed bug in removeTilesOlderThan where actually tiles newer than the specified expiry were being removed (#172)

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Fixed bug where any operation that attempted to delete tiles fatally crashed on some iOS devices
  This appears to be an ObjectBox issue where streaming the results of a database query caused the crash. Instead, FMTC now uses a custom chunking system to avoid streaming and also avoid loading potentially many tiles into memory.

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Fixed compilation on web platforms: FMTC now internally overrides the FMTCObjectBoxBackend and becomes a no-op on non-FFI platforms
• Minor documentation improvements

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Fixed bug where errors within the import functionality would not always be catchable by the original invoker
• Minor other improvements

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Upgraded to flutter_map v7 to support Flutter 3.22 (also upgraded other dependencies)
• Deprecated BaseRegion.toDrawable, and all implementations

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

• Fixed bug on initialisation, where using multiple/background FlutterEngines would attempt to re-open a single ObjectBox Store (aka. root) multiple times

See the full CHANGELOG for more information: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/main/CHANGELOG.md.

This update has essentially rewritten FMTC from the ground up, over hundreds of hours. It focuses on:

• improving future maintainability by improving modularity and seperation of concerns
• improving stability & performance across the board
• supporting a many-to-many relationship between tiles and stores to reduce duplication

I would hugely appricate any donations - please see the documentation site, GitHub repo, or pub.dev package.

I would also like to thank all those who have been waiting and contributing their feedback throughout the process: it means a lot to me that FMTC is such a crucial component to your application.

And without further ado, let's get the biggest changes out of the way first:

• Added support for modular storage/root backends through FMTCBackend

  • Removed Isar support
    Isar unfortunately caused too many stability issues, and is not as actively maintained as I would like (I can sympathise :D).
  • Added ObjectBox as the default backend (FMTCObjectBoxBackend)
    ObjectBox uses the same underlying database technology as Isar (MBDX), but is more maintained, and I'm hoping, more stable. Note that ObjectBox declares it only supports 64-bit systems, whereas Isar was just 'mostly unstable' on 32-bit systems until recently (where is also became 64-bit only): it's time for the future!
  • It is expected that backends support a many-to-many relationship between tiles and stores
    This has reduced duplication between stores and tiles massively, and now allows for smaller, fine-grained region control. The default backend supports this with as minimal hit to performance as possible, although of course, database operations are now considerably more complex than in previous versions, and so therefore will take slightly longer. In practise, there is no noticeable performance difference.
  • It is expected that backends cache statistics instead of calculating them at get time
    This has decreased the time spent fetching basic statistics, and allowed for increased efficiency when getting multiple stats at once. Of course, there is some impact on performance at write time: it must all be accurately tracked, else it will be inaccacurate/out-of-sync.

• Restructured top-level access APIs

  • Deprecated StoreDirectory & RootDirectory in favour of FMTCStore and FMTCRoot
    The term 'directory' has been misleading for a couple of years now, as it hasn't been actual filesystem directories storing information since the introduction of v7.
  • Removed the FlutterMapTileCaching/FMTC access object, in favour of FMTCStore and FMTCRoot direct constructors
    Much of the configuration and state management performed by this top-level object and it's close relatives were transferred to the backend, and as such, there is no longer a requirement for these objects.
  • Removed support for synchronous operations (and renamed asynchronous operations to reflect this)
    These were incompatible with the new Isolated FMTCObjectBoxBackend, and to keep scope reasonable, I decided to remove them, in favour of backends implementing their own Isolateion as well.

• Reimplemented bulk downloading

  • Added CustomPolygonRegion, a BaseRegion that is formed of any* outline
  • Added pause and resume functionality
  • Added rate limiting functionality
  • Added support for multiple simultaneous downloads
  • Improved developer experience by refactoring DownloadableRegion and startForeground
  • Improved download speed significantly
  • Fixed instability and bugs when cancelling buffering downloads
  • Fixed generation of LineRegion tiles by reducing number of redundant duplicate tiles
  • Fixed usage of obscuredQueryParams
  • Removed support for bulk download buffering by size capacity
  • Removed support for custom HttpClients

• Deprecated plugins

  • Transfered support for import/export operations to core (RootExternal)
  • Deprecated support for background bulk downloading

• Migrated to Flutter 3.19 and Dart 3.3

• Migrated to flutter_map v6

With those out of the way, we can take a look at the smaller changes:

• Improved recovery system to monitor which tiles can be skipped on re-downloading (via DownloadableRegion.start)
• Improved error handling (especially in backends)
• Added StoreManagement.pruneTilesOlderThan method
• Added shortcut for getting multiple stats: StoreStats.all
• Added secondary check to FMTCImageProvider to ensure responses are valid images
• Replaced public facing RegionType/type with Dart 3 exhaustive switch statements through BaseRegion/DownloadableRegion.when & RecoverableRegion.toRegion
• Removed HTTP/2 support
• Fixed a whole bunch of bugs

In addition, there's been more action in the surrounding enviroment:

• Created a miniature testing tile server
• Created automated tests for tile generation
• Improved & simplified example application
  • Removed update mechanism
  • Added tile-by-tile/live download following

• Fixed bugs when generating tiles for LineRegion

• Bulk downloading has been rewritten to use a new implementation that generates tile coordinates at the same time as downloading tiles
  • checking the number of tiles in a region now uses a significantly faster and more efficient implementation
  • Starting a download no longer causes significant memory bloat, which could crash the app on large regions
  • Starting a download is now much quicker and closer to constant time, as tile coordinates don't need to be pre-generated
  • Cancelling a download no longer causes many QueueCancelledExceptions to be thrown, which could crash the app
  • Removed reliance on 'queue' dependency in order to squeeze as much speed as possible out of the new implementation
• Other improvements
  • initialise now automatically renames databases if their filename ID doesn't match their name hash
  • initialise can now throw a more useful FMTCInitialisationException with improved clarity
  • Methods that require a valid store descriptor object to be present now throw FMTCDamagedStoreException if it is not present
• Other bug fixes
  • FMTCImageProvider's oldest tile deleter & bulk downloading now respects custom root directories
  • FMTCImageProvider now evicts failed images from Flutter's ImageCache to prevent errors
• Added support for custom HttpClients/BaseClients
• Added support for Isar v3.1 (bug fixes & stability improvements)

Version 7 was made unstable due to a non-semantic versioning compliant update of a dependency.
This means the pub version resolver can never resolve FMTC v7 without introducing compilation errors.

[7.2.0] - 2023/03/03

• Stability improvements
  • Starting multiple downloads no longer causes LateInitializationErrors
  • Migrator storage and memory usage no longer spikes as significantly as previously, thanks to transaction batching
  • Opening and processing of stores on initialisation is more robust and less error-prone to filename variations
  • Root statistic watching now works on all platforms
• Multiple minor bug fixes and documentation improvements
• Added maxStoreLength config to example app

[7.1.2] - 2023/02/18

• Minor bug fixes

[7.1.1] - 2023/02/16

• Major bug fixes
• Added debug mode

[7.1.0] - 2023/02/14

• Added URL query params obscurer feature
• Added headers and httpClient parameters to getTileProvider
• Minor documentation improvements
• Minor bug fixes

[7.0.2] - 2023/02/12

• Minor changes to example application

[7.0.1] - 2023/02/11

• Minor bug fixes
• Minor improvements

[7.0.0] - 2023/02/04

• Migrated to Isar database
• Major performance improvements, thanks to Isar
• Added buffering to bulk tile downloading
• Added method to catch tile retrieval errors
• Removed v4 -> v5 migrator & added v6 -> v7 migrator
• Removed some synchronous methods from structure management
• Removed 'fmtc_advanced' import file

Plus the usual:

• Minor performance improvements
• Bug fixes
• Dependency updates
• Documentation improvements

• Performance improvements
• Changed license to GPL v3
• Updated dependencies

• Fixed bugs within import functionality
• Added support for notifications on Android 13+
• Improved performance
• Updated dependencies

• Improved stability
• Updated dependencies

• Added support for flutter_map v3
• Fixed bugs
• Improved example application

• Fixed bugs
• Updated dependencies
• Improved example application

• Added import/export functionality
• Improved example application
• Improved documentation
• Fixed bugs

• Renamed and refactored internal and public APIs, ready for release
• Added successfulSize statistic to DownloadProgress
• Improved example application
• Improved documentation
• Fixed bugs

• Polish and preparations for full v5 release
• Moved migrator into rootDirectory to fix bugs
• Converted DownloadManagement to a singleton object
• Finished example application
• Improved documentation
• Fixed bugs

• Improved download time estimates and added tiles per second statistic
• Added custom filesystem string sanitiser setting to FMTCSettings
• Improved statistic watchers
• Improved example application
• Improved documentation
• Fixed bugs

• Added cache hits and misses statistics
• Added v4 to v5 file structure migration methods
• Reworked invalidateCachedStatistics
• Improved documentation
• Fixed bugs

• Improved example application
• Updated Gradle for example application
• Improved documentation
• Improved background downloading implementation
• Added support for 'flutter_map' v2.0.0
• Added Windows support to example application
• Fixed multiple bugs