What's Changed

• fix: do not stop crawling at currentDepth 0 by @SuperchupuDev in #149

Full Changelog: v6.4.5...v6.4.6

What's Changed

1. Ensure callback is called only once by @thecodrr in #144

Full Changelog: v6.4.4...v6.4.5

What's Changed

• Update documentation for withCallback() by @dave-swift in #133
• Fix hang when crawling root with direct symlinks by @pralkarz in #137
• Add prettier and run format by @43081j in #140

New Contributors

• @dave-swift made their first contribution in #133
• @pralkarz made their first contribution in #137

Full Changelog: v6.4.3...v6.4.4

Fixes

• Do not incorrectly use real paths on exclude predicate by @SuperchupuDev in #129

Fixes

1. Fix regression where fdir would never resolve when maxDepth was set to < 0 (#127)

Fixes

Recursive symlinks handling (#125)

Previously, fdir left it up to the OS to handle recursive symlinks. Unfortunately, this resulted in an infinite loop that'd either cause a crash or take too long to resolve (each OS has a different limit on resolving recursive symlinks). This has been fixed now in #126.

Recursive symlinks with resolvePaths: true

When resolvePaths is set to true, fdir does not crawl a directory it has already visited. To figure out whether we have visited a directory or not, fdir maintains a list of all the directories it has visited. This might result in slightly higher memory usage than before.

For a directory that looks like this:

/dir/
  file
  symlink -> /dir/

fdir will return:

[ "/dir/file" ]

In short, you won't see duplicated paths in the output which is the expected behavior when working with file systems since paths are unique.

Recursive symlinks with resolvePaths: false

When you set resolvePaths to false, the behavior differs because now all symlinks become part of the path.

For a directory that looks like this:

/dir/
  file
  symlink -> /dir/

fdir will return:

[ "/dir/file", "/dir/symlink/file" ]

To prevent recursion, all recursive symlinks are only resolved a single level deep making sure you never see something like /dir/symlink/symlink/symlink/file. This allows for glob patterns to work with recursive symlinks without creating a performance issue.

Relative recursive symlinks

Relative recursive symlinks work exactly as above except the returned paths are relative to the root. Everything else is exactly the same.

Thanks to @SuperchupuDev for bringing this to my attention.

Features

Exclude symlinks

You can now specifically exclude symlinks from the crawling process. Here's how you can do that:

new fdir({ excludeSymlinks: true }).crawl().sync();

Thanks to @SuperchupuDev in #115

Custom glob functions

Previously, fdir only supported picomatch for globbing disallowing any customization in this area. While that worked really well for most people, it wasn't super flexible. Starting from this version, fdir supports changing the default glob function:

// using a custom function
const customGlob = (patterns: string | string[]) => {
  return (test: string): boolean => test.endsWith('.js');
};
const crawler = new fdir().withGlobFunction(customGlob).globWithOptions("**/*.js");

withGlobFunction accepts a glob factory function which you can use to perform intensive work only once. For example, picomatch provides a glob factory that optimizes and preprocesses your glob patterns increasing match significantly.

Thanks to @43081j in #98

Fixes

• Support symlinks with relative paths enabled by @SuperchupuDev in #114
• Do not return an empty string for the root by @SuperchupuDev in #123

Other

• Add Video Hub App to "Used by" section of README by @whyboris in #121
• docs: update symlinks support by @SuperchupuDev in #119
• chore(deps): bump vite from 5.0.2 to 5.4.8 by @dependabot in #117
• chore(deps): bump braces from 3.0.2 to 3.0.3 by @dependabot in #118
• docs: list more packages using fdir by @benmccann in #112

New Contributors

• @benmccann made their first contribution in #112
• @43081j made their first contribution in #98
• @whyboris made their first contribution in #121

Full Changelog: v6.3.0...v6.4.0

Added

1. withSymlinks now supports the resolvePaths argument again. It was accidentally removed when migrating to TypeScript. Thanks to @SuperchupuDev (https://github.com/thecodrr/fdir/pull/104)

Fixes

1. Symlink paths no longer end with a leading slash when withSymlinks is enabled.

Other

• chore: exclude irrelevant files from published bundle by @SuperchupuDev in https://github.com/thecodrr/fdir/pull/105
• fix(docs): remove Immich CLI by @etnoy in https://github.com/thecodrr/fdir/pull/108
• docs: document missing options by @SuperchupuDev in https://github.com/thecodrr/fdir/pull/110
• chore: fix import in benchmark scripts by @SuperchupuDev in https://github.com/thecodrr/fdir/pull/111

New Contributors

• @etnoy made their first contribution in https://github.com/thecodrr/fdir/pull/108

Full Changelog: https://github.com/thecodrr/fdir/compare/v6.2.0...v6.3.0

1. Updated picomatch peerDependency to v4
2. Relative paths now work correctly when onlyDirs is enabled.

1. Fixed issue where consecutive backslashes would get collapsed (#93). (This also fixes crawling of WSL directories on Windows which start with \\wsl.localhost\Ubuntu)
2. Upgraded picomatch peerDependency requirement to 3.x from 2.x.

withPathSeparator(separator: "/" | "\") 🆕

withPathSeparator option allows you enforcing a specific path separator regardless of what platform the code is running on. This is especially useful if your test snapshots contain paths as those snapshots will fail on Windows because Node.js uses \\ path separator on Windows by default.

For example:

const files = await new fdir().withFullPaths().withPathSeparator("/").crawl("node_modules").withPromise();

More reliable globbing

This release contains a lot of fixes for glob making it much more reliable. (thanks to @bglw for reporting a reproducible test case #92)

For example, doing this would return an empty array:

const crawler = new fdir().withBasePath().glob("**/*.txt");
const files = await crawler.crawl(".").withPromise();

This was because picomatch and other globbing libraries don't deal too well with paths that start with . or ./. Starting from this version, fdir tries very hard to not include ./ or . at the beginning of the paths.

The end result is that fdir should now work similar to fast-glob and other globbing libraries.

Node v20 support

Starting from this version, fdir now officially supports Node v20 with all its tests running on it.

Other fixes

1. fdir now automatically fallbacks to crawling the current working directory if you pass an empty string as crawl root.
2. Using withRelativePaths with ./ as root path will now have no effect.

1. Fixed invalid reference of TypeScript types (#88)

Full Changelog: https://github.com/thecodrr/fdir/compare/v6.0.0...v6.0.1

Note: While fdir tries to strictly follow semver, this release doesn't actually break anything. It does deprecate a few things but overall, migrating from v5.3.0 to v6.0.0 should be seamless.

:star: Features & new stuff

Typescript rewrite

fdir has now been fully rewritten in TypeScript. This brings better clarity into what's happening and how its happening. The code in the project has also be reorganized & broken down so it's much easier to understand now.

Another benefit of using TypeScript is types. Since everything is now autogenerated by tsc, it is always in sync with the actual API. With the help of generics, the output type is now automatically inferred based on the method used. That means no more as string[] etc.

globWithOptions

Globbing support has always been barebones in fdir. This release brings in full support for passing picomatch options when globbing. Use it like this:

new fdir()
  .globWithOptions(["**/*.js"], { dot: true })
  .crawl("path/to/dir")
  .sync();

⚔️ Deprecations

A lot of unintuitive & badly designed API choices have been deprecated. They will continue to work as intended but they will eventually be removed in the upcoming major versions. These include:

crawlWithOptions

This function was added as a convenience for people who don't like the Builder API. This has now been replaced with the fdir constructor.

Instead of this:

new fdir()
  .crawlWithOptions("./", { includeDirs: true })
  .sync();

You should now do this:

new fdir({ includeDirs: true })
  .crawl("./")
  .sync();

P.S. I forgot to deprecate includeDirs and replace it with includeDirectories. Oh well, I'll do that in the next version.

directories instead of dirs

When using the onlyCounts() API, the resulting object will now contain directories instead of dirs. A minor change but I really don't like abbreviations unless absolutely necessary.

directory instead of dir

When using the group() API, the resulting object will now contain directory instead of dir.

And that's it. No other changes to the API. I am pretty sure that the TypeScript rewrite fixed some hidden & hard-to-debug bugs so you should find v6.0.0 much more stable.

:star: Features & new stuff

1. It is now possible to disable path resolution of files inside symlinked directories (#84). This release adds new optional parameter to the withSymlinks. Check the docs on how to use it:

This change is 100% backwards compatible and should not break any of your scripts that rely on symlink resolution.

:bug: Fixes & improvements

1. Add picomatch in peerDependencies by @kyleknighted in https://github.com/thecodrr/fdir/pull/76
2. Make picomatch an optional peerDependency by @IanVS in https://github.com/thecodrr/fdir/pull/80

New Contributors

• @kyleknighted made their first contribution in https://github.com/thecodrr/fdir/pull/76
• @IanVS made their first contribution in https://github.com/thecodrr/fdir/pull/80

Note: fdir follows semantic versoning hence this release is backward compatible with only version 5.x.

Features

1. withRelativePaths has been added to return paths relative to the root directory (closes #51)

Fixes

1. respect the original symboliclink if resolveSymlinks is false (#63)
2. only handle fs related errors (#56) — before we were swallowing all thrown errors.
3. Fixed a critical issue with async crawling that caused the crawler to return early.

Note: fdir follows semantic versoning hence this release is backward compatible with only version 5.x.

Features

1. Added symlink resolution support via .withSymlinks method (#53 & #58):

const files = new fdir().withSymlinks().crawl("/path/to/dir").sync();

1. Performance & memory usage has also been greatly improved due to the many internal refactorings.

Infrastructural improvements:

Aside from the symlinks support, this release has a lot of under-the-hood refactoring. This has improved the maintainability of the project from 83% to 97%.

Major improvements include:

1. Added ARCHITECTURE.md
2. Added CONTRIBUTING.md
3. Consistent internal naming for functions, variables & arguments.

And other changes you can see in PR: #59

Thanks to everyone who has used, supported, tested, & worked on this project. This is also a celebratory release since fdir is now being used by projects such as snowpackjs/snowpack & mdn/yari. So YAY!

Note: fdir follows semantic versoning hence this release is not backward compatible with any previous release.

Breaking Changes

1. Filters applied using new fdir().filter() are now joined via AND instead of OR. (#35)

Features

1. There is now a brand new onlyDirs builder function that allows you to grab (and filter) only directories ignoring all files. (#43)

const crawler = new fdir().onlyDirs();

1. As a result, filters now work with directories as well and include isDirectory as second parameter to .filter() function:

const crawler = new fdir()
  .filter((path, isDirectory) => path.startsWith("."))
  .filter((path, isDirectory) => path.endsWith(".js"));

1. Full path of the directory is now sent to the excludeFn of .exclude(excludeFn): (#46)

const crawler = new fdir().exclude((dirName, dirPath) =>
  dirName.startsWith(".")
);

Fixes

1. Use \0 instead of nothing to separate cached glob patterns (#42)
2. Resolved all concurrency issues that made the configuration of multiple concurrent instances of fdir to collide (#47)
3. Node was exiting whenever an error (like ENOENT) occured. (#48)

Thank you @papb for your help and PR #45!

What's Improved:

1. Performance improved by 8%.

Breaking Changes:

1. Due to issues with export default in the Type Definitions, I had to migrate to using named exports. (#37)

So this is now invalid:

import fdir from "fdir";

Instead do this:

ES6:

import { fdir } from "fdir";

CommonJS:

const { fdir } = require("fdir");

What's Fixed

1. Fixed issue where excludeFn was affected by Options (#31) Now fdir only will send the root directory name as the param to excludeFn.

What's Fixed:

1. Enabled dotfile matching in glob API by default (Fixes: #30)

What's New:

1. Multiple filters are now supported. So basically, this will now work as expected:

new fdir().filter(p => p.endsWith(".js")).filter(p => p.startsWith(".nim"))

1. I have finally added glob pattern caching so globbing performance should increase by about 5-8% if you are using the same glob pattern again and again.

Install using:

$ npm i fdir

Breaking Changes:

1. Due to performance reasons, I had to remove passing an Options object through the constructor. In place of that, now there is a new crawlWithOptions that takes an Options object in its second parameter:

new fdir().crawlWithOptions("path/to/dir", { includeBasePath: true }).sync();

Fixes:

1. fdir will now properly handle the case where picomatch is not installed.

Improvements:

1. Some minor memory improvements were made which should improve the performance slightly.

What's Improved

1. glob performance has been improved by about 40%.

What's New:

1. It is now possible to pass an Options object directly into the constructor, avoiding method chaining completely.

What's Fixed:

1. Added missing typescript definitions for withError & crawl.
2. Fixed another small issue where Synchronous API was causing a crash.

Install using:

npm i fdir

NOTE: This release follows semantic versioning and therefore, it is not backwards compatible with 2.x.

What's New:

1. Added a new glob crawler option for glob matching using picomatch. To use it you must install picomatch yourself.

yarn add picomatch

This was done because not everyone needs globbing.

NOTE: This release is backwards compatible with 3.x

This is a minor patch release.

What's fixed:

1. change TypeScript typings to use string instead of String (thanks to @OmgImAlexis) (#16)

What's New?

• Added ignoreErrors option to ignore errors (default: false)
• Added proper error handling (#8)

What's Fixed?

• Made Options optional in type defs. (#9)
• Fixed issue where input path with trailing slash did not get normalized (#10)

What's improved?

This release mainly included bug fixes and code refactoring. Nothing much has changed in terms of performance.

What's Changed?

• Renamed includeBasePath to excludeBasePath. You'll need to reverse the functionality.

What's Fixed?

• There was an issue where isExcludedDir function was sending only the directory name (not its path) in the argument. That's fixed now.

What's Improved?

As always, the performance. Also cleaned up the code quite a lot.

Nothing new. Just optimisations to make it even faster. New benchmarks can be found in README.md

What's New?

Nothing

What's Improved?

Migrated to util.promisify for fdir.async. This surprisingly improves performance on battery (when CPU is downclocked to save battery) so now both benchmarks are almost equal.