rubix/ml

• Integers are now considered a categorical data type
• K Nearest Neighbors and KNN Regressor inference is now parallelized
• Isolation Forest training and inference is now parallelized
• Added disk-based streaming neural network snapshotting
• Added validation interval parameter to MLPs and GBM Learners
• Cross Entropy loss function now split into Binary and Multiclass
• Logistic Regression, Softmax, and Adaline now use hold out set
• Adaboost now uses validation set with early stopping window
• Renamed TF-IDF dampening parameter to sublinear
• Exportable Extractors now append by default with option to overwrite
• RBX Serializer tracks major library version number, not minor
• Added Class/Cluster Purity clustering metrics
• V-measure, Completeness, and Homogeneity now use entropy-based formula
• Fixed KDTree edge pruning + optimize traversal
• Ball and Vantage Trees now require Subadditive kernels
• K-d Trees now require Monotonic distance kernels
• Optimize Dataset sort(), sorting is now unstable
• Added per-class smoothing to Gaussian Naive Bayes
• Added per-cluster smoothing to Gaussian Mixture
• You can now exclude certain categories from one-hot encoding
• Fixed SVC save/load using class map sidecar
• Parallel Backends now default to max physical cores not logical
• Added workers() method to the Backend interface
• No longer save/load Backend state, transient per environment
• Added Float Type Converter numeric string and ints to float
• Boolean Converter now converts truthy and falsy
• Interval Discretizer now encodes values as integers
• Polynomial Expander now limited to 10'th degree
• Updated to PSR-3 Log version 3
• Update Amp Backend to Amp version 2.0
• Removed Word Stemmer tokenizer
• Removed window early stopping from TSNE
• Removed output layer L2 Penalty parameter from MLP Learners
• RBX serializer now emits warning on class revision mismatch
• Class revision hash now compensates for circular references
• Filesystem Persister now does atomic writes
• Added cleanup() method to remove neural network residual state
• Murmur3 new default Token Hashing Vectorizer hash function
• Dataset fold() now returns excess samples in last fold
• Increase default Decision Tree max leaf node size from 3 to 5
• Canonicalized He initializer
• Xavier 2 now extends He as a deprecated alias
• Remove Softmax activation function
• Fix Multiclass layer gradient for non-Cross Entropy losses
• Optimizers now take a Scheduler rather than a raw learning rate
• MLP Learners now have gradient accumulation and clipping
• MLP Learners can now freeze first k layers for fine-tuning
• PlusPlus and KMC2 Seeders always return unique seeds
• KMeans and Fuzzy C Means now restrict non-Euclidean kernels
• NDJSON exporter now preserves zero decimal numbers as floats
• Add Dataset chunked() factory for online training
• Added L1 penalty to Dense layers
• Adaline, Logistic Regression, Softmax Classifier now elastic net
• Added Iterative interface with progress() method
• Iterative Learners renamed steps() method to progress()
• Changed default gradient-based minChange from 1e-4 to 1e-5
• Prior is now default Strategy of Missing Data Imputer
• DBSCAN is now a Learner, Probabilistic and Persistable
• Grid Search now has a fromNamedParams() factory method
• Grid Search now generates a results() table

• Integers are now considered a categorical data type
• K Nearest Neighbors and KNN Regressor inference is now parallelized
• Isolation Forest training and inference is now parallelized
• Added disk-based streaming neural network snapshotting
• Added validation interval parameter to MLPs and GBM Learners
• Cross Entropy loss function now split into Binary and Multiclass
• Logistic Regression, Softmax, and Adaline now use hold out set
• Adaboost now uses validation set with early stopping window
• Renamed TF-IDF dampening parameter to sublinear
• Exportable Extractors now append by default with option to overwrite
• RBX Serializer tracks major library version number, not minor
• Added Class/Cluster Purity clustering metrics
• V-measure, Completeness, and Homogeneity now use entropy-based formula
• Fixed KDTree edge pruning + optimize traversal
• Ball and Vantage Trees now require Subadditive kernels
• K-d Trees now require Monotonic distance kernels
• Optimize Dataset sort(), sorting is now unstable
• Added per-class smoothing to Gaussian Naive Bayes
• Added per-cluster smoothing to Gaussian Mixture
• You can now exclude certain categories from one-hot encoding
• Fixed SVC save/load using class map sidecar
• Parallel Backends now default to max physical cores not logical
• Added workers() method to the Backend interface
• No longer save/load Backend state, transient per environment
• Added Float Type Converter numeric string and ints to float
• Boolean Converter now converts truthy and falsy
• Interval Discretizer now encodes values as integers
• Polynomial Expander now limited to 10'th degree
• Updated to PSR-3 Log version 3
• Update Amp Backend to Amp version 2.0
• Removed Word Stemmer tokenizer
• Removed window early stopping from TSNE
• Removed output layer L2 Penalty parameter from MLP Learners
• RBX serializer now emits warning on class revision mismatch
• Class revision hash now compensates for circular references
• Filesystem Persister now does atomic writes
• Added cleanup() method to remove neural network residual state
• Murmur3 new default Token Hashing Vectorizer hash function
• Dataset fold() now returns excess samples in last fold
• Increase default Decision Tree max leaf node size from 3 to 5
• Canonicalized He initializer
• Xavier 2 now extends He as a deprecated alias
• Remove Softmax activation function
• Fix Multiclass layer gradient for non-Cross Entropy losses
• Optimizers now take a Scheduler rather than a raw learning rate
• MLP Learners now have gradient accumulation and clipping
• MLP Learners can now freeze first k layers for fine-tuning
• PlusPlus and KMC2 Seeders always return unique seeds
• KMeans and Fuzzy C Means now restrict non-Euclidean kernels
• NDJSON exporter now preserves zero decimal numbers as floats
• Add Dataset chunked() factory for online training

3.0 (#474)

* Relax syntax a bit

* Add AGENTS.md file

* ML-396 migrate Regressors to NumPower

* ML-396 stan check commented in git actions

* ML-396 added rule to phpcs to remove wornings on new_with_parentheses

* ML-396 restored files structure for phpstan

* ML-396 phpcs fix for Benchmarks\Regressors

* ML-396 phpcs fix for docs\Regressors

* ML-396 fix renamed Xavier1 to XavierNormal

* ML-396 reverted changes in AdalineBench for RuntimeException

* ML-396 renamed XavierUniform into Xavier1Uniform

* ML-396 fixed NumPower::truncatedNormal() signature

* ML-396 fixed Xavier1Uniform toString value

* ML-396 added Xavier2Uniform

* ML-396 refactored usage of packSamples with global function array_pack

* ML-396 refactored usage of packSamples with global function array_pack

* ML-396 refactored usage of packSamples with global function array_pack

* ML-396 refactored usage of packSamples with global function array_pack

* ML-396 improved RidgeTest with different types of tests

* Remove unused depth and maxDepth parameters from array_pack function

* ML-396 add Xavier2Normal, rename XavierNormal to Xavier1Normal, and update Xavier docs

* ML-396 refactoring array_pack

* ML-417 change location for tests cache

* ML-417 fix ogisticRegression::featureImportances() to NumPower

* ML-417 improved Softmax::activate in more efficient way

* ML-417 fix old Tensor API for NDArray

* ML-417 improved Softmax::activate in more efficient way

* ML-417 fix for MulticlassTest

* ML-417 align multiclass targets with network output

* ML-417 php cs fix

* ML-417 add diff option for php cs fixer

* ML-417 align Softmax with Multiclass column layout

* ML-417 change docs for Softmax

* Add check for NumPower at runtime instead of installtime (#415)

* Initial commit

* Add min version check

* Widen test tolerances

* Fix tests

* Revert "Fix tests"

This reverts commit f492b84.

* Widen tolerances

* Fix AI slop

* Remove more AI slop

* Fix KDTree Edge pruning + optimize traversal (#422)

* Initial commit

* Fix test

* Add BoxPrunable interface

* Optimize Dataset sort() (#431)

* Update CHANGELOG

* Purity to proper V-measure (#435)

* Initial commit

* Fix docs

* Update CHANGELOG

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix mangled Swish layer (#437)

* Fix Activation Function interface

* Fix initializers

* A little nicer

* Fix some tests

* Fix more tests

* Refactor data providers

* Fix more tests

* Appease Stan

* More appeasement

* More appeasement

* More appeasement

* More appeasement

* Update to Amp 2.0

* Better spatial tree testing

* Fix MLP Regressor test

* Fix tests

* Remove tsne window (#436)

* Initial commit

* Add tests

* Remove loss-based window (#443)

* Optimize defaults (#441)

* Initial commit

* Bump window for non-eval early stopping

* No hidden default

* Optimize tests

* More test workers

* Remove duplicate tests

* Fix epsilon smoothing bugs

* Fix Softmax NumPower version (#438)

* Disk based snapshots (#444)

* Initial commit

* A little nicer

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Don't unset snapshot path

* Stream params from disk

* No need to clone parameters anymore

* Clean up snapshot file after training

* Make snapshot path transient

* Better validation

* Add docs

* Better testing

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Update CHANGELOG

* Fix Batch Norm NumPower version (#445)

* Break apart cross entropy (#440)

* Initial commit

* Don't allow BCE on multiclass estimators

* Better comments

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix slop

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix Huber Loss NumPower version(#446)

* Update CHANGELOG

* Update CHANGELOG

* Remove Word Stemmer tokenizer

* Add numerical stability to SoftPlus

* Gaussian nb per class epsilon (#450)

* Initial commit

* Update CHANGELOG

* One hot encoder exclude categories (#454)

* Create a 'drop' parameter (#339)

* Bump version and fix coding style

* Optimize Labeled randomize() (#452)

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chris Lloyd <chrislloyd403@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Stop using deprecated contains()

* Prevent using wrong distance kernels

* Update CHANGELOG

* Initial commit

* A little nicer

* Nicer naming

* Appease Stan

* Fix test

* Fix CI

* Fix typo

* Fix SVC Save/Load missing class map (#449)

* Initial commit

* A little nicer

* Load model after classes

* Test sad path

* Prevent torn state on load

* Make saving as atomic as possible

* Better error handling

* More validation

* Optimize true-negative counting (#451)

* Adaboost defer sample weight renormalize (#457)

* Initial commit

* Unset variable

* Optimize N-gram and K-skip N-gram tokenizers (#459)

* Initial commit

* Fix degenerate settings

* Update CHANGELOG

* Modernize unit tests

* Appease Stan up to level 6

* Update AGENTS.md

* Update CHANGELOG

* Update CHANGELOG

* Ignore opencode stuff

* Add space

* Add new interfaces to docs

* Clean up a bit

* Isolate Swoole tests

* Tighten tolerances

* Optimize Loda binning (#455)

* Initial commit

* A little nicer

* Swoole workers now configurable (#464)

* Add windowed evals to Softmax Classifier, Logistic Regression, Adaline, Adaboost (#462)

* Initial commit

* A little nicer

* Appease Stan

* Remove loss change from logs

* Add to Adaboost

* Update CHANGELOG

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* A little nicer

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix Ridge tests

* PCA, LDA, SVD to Numpower (#461)

* Initial commit

* A little nicer

* Remove plan

* Add tests

* Check for empty dataset

* Update the docs

* Fix test

* A little nicer

* Update README

* A little nicer

* Improve the docs

* Update suggestions

* Appease Stan

* Integers are now Categorical (#463)

* Intial commit

* Fix benchmark

* Fix type-hint

* Fix test

* Tighten tolerances

* Fix tests

* Parallel KNN and Isolation Forest (#465)

* Intial commit

* Update the docs

* Parallelize Isolation Forest

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* A little nicer

* Fix coding style

* A little nicer

* Make Backend transient

* Fix exception class

* Nicer

* A little nicer

* A little nicer

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Appease Stan

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Remove unused script

* Fix test

* Tsne numpower (#466)

* Intial commit

* Tweak benchmark

* Fix test

* A little nicer

* Remove MAE loss function

* Make backend() public

* A little nicer

* Nicer

* Optimize Swoole backend task scheduling

* Nicer after callback

* Better logging

* Surface scores and combinations

* Warn if workers > physical cpu cores

* Autodetect Amp CPUs in test

* GMM Cluster Epsilon (#467)

* Intial commit

* Update CHANGELOG

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Numpower dtypes (#469)

* Initial commit

* Fix interface

* Fix typo

* Fix tests

* Remove comment slop

* Remove more comment slop

* Remove even more comment slop

* More comment slop

* Revert numpower (#470)

* Intial commit

* Save game

* Fix tests

* Remove

* Clean up a bit

* Update CHANGELOG

* A little nicer

* Add Float Type Converter

* Fix typo in CHANGELOG for categorical data type

* Fix grammar in CHANGELOG for Adaboost entry

* Update CHANGELOG.md

* Fix typehinting

* Change parameter name

* Interval discretizer now outputs ordered categories

* A little nicer

* Better validation

* Can now clear neural network Adaptive optimizer state

* Add more tests

* Add even more tests

* Limit degree of polynomial

* Even more tests

* Neural network cleanup (#472)

* Intial commit

* Update CHANGELOG

* Overall persistence improvements (#471)

* Initial commit

* A little nicer

* Save game

* Detect cycles in object gaph traversal

* A little nicer

* Use crc32 for payload

* Update the docs

* Update CHANGELOG

* A little nicer

* Change file mode to 'wb' for binary write

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Add type check for offset in offsetSet method

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Add type validation for Set members in offset methods

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Change checksum type from 'crc32' to 'crc32b'

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix serialization section in model persistence documentation

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Update RBX V1 reference to RBX V2 in documentation

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Update IteratorAggregate implementation for Set class

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Add tests

* A little nicer

* A little nicer

* Nicer

* Fix test namespaces

* Validate schemas

* Strict comparison

* Nicer

* Remove test

* Close handle if still open

* Recurse into array elements

* A little nicer

* Break out RBX V2

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Update CHANGELOG

* Update CHANGELOG

* Update CHANGELOG

* Add typehints

* Appease Stan

* Add missing type hints

* Use set object

* More set optimizations

* Fix adaboost snapshotting

* Murmur3 new default Token Hashing Vectorizer hash function

* Dataset fold() now returns excess samples in last fold

* Fix typehinting

* Optimize diagonal distance kernel

* Tighter typehinting

* Convert depcrecated object storage syntax

* Remove exception slop

* Remove window

* Upgrade guide (#476)

* Intial commit

* Expand on new features

* Update Pipeline initialization syntax in documentation

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Update cost function defaults in upgrade documentation

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* One Hot Encoder casts features to floats instead of integers

* Revert "One Hot Encoder casts features to floats instead of integers"

This reverts commit e8b794b.

* Improve the migration guide

* Improve the migration guide

* A little nicer

* Improve Tuple

* Improve the guide

* Check if committee fully-trained

* Check for empty array

* Guard empty dataset

* Consistent ITree search route

* Guard fold()

* Guard number of subsets

* Fix Bootstrap Aggregator

* Improve the docs

* Canonicalize He and Xavier 2 initializers

* Remove slop

* Fix more slop

* Update AGENTS.md

* Fix tests

* Fix AI slop

* Fix Multiclass layer gradient (#480)

* Initial commit

* Refactor loss calculation to use transposed output

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Update expected outputs in MulticlassTest

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix AI slop

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Scheduler Interface (#482)

* Optimizer refactor (#483)

* Initial commit

* Update the docs

* A little nicer

* Added some tests

* Fix typo

* A little nicer

* Add learning rate to logging

* MLP Smart final layer projections (#484)

* Initial commit

* A little nicer

* Neural Network refactor - decouple update step (#485)

* Initial commit

* Update the docs

* A little nicer

* Added some tests

* Fix typo

* A little nicer

* Add learning rate to logging

* Initial commit

* A little nicer

* Save game

* Revert

* Save game

* Save game

* Nicer

* Fix tests

* Clean up a bit

* Even nicer

* Even nicer

* Rename accumulate

* Clean up a bit

* Tick scheduler

* Optimize gradient norm calculation in LogisticRegression

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Enhance gradient accumulation step handling

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Optimize gradient norm calculation in SoftmaxClassifier

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Optimize gradient norm calculation in Adaline

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Refactor MLPRegressor constructor parameters

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix documentation for optimizer parameter update

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Cleanup GitHub CoPilot AI slop

* Remove more CoPilot AI slop

* Fix more AI slop

* Fix global L2 norm

* Nicer

* Nicer docs

* Clean up AI slop

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Revert "Neural Network refactor - decouple update step (#485)" (#487)

This reverts commit 1771da1.

* Neural Network refactor - gradient accumulation and clipping (#488)

* Initial commit

* Update the docs

* A little nicer

* Added some tests

* Fix typo

* A little nicer

* Add learning rate to logging

* Initial commit

* A little nicer

* Save game

* Revert

* Save game

* Save game

* Nicer

* Fix tests

* Clean up a bit

* Even nicer

* Even nicer

* Rename accumulate

* Clean up a bit

* Tick scheduler

* Optimize gradient norm calculation in LogisticRegression

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Enhance gradient accumulation step handling

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Optimize gradient norm calculation in SoftmaxClassifier

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Optimize gradient norm calculation in Adaline

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Refactor MLPRegressor constructor parameters

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix documentation for optimizer parameter update

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Cleanup GitHub CoPilot AI slop

* Remove more CoPilot AI slop

* Fix more AI slop

* Fix global L2 norm

* Nicer

* Nicer docs

* Clean up AI slop

* Remove gradient clipping from linear models

* Update CHANGELOG

* Add citations

* A little nicer

* Clean up a bit

* A little nicer

* Compensate for short batches

* Freeze first k layers of pretrained network

* Update CHANGELOG

* Add tests

* Optimize backprop over frozen layers

* Expand Network interface

* Fix typo

* Improve the docs

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Update the upgrade guide

* Improve the docs

* Add tests

* Nicer

* Improve the docs

* Update CHANGELOG

* Improve the upgrade guide

* A little nicer

* Remove duplicate

Removed the ability to clear neural network Adaptive optimizer state.

---------

Co-authored-by: Samuel Akopyan <samuel.akopyan@thrivedx.com>
Co-authored-by: Samuel Akopyan <sakopyan@oktopost.com>
Co-authored-by: Samuel Akopyan <apphp.com@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Chris Lloyd <chrislloyd403@gmail.com>

• SVD, PCA, LDA, throw exception on incompatible Tensor extension version.

• PCA, SVD, and LDA now work without Tensor extension

• Optimize random weighted subset with replacement
• Optimize Prior guessing strategy
• Added numerical stability to ELU and SELU

• Optimize Dataset validation
• Fix AdaBoost early stopping with dropped learners
• Fix Report and Tuple null values
• Fix Random Projector mangitudes
• Prevent division by zero in Sparse Random Projector

• Fix SQL Table offset syntax to work with Postgres
• Prevent SQL Table injection attack
• Pipeline add missing trained guard on score method
• Fix Column Picker/Filter on null columns
• Fix Image Rotator mangling non-90° rotations

• Fix Adaboost proba() probability normalization
• Optimize minmax operations
• Optimize Diagonal distance kernel

• Optimize DBSCAN inference
• Optimize K-means and FCM training

• Optimize brute-force nearest neighbors operations
• Optimize N-gram and K-skip N-gram tokenizers
• Optimize Adaboost, defer weight renormalization
• Optimize true-negative counting
• Added even more numerical stability to SoftPlus

• Symmetrize TSNE affinities matrix, fixes optimization
• Add numerical stability to SoftPlus activation function
• Fix Extra Tree leaf node entropy
• Add numerical stability to BM25 Transformer

• Optimize squaring operations
• Optimize Fuzzy C-means training
• Fix K-skip n-gram unigrams

• Optimize Decision Tree pruning
• Optimize K-means training
• Fix Classification Tree Gini impurity calculation
• Fix SELU negative branch derivative computation

• Fix Batch Norm normalize over batch size not features
• Fix Swish beta and input gradients
• Fix Swish gradient at zero input
• Fix GaussianNB variance smoothing when global max is zero
• Fix Prior strategy off-by-one bias

• Add numerical stability to logsumexp function
• Fix Mean Shift delta hyper-parameter
• Fix MAPE calculation in Error Analysis report
• Fix SiLU derivative at zero
• Remove spurious +1 in Relative Entropy gradient
• SoftPlus derivative expressed in terms of input
• Softmax added numerical stability

• Fix AdaBoost early stopping algorithm
• Fix zero quantile calculation
• Fix Jaccard zero distance on same vectors
• Fix Image Rotator zero jitter
• Fix CSV extractor crashing on blank body lines
• Fix K-means size calculation on partial train
• Fix true-nagatives undercounting in MCC, Informedness, and Multiclass Breakdown

• Update PHP Stemmer to version 4.0

• Fix bug in One-class SVM inferencing

• Fix bug in SVM (SVC and SVR) inferencing

• Add GELU activation function
• Add numParams() method to Network
• Neural Network Learners now report number of trainable parameters
• Regex Filter added pattern to match unicode emojis
• Custom escape character for CSV Extractor

• Optimize Adam and AdaMax Optimizers