evidently

• Supported dashboard visualization in Google Colab
• Supported dashboard visualization in python Pylab
• Added a parameter to switch on pylab visualization model: dashboard.show(mode='pylab')

• Added a way to calculate metrics with moving window
• Added metrics logging to Prometheus
• Added an example of Data Drift Monitoring with Prometheus and Grafana
• Added a config for Data drift dashboard at Grafana

Fix for https://github.com/evidentlyai/evidently/issues/61

• Added the source code for the UI (now it can be built from the source)
• Created utils.py with helper functions
• Added config for Pylint
• Added some unit tests

Added the following options to configure data drift report:

• 'drift_conf_level' confidence level for the individual features (default value = 0.95)
• 'drift_features_share' - share of the drifted features to detect dataset drift (default value = 0.5)
• 'xbins' - the custom bins to plot in the datadrift table
• 'nbinsx' - the custom number of bins to plot in the datadrift table

If share of the features drifted at the 'drift_conf_level' confidence level is higher than the 'drift_features_share' threshold, than Dataset drift will be detected. Otherwise Dataset drift will not be detected.

• When you use Evidently in the command-line interface, we collect basic telemetry. It includes data on the environment (e.g. Python version) and usage (type of report or profile generated).
• Our telemetry is intentionally limited in scope. We DO NOT collect any sensitive information and never see the data, its structure, or column names.
• You can read more about what we collect here.
• You can opt-out from telemetry collection by setting the environment variable EVIDENTLY_DISABLE_TELEMETRY=1

• Added proportion difference test for binary categorical target/prediction drift
• Added proportion difference test for data drift (categorical features)

• Fixed Regression Performance Analyzer (underperformance segments table)
• Fixed Prob Classification Performance Analyzer (precision-recall table)
• Fixed Classification Performance Analyzer (Classification Confusion Based Feature Distribution Table)
• Updated CatTargetDriftTab to use analyzers
• Updated NumTargetDriftTab to use analyzers
• Updated RegressionRerformanceTab to use analyzers
• Updated ClassificationPerformanceTab to use analyzers
• Updated ProbClassificationPerformanceTab to use Analyzers

• Sampling for large datasets: Sequential Sampling and Random Sampling
• Changed names: Production -> Current literally everywhere
• Fixed json serialisation issue
• Updated Ranges for plots inside of the “Classification Quality By Feature” table in Probabilistic Classification Performance Dashboard

• Released Profiles
• Added “calculate” step to separate Dashboard/Profile object creation from heavy calculation process.
• Changed name: DriftTab -> DataDriftTab

• Bug fixes and dependency updates
• Regression Performance Report: added a new plot to detect error bias per feature
• Probabilistic Classification Report: added a new plot to help choose the decision threshold