pyshacl

Added

• Support for validating multiple target data graphs.
  • Default "combine" mode loads all given data graphs into a single Dataset and validates all at once.
  • New "validate-each" mode validates each data graph independently and returns multiple results.
  • CLI now accepts multiple data graph paths and adds a --validate-each flag.
  • New validate_each() entrypoint for per-graph validation in library code.

Fixed

• SPARQL constraint validation no longer misidentifies VALUES in property paths or predicate names.
  • Fixes #301
• Validation report text output is now deterministic.
  • Fixes #304
• use_shapes filtering no longer fails when omitted PropertyShapes are referenced by included shapes.
  • Fixes #298

Fixed

• Fixed a bug in SHACL Shape Targeting feature, BNode shapes that are referenced by a shape in sh:or, sh:and or sh:xone were not getting selected.
  • Fixes #280

Fixed

• Finalize the decoupling of base_uri from graph identifier.
  • Blame the conflated naming of publicID in RDFLib for that confusion.
• load_from_source will now correctly detect and use the BaseURI of files passed in, for relative URIs.
  • Fixes #281

Changed

• Update to Poetry v2.0 and new pyproject.toml format.
• Removed "Black", switched to "Ruff" for formatting as well as linting.
• Switched to parsing file: IRIs in line with the RDF spec, and allow (base-less, or root-less) relative file: IRIs (as per the RDF spec).
  • But "<file:>" IRIs in Turtle files are now not made relative to BaseURI, because they are relative to the CWD.

Added

• Two new basic examples in the Examples folder.
  • "sparql_assert_datatype.py" shows how to use SPARQL-based Constraints to assert a datatype on a literal.
  • "remote_sparql.py" shows how to use SparqlConnector store to validate data on a remote SPARQL endpoint.

Fixed

• Fixed a bug where the identifier would become "None" (string) in the load_from_source function.
• Typos in the example Ontology files in the test suite.

Added

• When validating a Dataset instead of a bare Graph, PySHACL will now expand RDFS and OWL-RL inferences into a separate named graph, to avoid polluting the datagraph.
• When using SHACL Triple Rules from SHACL-AF spec, PySHACL will now add the expressed triples into a separate named graph. This allows you to more easily get the expanded triples back out again afterward.
  • This is implemented for TripleRules, SPARQLRules and JSRules

Changed

• PySHACL no longer supports older RDFLib versions
  • PySHACL relies on the latest OWL-RL version, that in-turn relies on the latest RDFLib version
  • Therefore PySHACL now requires RDFLib v7.1.1 or newer
• Dropped Python 3.8 support.
  • Python developers discontinued Python 3.8 last month
  • The next version of RDFLib and OWL-RL will not support Python 3.8
  • Removed Python 3.8 from the RDFLib test suite
  • Python 3.9-specific typing changes will be incrementally introduced

Fixed

• PySHACL no longer overwrites the Python root logger and removes all its handlers. How Rude.

Added

• owl:imports now works with bnode values, where it contains the following:
  • schema:url is the string where to find the imported ontology
  • schema:url (again) with a "file://" path, to a local copy of the ontology
  • schema:identifier that is the canonical name to use for the ontology at load time (this is the publicID)
• RDFUtil.loader load_from_source function now supports identifier that is akin to the publicID of the file being loaded, and that is passed to RDFLib parser to correctly do relative URIs, etc.

Changed

• Big change to how Milti-graph datasets (ie, rdflib.ConjunctiveGraph and rdflib.Dataset) are handled.
  • Instead of validating each named graph individually, PySHACL now sets defaultUnion=True and now validates the entire Dataset at once.
  • This allows you to logically segment your dataset as desired into many individual named graphs, and validation will still work as you expect it to.
  • This is in preparation for another big upcoming change in pySHACL that will allow OWL-RL inferencing to place inferred triples into a separate named graph, and SHACL Rules to place inferred triples into a separate named graph, and validation will still work as expected because validation is now against a union of the whole dataset.
• Pre-Compile Regexs in sh:Pattern constraints. This allows faster re-use of the constraint, if is applied to many different targets.

Fixed

• Attempting to stringify a focus_node ar a value_node from the datagraph, where that node doesn't actually exist in the datagraph, no longer crashes, it falls back to a different method.

Added

• SHACL Rules Expander Mode
  • A new alternative Run Mode for PySHACL
  • PySHACL will not validate the DataGraph against Shapes and Constraints, instead it will simply run all SHACL-AF Rules to expand the DataGraph.
  • By default it will output a new graph containing the existing DataGraph Triples plus the expanded triples
  • Run with inplace mode to expand the new triples directly into the input DataGraph
• Focus Node Filtering
  • You can now pass in a list of focus nodes to the validator, and it will only validate those focus nodes.
  • Note, you still need to pass in a SHACL Shapes Graph, and the shapes still need to target the focus nodes.
  • This feature will filter the Shapes' targeted focus nodes to include only those that are in the list of specified focus nodes.
• SHACL Shape selection
  • You can now pass in a list of SHACL Shapes to the validator, and it will use only those Shapes for validation.
  • This is useful for testing new shapes in your shapes graph, or for many other procedure-driven use cases.
• Combined Shape Selection with Focus Node filtering
  • The combination of the above two new features is especially powerful.
  • If you give the validator a list of Shapes to use, and a list of focus nodes, the validator will operate in a highly-targeted mode, it feeds those focus nodes directly into those given Shapes for validation.
  • In this mode, the selected SHACL Shape does not need to specify any focus-targeting mechanisms of its own.
• Combined Rules Expander Mode with Shape Selection
  • The combination of SHACL Rules Expander Mode and Shape Selection will allow specialised workflows.
  • For example, you can run specific expansion rules from a SHACL Shapes File, based on the new triples required.

Changed

• Don't make a clone of the DataGraph if the input data graph is ephemeral.
  • An ephemeral graph is one that is loaded from a string or file location by PySHACL
  • This includes all files opened by the PySHACL CLI validator tool
  • We don't need to make a copy because PySHACL parsed the Graph into memory itself already, so we are not concerned about not polluting the user's graph.
• Refactorings
  • shacl_path_to_sparql_path code to a reusable importable function
  • move sht_validate and dash_validate routes to validator_conformance.py module.
    • Removes some complexity from the main validate function.
• Typing
  • A whole swathe of python typing fixes and new type annotations. Thanks @ajnelson-nist

Fixed

• Fixed SHACL Path generation where sh:inversePath is wrapping a different kind of SHACL Path.
  • This probably fixes lots of unreported sh:inversePath bugs
  • Fixes #227
• Fixed generic message generation when there are multiple sh:and, sh:or, or sh:xone constraints on a Shape.
  • Fixes #220
• Fix logic determining if a datagraph is ephemeral.

Added

• Added ability to specify a custom max-evaluation-depth. If you find yourself with a legitimate use case, and you are certain you need to increase this limit, and you are cetain you know what you are doing:
  • use --max-depth i argument on the commandline where i is an integer 1 to 999.
  • or use max_validation_depth=i in the validator arguments if using PySHACL as a library.
  • Fixes #224
• Add a "SHACLExecutor" context to be passed down to all Shape and Constraint objects, to allow them to operate in a common mode with common configuration values, as chosen by the mechanism that starts them (eg, the Validator).
  • This will allow for further configuration of "modes of operation" for PySHACL in the future, to allow for forms of exectors other than "Validator".

Changed

• Bumped Black version to 24.3.0 to mitigate CVE.
• Bumped Poetry and Poetry-Core version to latest-and-greatest (fixes a bunch of long-standing bugs, I highly recommend updating).
• cli main() no longer returns and int, it now emits sys.exit(code) and never returns.

Fixed

• Fixed typing on Python 3.8 and 3.12 (PRs #192 and #223). Thanks @ajnelson-nist
• Fixed auto-generated validation message for sh:not when more than one sh:not is used on a constraint.

Changed

• Dropped support for Python 3.7
  • Note, for compatibility with RDFLib 7.0, we specify a minimum Python version of v3.8.1
  • https://github.com/RDFLib/rdflib/blob/3bee979cd0e5b6efc57296b4fc43dd8ede8cf375/CHANGELOG.md?plain=1#L53
• Add preliminary support for Python 3.12
• Dropped support for RDFLib v6.2.0 and earlier
  • Only RDFLib v6.3.2 and v7.0 are supported by PySHACL v0.25.0+
• Bumped to updated version of Black and Ruff
  • Reformatted everything according to py38 codestyle

Fixed

• Do not hard-pin to importlib-metadata. Fixes #214

Added

• Added Python 3.11 support (use it, internal benchmarking shows its 25-30% faster than Python 3.8)
• sh:node NodeConstraint now includes details of its child validation results, that were normally not included in the validation report.
  • exposed via the sh:detail property on the NodeConstraint validation report

Changed

• Added compatibility with Python 3.11, this requires:
  • RDFLib v6.3 or greater (recommended v6.3.2)
  • PyDuktape v0.4.3 for python 3.11 support
  • Poetry v1.5.0 (or poetry-core v1.6.0)
• Graph Namespace manager now only registers 'core' namespaces, this avoids having inconsistencies and incompatibilities with your own namespaces.
• Replaced Flake8 and isort with Ruff
• Updated to latest Black version for formatting

Fixed

• Extend ontology inoculation to include triples where NamedIndividual URI is object.
• Re-black all files, re-sort with new Ruff isort, fix some Mypy typing inconsistencies

In this release:

Fixed

• Inoculating the datagraph using an extra ontology graph now copies over any missing namespace prefixes from the ontology graph to the datagraph.
  • This is to match old ontology-graph-mixin behaviour that had this side-effect.
  • Added a test to ensure this behaviour is not broken again.
• Stringifying nodes in QualifiedValueShape default message was using wrong stringification operation.

In this release:

Fixed

• Clone full contents of an OWL:NamedIndividual from the ontology graph to the datagraph, during the inoculation procedure.
  • This fixes the case where an NamedIndividual in an OWL ontology had properties that were required in the datagraph at runtime to ensure successful validation
• Avoid hitting the recursion limit when stringifying a blank node, when OWL inferencing has inserted owl:sameAs the same blank node as is being serialized.
• Avoid hitting the recursion limit when cloning a graph with a blank node, when OWL inferencing has inserted owl:sameAs the same blank node as is being cloned.

Added

• Added a default message for QualifiedValueShape constraint component. It never had one before.

Changed

• Lots more debug messaging. Debugging is now much more verbose.
  • This gives more insight into how PySHACL runs, what it is doing, and how long each step takes.
  • All constraint evaluations will now output their results, regardless of whether are conformant or non-conformant or if they are used in the final conformance report.
  • You will probably want debug turned off unless you are tracking down the source of a problem or performance issue.

In this release:

Changed

• Big change to how ontology mix-in mechanism works
• Feature is now called datagraph inoculation
• Inoculation copies only RDFS and OWL axioms (classes, properties and relationships) from the extra-ontology file into the datagraph
  • This mitigates a class of errors that cause the validator to perform validation on Nodes that should not be in the datagraph
  • Such as cases where the Shapes graph and Extra-ontology graph are the same graph, but having SHACL Shapes and constraints in the datagraph is undesired.
• Details around automatically cloning the datagraph before modification (inoculation) remain unchanged.
• If you preferred the old behaviour, where the whole extra-ontology file was mixed-in to the datafile, please file a Github issue outlining your need for that.

In this release:

Added

• New HTTP Server functionality
  • run PySHACL as a persistent service, exposing an OpenAPI3.0-compatible REST interface
• Detection of filename when downloading web attachments is added
• Better detection of invalid integer values for sh:minLength and sh:maxLength string constraints.

Fixed

• Opening a http link that has chunked encoding, or content-disposition attachment will now work correctly.

In this release:

Fixed

• Ill-typed/Ill-formed literals now fail the DataType test, as expected
  • Requires RDFLib v6.2.0+
  • Fixes #140 (and possibly fixes #151)
  • Unskipped one of the remaining skipped shacl-test-suite SHT tests (datatype-ill-formed_ttl.ttl)
• Fixed detection of recursion to be more lenient of deliberately recursive (but not infinitely recursive) shapes.
  • Fixes #154
• MetaShacl works again now with RDFLib >= v6.2.0
  • Fixes #153
• Fixed typing issues affecting interoperability of new version of RDFLib and PySHACL.

Changed

• RDFLib v6.2.0 or greater is now required to run PySHACL
  • This new version of RDFLib implements the ill-typed Literals feature, that helps with sh:datatype constraint validation.
  • Removing support for older versions of RDFLib allows PySHACL to implement new features, and have less unnecessary code
• Bumped to using new Poetry v1.2.0 (or newest poetry-core v1.1.0)
  • Changed pytest-cov and coverage tests to be optional dependencies for dev
• Bumped version of Black to 22.8.0, and re-blacked all files
• Removed old monkey patches, no longer needed for the latest version of RDFLib
• Removed bundled "Memory2" store, now using the default "Memory" from RDFLib
  • Regenerated bundled pickled triplestores, to use Memory instead of Memory2
• Updated official dockerfile with newest version of PySHACL and RDFLib
  • Published to dockerhub at ashleysommer/pyshacl
  • docker pull docker.io/ashleysommer/pyshacl:latest

In this release:

Fixed

• CLI Output Table formatting crashed when report graph did not contain a resultMessage
  • Fixes #145
• Executing advanced-mode triples rules can sometimes skip the graph clone step, and incorrectly emits new triples directly into the input data-graph
  • Discovered when investigating #148

Changed

• Executing advanced triples rules no longer incorrectly emits new triples directly into the input data-graph
  • This may been seen as a breaking change, if your workflow relied on this incorrect behaviour.
  • If you really the rules engine to emit new triples into your input data graph, use the inplace validator option.
• Updated built-in schema.ttl file to newer version that doesn't have UTF-8 encoding issues

Added

• Official Dockerfile is now included in the repository
  • Thanks @KonradHoeffner; Fixes #135
  • Published to dockerhub at ashleysommer/pyshacl
  • docker pull docker.io/ashleysommer/pyshacl:latest

In this release:

Fixed

• Fixed a long-standing oversight where ShapeLoadErrors and ConstraintLoadErrors were not reported correctly when running PySHACL in CLI mode.
  • Sorry about that. Thanks lots of people for reporting this over the last year. I wish I fixed it sooner.
• Fixed a long-standing bug where using $PATH in a sh:sparql query on a PropertyShape would not work correctly.
  • Fixes #124, Thanks @Martijn-Y-ai
• Fixed a long-standing bug, that allows PySHACL to more reliably determine if graph source is a file path, or a graph string.
  • Fixes #132, Thanks @Zezombye
• Fixed an issue where sh:pattern could not be applied to a Literal that was not an xsd:string or URI.
  • Fixes #133, Thanks @nicholascar
• Fixed the outdated/incorrect error reported when a PropertyShape's sh:path value gets an unknown path type.
  • Fixes #129, Thanks @edmondchuc

Added

• New --allow-infos option in CLI mode and Python Module mode.
  • This is like --allow-warnings except it only allows violations with severity of sh:Info.
  • (--allow-warnings continues to allow both sh:Warning and sh:Info as it used to.)
  • Fixes #126, Thanks @ajnelson-nist
• SPARQL-based Constraints can now substitute arbitrary bound SPARQL variables into their sh:message
  • Fixes #120

Changed

• --allow-infos and --allow-warnings can now also be enabled with --allow-info and --allow-warning respectively.
• Removed Snyk check on CI/CD pipeline, because there is an RDFLib issue blocking Snyk on PySHACL from passing.

Added

• Added the ability to pipe in SHACL file or ONT file via stdin on Linux or MacOS

Fixed

• Fixed an issue where the filetype detection routine in the RDF loader would fail to reset the file back to the start.

Added

• Added Python 3.10 support (when using RDFLib v6.1.1 or greater)
• Added more type hinting, to conform to the new type hinting added by RDFLib 6.1.1

Changed

• Subtle correction in the way sh:prefixs works with sh:declare on the given named ontology.
• Bumped some min versions of libraries, to gain compatibility with Python 3.10

Fixed

• Fixed test for issue #76
• Fixed #76 again (after fixed test)

Fixed

• Don't crash when a SHACL function is registered more than once (eg, if a function is both SPARQLFunction and JSFunction), fixes #108, thanks Gabe Fierro
• Fixed typo in CLI help output, thanks Alex Nelson
• Don't print env vars when importing JS module, thanks MPolitze
• Fix typo preventing OWL-RL >=6.0 to be used with pySHACL, Fixes #111

Added

• Add Snyk checks to CI/CD pipeline

Fixed

• SPARQL queries with words "values", "minus", or "service" in its comments no longer incorrectly throw an exception.

Changed

• Switched from Travis to Drone for CI testing

Added

• New Table output type for commandline tool. Thanks @nicholascar

Fixed

• Handle transitive subclasses when evaluating sh:targetClass - @gtfierro
  • Fixes #96
• Improve detection of RDF/XML files when loading unknown content
  • Fixes #98
• Imported type stubs and resolved ALL MyPy issues! (this was a big effort)
• Logic fixes in the dataset loader (thanks to inconsistencies exposed by MyPy)

Changed

• Add special cases to sh:dataclass constraint, when the given shape uses rdfs:Literal or rdfs:Dataclass as the dataclass to match on
  • Fixes #71

Added

• Add datashapes.org/schema as a built-in graph
  • Fixes #98
• Added ability to pass a TextIO or TextIOWrapper object into the dataset loader

Added

• ExpressionConstraintComponent is implemented!
  • Use your previously defined SHACL Functions to express complex constraints
  • Added DASH-tests for ExpressionConstraintComponent
  • Added advanced tests for ExpressionConstraintComponent, SHACLRules, and SHACLFunctions.
• New Advanced features example, showcasing ExpressionConstraint and others features

Changed

• Allow sh:message to be attached to an expression block, without breaking its functionality
• A SHACL Function within a SHACL Expression now must be a list-valued property.
• Refactored node-expression and path-expression methods to be common and reusable code
• Re-black and isort all source files

Changed

• sh:class Constraint now applies transitively.
  • This means it will follow rdfs:subClassOf relationships right to the top of the hierarchy.
  • Be careful with this, could lead to recursion or infinite loops!
  • This requires a big version number bump because it's technically a breaking change.
  • Fixes #87, thanks @gtfierro

Fixed

• Compatibility with RDFLib 6.0.0
  • Don't use .term() (PR #84)
  • Use Namespaces in a way that works on both RDFLib 5 and 6.

Changed

• Do not patch rdflib with Memory2 store on RDFLib 6.0.0+

Added

• Allow-Warnings is finally available. (Closes #64)
  • Setting this option puts PySHACL into a non-standard operation mode, where Shapes marked with severity of sh:Warning or sh:Info will not cause result to be invalid.
  • Despite the name, it allows both sh:Info and sh:Warning.
  • Try it with allow_warnings=True on validate() or -w in the CLI tool.

Fixed

• Fixed Abort Mode. (Fixes #75)
  • This optional mode allows the validator to exit early, on the first time your data fails to validate against a Constraint.
  • Name changed from abort_on_error to abort_on_first
  • Try it out with abort_on_first=True on validate() or --abort in the CLI tool.

Added

• Added an iterate_rules option, that causes SHACL Rules to run repeatedly until reaching a steady state. (Closes #76)
  • Works with SPARQLRules, TripleRules, and JSRules.
• Variables {$this}, {$path}, and {$value} will be populated in the sh:message of a SPARQL Constraint. (Closes #30)

Changed

• Relaxed the Max Evaluation Depth from 28 to 30, we were seeing some real-world cases where meta-shacl was failing on large Shapes Graphs at 28 levels deep.
• sh:namespace values can now be xsd:anyURI or xsd:string or "literal string", but now cannot be <URI nodes>.
• sh:order can now support xsd:decimal values and xsd:integer values, and can be interchanged at will.

Added

• Potential speedups when executing validation by lazy-loading large modules which may never be required in a normal validation run.

Fixed

• Black and Flake8 issues outstanding from 0.14.1 release.
• Workaround a RDFLib bug trying to import requests when requests is not required to be installed.
  • This bug will still be observed if you use SPARQLConstraints, SPARQLFunction or JSFunction features, but it can be worked around by simply installing requests in your python environment.

Added

• Inplace Mode, for when cloning your datagraph is undesirable
  • Normally pyshacl will create an in-memory copy of your datagraph before modifying it (when using ontology mixin, or inferencing features)
  • This might be unwanted if your datagraph is very large or remote and cloning it into memory is not a good option
  • Enabling inplace mode will bypass this clone step, and apply modification operations directly on your data_graph (use with caution!)
  • Enable with inplace=True kwarg on validate().
  • Inplace mode is not yet available via the CLI application, and perhaps doesn't even make sense to have it available there.

Fixed

• Inferencing will no longer incorrectly place expanded triples into your original data_graph, unless you enable 'inplace'
• SHACL-JS loader will no longer fail if the regex module is not installed (it will fall back to using builtin re)
• SHACL-Rule DASH-tests will now pass when the SHACL-rule is applied on multigraph (Dataset or ConjunctiveGraph)

Added

• SHACL-JS Support!
• Implements all of the features in the SHACL-JS SHACL Extension specification: https://www.w3.org/TR/shacl-js/
• Includes:
  • JS Constraints
  • JS ConstraintComponents
  • JS SHACL Functions
  • JS SHACL Rules
  • JS Target
  • JS TargetType
• To install it, make sure you do pip3 install pyshacl[js] to get the correct extra packages.

Changed

• Added JS flag to the CLI tool to enable SHACL-JS features
• Updated README and FEATURES matrix

Fixed

• Fixed a long standing issue where our fancy loader would try to seek() on a file, after the file was closed by the JSON-LD parser
  • (thanks @nicholsn for reporting it)
• Fixed https://github.com/RDFLib/pySHACL/issues/62

Added

• Added the ability for PySHACL to use baked in graphs instead of fetching them from a HTTP endpoint when a known graph is imported using owl:imports
  • This allows for time savings on graph-load and saves a HTTP request
  • Also allows us to embed fixed errata versions of files in place of release-time ones online

Fixed

• With new features, comes new bugs
• With the ability to now load SPARQLFunctions, this removes the barrier for loading Schema.org SHACL in advanced mode
• But when doing so revealed more issues. They are now fixed:
• Fixed SPARQLConstraintComponent getting confused when shacl.ttl was loaded into your Shapes file using owl:imports
• Fixed https://github.com/RDFLib/pySHACL/issues/61

Changed

• Refactored SPARQLConstraintComponent code, to allow for other custom constraint components in the future
  • This prevented SPARQLConstraintComponent getting confused when shacl.ttl was loaded into the Shapes file using owl:imports

Added

• SPARQLTargetType
  • New SPARQL-based Target Type feature
  • The Paramaterisable form of SPARQLTarget from the SHACL Advanced Features spec
  • https://www.w3.org/TR/shacl-af/#SPARQLTargetType
• Added a test for SPARQLTargetType - Theres none in the SHT suite, or the DASH suite.

Changed

• Refactored sh:parameter code in SPARQL-based Constraint Components, SHACLFunctions, and SPARQL-Based Target Types
  • They all now share a common SHACLParameter helper class, reducing code duplication
• Refactored SPARQLQueryHelper
  • SPARQLQueryHelper internal class is now more helpful
  • query_helper can now extract param bindings into param-value pairs for parameterised queries
  • Reduces more code duplication

Added

• New SHACL Advanced Spec Features!
• All NodeExpressions can now be used in SHACL Rules
  • Focus Node (sh:this)
  • FilterShape (sh:filterShape)
  • Function Expressions (any sh:SHACLFunction and args)
  • Path Expressions (use sh:path in a NodeExpression)
  • Intersection Expressions (sh:intersection)
  • Union Expressions (sh:union)
• SHACLFunctions (including SPARQLFunction)
  • Both SHACLFunction and SPARQLFunction are now fully implemented including unit tests and edge cases
  • SHACLFunctions are bound to PySHACL and can be used in SHACL Rules and NodeExpressions
  • SPARQLFunctions are bound to the RDFLib SPARQL Engine, so they can be used in other SPARQL queries
  • Read the manual for more info: https://www.w3.org/TR/shacl-af/#functions

Fixed

• Short versions of uris were sometimes not used in the Validation Report when they should've been
• Checking results of some tests was being skipped! Lucky this wasn't letting through any SHACL errors.
• Fixed error message when using sh:ignoredProperties on a node that isn't sh:closed issue #58

Fixed

• In a validation report graph, when FocusNode and ValueNode are the same node, and are a blank node, when they get copied into the validation report graph they will have the same bnode id as each other.
• Optimised the algorithm for copying different kinds of rdf nodes into the validation graph.

Changed

• When the FocusNode and ValueNode are copied into the validation graph from the data graph, they will try to keep the same bnode id they had before, if possible.

Fixed

• A couple of autogenerated sh:message strings were trying to serialize from dataGraph rather than shapeGraph

Fixed

• A couple of autogenerated sh:message strings were missing their focusNode element in v0.12.1

Added

• All SHACL Core constraints now have their own autogenerated sh:message.

  • This is used as a fallback when your Shape does not provide its own sh:message
  • See the new sh:resultMessage entries in the Validation Report output
  • These are hopefully more human-readable than the other fields of the Validation Report results

• Added a copy of the implementation of the new 'Memory2' rdflib triplestore backend.

  • This when using Python 3.6 or above, this is faster than the default 'IOMemory' store by:
    • 10.3% when benchmarking validation with no inferencing
    • 17% when benchmarking validation with rdfs inferencing
    • 19.5% when benchmarking validation with rdfs+owlrl inferencing

Changed

• PySHACL is now categorised as Production/Stable.
  • This marks a level of maturity in PySHACL we are happy to no longer consider a beta
  • A v1.0.0 might be coming soon, but its just a version number, doesn't mean anything special
• Changed default rdflib triplestore backend to 'Memory2' as above.
• Tiny optimisations in the way sh:message items are added to a validation report graph.

Fixed

• Regression since v0.11.0, sh:value and sh:focusNode from the datagraph were not included in the validation report graph if the datagraph was of type rdflib.ConjunctiveGraph or rdflib.Dataset.

Removed

• Python 3.5 support is removed. PySHACL now requires Python 3.6 or above.
  • Routine tests are run using Python 3.6.11, 3.7.8, and 3.8.2.
  • Python 3.9 might work but is not yet supported.

Added

• Python 3.6-compatible type hinting is added throughout the codebase
• MyPy library is used to run type checking during testing process
• Flake8 linting is added to enforce PEP8
• isort is added to enforce imports linting
• Black is added to keep formatting consistent across releases

Changed

• PySHACL is no longer a setuptools-based project with a setup.py and requirements.txt file.
• PySHACL is now a PEP518 & PEP517 project, it uses pyproject.toml and poetry to manage dependencies, build and install.
• For best compatibility when installing from PyPI with pip, upgrade to pip v18.1.0 or above.
  • If you're on Ubuntu 16.04 or 18.04, you will need to run sudo pip3 install --upgrade pip
• Editor Line Length for PySHACL code is now set to 119 as opposed to 79 chars.

Added

• New feature to CLI tool
  • -V shows the PySHACL version
• Run module directly
  • You can get access to the same CLI tool if you install the module and run it using python3 -m pyshacl
  • See python3 -m pyshacl --help for more details

Deprecated

Announcement

• This is the final version with Python v3.5 support
  • Versions 0.12.0 and above will have newer package management and dependency management, and will require Python v3.6+.

Fixed

• Fixed a bug present since v0.11.0. If the data graph has multiple named graphs, and an extra ontology mixin source used and that also has multiple named graphs, then only the first graph in the mixins source was added to the datagraph.
  • Now all named graphs from the mixin source are mixed into all named graphs of the datagraph, as originally intended.
  • Fixed one unit test which had been intermittently failing
• Cleaned up the behaviour around performing patch to Boolean Literal parsing on rdflib 5.0.0

Fixed

• Preparatory changes for the incoming rdflib 5.0.0 release
• Changed to a new more predictable literal comparison routine for minInclusive, minExclusive, maxInclusive, and maxExclusive. This removes the need for one monkey-patch in rdflib 4.2.2 and works around the TOTAL_ORDER_CASTERS special cases in rdflib 5.0.0.

Fixed

• Fixed Issue #040
• Fixed badly-formatted dates in the changelog

Added

• Added ability for pySHACL to track and monitor its evaluation path during validation
  • This allows for the validator to detect two different scenarios:
    • A recursive shape has triggered an infinitely-recursive validation, back out
    • Evaluation Path too deep (error generated, prevents python recursion depth errors)
• Added a test for Issue #40

Fixed

• Fixed Issue #036

Added

• Added test for #036

Changed

• Nodes defined as TargetNode by a SHACL Shape no longer is required to be present in the DataGraph.

Fixed

• Fixed Issue #032
• Stringification of Focus Node, and Value Node in the results text string now works correctly
  • This is an old bug, that has been around since the first versions of pySHACL
  • Manifests when the DataGraph is a different graph than the ShapesGraph
  • Recent change from using Graphs by default to using Datasets by default helped to expose this bug
  • Thanks to @jameshowison for reporting the bug

Changed

• Stringification of a blank node now operates on a rdflib.Graph only, rather than a Dataset.
  • Added mechanism to extract the correct named graph from a dataset when stringifying a blank node.
• Added a workaround for a json-ld loader bug where the namespace_manager for named graphs within a conjunctive graph is set to the parent conjunctive graph.
  • This necessary workaround was exposed only after changing the blank node stringification above. (Fixing one bug exposed another bug!)

Changed

• Bumped min OWL-RL version to 5.2.1 to bring in some new bugfixes
• Corrected some tiny typos in readme

Fixed

• Fixed an inferencing bug introduced in v0.11.0 (present in v0.11.0 and v0.11.1)
  • OWL or RDFS or BOTH inferencing wasn't being applied correctly because RDF data is now loaded in as a Dataset rather than a Graph.

Changed

• Fixed the "extras" tests to take advantage of RDFS inferencing to better expose this kind of bug in the future.