NewYour coding agent can read the release notes before it upgrades.Set up the MCP server β
PyPI, #3749 most downloaded on PyPI
A framework for building Singer taps and targets
Latest stable
0.54.9
Last release 24 days ago
22 Oct 2026
Ships on a steady schedule
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
3 versions withdrawn
withdrawn after publishing
6 years old
417 releases Β· first in 2021
Drag across the ruler to compare, or tap a release to open it.
The Singer SDK now provides an extensible API for loading schemas from various sources through the SchemaSource API. For example,
The Singer SDK now provides an extensible API for loading schemas from various sources through the SchemaSource API. For example,
from typing import ClassVar
from singer_sdk import OpenAPISchema, StreamSchema
# Load from OpenAPI spec
openapi_source = OpenAPISchema("https://api.example.com/openapi.json")
class UsersStream(RESTStream):
name = "users"
schema: ClassVar[StreamSchema] = StreamSchema(openapi_source, key="User") # Load "User" component
Docs: https://sdk.meltano.com/en/v0.48.0/guides/schema-sources.html
__original_stream_name__ in stream maps -- Thanks @ReubenFrankel!.env.example files to tap, target and mapper templatesstream_maps--about --format=markdown compliant with CommonMarkRESTStream.get_new_paginator to indicate returning None is supported to indicate no pagination- in SQL targets when default_target_schema is set, introduced in #3020RESTStream.records_jsonpath read-only by defaultget_stream_schema the public API of the StreamSchema descriptorSchemaSource generic on any hashable key typecapabilities a proper class attribute for tap, target and mapper classesBaseTestClass subclass when generating built-in testsSingerTestRunner classjsonschema validator instance directly instead of routing through jsonschema.validators.validator_forTestSuite β SingerTestSuite to prevent a pytest test collection warningclick callback to parse config input arguments and deprecate PluginBase.config_from_cli_argsFaker class from stream mapssinger_sdk._singerlibYour coding agent can read these notes before it upgrades. Set up the MCP server β
## v0.47.4 (2025-06-25) ### π Fixes - #3138 Ensure Singer SDK warnings are logged
## v0.47.3 (2025-06-19) ### π Fixes - #3113 Remove redundant tool.ruff.target-version from tap template ### π Documentation Improvements - #3116 Docum
## v0.47.2 (2025-06-17) ### π Fixes - #3111 Ensure state progress markers in child streams are finalized - #3108 Suppress unmapped warning for additio
## v0.47.1 (2025-06-16) ### π Fixes - #3106 Apply .upper() to metrics_log_level setting value - #3102 Bump "build and inspect python package" action i
Taps can now emit `ACTIVATE_VERSION` messages when the builtin emit_activate_version_messages config is set to True.
ACTIVATE_VERSIONTaps can now emit ACTIVATE_VERSION messages when the builtin emit_activate_version_messages config is set to True.
Setting the <APP_NAME>_LOGLEVEL or LOGLEVEL environment variables will now apply that level to all logs generated by taps , mappers and targets.
post_processThe stream post_process method is now executed for all stream types automatically. This can result in duplicate calls to the function if you have a stream with a custom get_records implementation such as
def get_records(self, context):
"""Get records from the API response."""
for record in self.request_records(context):
transformed_record = self.post_process(record, context)
if transformed_record is None:
continue
yield transformed_record
So you may want to streamline it:
def get_records(self, context: dict | None) -> t.Iterable[dict[str, t.Any]]:
"""Get records from the API response."""
yield from self.request_records(context):
SELECT statement filter and limit clauses without modifying extraction logic<APP_NAME>_LOGLEVEL and LOGLEVEL environment variables across all default loggingACTIVATE_VERSION messages for streams with FULL_TABLE replicationpost_process automatically to all stream typesrequired properties list in the JSON schema- in SQL targets when default_target_schema is set, introduced in #3020BATCH extra dependency in templatesauth callable/tuple to each tap API requestdict.setdefault to fill in state dictionarypyproject.toml in tap, mapper and target templatescontext parameter of SQLStream.build_query a keyword-only argumentSQLStream.build_queryTypedDict for tap stateStream.get_recordsINFO to DEBUGBATCH storage object only once per set of filesStream.primary_keys to always be a sequence## v0.46.5 (2025-06-24) ### π Fixes - #3135 Ensure Singer SDK warnings are logged
## v0.46.4 (2025-05-28) ### π Fixes - #3066 Revert table name splitting by - in SQL targets when default_target_schema is set, introduced in #3020
- in SQL targets when default_target_schema is set, introduced in #3020## v0.46.3 (2025-05-15) ### π Fixes - #3042 Revert "fix(targets): Avoid emitting message "No schema for record field." when "additionalProperties" is
## v0.46.2 (2025-05-15) ### π Fixes - #3037 Avoid aborting parent stream before child stream is finished in standard tap tests - #3031 Avoid emitting
"No schema for record field." when "additionalProperties" is set to true in stream schema -- Thanks @SidduHussain!## v0.46.1 (2025-05-13) ### π Fixes - #3034 Avoid emitting an empty state from the target if the tap has not sent any state messages - #3030 Bump stan
## v0.46.0 (2025-05-06) ### β¨ New - #2162 Expose metric dictionary to make logging metrics as JSON easier - #2977 Limit extraction to 25 records by de
default_target_schema is specifiednan, inf and -inf values as null in Singer stream## v0.45.12 (2025-06-24) ### π Fixes - #3132 Ensure Singer SDK warnings are logged
## v0.45.11 (2025-04-28) ### π Fixes - #3004 Ensure fields with an array of JSON schema types are properly handled by SQL targets - #2995 Pin GitHub a
Faker from stream maps contextseffective_schema to apply schema manipulations even on the input catalogmsgspec extra as a prerequisite to using the corresponding writer/reader classes -- Thanks @ReubenFrankel!## v0.45.10 (2025-04-16) ### π Fixes - #2978 Restore private _http_headers attribute of REST streams
## v0.45.9 (2025-04-15) ### π Fixes - #2975 Update cookiecutter templates - #2974 Do not log a warning if null cannot be appended to schema types - #2
## v0.45.8 (2025-04-14) ### π Fixes - #2965 AttributeError: object has no attribute '_requests_session' when accessing public property during stream i
## v0.45.7 (2025-04-14) ### π Fixes - #2968 singer_sdk.singerlib.resolve_schema_references now resolves oneOf schema references - #2924 Generate stand
singer_sdk.singerlib.resolve_schema_references now resolves oneOf schema referencesschema attributes (e.g. REST and GraphQL streams) can have their schema overridden by the catalogfallback-version for hatch vcs## v0.45.6 (2025-04-11) ### π Fixes - #2961 Conform NaN, Infinity and -Infinity to null values ### π Documentation Improvements - #2960 Documented usi
## v0.45.5 (2025-04-04) ### π Fixes - #2952 Adapt field schema type to string when transformed with stream maps using json.dumps ### βοΈ Under the Hood
## v0.45.4 (2025-04-01) ### π Fixes - #2946 In the built-in tap tests, do not emit a warning for missing records on ignored streams
## v0.45.3 (2025-03-29) ### π Fixes - #2937 Revert "fix: Ensure stream schema is overridden by the input catalog (#2932)" ### π Documentation Improvem
## v0.45.2 (2025-03-28) ### π Fixes - #2935 Added uv venv backend to Meltano project in templates - #2934 Updated cookiecutter pre-commit hooks and ad
## v0.45.1 (2025-03-27) ### π Fixes - #2925 Let downstream packages import from deprecated singer_sdk._singerlib.catalog
singer_sdk._singerlib.catalogβ¦singer_sdk.singerlib. If you're using the now deprecated _singerlib module, please update your code to use the new public module.
Taps and targets now support custom message writing and reading interfaces, and the new (experimental) msgspec-based encoding is available.
See the docs for more information.
Thanks to @BuzzCutNorman and @s7clarke10 for the thoughts and suggestions!
singerlib is now a public moduleThe low-level Singer library is now available as a public module and can be imported directly from singer_sdk.singerlib. If you're using the now deprecated _singerlib module, please update your code to use the new public module.
The x-singer.decimal JSON Schema extension is now supported by the SDK. Read the docs to learn how to use it with taps and targets.
x-singer.decimal JSON Schema extensionsingerlib to public modulepytest-github-actions-annotate-failures for GitHub workflows of tap, target and mapper templatesNo records were available to test warning when no records are ignored for all streams -- Thanks @ReubenFrankel!No records were available to test warnings when empty stream is ignored -- Thanks @ReubenFrankel!JSONSchemaToSQL from the target configurationNothing published for this version
Nothing published for this version
Nothing published for this version
## v0.44.4 (2025-06-24) ### π Fixes - #3128 Ensure Singer SDK warnings are logged ### π¦ Packaging changes - #3129 Test with click 8.2.x
## v0.44.3 (2025-02-05) ### π Fixes - #2870 Do not emit a warning needlessly when rest_method is not set in a stream class
rest_method is not set in a stream class## v0.44.2 (2025-01-31) ### π Fixes - #2862 Include package license metadata conditionally in templates -- _Thanks @ReubenFrankel!_ - #2859 Use uv in
Nothing published for this version
## v0.44.1 (2025-01-29) ### π Fixes - #2847 Update Cookiecutter templates - #2844 Avoid writing an empty state -- _Thanks @joaopamaral!_ - #2843 Use S
## v0.44.0 (2025-01-23) ### β¨ New - #2830 Allow developers to mark stream schema and settings fields as deprecated - #2829 Support a x-sql-datatype JS
x-sql-datatype JSON Schema annotation to let targets customize SQL type handlingNothing published for this version
## v0.43.1 (2024-12-10) ### π Fixes - #2807 Allow developers to set RESTStream.http_method
The RESTStream.rest_method attribute is deprecated in favor of `RESTStream.http_method`.
This release introduces a few improvements and changes to the SDK that should make it easier to work with SQL taps, REST streams, and targets.
A new SQL tap class attribute exclude_streams allows developers to exclude certain database schemas by default from discovery. This can speed up discovery and avoid issues when mapping types from database-internal schemas.
Discovery for SQL taps should be about 35% faster now, thanks to using SQLAlchemy 2.0+ reflection features.
A new target built-in setting process_activate_version_messages lets users disable processing of ACTIVATE_VERSION messages.
Support for other content-types in REST streams with the new RESTStream.payload_as_json attribute. This should make it easier to work with APIs that expect non-JSON payloads, such as XML.
The RESTStream.rest_method attribute is deprecated in favor of RESTStream.http_method.
[!IMPORTANT] This release drops support for Python 3.8. Please make sure to upgrade your tap or target to support Python 3.9+.
[!NOTE] SQL taps now require SQLAlchemy 2.0+.
activate_version for targets to optionally disable processing of ACTIVATE_VERSION messagesdecimal.Decimal in REST and GraphQL stream responsesmaxLength limit for VARCHARsversioning-strategy to dependabot config of Cookiecutter templateslistdir in folder taptable-key-properties, replication-key and forced-replication-methodactivate_version to process_activate_version_messages--catalog or --state do not existSQLToJSONSchema from the tap configurationNothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
## v0.42.1 (2024-11-11) ### π Fixes - #2756 Safely compare UUID replication keys with state bookmarks -- _Thanks @nikzavada!_ Full Changelog: https://
Full Changelog: https://github.com/meltano/sdk/compare/v0.42.0...v0.42.1
For SQL target developers, there is now a public and official API for overriding how JSON schema types are mapped into SQL types. Let us know if there
__stream_name__, including in the context of __alias__ expressions.SQLConnector.prepare_primary_key for target to implement for custom table primary key adaptationtitle is now supported in configuration and stream properties__alias__ context of stream maps -- Thanks @holly-evans!datetime.datetime instances in stream maps are now correctly mapped to date-time JSON schema stringsNothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
It's easier now for SQL tap developers to customize the mapping from SQL column types to JSON schema. See the guide for details.
BATCH messages via stream maps -- Thanks @ReubenFrankel!maxLength when applicableflattening_max_depth is not setdate-time strings in the schema message -- Thanks @gregkoutsimp!None or __NULL__ in stream maps are now also removed from the schema required arrayFileStream (still in active development!)Faker class in stream mapsrecord-flattening capability -- Thanks @ReubenFrankel!SQLConnector.type_mapping to SQLConnector.sql_to_jsonschemawarnings.deprecated decoratorNothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Targets now emit basic metrics. There's nothing to do on your end, they're enabled by default!
Faker class is available in stream maps context. Note that like the faker instance added in v0.35.0, this requires installing the faker library in the tap/target environment. For example, with tap-example[faker].schema_is_valid built-in tap test validates stream schemas against the JSON Schema specificationFaker class, rather than just a faker instancepytest-durations dependency from testing and use native pytest option --durations@override decoratorauth_headers and auth_params of APIAuthenticatorBase simple instance attributes instead of decorated propertiesfake and Faker objects were added to the stream maps contextget_starting_timestamp in incremental replication guidefaker extraNothing published for this version
Nothing published for this version
## v0.39.1 (2024-08-07) - PyPI π ### π Fixes - #2589 Make sink assertion compatible with stream maps -- _Thanks @JCotton1123!_ - #2592 Fixed typos in
--about plain text output_ (underscore) based on engine -- Thanks @haleemur!pendulum and python-dateutilpackaging semantic type in contributing guideThe json module can now be used within stream maps
The json module can now be used within stream maps
stream_maps:
animals:
id: id
data: "json.dumps({'description': description, 'verified': verified, 'views': views, 'created_at': created_at})"
__else__: null
The crypto dependencies PyJWT and cryptography are now optional and can be installed by requesting the jwt extra, i.e. singer-sdk[jwt]. These are only currently required for JWT authentication, but if your package depends on any of these, make sure the dependency is explicit.
The pendulum and python-dateutil dependencies are no longer used by this library and will be removed in a future release. If your package relies on any of them, make sure the dependency is explicit.
We're working on supporting msgspec for more performant IO, so keep an eye out for that π.
default_logging.yml to their packagejson module is now avaiable to stream maps -- Thanks @grigi!Stream.context attributeACTIVATE_VERSION messages in the base target class--aboutSQLSink a generic with a SQLConnector type parameterserialize_message for Singer writersPyJWT and cryptography dependencies optionalget_starting_timestamp requires setting a non-null replication_keyurl_base, path and the sync contextOne column per quarter.