NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
pub.dev
pub.dev, #3457 most downloaded on pub.dev
Bones_API - A powerful API backend framework for Dart. It comes with a built-in HTTP Server, route handler, entity handler, SQL translator, and DB adapters.
Latest stable
1.35.1
Last release 13 days ago
08 Nov 2026
Ships fairly regularly
a new release about every 3 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
5 years old
388 releases · first in 2021
Drag across the ruler to compare, or tap a release to open it.
Private fields that only memoize a public getter or method of the same name (String? _origin; String get origin => _origin ??= ...) take that name aga
String? _origin; String get origin => _origin ??= ...) take that
name again, as before 1.35.0 renamed them to *Cache: 27 fields (e.g.
_hostname, _port, _origin, _lastPathPart, _fieldsNames,
_fieldsWithTypeEntity, _totalRules, _tablesVersions), and
APIRequest._requesterAddressResolved (its getter's name is taken by the
given address). Caches filled by a private helper or a differently named
member keep *Cache. Behavior unchanged (private names only).sqlite3: ^3.9.1 (was ^3.8.0): its build hook now reports a failed
native-library download from GitHub releases as the HTTP error (e.g.
504), not as a hash mismatch (simolus3/sqlite3.dart#435). 3.9.0 also
added CommonDatabase.trace and Database.interrupt().Your coding agent can read these notes before it upgrades. Set up the MCP server →
…(the typo getEntityRepositoresTables kept, @Deprecated).
API/server layer clean-up (behavior-preserving):
APIServerWorker: both branches of _processAPIResponse end in one
_finishResponse; helpers renamed to what they do (_buildShelfResponse,
_disposeAndCompressResponse, _startServersAndInitializeAPIRoot,
_errorResponse). The CORS headers are set once (they were set again
when building the headers)._readPayloadBytes for text and binary payloads, one
maxPayloadLength check, done before copying the blocks (a single
Uint8List block is still used as is).APIServer.toAPIRequest: headers and query parameters flattened by one
helper. New constants: APIServer.sessionIDCookieName (SESSIONID),
headerServerTiming, headerKeepAlive, headerContentLength,
headerXAPIServerWorker; APIRoot.apiInfoRoute (API-INFO) and
APIRoot.defaultAuthenticationRoute (authenticate).APISecurity: one resolveMapped chain resolving a token's credential;
the disposeAuthentication* methods share one helper.APIRoot: the default-module fallback shared by acceptsRequest and the
call.APIServerResponseCache: an evicted entry is removed once (not per
alternative); one GET check in cacheResponse.APICronScheduler._execute split (_runCalls, _storeRunResult).FieldsFromMap: one lookup helper for the lower-case and simple keys.*Cache, documented, declared before their
users.Conditions and SQL generation clean-up (behavior-preserving, the same SQL):
Condition: the ordering comparisons (>, >=, <, <=) written once
(a private enum and a private base of the KeyCondition{GreaterThan,...}
classes; public API unchanged); KeyCondition.encodeValue in the base
class; ConditionParameter.matchesIn resolves its parameters once.EncodingContext.parameterValue; isReservedWord by a set;
ConditionSQLEncoder.qualifiedColumnSQL and shared ID/empty-set helpers.SQLDialect.qualifiedColumnSQL; SQLGenerator.uniqueConstraintName,
indexName, foreignKeyName and relationshipTableName (the names of
the generated indexes, constraints and relationship tables, in one place).DBSQLAdapter: the table check (_checkDBTableScheme), the relationship
SQL generators and the JOINs split into documented helpers.DBSQLMemoryAdapter: one select matcher; shared delete helpers.APIDBModule.select: directives, table and pagination helpers.*Cache and documented.Entity model clean-up (behavior-preserving, except the fixes below):
EntityResolutionRules.merge/fetchTypes and
EntityAccessRules (group flattening, condition check, reversed scan)
through shared helpers.EntityHandler: one scalar equality chain for equalsValues and
equalsValuesBasic; the comparisons through one numeric helper; a shared
cached field-type lookup for GenericEntityHandler and
ClassReflectionEntityHandler; private caches named *Cache and
documented.Transaction: identity sets beside the operation lists, for O(1)
membership checks (a bulk storeAll was O(n²)), created once a list
passes 16 operations (a smaller transaction scans its list, cheaper than
allocating the sets). Memory remove in the benchmark: 4.9× faster.storeAllFromJson (named block-size constants), the seed
JSON helpers, the provider lookups, IterableEntityRepository and
TransactionOperation.toString: duplicated blocks moved into helpers.EntityReference/EntityReferenceList, TypeInfo and Map getter
extensions, Time, EntityPagination: duplicated code merged.isEqualsIterableDeep/intersectsIterableDeep: iterate instead of
elementAt(i) (O(n²) on a non-List Iterable).Adapters clean-up (behavior-preserving):
DBObjectAdapter: the implementation copy-pasted in the memory,
directory and GCS object adapters (table schemes, transactions,
order/pagination, ID/table names and entity JSON normalization, operation
execution), with the hooks createAdapterException (each adapter keeps
its exception type), encodeBytesAsDataURL and adapterLogger.DBSQLAdapter: the schema introspection of the PostgreSQL, MySQL and
SQLite adapters (tables names and fields references caches, relationship
tables) over listTablesNamesImpl/findFieldsReferencedTablesImpl; the
30 s fields references cache TTL named; openTransactionWithPool, the
shared transaction defaults and autoInsertAfterNoopUpdate.DBEntityRepository: one ensureStored implementation (hooks
storeWithParentOperation, onEnsureStoredExisting) and
newSelectOperation; getEntityRepositoriesTables (the typo
getEntityRepositoresTables kept, @Deprecated).*Cache and documented.DBSQLAdapter: @protected openTransactionWithPool,
listTablesNamesImpl, findFieldsReferencedTablesImpl (default: throw
UnsupportedError, for an adapter that doesn't introspect a database),
listTablesNamesCached, findFieldsReferencedTablesCached,
findRelationshipTables, autoInsertAfterNoopUpdate; the static
sqlStringLiteral.DBSQLAdapter defaults for what was abstract: cancelTransaction
(true), closeTransaction (nothing), callCloseTransactionRequired
(false), cancelTransactionResultWithError (false),
throwTransactionResultWithError (true), the behavior of a pooled
driver whose transaction callback commits or rolls back (as the
PostgreSQL, MySQL and SQLite adapters): a subclass that relied on them
being abstract now inherits these.DBObjectAdapter: @protected executeOperation,
normalizeEntityJSON, normalizeObjectID, normalizeTableName,
getTableIDFieldName, applyTableOrderAndPagination,
createAdapterException, encodeBytesAsDataURL, adapterLogger,
onTransactionContextCanceled, onTransactionContextClosed.openTransaction stays typed as their context
(synchronous); cancelTransaction of the MySQL/SQLite adapters stays
bool.DBSQLMemoryAdapter keeps its own relationship-tables resolution (it
doesn't introspect a database).Deprecated (still working):
EntityHandler.graterThanValue/graterThanOrEqualValue: use
greaterThanValue/greaterThanOrEqualValue.EntityRulesResolver.cotextProviders: use contextProviders.FieldsFromMap.getFieldsKeysInMap's fieldLC/fieldSimple (ignored).DBAdapter.getEntityRepositoresTables: use
getEntityRepositoriesTables.Fixes:
APIServerResponseCache: the eviction (maxMemorySize reached) sorted
the cached values, which hold a List for an entry with compressed
alternatives: a TypeError. The entries are now sorted with all their
alternatives (by total usage and memory size).APIServerResponseCache: the eviction removed the most used entries
first (usage sorted descending): now the least used first, and among
equally used, the largest first.APIServerResponseCache: an expired entry (file modified) was removed
without subtracting its memory size from totalMemorySize.APIServerConfig: the user directory (Let's Encrypt paths) fell back to
Directory('~/'), a literal ~ directory, not the home: it's resolved
from the environment only (resolveUserDirectory): USERPROFILE then
HOME on Windows, HOME then USERPROFILE elsewhere (other platforms,
e.g. Android, used to have none).APIServerConfig.normalizeHeaderValue: the k3 path key was ignored (a
value at a 4-key config path returned the default): now used.APIServer: the JSON-decode error log printed the stack trace in place
of the invalid payload (a shadowed catch variable).IdenticalSet: grouped elements by hashCode, missing an element whose
hashCode changed: now by identityHashCode.FieldsFromMap.getFieldsKeysInMap: the given fieldLC/fieldSimple
(of one field) were used for every field: now computed per field (the
parameters are ignored and @Deprecated).Condition.parse: an escaped quote in a single-quoted string
(name == 'a\'b') failed to parse.SQLDialect.toString: lists all its fields (and a missing space).APIDBModule.select: the entities' values are HTML-escaped in its page
(they were written as is: a < or & broke the page, and could inject
markup).GenericEntityHandler.getFieldType with resolveFiledName: the entity is
asked for the resolved field name (it was asked for the key again).EntityRepositoryProvider.populateFromSource: workingPath is used to
resolve the source (and the variables) file.EntityRepositoryProvider.storeAllFromJson/storeAllFromJsonEncoded:
transaction is used, and the %name% variables are resolved (they
were ignored).EntityReference.from/EntityReferenceList.from: entityHandlerProvider
is forwarded.IterableEntityRepository.deleteEntity: transaction is forwarded.EntityAccessRules ==/hashCode: entityFields compared element-wise
(it was identity).EntityResolutionRules with allEager and allLazy both true: invalid
(validate throws ValidateEntityRulesError);
isEagerEntityType/isLazyEntityType recursed forever on it. A merge
of one side's allEager with the other's allLazy drops the conflict as
a same-flag one: both sides tolerant, both flags dropped; one tolerant, it
yields its flag; neither, it throws.DBPostgreSQLAdapter.doCountSQL: a result without a row threw a
StateError: now 0, as the SQLite and MySQL adapters.DBPostgreSQLAdapter: the ID sequence fix after an update's auto-insert
used max(id), failing for a table whose ID column isn't id: now the
table's (quoted) ID column.DBPostgreSQLAdapter, DBMySQLAdapter: the schema introspection
queries interpolated the table (and database) names into SQL string
literals unescaped: a ' in a name broke (or changed) the query. Now
escaped by DBSQLAdapter.sqlStringLiteral: the ' doubled, and for
MySQL (DBMySQLAdapter.mysqlStringLiteral; its default sql_mode makes
\ an escape in literals) also the \. The queries are otherwise the
same (MySQL's foreign keys query lost two trailing spaces).null and [] mean the same in a condition: an empty set for =~/!~, a null for ==/!=. Behavior changes, on every DB adapter, for any field (a number, a
null and [] mean the same in a condition: an empty set for =~/!~, a
null for ==/!=. Behavior changes, on every DB adapter, for any field
(a number, a text, a relationship, a path through one):
| Condition | Now | Was |
|---|---|---|
x =~ null, x =~ ? with null |
none (as x =~ []) |
IS NULL |
x !~ null, x !~ ? with null |
all (as x !~ []) |
IS NOT NULL |
x == [], x == ? with [] |
IS NULL (as x == null) |
memory: none |
x != [], x != ? with [] |
IS NOT NULL (as x != null) |
memory: all |
(SQL already encoded == [] / != [] as IS NULL / IS NOT NULL.)
An explicit null element is kept: number =~ [null] / =~ [3, null]
(also a parameter [null]) still match a null (IS NULL), except for a
relationship field:
EntityReference; a
to-many: a List of entities or an EntityReferenceList) is compared by
the IDs of its related entities, which are never null: a null element
is ignored. store =~ [null] matches none and store =~ [1, null] is
store =~ [1] (it was IS NULL, "without a related entity"). Use
store == null / store != null (or == [] / != []) for "without" /
"with" a related entity. (KeyCondition.isRelationshipKey.)
A relationship field is resolved by its declared type, on memory and SQL
alike (KeyCondition.isRelationshipType): an EntityReference, an
EntityReferenceList, an entity type, or a List of one (not a List
of enums or of other classes); on SQL not by the DB's foreign keys (a
to-one may have none, e.g. User.photo, a String ID).==/!= with a single-element list is with its element, as SQL
already did: store == [7] is store == 7 (memory matched none). Not
for a list or binary field (a List<String>, a Uint8List): it's
compared with the list as a whole (tags == ["a"] doesn't match
["a", "b"]).EntityReference (or an EntityReferenceList) as a parameter is
compared by its ID(s), with =~/!~ (also of a path ending in a
reference: owner.userInfo =~ ?) and with ==/!= (a reference, or a
reference list with up to 1 ID: as == [1, 3], more IDs aren't a
single value). They matched none (SQL: IS NULL).== [] is IS NULL also for a list or binary field (an empty
List<String>, an empty Uint8List, is not == []). A missing named
parameter (x =~ ?:a without a) is a null: an empty set.null for other fields, as a SQL outer
join; it failed.==/!= with a parameter list of 2 or more values isn't a single value:
SQL fails (unchanged), and memory matches none (it compared any element).roles.type == ?
with [RoleType.admin], photo == ? with [photo]) was compared as its
text (toString), not its name / ID.null element (['a', null]) was
== null (any element null); a list is a value, not null.orderByID: null), lost the ID order its
JOIN had: its IDs subquery (id IN ( SELECT ... )) may not keep the
table order (PostgreSQL's hash semi-join). It's ordered by ID again (as
memory).store == ? with [] (a to-one entity) failed resolving [] as an
entity's ID (memory). A list as a single entity's value: [] is null,
[x] is x.String/int ID column (a
to-one without a foreign key, photo == ? with a Photo) was compared
as its text (toString), not its ID.Photo in an object storage) was resolved as null: photo == null
matched every user, and photo =~ [id] none. Its ID is kept, as a SQL
column. And a String ID for an entity-typed key (photo =~ ["<id>"])
failed resolving it as an entity.KeyConditionINBase.valuesAsList(null) is [] (it was [null]).Tests:
!(...): =~/!~ with
[], null, [null], an ID, [id], [id1, id2], [id, null] (as
literals, and as ? and ?:name parameters), and entities as parameters;
==/!= with null, [] (literals and parameters) and IDs; Set
parameters, EntityReference/EntityReferenceList parameters, == [id],
and &&/|| combinations. Over a to-many EntityReferenceList
(Address.closedStores), a to-many List (Address.stores), a to-one
entity (Store.owner), a to-one EntityReference (User.userInfo), and
a to-one with a String ID without a foreign key (User.photo); a path
ending in a reference (owner.userInfo) with references as parameters;
and a to-many's countByQuery and limit/offset pages.=~ null, !~ null,
=~ [null], == [], != [], == [v], =~ [v] and their parameters,
over text, numbers, a to-one's and a to-many's fields.==/!= with a list on a list or binary field (as a whole).Known differences (unchanged):
closedStores =~ [1] && stores =~ [2]) fail on SQL (one relationship table is joined): see
1.33.0.String ID for an integer-ID relationship (owner == ? with '1')
matches on SQL (the DB converts it), and not in memory ('1' isn't
1).Condition.matchesEntity)
doesn't unpack an EntityReferenceList field to its IDs.photo.data, a
Photo in an object storage) isn't supported (an error, on every
adapter).The condition operators give the same results on every DB adapter (memory, SQLite, PostgreSQL, MySQL), except the known differences below. Behavior ch
The condition operators give the same results on every DB adapter (memory, SQLite, PostgreSQL, MySQL), except the known differences below. Behavior changes:
JOINs are in a subquery,
id IN ( SELECT id ... JOIN ... WHERE cond ). So countByQuery, limit
and pagination also count each entity once.null through a relationship (owner.level == null, =~ null, =~ [x, null], a null parameter) uses an outer join:
a missing related entity (a null owner, or no rows of a to-many
relationship) is a null field, so it matches, as in memory, and as !=
and !~ already did (since 1.32.0). It was an inner join: == null
missed it while != x matched it.owner.roles.type) joined its relationship table to the main table
(user__roles__rel.role = store.id), not to the to-one's
(user__roles__rel.user = user.id): it matched the wrong rows.DELETE through a relationship (to-one, deleteByQuery( 'owner.email == ?'), or to-many) was invalid SQL (DELETE ... JOIN): its
JOINs are now in the subquery of the IDs too (MySQL: in a derived
table, as a DELETE can't select from its own table).String compared with a Decimal field
(roles.value == "1.5") was resolved as null (IS NULL).nulls (as a SQL outer join):
roles.type == null and roles.type =~ [null] match a user without
roles, and roles.type != null and !~ [null] don't (they did).
!= "admin" and !~ ["admin"] still match it.roles.type == ? with a list parameter ([['guest']], [[null]])
compares each related entity's value with it, as a field's
number == ? does (it matched nothing).KeyCondition.hasToManyInKeyPath, by the
declared field types): an empty list field (List<String>,
Uint8List) is still a value, not a null.deleteByQuery through a to-many relationship, of a table
without to-one references (address: stores.name == ?), deleted
nothing (its relationships weren't resolved, as a select does).Decimal field compares by value: value == 1.5,
value =~ [1.5, 3] and value == ? with a Decimal matched nothing (an
entity's map keeps it as a String; EntityHandler.equalsValues* now
compare a DynamicNumber with a number or a plain decimal String by
value).Known differences (SQL):
2 conditions over the same to-many relationship
(roles.type == "admin" && roles.type == "guest") are, in memory, each
any related entity, and on SQL the same joined row (so this matches
nothing on SQL). Prefer one condition (roles.type =~ [...]), or !(...)
(a subquery each).
2 to-many relationships to the same table in one query
(address.stores and address.closedStores, both to store) share one
JOIN: stores.name == "a" && closedStores.name == "b" compares both
with the same row.
A path that reaches the main table again (store: owner.address.stores .name) has a duplicated table alias: an error on SQL.
Tests: 'Operators: the same results on every adapter', on all the DB
adapters: ~240 conditions, each also checked through !(...) (its
complement), as a sorted list (an entity selected twice fails), and
x == null / x != null checked to split the entities. ==, !=,
=~, !~, >, >=, <, <=, !, &&, || and groups; a literal, a
single value, a list, null, [null], a list with a null, [], a
regexp (/.../, /.../i), and the parameters ?, ?:name, /?/ and
/?:name/ (also null, [] and a list with ==). Over text, a nullable
number, a date, an enum, a bool, a nullable decimal (also as a String),
a nullable to-one relationship (by its ID, a field of it,
owner.address.city 2 relationships away), a to-many one (roles, also
a user without any), and a to-many through a to-one (owner.roles). And
countByQuery, limit/offset, and deleteByQuery through a to-one
and a to-many relationship.
Fix (1.32.0 regression): on SQL an empty IN list was written as a null: x =~ ? with [] was x IS NULL (matching the rows with a NULL, instead of none),
IN list was written as a
null: x =~ ? with [] was x IS NULL (matching the rows with a
NULL, instead of none), and x !~ ? with [] was x IS NOT NULL
(excluding them). Now =~ [] is FALSE and !~ [] is TRUE, as when
matched in memory. (1.31.0 wrote x IN ( null ) and x NOT IN ( null ),
both matching nothing.) Also a literal x =~ [] (was x IS NULL).
null is still IS NULL: x =~ null (a parameter's null) and
x =~ [null]. resolveValueToType(null, valueAsList: true) is now
[null] (was [], indistinguishable from an empty list).deleteByQuery('x !~ ?', [[]]) now deletes every row (as
matched in memory); 1.31.0's NOT IN ( null ) deleted none.bones_api_condition_not_test.dart), and
on every SQLAdapter (memory, SQLite, PostgreSQL, MySQL): =~ [],
!~ [], !(x =~ []), a reference column (owner =~ []), through a
relationship (owner.email =~ [], !~ [] and !( ... ): the JOIN and
the subquery) and a to-many one (roles =~ [], !(roles =~ [])), in an
||, and a null parameter.Breaking (query results): the negated operators also match a null field (or a missing related entity), the same in every SQLAdapter:
null
field (or a missing related entity), the same in every SQLAdapter:
x != v is ( x != v OR x IS NULL ), x !~ [...] is
( x NOT IN (...) OR x IS NULL ) and x !~ /re/ is
( NOT <match> OR x IS NULL ). Before, a null column didn't match
them in SQL, but did in DBSQLMemoryAdapter. x != null is still
IS NOT NULL.x !~ /re/ (1.31.0: matched neither =~ nor !~) now matches a null.JOINs are outer (KeyCondition.isNegated).roles.type != "admin" matches the
users with some role that isn't admin, or without roles (memory: before,
!= matched any user, and !~ [...] "none in").!, of a parenthesized condition or a single comparison:
!(a == 1 || b == 2), !name =~ /^x/, nested !(a && !(b)).
!a == 1 && b == 2 is (!(a == 1)) && b == 2.
ConditionNOT: the complement of its condition, also when it's unknown
(a null field or a missing related entity). Through a to-many
relationship, "none matches": !(roles.type == "admin") matches the users
without an admin role.NOT COALESCE(( cond ), FALSE); through
other tables, id NOT IN ( SELECT id FROM ... WHERE cond ), with the
subquery's own JOINs (MySQL, SQLDialect.subqueryAsDerivedTable: in a
derived table, also for a DELETE).EncodingContext.sub: a sub-context sharing its parent's parameters and
placeholders. ConditionSQLEncoder.subqueryFromSQL: the FROM clause of
a subquery, from the adapter (DBSQLAdapter._fromSQL, also used by
_generateSQLFrom).ConditionEncoder.encodeConditionNOT: not abstract (throws a
ConditionEncodingError), for the existing subclasses.== over a to-many relationship compared the whole list
with the value (roles.type == "admin" matched nothing); now any of them,
as in SQL.>, >=, <, <=) with a null field
threw an ArgumentError; now it doesn't match, as in SQL. Over a to-many
relationship (a list) it also threw; now any element, as in SQL.null in an IN list is as IS NULL, as in memory: SQL
x =~ [1, null] is ( x IN (1) OR x IS NULL ) (IN (1, NULL) didn't
match a NULL), and x !~ [1, null] is ( x NOT IN (1) AND x IS NOT NULL )
(NOT IN (1, NULL) matches nothing). Also of a parameter's list.&&, ||) are encoded one after another:
one resolved asynchronously (e.g. a TableScheme not cached yet) could be
written after the next one.NULL) has no OR id IS NULL, and ! of an ID
condition is a plain NOT ( ... ).null".bones_api_condition_not_test.dart (parsing, round trips,
matching, the SQL), and on every SQLAdapter (memory, SQLite, PostgreSQL,
MySQL): ! over fields, to-one and to-many relationships, nested, with
parameters, a DELETE, the complement of each condition, and the nulls
of the negated operators.New: regular expression conditions in queries: field =~ /pattern/flags (matches) and field !~ /pattern/flags (does not match), e.g. email =~ /@corp\.c
field =~ /pattern/flags
(matches) and field !~ /pattern/flags (does not match), e.g.
email =~ /@corp\.com$/i. A value between / makes =~/!~ a regexp;
any other value is still IN/NOT IN.
/?/, /?#1/ or /?:name/, whose value
is a String or a RegExp (its case flag is used).i. In a pattern, \/ is a /.KeyConditionRegExp: the same matching in every SQLAdapter
(PostgreSQL's semantics): anywhere in the value, . also matches a line
terminator, ^/$ only at the start/end of the whole value. A non-text
field is matched by its text (an enum by its name, an entity reference by
its ID); a null field matches neither =~ nor !~. The pattern is
always a SQL parameter.SQLDialect.regExpSyntax (SQLRegExpSyntax): PostgreSQL ~/~* (a
non-text column cast to TEXT), with \b/\B rewritten as \y/\Y
(a backspace/backslash in PostgreSQL); MySQL 8.0.4+
REGEXP_LIKE(..., 'cn'/'in') with $ rewritten as \z (ICU's $ also
matches before a final line terminator); SQLite a
regexp(pattern, value, flags) function, registered by DBSQLiteAdapter
on its connection (also value REGEXP pattern); and DBSQLMemoryAdapter
matches the condition in memory.ConditionEncoder.encodeKeyConditionRegExp: not abstract (throws a
ConditionEncodingError), for the existing subclasses.RegExp as an ==/!=/IN parameter (field =~ ?) is now an
ArgumentError (it was compared as its toString): a regexp parameter is
written between / (field =~ /?/).?) of the same field shared one SQL
placeholder, bound to the 1st value: id > ? && id < ? with [1, 10] was
id > 1 AND id < 1 (also =~ ?/!~ ? lists). resolveParameterValue
reuses a placeholder only for the same value (otherwise a new key,
ConditionEncoder.newParameterKey), and ConditionParameter.getValue
resolves a positional ? by its position when there are positional
parameters (DBSQLMemoryAdapter also passes them when deleting).selectFirstByQuery (EntitySource, EntityRepository) dropped the
positionalParameters (not passed to selectByQuery).bones_api_condition_regexp_test.dart (parsing, encoding,
matching and the SQL of each dialect), and the regexp conditions on every
SQLAdapter (memory, SQLite, PostgreSQL, MySQL).Fix: a transaction opened before Transaction.execute/executeOrError (by its operations, e.g. store(o, transaction: t) then t.execute(...)) hung the ca
Transaction.execute/executeOrError
(by its operations, e.g. store(o, transaction: t) then t.execute(...))
hung the caller when its adapter's COMMIT failed asynchronously
(PostgreSQL, MySQL, SQLite). The adapter sets the transactionResult (its
COMMIT) when it opens the transaction, in the caller's error Zone, and
the commit listened to it inside the transaction's error Zone: the error
was only reported as uncaught in the caller's zone, and execute never
completed. The same for an aborted transaction (a block error) whose
rollback fails (as the adapters do): execute and its resultFuture never
completed. And for commit called from another error zone.
Transaction.transactionResult is now a getter/setter that records the
Zone where it's set: a Future is listened to inside that zone, and
its value or error forwarded to the zone of the commit (or abort).DBSQLMemoryAdapter.resolveTransactionResult checked
is! TransactionAbortedError on the Future of
super.resolveTransactionResult (always true): the context of an aborted
transaction, already cancelled (rolled back and disposed), was also
"consolidated" and disposed a 2nd time (miscounting
poolDisposedElementsCount). The check is now on the transaction's result.COMMIT of a transaction opened before execute /
executeOrError (memory, SQLite, PostgreSQL, MySQL), with commit called
directly, and its rollback; a transactionResult set in another error zone
(execute, concurrent commits, abort); the memory adapter's aborted and
committed resolveTransactionResult.PR: #200
Fix: an asynchronous commit failure in Transaction.execute/executeOrError hung the caller; it now reaches the caller, and the transaction is finished
Fix: an asynchronous commit failure in Transaction.execute/executeOrError
hung the caller; it now reaches the caller, and the transaction is finished
(isFinished, resultFuture). The commit runs inside the transaction's
error Zone, and Dart never delivers a Future's error across error zones:
it was only reported as an uncaught error ([Transaction ERROR]), and
execute never completed. This covers both a failing closer and an adapter's
failing COMMIT (its transactionResult: PostgreSQL, MySQL, SQLite), and
also Transaction.executeBlock and the repository operations that use it.
A synchronous result (or error) is still returned (or thrown) synchronously,
and a commit error is still never passed to onError.
Transaction: a failed commit (sync or async, also from commit called
directly) finishes the transaction as aborted, with a
TransactionAbortedError (reason: 'Commit failed') wrapping the commit
error (also in error): isCommitting is no longer left true, the
transactionFuture (if not completed yet) and resultFuture complete, and
executingTransaction is cleared. A concurrent 2nd commit then returns
null (as for an aborted transaction).
Transaction: the resultFuture of an aborted transaction now completes
also when the adapter fails its transactionResult (the rollback of the
PostgreSQL/MySQL/SQLite adapters): it never completed.
Transaction: an adapter's COMMIT error notified to an auto-commit
transaction no longer re-commits it (which only returned the failing
transactionResult again, as an unlistened, uncaught error).
Transaction.toString: no infinite recursion when its abortError wraps an
error that describes the transaction (a failed COMMIT's
DBAdapterException).
New tests: an asynchronous and a synchronous failing commit (execute,
executeOrError, executeBlock, commit, auto-commit), a failing
transactionResult (with and without a block error), the transactions after
a failed commit, an asynchronous successful commit, and synchronous results.
With a real failing COMMIT: SQLite (a deferred foreign key), PostgreSQL (a
deferred unique constraint) and MySQL (the connection closed before the
COMMIT).
Dependencies:
swiss_knife to ^3.4.2 (bug fixes; of them bones_api is only
affected by MimeType.parse: mp3 is audio/mpeg, the jpeg/txt
aliases keep the charset, and '/foo' is invalid).mercury_client to ^2.4.1: a String request body is encoded
with the Content-Type charset (APIModuleProxyHttpCaller sends JSON,
still UTF-8), and on the VM binary responses (images other than SVG,
audio, video, fonts) stay bytes even with a charset.args_simple to ^1.3.0 (-- ends the options, --key=value is
split, -D/-P properties only with =/:): APIServerConfig.fromArgs
reads only named options and flags, unaffected.graph_explorer to ^1.2.0 and ascii_art_tree to ^1.1.3: the
SQLBuilder hierarchical order (toHierarchicalOrder) uses a stable
sortByInputDependency (tied SQLs keep their order; the order is still
validated, with its fallback).dart_spawner to ^1.3.0: the CLI's spawned isolates
(spawnDartScript, used by bones_api serve/console) are no longer
reported as exited (exitCode 0) when they don't answer a ping within
2s with the VM service enabled.build_runner kept at ^2.16.2: 2.17 requires build 5, and
reflection_factory 2.13 requires build ^4.Adapted to mysql1 0.22: Result (was Results, deprecated), ResultRow.toColumnMap()/row[name] (was the deprecated fields), a non-null affectedRows. DBMy…
Transaction.execute/executeOrError: a commit that fails synchronously
(e.g. a synchronous closer, as with the memory/SQLite adapters) now reaches
the caller. It was passed to the onError of the internal asyncTry and
swallowed: the block's result was returned as if the commit succeeded
(async_extension 1.2.24 fix: an onFinally error replaces the result, as
in a Dart try/finally). A commit error is never passed to onError.
DBMySQLAdapter:
mysql1 0.22: Result (was Results, deprecated),
ResultRow.toColumnMap()/row[name] (was the deprecated fields), a
non-null affectedRows. DBMySqlConnectionWrapper.query/queryMulti
return Future<Result> (the same type: Results is a typedef of it).MySqlException (a
timeout, a socket error) is marked closed, so the pool discards it:
mysql1 (since 0.21) closes it in that case, without exposing it.mysql1 0.21 query parameters are written into the SQL as
literals (no prepared statements), and a DateTime is sent with its
fractional seconds, which the server rounds to the column precision
(TIMESTAMP: whole seconds). It used to be truncated.Time: the comparison operators are typed bool, and +/- Time
(they were dynamic).
lints 6: explicit types (strict_top_level_inference) and null-aware
elements (use_null_aware_elements).
decodeQueryStringParameters: split each pair on the first = only
(a=b=c decoded to {'a': 'b'}; now {'a': 'b=c'}), as swiss_knife
3.4.1 fixed in its decodeQueryString.
APIModuleProxyHttpCaller (responses): with swiss_knife 3.4.1 an
application/xhtml+xml response is a MimeType with isXHTML (it parsed as
application/xhtml): it's read as text, never decoded as JSON.
New tests for a failing commit in Transaction.execute (sync and async),
and for a MySQL connection closed by mysql1.
GitHub Actions: dart-lang/setup-dart@v1.8.1.
test_vm (unit) job:
bones_api_utils_weaklist_test opened a VM service connection per GC
and never disposed them. The test_vm jobs' timeout is now 12 min.Dependencies:
async_extension to ^1.2.24.swiss_knife to ^3.4.1 (of its behaviour fixes only the
MimeType one above affects bones_api).mysql1 to ^0.22.0.gcloud to ^0.9.0.archive to ^4.6.0 (only its encoder is used: its decoders'
new error contract doesn't apply).args_simple to ^1.2.1.ascii_art_tree to ^1.1.1.dart_spawner to ^1.2.1.graph_explorer to ^1.1.2.lints to ^6.1.0.build_runner kept at ^2.16.2: 2.17 requires build 5, and
reflection_factory 2.13 requires build ^4.Updated async_events to ^1.4.0.
async_events to ^1.4.0.statistics to ^1.2.2.data_serializer to ^1.2.3.mercury_client to ^2.4.0.resource_portable to ^3.1.4.shelf_letsencrypt to ^2.0.8: since 2.0.4 (owner-only private
keys) it imported dart:ffi, failing the web compilation of code
importing bones_api_server.dart (like the browser tests).shelf to ^1.4.3.shelf_static to ^1.2.0.meta to ^1.19.0.petitparser to ^7.1.0.mime to ^2.1.0.postgres to ^3.5.20.sqlite3 to ^3.8.0.yaml to ^3.1.4.crypto to ^3.0.8.archive to ^4.5.2.googleapis_auth to ^2.3.5._discoveryapis_commons to ^1.0.8.web to ^1.1.2.test to ^1.33.0.coverage to ^1.16.0.vm_service to ^15.4.0.A route refused by an APIRouteAuthenticatedRule (or a subclass, like APIRoutePermissionTypeRule) because the request is not authenticated (no credenti
APIRouteAuthenticatedRule (or a subclass, like
APIRoutePermissionTypeRule) because the request is not authenticated (no
credential, or an invalid or expired token) now requires authentication:
401 with WWW-Authenticate: Bearer realm="API" and the body
UNAUTHORIZED: Not authenticated. It was a 403, the same as an
authenticated request refused, so a client could not tell a lost login from
a refused access. Bearer: no browser prompt (unlike Basic).
403
(UNAUTHORIZED: Rules issues ...).APIRoot.call / doCall of such a route now returns
the message as the APIResponse.error (was the payload), with
requiresAuthentication and authenticationType Bearer.APIRouteHandler.isUnauthenticatedRefusal.APIRouteAuthenticatedRule.isAuthenticated.APIServer: a blank authenticationRealm now defaults to API (it
checked the authentication type instead).Fix: generateTables failing the initialization (PostgreSQL: 42703: column "..." does not exist) for an existing table without a column added to the en
generateTables failing the initialization (PostgreSQL:
42703: column "..." does not exist) for an existing table without a
column added to the entity later, with an index over it: the
CREATE TABLE IF NOT EXISTS doesn't add the column, and its
CREATE INDEX failed.
DBSQLAdapter.populateTables: a failing CREATE INDEX over columns
missing in its (existing) table is skipped, with a warning, when
checkTables is pending: it then reports the missing columns (and
indexes), with the SQLs to fix them. Without checkTables: an error
naming the missing columns.DBSQLAdapter.extractCreateIndexInSQL.Brotli (br) response compression, using brotli_fast (pure Dart), next to GZip:
br) response compression, using brotli_fast (pure Dart),
next to GZip:
Accept-Encoding
(q-values, q=0 and * respected): the best one in
APIServerConfig.compressionEncodings (default [br, gzip]; on a tie
the server order wins).compressResponse), and static files
by createCompressionMiddleware. HEAD responses only get the Vary.Isolate
(APIBrotli.encodeAsync, APIGZip.encodeAsync), not blocking the
server's event loop: from about 1.5ms of encoding
(APIBrotli.isolateMinLength: 32 KB at quality 9+, 64 KB at 6+,
256 KB below; APIGZip.isolateMinLength: 192 KB at level 9, 384 KB at
6+, 768 KB below). Concurrent requests for the same cached file share
one encoding.compressionIsolatePool (shared by Brotli and
GZip), a new APIIsolatePool (bones_api_utils_isolate_pool.dart):
long-lived workers (the Brotli encoder tables are initialized once per
worker, ~2.5ms saved per encoding at quality 9), at most
maxIsolates (default: half of the processors divided by the server
workers, 2..4, APIIsolatePool.defaultMaxIsolatesFor; set by each
APIServerWorker, configureCompressionIsolatePool) with the other
encodings queued, and workers idle for idleTimeout (default 30s) exit.
With 30 concurrent 256 KB q9 encodings, the event loop stalls at most
~2ms (~60-100ms spawning an isolate per encoding, ~400ms encoding
inline); with 30 concurrent 1 MB GZip level 4, ~2ms (~67ms inline).gzip and br variants of
each file (a gzip file is transcoded to br when preferred).deflate) is only served
to the requests accepting it.9+, GZip 7) is only used
for the responses that will be cached (APIServerResponseCache.isCacheable,
which now checks the uncompressed length, uncompressedContentLength).Accept-Encoding: gzip in the request. Now it's compressed only
with an accepted encoding.
APIServer.resolveBody/resolveBodySync called without the new
compression parameter keep the old behavior (GZip).APIResponse.error) was GZip-compressed
without a Content-Encoding header. Now it's compressed once, with it.206 Partial Content, a
Content-Range, 204, 304) are not compressed (Range requests of
static files).404 body is served with its Content-Encoding,
and only to requests accepting it.APIResponse has a failing Future payload never
got a response (it hung): the error of a Future created in the
APIRoot.call error Zone was not delivered to APIServer.resolveBody
(only logged as an uncaught error). Now it's a 500 error response.
APIResponse.payloadZone/runInPayloadZone: the Zone where a
Future payload was created, to listen to it.Accept-Encoding is parsed (it was a substring match: gzip;q=0 was
taken as accepting gzip). x-gzip is taken as gzip.Vary: Accept-Encoding is sent with responses that can be compressed.APIServerConfig:
compressionEncodings (--compression-encodings br,gzip,
server.compression_encodings): an empty list, false (or none,
off, no) disables the response compression (also the HttpServer.autoCompress GZip, which is
disabled when gzip is not listed). Unknown encodings are ignored
(logged); if none is known, the defaults are used.brotliQuality (--brotli-quality 5, server.brotli_quality;
0..11, default 5). Cached static files use at least 9
(and GZip level 7, as before).Content-Encoding: br are decompressed
(decompressPayload), with maxPayloadLength enforced while decoding
(APIServer.defaultMaxBrotliPayloadLength, 64 MB, if not defined: a few
bytes of Brotli can decode to GBs). Decoded in the compressionIsolatePool
(APIBrotli.decodeAsync), not blocking the event loop.bones_api_utils_compression.dart (exported by
bones_api_server.dart): APIContentEncoding, APIResponseCompression,
APIBrotli, APIGZip, compressionIsolatePool,
configureCompressionIsolatePool, parseAcceptEncoding,
resolveContentEncoding, normalizeContentCoding, compressResponse,
createCompressionMiddleware, mergeVaryHeader. And
bones_api_utils_isolate_pool.dart: APIIsolatePool.>=3.13.5 (required by brotli_fast).
prefer_initializing_formals applied (private named parameters: the
public parameter names are unchanged).APICronStoreDB.buildRepositories returns List (was List >): it can be added to a provider's list of DBSQLEntityRepository with no cast, and still to a
APICronStoreDB.buildRepositories returns List<DBSQLEntityRepository>
(was List<EntityRepository<Object>>): it can be added to a provider's
list of DBSQLEntityRepository with no cast, and still to a list of
EntityRepository (lists are covariant).Cron time zones: the full timezone database is used (data/latest_all.dart). The default one (data/latest.dart, loaded until now) has no links/aliases,
timezone database is used
(data/latest_all.dart). The default one (data/latest.dart, loaded
until now) has no links/aliases, so a job with GMT, Europe/Oslo or
Atlantic/Reykjavik was rejected as an unknown time zone.
Cron: scheduled calls of API routes (APICronScheduler), see the README section "Cron: scheduled routes".
Cron: scheduled calls of API routes (APICronScheduler), see the README
section "Cron: scheduled routes".
APICronJob: a route called on a 5 fields cron expression
(CronExpression), in an IANA time zone (daylight saving aware).
@APICron on a reflected route method, by
APIRoot.loadCronJobs(), or by the cron.jobs config (which can also
override or disable a job of the code by name). An invalid config entry,
or one for an unknown job, is skipped (logged): the other jobs run.timeout, maxLateness and leaseDuration must be positive, and
retry.attempts at least 1. The time zone is resolved when the scheduler
starts (a job with an unknown one is skipped).@APICron('30 23 * * *', ...) takes a cron expression, and
@APICron.at(minute: '30', hour: '23', ...) the schedule by its fields
(minute, hour, dayOfMonth, month, dayOfWeek; default *).
Both accept name, timeZone, parameters, enabled, maxCalls,
attempts, timeout, maxLateness, leaseDuration and retry.APIRoot.call, with an internal system authentication
(APIAuthentication.system, passes APIRouteAuthenticatedRule), the
requester source internal and the headers X-Cron-Job,
X-Cron-Scheduled-At, X-Cron-Run and X-Cron-Attempt.X-Continuation
(APIResponse.continuation), the route is called again with the
parameter continuation, up to maxCalls per run.APICronRetry): exponential backoff (default 3 attempts: 30s,
2m), honoring Retry-After (capped by maxDelay). UNAUTHORIZED,
BAD_REQUEST and NOT_FOUND are not retried.timeout: a slower call is logged, but waited for (a route can't be
cancelled): never retried or overlapped while running.APICronScheduler: started when the config cron.enabled (or
cron: true) is true, once APIRoot and its modules are successfully
initialized (cronSchedulerReady); not in auxiliary worker isolates (one
per process). A cron config of another type is ignored (disabled), and
the scheduler never fails the API initialization. Stopped by
APIRoot.close: a running job stops after its current call (left
resumable, its lock released; see waitRunning), and no other check or
runNow runs until started again.
APICronStore
(unique job + scheduled time), also among several instances.maxLateness (default 1 day); older ones are recorded as skipped
(at most the 100 after the last run, per check).runNow(job), selectRuns(job), and history pruning (history_days,
default 90; never a running run, nor a run still in its job's
maxLateness window, that would run again).APICron.APICronStore: APICronStoreMemory (single instance) and
APICronStoreDB (tables api_cron_job_run and api_cron_job_lock; add
APICronStoreDB.buildRepositories(adapter) to the repositories of the
API). APIRoot.loadCronStore() uses the DB store when its repositories
are registered (otherwise the in-memory store, with a warning: single
instance only).APIModule.cronJobs and addCronJob (a duplicated job name in a module
is an error; an @APICron job is only added with its route). Note: modules with generated
reflection (routes.anyFrom(reflection)) need the reflection code
regenerated (build_runner) to include the new cronJobs field.timezone.Fix: a route method with an APIRequest as its only positional parameter was always registered as a raw handler, invoked with the APIRequest only. Its
APIRequest as its only positional parameter
was always registered as a raw handler, invoked with the APIRequest only.
Its optional and named parameters were never passed: optional ones were
silently null (or their default value), with no error or warning, and a
required named parameter failed every call (NoSuchMethodError).
APIRequest (see MethodReflectionExtension.isRawAPIRequestHandler).APIRequest plus optional or named parameters is routed
like any other API method: its parameters are resolved from the request
(parameters, JSON payload, path parts), with the APIRequest injected.
The injected APIRequest doesn't consume a path part.APICredential positional
parameter) are no longer needed, and still work.APIRequest, APICredential or APIAuthentication parameter name is no
longer counted as a parameter key).PR: #188
Static files: virtual file and directory aliases, defined by .aliases files inside the served directory tree (see the README section "Static files: .a
Static files: virtual file and directory aliases, defined by .aliases files
inside the served directory tree (see the README section
"Static files: .aliases").
StaticFilesAliases: a middleware in front of the static files handler.
It rewrites a request without a physical file to its alias target: exact
file alias, directory prefix, pattern (*, ?, **/) or overlay (./).
RegExp)..aliases directory (.. and absolute paths
are rejected), and symlinks follow the existing root containment./ is redirected with a 302..aliases files (of existing directories), missing directories and
resolutions are cached and re-checked every 10s, in bounded caches
(evicting the oldest entries). A cached resolution costs ~130ns, and a
path without any applicable alias ~300ns.Cache-Control, cross-origin and cookieless
headers), sharing the physical path's cached response..aliases files are never served: they respond 404. Before, a
.aliases file in a served directory was served like any other file.DBObjectDirectoryAdapter: selectAll and selectByIDs read all
their object files in parallel, so concurrent selects of a large table
opened thousands of files at once (not a leak, but an unbounded burst),
failing with Too many open files (errno = 24) under a low limit (macOS
default: 256). The reads are now limited by a FileLimited semaphore (50),
in batches of that limit per select.
DBObjectDirectoryAdapter.fileLimited: the shared semaphore, tunable with
setSemaphoreLimit (FileLimited is also exported by
bones_api_db_directory.dart).existsSync check before the read: a missing file reads as null),
and concurrent deletes of the same object delete it once (the others
delete nothing, without an exception).FileLimited.readAsString: like File.readAsString, under the semaphore.FileLimited: an operation in flight while setSemaphoreLimit
replaced the semaphore released the new one (raising it above its limit);
it now releases the semaphore it acquired.Pull requests: #186, #187.
Faster entity decoding from JSON, e.g. a module proxy decoding a response with many entities in the browser. The decoded entities are the same.
Faster entity decoding from JSON, e.g. a module proxy decoding a response with many entities in the browser. The decoded entities are the same.
reflection_factory: ^2.12.2 → ^2.13.0, with reflection regenerated:
constructors called directly with resolved values, map shapes reused, and
JSON objects no longer copied before decoding.Json.defaultFieldValueResolver: a String, bool or int value of a
field of that type is returned without runtimeType (costly compiled to
JavaScript), and isListEntity is only checked for an Iterable.EntityHandler.createFromMap: when the values are resolved synchronously,
the instantiation steps are called directly, without a closure per entity.ClassReflectionEntityHandler.getID: reads the ID field kept once,
instead of looking it up by name for each entity.benchmark/db_users/bin/json_decode_benchmark.dart, decoding
a JSON list of users (each with an address and 2 roles) as a response of
user/list, by layers and as a module proxy does (decodeJson +
resolveResponse). It runs on the VM, and compiled to JavaScript and Wasm.
A list of 1000 users through the proxy, µs per user:
-O1 --minify): 27.4 → 11.8 (−57%).-O2 --minify): 6.5 → 3.7 (−42%).reflection_factory: ^2.12.0 → ^2.12.2, with reflection regenerated (2.12.1: fixes a build-time regression of 2.12.0; 2.12.2: faster ClassProxy generat
reflection_factory: ^2.12.0 → ^2.12.2, with reflection regenerated
(2.12.1: fixes a build-time regression of 2.12.0; 2.12.2: faster
ClassProxy generation).reflection_factory: ^2.11.0 → ^2.12.0, with reflection regenerated: its generated constructor invokers call constructors directly instead of through F
reflection_factory: ^2.11.0 → ^2.12.0, with reflection regenerated:
its generated constructor invokers call constructors directly instead
of through Function.apply.
Optimization: ClassReflectionEntityHandler.instantiateFromMapImpl
resolves the values up front, with the same
Json.defaultFieldValueResolver (by each field's type), and calls
createInstanceFromMap(valuesResolved: true): constructors are called
straight from the map, without reflection_factory's MethodInvocation
building. The resolver path is used as before when the map has a key that
isn't a field (e.g. a constructor parameter with another name), when a
constructor parameter named as a field has another type than the field,
or when a value doesn't resolve synchronously (or resolves to null).
user/list +17–20%,
user/byState +12–18%, user/byId +12–15%.Fix: Json.defaultFieldValueResolver resolves:
EntityReferenceList<T> field or parameter from a list of IDs,
entities or entities maps (as an EntityReferenceList<T>). It gave a
List, so instantiating with such a value failed.Set<T> field or parameter from a list or a set (as a Set<T>, with
its elements resolved as a List<T>). It gave a List.TypeInfo.toEntityReferenceList: also from entities in an untyped
List (e.g. List<dynamic>).DBSQLAdapter.executedSQLsCount: the number of SQL statements (round trips) executed for entity and relationship operations. The shared DB tests pin th
DBSQLAdapter.executedSQLsCount: the number of SQL statements (round
trips) executed for entity and relationship operations. The shared DB
tests pin the statements of each operation (insert, selectByID,
selectByQuery, length, update, deleteByID) on every SQL adapter, so
an optimization or a regression changes them visibly.
benchmark/db_users: --repeat N (the median of N rounds per
operation, with their range). The users registered by each register
round are removed after it, so every round (and update/remove after
it) runs on the same table sizes. The benchmark's User.create() no
longer hashes an empty password for every user loaded, and
Address.state is indexed (user/byState).Optimization: storing the relationships of a just inserted entity, with
an ID generated by that insert, no longer runs a DELETE of its other
relationship rows (it has none) on the adapters whose generated IDs are
never reused (DBSQLAdapter.generatedIDsHaveNoRelationships: PostgreSQL,
MySQL): one statement (round trip) less per relationship field on every
insert. An insert of a user with an address (2 relationship fields) and
2 roles: from 9 statements to 6.
TransactionOperationStoreRelationship.newEntity.ON DELETE CASCADE). A relationship table without it, on a
database that can reuse an ID (MySQL before 8.0 resets AUTO_INCREMENT
on restart; a reset PostgreSQL sequence), could give a new entity a
deleted one's relationship rows: override
generatedIDsHaveNoRelationships as false then.Optimization: generated relationship tables no longer get an index on the
source column (<table>__<source>__idx): it's the first column of their
UNIQUE (source, target) constraint's index, which already serves
lookups by it (one index less to update on every write). Existing tables
keep it; checkTables doesn't require it (PostgreSQL resolves an index
by its first column).
Optimization: Transaction no longer captures a StackTrace on every
commit (it only located an unexpected error): only with asserts enabled
(debug, tests). user/count of the users benchmark: +62% (memory), +32%
(SQLite).
Optimization: SQL.sqlPositional (SQLite) is computed once per SQL text
(with the default placeholder), not with a regular expression per
statement.
Optimization: DBPostgreSQLAdapter resolves the column names of a result
once for all its rows.
Optimization: @pragma('vm:prefer-inline') on small functions of hot
paths that a CPU profile showed not inlined: ZoneField.get (the
executing Transaction), the row layout match of createFromMap, the
digits of parseDateTimeISOUTCFast (a top-level function, not a closure)
and the common values (null, num, String, bool) of the tracking
copy of an entity's fields. Users benchmark: byState (a join, 20 users)
+3% (SQLite), +1% (memory); the others within their variation.
Behavior change: a SQL select with a limit (and no orderByID) is
now ordered by the ID column (ORDER BY id ... LIMIT n), as one with an
offset already was: which rows an unordered LIMIT returns is
undefined in SQL (it can change with the query plan), e.g. which row a
selectFirstByQuery returns. Pass orderByID: false to opt out. The
in-memory adapters (DBSQLMemoryAdapter, the object adapters,
IterableEntityRepository) keep their rows in insertion order and still
order only for an offset. A limit of 0 (no LIMIT clause) doesn't
order.
OrderDirection.resolveOrderByID(orderByID, offset, [limit]).Fix: DBObjectGCSAdapter: parallel cache writes over the cache limits
(cacheLimit/cacheFilesLimit) started concurrent limit checks, which
listed and deleted the same files: the second delete threw a
PathNotFoundException in a future nobody awaited (an unhandled error).
The checks now run one at a time: a call during a check queues one more
check after it (shared by the calls until then), since the running one
may have listed the files before that call's write. A cached file already
removed counts as deleted, and an error of a check (nobody awaits it: e.g.
the cache directory removed) is logged, not left as an unhandled error.
Optimization: turning rows (or any Map) into entities
(EntityHandler.createFromMap), for every adapter:
resolveFieldsValues resolves the values into one map, synchronously
when none of them is a Future (no forEachAsync, entries and
intermediate maps per row).resolveValueByType no longer builds a TypeInfo of every value: the
conditions that don't depend on the value are checked first.parseDateTimeISOUTCFast: the DateTimes DBSQLiteAdapter stores
(toUtc().toIso8601String()) are parsed without a regular expression.list/byState
(20 users per call) +31%, single-user reads +11–12%; memory byId
+12%.Fix: DBMySQLAdapter: a NULL column of a select with parameters (e.g.
selectByQuery(' name == ? ')) was read as absent, so the entity field
got its default (e.g. 0) instead of null: mysql1 leaves it out of a
row's fields in its binary protocol (a statement with parameters). The
rows are now read from their values.
Fix: ConditionSQLEncoder.encodeIDCondition/encodeIDConditionIN built
the ID column from the encoding parameter key, which is renamed (e.g.
"param_id") when the query has a parameter named like the ID column.
SQL: the IDs of an ID condition (#ID == x, selectByID, selectByIDs
and the selects of references) are now parameters (placeholders), not
values written into the statement: its text is the same for any ID. An
IN list of IDs is padded (repeating its last ID) to 1, 2, 4, 8
... 256, then multiples of 256 (ConditionSQLEncoder.idsListPaddedLength),
so its text is the same for lists of similar lengths. An ID that is
already a parameter of the query (#ID == ?) keeps it. Not faster by
itself (users benchmark: within −1.5% and +0.6%): the groundwork for
reusing statements (prepared statements per connection).
SQLDialect.intParameterTypeSuffix (PostgreSQL: :int8): an integer
ID parameter is typed, so an ID beyond the column's type matches
nothing (as a literal did) instead of failing ("out of range").SQLDialect.maxIDsAsParameters: an IN list padded beyond it is
written as values. A parameter is only faster when its statement is
reused, which a long list's length seldom is, and a statement's
parameters are limited (SQLite: 32766; PostgreSQL: 65535; e.g. a
selectByIDs of 40000 IDs). Measured selecting N rows by ID (reused
parameters vs values): PostgreSQL 512 (parameters 16–29% faster up
to ~400 IDs, about even at ~640–768, values faster from ~1024); SQLite
1024 (the same cost per ID, ~10–15% less when reused); MySQL 0, which
keeps the IDs written as values: mysql1 runs a statement with
parameters as prepare + execute (2 round trips, not reused), one
without as 1. Splitting a list into batches is never faster than
values (PostgreSQL: a round trip per batch).Behavior change: prepared statements, reused by SQL text
(PreparedStatementsCache; the least recently used released over
preparedStatementsCacheSize, default 100 per connection). On by
default; db config preparedStatements: false disables them. Behind
PgBouncer in transaction pooling mode before 1.21, disable them: a
prepared statement's name can then refer, on a shared server connection,
to another client's statement, which runs instead (silently). DBSQLAdapter.preparedStatementsInfo: hits, prepared, evicted and
stale counters. An invalid preparedStatementsCacheSize (< 1) passed to
an adapter's constructor throws an ArgumentError there (a config value
falls back to the default).
DBPostgreSQLAdapter: per connection. A SQL text is prepared when it
repeats (preparing costs a round trip; a statement run once doesn't pay
it), and only outside a transaction (it would be prepared on the
transaction's session, ending with it); a prepared statement is reused
also inside the connection's transactions. A stale one (0A000: a
schema change altered its result columns; 26000) is replaced, and the
statement run again unprepared (outside a transaction: inside one the
error aborted it). A cached statement runs only while its session is
open: on the session of a finished transaction it fails, as an
unprepared one does, instead of running outside that transaction.DBSQLiteAdapter: per database handle, on the first use (no round
trip); SQLite recompiles a prepared statement after a schema change.
Released before the handle is closed. A SQL with more than one
statement is still rejected (checkNoTail), not run up to its first.DBMySQLAdapter: none (mysql1 doesn't reuse prepared statements).list/byState +4–5%; PostgreSQL reads
+8–12%, writes about the same.reflection_factory: ^2.11.0 (requires Dart 3.13.2+): Json.decode / decodeAsync with nested JSON lists (previously lazily cast lists that threw when re
reflection_factory: ^2.11.0 (requires Dart 3.13.2+): Json.decode<Map> /
decodeAsync<Map> with nested JSON lists (previously lazily cast lists that
threw when read). Reflection code regenerated (only the builder version
changed).
build_runner: ^2.16.2 (analyzer 14, required by it).Fix: DBSQLiteAdapter: concurrent transactions failed (no such savepoint)
or reverted each other's writes. Every pooled connection shares one native
handle, so they were nested as SAVEPOINTs of a single stack: a COMMIT
also released another transaction's savepoint, and a ROLLBACK reverted
its writes. They now run one at a time (SQLite has a single writer
anyway); a transaction opened inside another one still nests as a
SAVEPOINT.
Fix: DBRelationalEntityRepository.store/update: an error raised by the
repository, not by the DB (e.g. a RecursiveRelationshipLoopError), left
its auto-commit transaction open forever, with the operations already
executed in it never committed or rolled back (holding its connection;
with SQLite, every later transaction nested inside it). It's now aborted
(rolled back).
Fix: the connection pool (Pool.releaseIntoPool) checked its size before
adding the released connection, so it could keep maxConnections + 1
connections.
Tests: concurrency integration tests (bones_api_concurrency_*_test.dart,
tag concurrency): parallel inserts, reads, partial updates (also of the
same entity), entities with references, connection pool pressure and
unique conflicts; transactions committed, aborted (an error or abort()),
in parallel, with parallel operations inside, a failing operation, and no
dirty reads between concurrent transactions. On the SQL memory, SQLite,
PostgreSQL and MySQL adapters, and the object memory, directory and GCS
(fake) adapters, each with its documented limitations (e.g. no rollback or
unique fields on object storage).
test_vm (concurrency) job (its 3 files at once); the workflow
also runs on PRs to any branch (stacked PRs).Fix: APIRequest.isAuthenticated was true only when the token was
expired (it is now true only when it isn't).
Fix: APISecurity.autoValidateAllTokens never ran (the elapsed time was
computed backwards, always ≤ 0): the expired tokens are now validated
(removed) at most every 5 minutes, as intended.
Fix: APIToken.fromJson parsed the duration from issueTime; it now
reads duration (seconds), defaulting to 3 hours when absent.
Fix: APIRequest.pathPartAt with a negative index (from the end) always
returned null.
Fix: parseAPIRequestMethod('options') returned null (a 'optionS'
typo).
Fix: APIRequest.getHeaderFirstOf: the case-insensitive match of the
first key returned a parameter instead of the header.
Fix: APIRouteBuilder.from with an Iterable of providers threw a
TypeError (swapped arguments).
Fix: DBSQLAdapter.parseConfigLogSQL with log: {sql: ...} parsed the
log Map (always false) instead of its sql value.
Fix: DBSQLAdapter.removeSQLsComments threw a RangeError on any
/* ... */ comment (now replaced by a space).
Fix: EntityReferenceList.toEntityReference threw a RangeError on an
empty list with entities; EntityReferenceList.getAt with an
entitiesFetcher fetched all the IDs (as a nested list) instead of the one
at index.
Fix: TransactionOperationTypeExtension.name threw for
selectRelationships.
Fix: APIRepository.delete ignored its transaction.
Fix: APIConfig.fromAsync (loadAPIConfigFromUri) of an http(s) URL
discarded the downloaded configuration (and requested the URL with a
trailing /): now it's returned (null, or the default, if not found).
Fix: APIServer with totalWorkers > 1 failed to spawn the workers
(object is unsendable - _CustomZone) when the APIRoot was created (or
initialized) in a Zone other than Zone.root (e.g. runZoned,
runZonedGuarded, a test): APIRoot.currentAPIRequest (a ZoneField,
holding its parent Zone) is moved out of the instance while it's copied.
APIRoot.prepareForIsolateCopy / finishIsolateCopy: called by
APIServer before and after spawning the workers. In between, the
ZoneField of this Isolate is kept out of the instance (in a static
Expando: not copied), and still used by this Isolate. Otherwise it's
an instance field (the fast path), created by the constructor, as before
(the calls' zones forked from the constructor's Zone).APIRoot.initializeIsolateCopy: initializes a copy in its Isolate
(its currentAPIRequest forked from that Isolate's Zone), called by
an auxiliary APIServerWorker before its start (otherwise created when
first used there).APIRoot.getCurrentAPIRequest([contextZone]): the current APIRequest
(or null), without creating the ZoneField.EntityRulesResolver context provider of an APIRoot is registered
once per Isolate (also by a copy with its modules already loaded), and
not again after close.Fix: APIServerResponseCache wrote a double comma (,,) in a cached
response's server-timing header.
Fix: AlterTableSQL with ifNotExists cut the column name of an entry
with leading spaces (line.substring(11) over the untrimmed line).
Fix: APIConfig.getPath threw a RangeError for a List index out of
range (now null).
Fix: registerAsDbLogger of a logger with a dotted (hierarchical) name:
its records weren't identified as DB logs (LogRecord.loggerName is the
full name).
Fix: runTestAPIServer left the server running on an unknown channel
command (now stopped, also closing its APIRoot, as on stop).
Fix: checkTables (SQLite) suggested SQLs that SQLite can't run:
SQLDialect.alterTableAddColumnIfNotExists (false for SQLite): no
ADD COLUMN IF NOT EXISTS.SQLDialect.alterTableAddConstraint, a missing column's foreign
key is in the column (REFERENCES), and a UNIQUE one is a unique
index (no ADD CONSTRAINT).Fix: checkTables errors with only missing enum CHECK constraints or
values had no description (now missingEnumConstraints and
missingEnumValues); missingCollectionReferenceColumns also listed the
non-collection reference fields.
Fix: checkDBTables with a missing table threw a null-check TypeError
(getRepositoriesSchemes) instead of reporting No scheme for repository.
Fix: Transaction: an error after the first one (e.g. of an operation
after the abort) was returned as the operation's result (as R: a
TypeError, or the error object as the result); now it's rethrown (and
still added to extraErrors).
Fix: IterableEntityRepository.ensureReferencesStored (store,
ensureStored) didn't store the entities of List<Entity> fields.
Fix: EntityHandler.decodeObjectListJson couldn't decode a list (cast to
O): each element is now resolved as an entity.
Fix: DBObjectMemoryAdapter/DBObjectDirectoryAdapter stored an
EntityReferenceList field as the object itself (wrapped in a List),
not its IDs: the entity couldn't be read back.
Fix: DBObjectDirectoryAdapter/DBObjectGCSAdapter couldn't store a
DateTime, enum or Decimal field (not JSON encodable): now encoded as
Json.toJson (DBObjectAdapter.toEncodableJson), and parsed back by the
entity's handler.
Fix: DBObjectDirectoryAdapter/DBObjectGCSAdapter: selecting an entity
with sub-entities (e.g. a User, with its Address and Roles) never
completed: the sub-entities' selects (by the operation's preFinish), in
the same transaction, waited for the operation, that waited for them. The
object adapters (also DBObjectMemoryAdapter) now finish an operation
after its transaction execution (as the SQL adapters).
Fix: onStore never notified the stores of DBEntityRepository and
DBRelationalEntityRepository (inserts and updates with changes; an
unchanged entity isn't stored, nor notified).
Fix: APIDBModule: select, insert, update and delete without a
table (e.g. /db/select) threw a RangeError (an error response), now
notFound.
Fix: EntityHandler.equalsValuesBasic/equalsValuesEntity/
equalsValuesEntityMap: values without an ID (or a resolvable one) were
equal (null == null), e.g. two unrelated non-entity objects, or two
entities not stored yet: a null ID now never matches (as
EntityHandler.equalsValues).
Fix: APIServer always sent the X-Access-Token header back, also when
the client sent the token (the Authorization header was checked in the
response headers, not in the request's).
Fix: APIAuthentication.isExpired(now:) ignored now.
Fix: DBAdapter.instantiateAdaptor ignored an async instantiator
(returning a Future<A?>, as declared by DBAdapterInstantiator), and
didn't skip an async null.
Fix: IterableEntityRepository (e.g. SetEntityRepository): re-storing
an entity with a unique field (@EntityField.unique()) threw
EntityFieldInvalid(unique) against itself: the unique values are now
checked against the other entities (not the same instance or ID).
Fix: SetEntityRepository (IterableEntityRepository.store): an entity
with an ID (an update, or an insert with a defined ID) was only "stored"
if it was the same instance already stored: an entity with a new ID wasn't
stored, and another instance with the same ID (an updated copy) didn't
replace the stored one. SetEntityRepository.nextID (length + 1) could
reuse an ID after a remove: now the greatest ID + 1.
Fix: DBObjectMemoryAdapter/DBObjectDirectoryAdapter/DBObjectGCSAdapter:
a partial update (store of a tracked entity: only its changed fields)
replaced the stored object with only those fields (the others lost): now
merged into the stored object.
Fix: APITokenStore.storeAPIToken: a Map<String, Object> data became an
UnmodifiableMapView<String, Object?> (not a Map<String, Object>): its
check was after the Map<String, Object?> one (that also matches it).
Removed unreachable code: LoggerHandler.resolveLogDestiny's 2-parameter
function branches (already MessageLoggers), APIConfig's %var%
fallback (the group always matches) and a duplicated
DBAdapterException.resolveToString branch.
APIServerWorker._startLetsEncrypt: removed the unreachable null
directory check (only called with a directory: canUseLetsEncrypt); the
"directory doesn't exist" check (removed after the worker's constructor)
is now tested.
Fix: DBObjectDirectoryAdapter: a partial update of an object whose file
can't be parsed (e.g. truncated by a crash in the middle of a write) failed
(the merge read it): now the update rewrites it (logged).
SetEntityRepository: its entries by ID (a LinkedHashMap): selectByID,
existsID and store look up the ID without a scan
(IterableEntityRepository.getStoredByID), and an updated copy replaces
the stored entity in its position (IterableEntityRepository.replace;
previously moved to the end of the iteration order).
Fix: checkTables reported a missing unique column's constraint also as a
missing unique constraint (its ADD COLUMN brings it): the suggested SQLs
had it twice.
Fix: an error in an operation's preFinish (run after its transaction
execution: e.g. building the entities of a select, or storing the
relationships of an insert) didn't abort the transaction: the operation
was never finished, and the transaction never committed nor aborted
(in an auto-commit transaction, a SQL adapter's connection held). Now
DBAdapter.finishOperation (used by all the adapters) notifies it to the
transaction (notifyExecutionError): aborted, and the error (resolved by
the adapter) rethrown.
Docs: the DBObjectAdapters (memory, directory, GCS) don't enforce unique
fields (@EntityField.unique(), @EntityIndex(unique: true)): object
storages (e.g. GCS, S3) are not designed for that (no index over the
objects' values, nor transactions over multiple objects). Documented in
DBObjectAdapter, each object adapter, EntityField/EntityIndex.unique
and the README (new ObjectAdapter section).
Fix: DBObjectGCSAdapter: a partial update with an error reading the
stored object (other than not found: e.g. a timeout or a 5xx response)
overwrote it with only the changed fields: now the update fails (the
stored object is kept). A select still handles a read error as not found.
Tests: DBObjectGCSAdapter (with an in-memory fake of the GCS JSON API):
insert, partial updates, updated copies, read errors, enum/Decimal
fields, deletes, the local cache (limits, cacheDevelopment) and
fromConfig; also the shared object adapters tests (without and with a
cache).
Fix: DBObjectGCSAdapter stored an EntityReferenceList field as the
object itself (wrapped in a List), not its IDs (as fixed for the memory
and directory adapters).
Fix: checkTables (MySQL) suggested SQLs that MySQL can't run:
ADD COLUMN IF NOT EXISTS (a syntax error, also in the latest MySQL):
SQLDialect.alterTableAddColumnIfNotExists is false for MySQL." (ALTER TABLE "store" ...: a string in
MySQL): AlterTableSQL/CreateTableSQL/CreateIndexSQL (SQLBuilder)
now use the dialect's elementQuote by default (" if it has none).Fix: DBMySQLAdapter.getTableScheme/getTableFieldsTypes threw for a
table that doesn't exist (ER_NO_SUCH_TABLE), instead of null (as the
other adapters).
APIDBModule.select: paginated by default:
defaultSelectLimit (default: 1000): the LIMIT of a request
without one (LIMIT=<n> in the URL query: a custom one; LIMIT=0: no
limit). PAGE=<n> without a LIMIT is now a page of
defaultSelectLimit entities.previous/next page links (keeping the query and
the EAGER/ORDER directives), and the [first - last] entities shown;
its JSON link is for the same page (and with the query now URL-encoded).Tests: line coverage raised from 70.3% to 95.1% (VM suite, including the PostgreSQL/MySQL Docker tests), with new test/bones_api_cov_*_test.dart files
test/bones_api_cov_*_test.dart
files (all VM, no Docker):
APIDBModule routes, APIRootStarter.APIServer (real HTTP servers on free ports: status branches, static
files, cache headers, workers), APIServerResponseCache, HTMLDocument.Transaction, EntityReference/
EntityReferenceList, entity rules, JSON and collection utilities.checkTables,
SQL builders and condition encoders.checkTables: duplicated indexes and CHECK constraints in a table are warned (never an error), with suggested SQLs to drop them:
checkTables: duplicated indexes and CHECK constraints in a table are
warned (never an error), with suggested SQLs to drop them:
INCLUDE columns isn't a duplicate), also a plain index over the
columns of a unique/primary key one. Kept: the primary key (the
introspected one, whatever its name; or a SQLite sqlite_autoindex_*),
else a unique one (the default name, <table>__<columns>__unique,
first), else the default name (<table>__<columns>__idx), else the
first by name. DROP INDEX IF EXISTS (MySQL: DROP INDEX ... ON table); a unique one also ALTER TABLE ... DROP CONSTRAINT IF EXISTS
(a constraint's index), noting that it fails if a foreign key references
it. MySQL SERIAL's unique index (with PRIMARY KEY, of the generated
CREATE TABLE) isn't reported.CHECK constraints over the same column (PostgreSQL), with the
same values or not (an old enum CHECK never dropped still restricts
the writes). Kept: the one with exactly the entity's enum values, else
one with all of them (and others), else a default name
(<table>_<column>_check, or <table>_<column>__check of the
checkTables suggestion), else the first by name; if none has the
entity's values, the suggestion says to review it. The column's field
is the one checkTables maps to it.CHECK over the column has
all of them (an old duplicated CHECK is warned, not an error).DBSQLAdapter.tablesDuplicates (of the last checkDBTables, by table)
and findTableDuplicates(scheme, entityHandler:): TableDuplicates
(indexes, checkConstraints: each group with the one kept first;
dropSQLs).TableScheme.duplicatedIndexes, duplicatedCheckConstraints,
checkConstraints (the enum CHECKs by name, not merged as in
constraints; PostgreSQL: conname, only the table visible in the
search path, as the indexes), primaryKeyIndex and indexesKinds
(introspected by the PostgreSQL, MySQL and SQLite adapters).SQLDialect.dropIndexOnTable (MySQL);
SQLGenerator.generateDropIndexSQL / generateDropConstraintSQL.checkTables (warned, not an error);
integration (SQLite, PostgreSQL, MySQL): the duplicates and the primary
key introspected, another kind (DESC) not reported, an old CHECK
not an error, and the suggested SQLs, run in each DB, drop them.Initializable (review of the 1.20.0 resolver):
Initializable (review of the 1.20.0 resolver):
checkInitialized(): back to the pre-1.20.0 default for an instance
initializing elsewhere: it returns (allowInProgress defaults to
true). allowInProgress: false throws (unless in a cycle with the
caller). A cycle it finds is recorded in the caller's
cyclicDependencies; when it throws for an async start, the caller isn't
left waiting for that instance.parent is a waiter too (with the calling hook): a cycle
through either is detected.DBEntityRepository / DBSQLEntityRepository: their dependencies are the
adapter and the repository adapter, not all the repositories (removing the
clique of every repository waiting for every other, still present in
1.20.0).Initializable: a new dependency resolver (the public API is unchanged, with additions). Cycles (self-referenced graphs) are resolved deterministically
Initializable: a new dependency resolver (the public API is unchanged,
with additions). Cycles (self-referenced graphs) are resolved
deterministically by a waits-for graph: a dependency already waiting
(transitively) for the requester isn't waited for (a cycle), anything else
is. No delays, timeouts or fabricated results; the framework's waits can't
deadlock. The framework is described in the library dartdoc.
checkInitialized passed. Now a
failed instance is failed (and idle), and is initialized again on the
next call.checkInitialized ignored a synchronous error, and executeInitialized
lost the original error.ok == false, with the error/stackTrace): the
futures never fail.checkInitialized throws if the instance is initializing elsewhere
(not in a cycle with the caller), unless allowInProgress: true (used
by the repository lookups).InitializationResult.cyclic, Initializable.cyclicDependencies
(the dependencies not waited for, in a cycle), InitializationStatus
failed/error/stackTrace, InitializationError.cause /
causeStackTrace, checkInitialized(allowInProgress:).checkInitialized/executeInitialized,
merged results, zones/waiter attribution, a DB-shaped graph), and the
initialization of the real providers/adapters/repositories (integration).The StateError has the suggested SQLs to fix the tables (they were only logged).
checkTables:
StateError has the suggested SQLs to fix the tables (they were
only logged).@EntityIndex isn't reported missing when the adapter's unique
indexes are unknown (TableScheme.uniqueIndexes == null, e.g. a custom
adapter): only the indexes known to be missing are.checkTables against controlled table schemes (all present, unknown
indexes/unique indexes, a non-unique index for a unique one, order /
superset / subset, the primary key, name conflicts, names in use
elsewhere, legacy columns, not reported twice, field unique and its
fallback to the constraints), and the fix SQLs per dialect.DateTime, null, update).allRepositories of an adapter.EntityRepositoryProvider / DBAdapter: one generic lookup, EntityRepositoryProvider.lookupEntityRepository (replacing traverseLookup): the own reposito
EntityRepositoryProvider / DBAdapter: one generic lookup,
EntityRepositoryProvider.lookupEntityRepository (replacing
traverseLookup): the own repositories, then the parent provider, the
known providers (several candidates resolved) and the global provider.
getEntityRepository / getEntityRepositoryByType only say how to search
locally and how to ask another provider.
traversed, a new optional named parameter, internal), so a lookup
started inside another one (e.g. by a provider's initialization) is
independent of it.EntityRepositoryProvider / DBAdapter: an EntityRepository lookup (getEntityRepository, getEntityRepositoryByType) asks each provider at most once. A p
EntityRepositoryProvider / DBAdapter: an EntityRepository lookup
(getEntityRepository, getEntityRepositoryByType) asks each provider at
most once. A per-provider "calling" flag only blocked a provider while it
was on the stack, and the fallback to the global provider (which knows all
of them) skipped even that, so a miss (a type no provider has) walked every
order of the open providers: N!, a hang with a dozen of them (e.g. tests
opening a provider each, or many adapters). Now one set of the providers
asked per lookup (EntityRepositoryProvider.traverseLookup); a provider's
initialization still runs outside it.allRepositories: passes the traversed providers on (it restarted them for
each known provider: endless with providers known to each other).@EntityIndex(..., unique: true): a composite unique constraint, which @EntityField.unique() (one field) can't express: `dart @EntityIndex(['establishm
@EntityIndex(..., unique: true): a composite unique constraint, which
@EntityField.unique() (one field) can't express:@EntityIndex(['establishment', 'day'], unique: true)
class SalesDay extends Entity { ... }
CREATE TABLE as the single-field ones:
CONSTRAINT "sales_day__establishment_day__unique" UNIQUE ("establishment", "day").
Default name: <table>__<column1>_<column2>__unique; or name:.@EntityIndex over a field already
@EntityField.unique() (the same constraint); a non-unique @EntityIndex
over the same columns (served by the unique one). A non-unique index over
the same columns (e.g. an indexed field, a reference) is dropped: the
constraint's index serves it.checkTables: only a unique index over these columns (in this order, any
name: also a UNIQUE constraint's) satisfies it. A missing one is
reported with ALTER TABLE ... ADD CONSTRAINT ... UNIQUE (as in the
CREATE TABLE; SQLite, which can't add a constraint to a table:
CREATE UNIQUE INDEX). A non-unique index with its name: reported as a
conflict. Not reported twice (as generateCreateTableSQL): an
@EntityIndex over the same columns as a missing unique field's
constraint, or as another missing one.DBSQLMemoryAdapter), as the SQL
databases (a null value never conflicts).EntityFieldInvalid('unique') with all its fields:
fieldsNames (new) and fieldName (establishment, day); from SQLite's
table.column, table.column, PostgreSQL's Key (a, b)=(...) and MySQL's
key name (its columns from the loaded table scheme:
DBSQLAdapter.loadedIndexColumns; a single-field one: its column, not
the key name).checkTables: @EntityField.unique() fields are checked by all the SQL
adapters (from TableScheme.uniqueIndexes; it was PostgreSQL only, from its
constraints), with the CREATE TABLE constraint name
(<table>__<column>__unique, was <table>_<column>__unique) and reported
in the error message (missingUniqueConstraints).UNIQUE (a, b) isn't a field's
TableUniqueConstraint (was one for a field named a, b).TableScheme.uniqueIndexes (the names of the unique indexes, also of
UNIQUE/PRIMARY KEY constraints) and hasIndexOn(columns, unique:):
introspected by the PostgreSQL (indisunique), MySQL (Non_unique) and
SQLite (pragma_index_list.unique) adapters.CreateIndexSQL.unique (CREATE UNIQUE INDEX);
SQLGenerator.generateAddUniqueConstraintSQL;
SQLDialect.alterTableAddConstraint (false: SQLite).DBSQLiteAdapter.parseConstraintTableFields,
DBPostgreSQLAdapter.parseUniqueViolationColumns.@EntityIndex: indexes over one or more fields (composite indexes), declared on the entity class, which @EntityField.indexed() (one field) can't expres
@EntityIndex: indexes over one or more fields (composite indexes),
declared on the entity class, which @EntityField.indexed() (one field)
can't express:@EnableReflection()
@EntityIndex(['establishment', 'creationDate'])
class OrderClosing extends Entity { ... }
<table>__<column1>_<column2>__idx; or name:. A default
name past the dialect's identifier limit (SQLDialect.maxIdentifierLength:
PostgreSQL 63, MySQL 64) is shortened with a stable hash
(<prefix>_<hash>__idx); a too long name: is an ArgumentError.generateCreateTableSQL /
generateFullCreateTableSQLs, generateTables). An @EntityIndex over
the same columns as another index of the table (e.g. a single field also
indexed or a reference) isn't created twice; the same name over other
columns is an ArgumentError.checkTables: an existing table without an index over these columns (in
this order, any name) is reported (missingEntityIndexes), with its
CREATE INDEX SQL. The columns are the table's ones mapped to the fields,
as for the other checks (e.g. a legacy creationDate column). If its name
is in use (drop it first: IF NOT EXISTS would skip it), that's reported
too: by an index over other columns, an index not usable as it, or
another table's index or relation (PostgreSQL, SQLite: names unique per
schema; DBSQLAdapter.findIndexesNamesInUse).ArgumentError.EntityHandler.getEntityIndexes(): the class annotations
(ClassReflectionEntityHandler), or GenericEntityHandler(entityIndexes:).CreateIndexSQL.columns: an index over several columns (columns,
isComposite); SQLGenerator.resolveEntityIndexColumns /
generateEntityIndexesSQLs.TableScheme.indexes (by name, its key columns in order) and hasIndexOn:
introspected by the PostgreSQL (pg_index: the table visible in the search
path, also partitioned tables), MySQL (SHOW INDEX) and SQLite
(pragma_index_list/pragma_index_info) adapters; null when unknown
(memory). Only usable plain-column indexes: partial, expression/functional,
(MySQL) prefix and (PostgreSQL) invalid indexes are ignored, and INCLUDE
columns aren't key columns. PostgreSQL before 11 (no indnkeyatts) is
supported; indexes that can't be read are null, not a scheme error.DBSQLAdapter.checkDBTables({force}): checks again on demand (e.g. after a
migration), reading all the loaded table schemes (also referenced and
relationship tables), fields types and references again
(SchemeProvider.disposeTablesSchemesCaches; the adapters' references
cache too). checkedTables is false until a forced check passes.benchmark/db_users: a Users API benchmark sub-project — an APIRoot with reflection-wired modules and Role/Address/User entities, measured end to end a
benchmark/db_users: a Users API benchmark sub-project — an APIRoot with
reflection-wired modules and Role/Address/User entities, measured end
to end against memory, SQLite or PostgreSQL (--docker or an existing
server), with latency percentiles and a results history (HISTORY.md).
Performance, found and measured with benchmark/db_users:
BEGIN/COMMIT — a select of an entity with references was
BEGIN, its queries and COMMIT. Its first write opens the transaction.
DBSQLAdapter.readsOutsideTransaction, on by default for PostgreSQL
(READ COMMITTED), opt-out with readsOutsideTransaction: false.
Single-entity reads 2x faster.DBSQLMemoryAdapter: relationship selects and inserts go through the
relationship indexes; a delete by ID takes its row by key and resolves
only it; a relationship delete is narrowed through the index. Reads up
to 3x, writes up to 7x faster.Fixes:
LoggerHandler.root threw a LateInitializationError when it was the
first logging access.DBSQLMemoryAdapter: after a rollback, a relationship restored in its
table was missing from the relationship index (an index entry is a Set
changed in place, which MapHistory can't roll back), so queries through
the relationship missed it. Its indexes versions snapshot was also never
refreshed.Dependency updates:
reflection_factory: ^2.10.1 (faster constructor lookup when decoding
entities: +10–15% on reads)map_history: ^1.0.7 (consolidate visits only the changed keys: the
memory adapter's commits no longer scale with the table size)APITestConfigDockerDB (and the PostgreSQL and MySQL configs):
APITestConfigDockerDB (and the PostgreSQL and MySQL configs):
dockerChosenPort: Docker chooses a free host port for the database
when the container starts, and it is written to the port of the DB
config. Unlike a port resolved before starting the container, parallel
test runs can't pick the same one. The container is named
<prefix>_<session>_<n>, so a container left by an interrupted run
can't take its name.runOptions: further docker run options for the container
(DockerRunOptions: tmpfs, labels, resources, health check...).APITestConfigDockerPostgreSQL: settings (each -c key=value),
initdbArgs, extraEnvironment, and ephemeral (a throwaway database:
durability off, data directory in memory).
APITestConfigDockerMySQL: settings, extraEnvironment and ephemeral.
Dependency updates:
docker_commander: ^3.1.0A route parameter typed as an enum no longer fails when the value arrives as a qualified name (Currency.brl), the form an enum.toString() produces and
A route parameter typed as an enum no longer fails when the value arrives as
a qualified name (Currency.brl), the form an enum.toString() produces and
the one external callers tend to send:
type 'String' is not a subtype of type 'Currency?' of 'chargedCurrency'
EnumReflection.from only read a bare value name, so the qualified form
parsed to null, APIRouteBuilder.resolveValueByType fell back to the value
it was given, and the raw String reached Function.apply. The bare name
(brl), in any case, already resolved.
deps: reflection_factory: ^2.10.0, which resolves the qualified enum name.
Added the cross_origin configuration entry, grouping every cross-origin response header and splitting it by the kind of response it applies to, the sa
Added the cross_origin configuration entry, grouping every cross-origin
response header and splitting it by the kind of response it applies to, the
same shape the cache entry already uses:
cross_origin:
api:
allow_origin: ''
allow_methods: 'GET,HEAD,PUT,POST,PATCH,DELETE,OPTIONS'
allow_headers: 'Content-Type, Access-Control-Allow-Headers, Authorization'
allow_credentials: true
expose_headers: 'Content-Length, Content-Type, Last-Modified, X-Access-Token, X-Access-Token-Expiration'
max_age: 0
vary_origin: true
static_files:
opener_policy: 'same-origin-allow-popups'
embedder_policy: ''
resource_policy: ''
The split is not cosmetic: CORS is set on API responses only, while the
cross-origin isolation policies are set on static file responses only, and
the two never meet on the same response.
Every entry is also a command-line option
(--cross-origin-api-max-age, --cross-origin-static-files-opener-policy,
...), and an APICrossOriginConfig can be passed straight to APIServer.
See APICORSConfig and APICrossOriginPolicies.
Until now all of these values were hardcoded in APIServer.setCORS, with no
way to change them short of rewriting the response.
Behavior change: static text/html responses are now served with
Cross-Origin-Opener-Policy: same-origin-allow-popups.
It severs the opener relationship with a cross-origin document that opened
the page — the isolation of same-origin — while still allowing the popups
the page itself opens to keep a handle back to it. That is what popup-based
sign-in flows need (Sign in with Google, OAuth popups): the popup reports
its result by calling back into its opener.
A document served by this server that is itself opened as a cross-origin
popup and calls window.opener (an OAuth callback landing page) must now opt
out with opener_policy: none.
COOP and COEP are only sent for text/html, since they are document
headers, while CORP applies to every static file. COEP and CORP are
disabled by default.
Behavior change: API responses are now served with Vary: Origin.
Access-Control-Allow-Origin reflects the request Origin, and without
Vary a shared cache is free to serve one origin's
Access-Control-Allow-Origin to another. Disable with vary_origin: false.
allow_origin accepts an allowlist. When set, the request Origin is
reflected only if it matches, otherwise no Access-Control-Allow-Origin is
sent and the browser blocks the cross-origin read.
Worth noting for anyone leaving it empty: reflecting any origin together with
Access-Control-Allow-Credentials: true — the behavior before this version,
and still the default — lets any site make credentialed calls to the API and
read the responses. The browser only rejects that pairing for a literal *.
Access-Control-Max-Age is now available through max_age, sent only on
OPTIONS responses, the only ones a browser caches. Defaults to 0, which
omits the header.
Fixed: a cookieless server did not apply the cookieless guarantee to static
file responses.
Set-Cookie was dropped and X-Cookieless-Server: Blocking all cookies was
added by the API response builder, but a static file response is built by the
shelf handler and returned before reaching it. So the header was missing
from every static file, and nothing enforced the absence of Set-Cookie
there. Both now happen for static files as well, including the non-2xx
responses.
The placeholder pruning added in 1.15.1 no longer runs on every encoded condition.
The placeholder pruning added in 1.15.1 no longer runs on every encoded condition.
Rewriting field == ? bound to null into field IS NULL leaves that
parameter unreferenced, and 1.15.1 found it by materializing the encoded
output and scanning it for every placeholder. That cost was paid by every
query, including the overwhelmingly common one that compares nothing against
null: ~0.30us per encoded condition (measured on a 3-placeholder
condition), against the ~0.90us of a whole logged route call after the 1.15.0
dispatch work.
Only a placeholder actually rewritten to IS NULL/IS NOT NULL can become
unreferenced, so those keys are now recorded as they are written, and a
condition that compares nothing against null returns immediately — without
materializing the output or scanning it. The output is materialized lazily
even then, since a recorded key may still be referenced by another operator.
No behaviour change: same statements, same bound parameters.
Fixed: a condition comparing a field to null passed as a parameter was encoded as field = ? bound to null. = NULL is never true in SQL, so the query r
Fixed: a condition comparing a field to null passed as a parameter was
encoded as field = ? bound to null. = NULL is never true in SQL, so the
query returned no rows instead of the rows whose column is null.
ConditionSQLEncoder did turn =/IN against null into IS NULL (and
!=/NOT IN into IS NOT NULL), but only when the null was written
straight into the statement. A null arriving as a parameter is encoded as a
placeholder, whose text never equals 'null', so the conversion was skipped.
Entity queries take the parameter form, which is why it surfaced there:
// Returned [] with matching rows present; now returns the rows whose
// `state` is null.
repository.selectByQuery(' state == ? && active == ? ',
parameters: {'state': null, 'active': true});
The encoder is shared, so this affected every SQL adapter — SQLite, PostgreSQL and MySQL alike — and any condition compared against a null parameter, including compound ones whose other terms matched.
Rewriting the comparison also leaves the parameter unmentioned by the
statement, so it is now dropped once the condition is encoded: PostgreSQL
rejects a statement carrying variables it does not use. A placeholder still
referenced by another operator (field > ? bound to null) keeps its binding.
Covered now by the shared adapter test suite, so all three adapters exercise it.
reflection_factory: ^2.8.1 → ^2.9.0.
dart_style to the formatter bundled with Dart 3.12, so on
Dart 3.13 the generated *.reflection.g.dart no longer matched this
package's own dart format, making dart format --set-exit-if-changed and
test/ensure_build_test.dart mutually exclusive. 2.9.0 tracks the
formatter the SDK ships.Faster request dispatch. A logged route call is ~2.9x faster (measured in-process, APIRoot.call on a trivial route: 2.76us -> 0.90us).
Faster request dispatch. A logged route call is ~2.9x faster
(measured in-process, APIRoot.call on a trivial route: 2.76us -> 0.90us).
LoggerHandler no longer builds the formatted log message when nothing
would consume it. Every record reaching the root listener was fully
formatted — timestamp, padded/truncated isolate and logger names, plus a
Zone lookup for the current APIRequest id — and then discarded when no
destination (logAllTo/logErrorTo/logDbTo/console) was configured,
which is the default. This was ~1us per record, and a route call emits two
(CALL> and RESPONSE>).APIRouteHandler caches its CALL> message and RESPONSE> prefix. Both
are fixed once a route is registered, but were re-interpolated per request
(including stringifying the declared parameters Map).APIRoot._callImpl no longer copies the path parts list just to read the
first one.APIServer.toAPIRequest no longer copies the query-parameters Map a
second time.The routes builder now accepts config: on any/get/post/put/
delete/patch/head, matching APIModule.addRoute. Previously an
APIRouteConfig could only be set through addRoute, so per-route logging
could not be turned off through the usual API:
routes.get('ping', handler, config: const APIRouteConfig(log: false));
Route logging is on by default and costs roughly 4x the rest of a trivial dispatch, so this is worth setting on hot routes.
New benchmark/ suites, with layered breakdowns so a regression can be
attributed rather than just observed. See benchmark/README.md.
dart run benchmark/bones_api_benchmark.dart # request path
dart run benchmark/json_benchmark.dart # JSON request/response
dart run benchmark/db_benchmark.dart # DB entity path
They record that query parsing is well cached (~300x cheaper than parsing)
and SQL generation is under a microsecond. JSON encoding already runs close
to a bare dart:convert encode, and request bodies use dart:convert
directly, so no JSON optimization came out of that suite.
DBSQLMemoryAdapter now answers a select by ID with a direct lookup in the
table Map, which is already keyed by ID, instead of scanning it. A miss
still falls through to the scan, so results are unchanged.
selectByID was O(rows) and is now flat: 7.8us -> 7.1us at 10 rows,
9.7us -> 7.2us at 50, and 25.3us -> 7.2us at 400. This mostly speeds up the
test suite and development, since the memory adapter is where those run.
New DBSQLiteAdapter: an embedded SQLite DB adapter, backed by the [sqlite3][sqlite3_pkg] package.
New DBSQLiteAdapter: an embedded SQLite DB adapter, backed by the
sqlite3 package.
import 'package:bones_api/bones_api_db_sqlite.dart';
var adapter = DBSQLiteAdapter('/var/lib/myapp/db.sqlite',
generateTables: true);
// Or an in-memory database:
var memoryAdapter = DBSQLiteAdapter(':memory:', generateTables: true);
Registered as sqlite, sqlite3, sql.sqlite and sql.sqlite3, so a
config block db: { sqlite: {...} } resolves it.
fromConfig accepts path/file/database/db for the database file,
and memory: true (or the path :memory:) for an in-memory database, plus
the usual generateTables/checkTables/populate/log.sql keys.
Irrelevant keys (host, port, username, password) are accepted and
ignored, so a config can be pointed at SQLite without being rewritten.
No server and no native library to install: the sqlite3 package
bundles SQLite (3.53.4) through Dart's build hooks.
Runs the same entity test-suite as the PostgreSQL and MySQL adapters, and
needs no Docker container to do it. New APITestConfigSQLite, exported by
package:bones_api/bones_api_test_sqlite.dart.
Notes on the SQLite dialect:
INTEGER PRIMARY KEY AUTOINCREMENT:
SQLite has no SERIAL/AUTO_INCREMENT, only a column declared exactly
INTEGER PRIMARY KEY aliases the rowid, and without AUTOINCREMENT
SQLite reuses the ID of a deleted row.ENUM is emulated with a VARCHAR CHECK (col IN (...)) constraint.sqlite3 is a synchronous driver, and SQLite allows a single
writer, the adapter uses one native handle shared by every pooled
connection: a second handle blocking on a lock would stall the isolate
holding it, and offers nothing to gain when there is no I/O to overlap.
Nested transactions use SAVEPOINT.New SQLDialect.returningAcceptsTableWildcard (default true, so the
PostgreSQL/MySQL/memory dialects are unchanged). SQLite rejects the
table-qualified wildcard that DELETE ... RETURNING emits
("RETURNING may not use TABLE.* wildcards") and needs a bare
RETURNING *.
Fixed DBObjectDirectoryAdapter losing objects written just before a read:
_saveObject was async and its Future was dropped by doInsert/
doUpdate, while every reader in the adapter inspects the filesystem
synchronously. A store could therefore return before its object was on
disk, and selectAll would silently omit it (a not-yet-written file reads
back as null and was discarded). The write is now synchronous.
Breaking: the minimum Dart SDK is now 3.10.0 (was 3.7.0), required by
sqlite3 and its build hooks.
Dependencies:
sqlite3: ^3.5.1New EntityPagination.onEvent: an optional hook notified of what is being fetched, for progress reporting and logging.
New EntityPagination.onEvent: an optional hook notified of what is being
fetched, for progress reporting and logging.
var p = userRepository.paginateByQuery(' state == ? ',
parameters: ['NY'], limit: 20, onEvent: (event) {
switch (event) {
case EntityPaginationPageLoading(:var page):
print('fetching page $page...');
case EntityPaginationPageLoaded(:var page, :var entriesLength):
print('page $page: $entriesLength entries');
case EntityPaginationPageError(:var page, :var error):
print('page $page failed: $error');
case EntityPaginationPageSkipped(:var page, :var reason):
print('page $page not fetched: ${reason.name}');
case EntityPaginationEnd(:var totalLength):
print('done: $totalLength entries');
case EntityPaginationReset(:var discardedPages):
print('discarded ${discardedPages.length} pages');
}
});
EntityPageLoader (a Stream would
only deliver in a later microtask, after a sync read already finished).
To consume it as a stream, forward it: onEvent: myEventStream.add.onEvent is not final, so it can also be attached to an already built
EntityPagination. Only events emitted afterwards are seen.Zone and
does not break the fetch.onEvent is null.New EntityPaginationEvent<O>, a sealed hierarchy so a switch over it is
exhaustive, with EntityPaginationListener<O> as the callback type:
EntityPaginationPageLoading: a fetch is about to start. Emitted once per
actual fetch.EntityPaginationPageLoaded: a fetch finished, with the entries, the
entriesLength, the elapsedTime of the pageLoader and isFinalPage.EntityPaginationPageError: a fetch failed, with the error, the
stackTrace and the elapsedTime. The error is rethrown to the caller
right after the event.EntityPaginationPageSkipped: a page was served without a fetch, with an
EntityPaginationSkipReason: alreadyLoaded, inFlight (a concurrent
request shares the fetch) or knownEmpty (past the resolved end). Not an
error — it is what makes a repeated, concurrent or past-the-end read free.EntityPaginationEnd: the end was resolved, with the finalPage and the
totalLength. Emitted once, immediately after the
EntityPaginationPageLoaded that resolved it — which is not necessarily
the final page itself, since an empty page can pin the end at its
predecessor.EntityPaginationReset: reset() or refresh() discarded the loaded
pages, with the discardedPages, the discardedEntitiesLength and
isRefresh — true while it is the reset of a refresh(), which
re-fetches those pages right after, so a consumer can tell an in-progress
refresh from a pagination that was simply emptied. Emitted after the
state is cleared, so the pagination already reads as empty and the
discarded state is on the event.Every event but EntityPaginationReset is about a page, and is an
EntityPaginationPageEvent (also sealed) carrying the page.
Note that concurrent page loads interleave: getRange and refresh start
every page at once, so all the fetches are announced before any completes.
paginateByQuery, paginate and paginateAll gained the optional onEvent
parameter, on EntitySource, EntityRepository and APIRepository, so the
hook is reachable without building an EntityPagination by hand.
Tests: 15 new cases in bones_api_entity_pagination_test.dart (the event
sequence of a full read, of an exact multiple of the page size, of an empty
result and of a failure; the 3 skip reasons; the synchronous delivery; a
listener attached after construction; a throwing listener; and the
reset/refresh events).
New EntityPagination : a lazily loaded, paginated view over a select, for reading a result page by page without knowing its total length upfront.
New EntityPagination<O>: a lazily loaded, paginated view over a select,
for reading a result page by page without knowing its total length upfront.
var p = userRepository.paginateByQuery(' state == ? ',
parameters: ['NY'], limit: 20);
await p.loadNextPage(); // page 1
p[0]; // sync, already loaded
await p.getAt(45); // loads page 3 on demand, leaving page 2 a gap
await p.loadAll(); // fills the gaps and resolves the total
page parameter of the select*
methods); entry indexes are 0-based (matching a Dart List). See
indexOfPage / pageOfIndex.getAt(45) with a limit
of 20 loads only page 3.operator [], loadedEntities) never fetches;
only the FutureOr methods (getAt, getPage, getRange,
loadNextPage, loadPage, loadAll, stream) do. operator [] returns
null for a gap, an unloaded page or an out-of-range index alike; use
isPageLoaded / isIndexKnownOutOfRange to tell them apart.List or an Iterable: both require a
length, which is exactly what a paginated select can't answer until it
reaches the end. Use loadAll when a complete list is really needed.What it knows: loadedPages, loadedPagesLength, loadedEntities,
loadedEntitiesLength, maxLoadedPage, maxLoadedIndex, maxKnownPage,
isFinalPageResolved, finalPage, totalLength, isKnownEmpty,
and information().
Since every page except the last holds exactly limit entries, identifying
the final page yields the total even with gaps:
totalLength == (finalPage - 1) * limit + entries(finalPage).
The end resolves when a page comes back short, when an empty page has a
loaded and full predecessor, or when page 1 comes back empty. An empty page
without a loaded predecessor does not resolve it — jumping to page 50
of a 3-page result only proves the end is somewhere before page 50 — but it
is still recorded, to avoid re-fetching that page or any page after it.
Concurrent requests for the same page share a single fetch, and a failed load is evicted so a retry actually retries.
New paginateByQuery, paginate and paginateAll on EntitySource,
EntityRepository (with resolutionRules) and APIRepository. They return
immediately without loading anything. orderByID defaults to true there,
rather than following the offset != null rule of the select* methods:
a paginated read is only meaningful over a stable order.
Note: each page is an independent select, without a shared Transaction.
Entries inserted or deleted between two page loads shift the offsets, so a
page loaded later can repeat or skip entries. This is inherent to
offset-based pagination; ordering by ID makes it as stable as it can be.
selectByQuery and its siblings gained 4 optional parameters, for pagination and ordering:
selectByQuery and its siblings gained 4 optional parameters, for pagination
and ordering:
offset: the return offset.page: the 1-based page to return, an ergonomic alternative to offset
that computes it from the page size: (page - 1) * limit.orderByID: orders the result by the table's ID column, resolved
automatically from the existing scheme machinery (TableScheme.idFieldName
→ EncodingContext.tableFieldID, or EntityHandler.idFieldName).orderDirection: the new OrderDirection enum, ascending (default) or
descending.Semantics:
orderByID ?? (offset != null): a non-null
offset turns the ordering on by default, since an offset-based
pagination needs a stable order to be correct. Pass orderByID: false to
opt out and get a bare OFFSET.orderDirection is ignored while the ordering is not active.page is a public convenience resolved to an offset at the repository
layer (see resolveSelectOffset); the adapter contract keeps taking only
offset. It throws an ArgumentError when combined with an offset (two
spellings of one thing), when there is no positive limit to use as the
page size, or when it is < 1. page: 1 resolves to offset: 0, which
still activates the ordering, so even the first page is stable.Added to EntitySource/EntityRepository (selectByQuery,
selectFirstByQuery, select, selectIDsByQuery, selectIDsBy,
selectAll), APIRepository, IterableEntityRepository
(matches/all included), DBEntityRepository, DBRelationalAdapter/
DBRelationalRepositoryAdapter/DBRelationalEntityRepository,
DBAdapter.doSelectAll/doSelectByIDs, DBSQLAdapter.doSelect/
doSelectIDsBy/generateSelectSQL/generateSelectIDsSQL and
DBSQLRepositoryAdapter.generateSelectSQL.
New OrderDirection enum (bones_api_types.dart), with sqlKeyword,
parse and the resolvers resolve and resolveOrderByID that state the
semantics above exactly once.
New compareEntityIDs and applySelectOrderAndPagination
(bones_api_entity.dart): the shared Dart-side "order by ID → skip → take"
used by every adapter that can't delegate the ordering to a DB engine.
New resolveSelectOffset (bones_api_entity.dart): resolves page to an
offset, and states the page/offset/limit validation rules once.
SQLDialect:
orderBySQL and limitOffsetSQL clause builders, so all the
dialect-specific SELECT tail syntax lives in one place.offsetRequiresLimit and offsetMaxLimitValue capabilities. MySQL sets
offsetRequiresLimit: true since it can't parse an OFFSET that is not
preceded by a LIMIT; an offset-only select there emits
LIMIT 18446744073709551615 OFFSET n. PostgreSQL and the generic
(in-memory) dialect emit a bare OFFSET n.SQL: new offset, orderByID and orderDirection fields (carried by
copy()), read by DBSQLMemoryAdapter to apply the same semantics in Dart.
APIDBModule.select (/db/select/<table>): new LIMIT=<n>, OFFSET=<n>,
PAGE=<n> and ORDER=asc|desc query directives
(see APIDBModule.selectQueryDirectives),
parsed from the query String alongside the pre-existing EAGER=true and
stripped before the remainder is parsed as the entity condition query. The
endpoint no longer selects the whole table and sorts it in Dart — the ordering
is now resolved by the DB. Its output order is unchanged. An invalid PAGE
becomes an error response rather than an uncaught ArgumentError.
Behavior change: limit is now honored on the paths that previously
accepted and silently ignored it — DBEntityRepository.select's
ConditionID/ConditionIdIN/ConditionANY/KeyConditionEQ fast paths,
DBAdapter.doSelectAll/doSelectByIDs, and the DBObjectMemoryAdapter,
DBObjectDirectoryAdapter and DBObjectGCSAdapter adapters. For example,
selectAll(limit: 2) on an object adapter returned every row before this
release; it now returns 2.
Source-breaking for external subclasses: new named parameters were added
to abstract members (EntitySource.select/selectIDsBy/selectAll,
DBAdapter.doSelectAll/doSelectByIDs,
DBRelationalAdapter.doSelect/doSelectIDsBy). Dart requires an override to
accept every named parameter of the supertype, so third-party
EntityRepository/DBAdapter implementations must widen their overrides.
Known limitation: a query over a to-many relationship generates a JOIN
without a DISTINCT, so it can return the same entity more than once
(pre-existing). Paginating such a query is therefore best-effort.
Tests:
bones_api_entity_select_order_test.dart (OrderDirection,
compareEntityIDs, applySelectOrderAndPagination, SQLDialect clause
builders), bones_api_entity_db_sql_select_test.dart (exact generated SQL
per case + end-to-end paging over the in-memory SQL adapter) and
bones_api_db_module_test.dart (first coverage of APIDBModule).bones_api_entity_db_tests_base.dart: 3 new tests in the shared adapter
template, so the generated SQL and real page-by-page reads are asserted for
the in-memory, PostgreSQL, MySQL, object-memory and object-directory
adapters. Verified against real PostgreSQL and MySQL containers.docker_commander: ^2.1.8 → ^3.0.0.
docker_commander: ^2.1.8 → ^3.0.0.
wasm_run and flutter_rust_bridge 1.x from the dependency graph
(they came in via docker_commander → apollovm, and were only ever needed
to execute Wasm — which nothing here does).shelf_web_socket ^1.0.2 and
web_socket_channel ^2.2.0, so every bones_api application was locked
out of a modern shelf/WebSocket stack. That constraint is now gone.docker_commander's own API is unchanged, so this is a minor release: the
DockerHost types exposed by the test utils keep the same shape.petitparser: ^6.1.0 → ^7.0.2 (required by apollovm 2.0.0).
JsonGrammarLexer.token: flatten() takes its message as a named
parameter in petitparser 7. Same behaviour, new call shape.ConditionSQLEncoder.valueToParameterValue: fixed encoding of a List of values containing ConditionParameters; each element is now resolved individuall
Bug fixes:
ConditionSQLEncoder.valueToParameterValue: fixed encoding of a List of values containing ConditionParameters; each element is now resolved individually instead of passing the whole list to every element.ConditionEncoder.resolveValueToType: fixed resolution of a single-element Iterable to a primitive type (was a no-op comparison instead of an assignment, leaving the value as a List).MapGetterExtension.matchKeyIgnoreCase: fixed case-insensitive key matching that always returned null (empty loop body); now returns the matching key. Also fixes setMultiValue(..., ignoreCase: true).Time: millisecond/microsecond range validation now correctly rejects 1000 (valid range is 0..999).Time._bytesInStringFormat: fixed the second-byte digit check that was effectively disabled (length < 2 instead of length >= 2).APISession.isExpired: now honors the provided now argument instead of always using DateTime.now().APIServerResponseCache cached entry: replaceFileStat no longer compares a variable to itself (identical(myFileStat, myFileStat)), so the file stat is correctly replaced.WeakList.set: now increments the internal modification counter, consistent with the other mutating methods.Tests:
Time range/string parsing, matchKeyIgnoreCase/getIgnoreCase/setMultiValue, APISession.isExpired, and ConditionSQLEncoder/ConditionEncoder value resolution).Added logging of severe errors when apiRoot.ensureInitialized() returns a failure with an error.
v1.9.30
APIRootStarter:
start:
apiRoot.ensureInitialized() returns a failure with an error.Project template:
update_project_template.sh:
project_template prepare command to exclude IDE module files matching ^\w+\.iml$ from the template archive.Dependencies:
build_runner to ^2.15.0.test to ^1.31.1.vm_service to ^15.2.0.Added method resolveIDValue to resolve the ID value from parameters or ConditionParameter.
ConditionID:
resolveIDValue to resolve the ID value from parameters or ConditionParameter.DBEntityRepository:
selectIDsBy and _selectByID to use ConditionID.resolveIDValue for ID resolution.select:
KeyConditionEQ matcher with a single key matching the entity ID field._selectByID to fetch the entity by ID and returns a single-element list or empty list accordingly.DBObjectDirectoryAdapter:
_doCountImpl and _doDeleteImpl to use ConditionID.resolveIDValue for ID resolution.parameters ?? namedParameters) to internal implementations.DBObjectGCSAdapter:
_doCountImpl and _doDeleteImpl to use ConditionID.resolveIDValue for ID resolution.parameters ?? namedParameters) to internal implementations.DBObjectMemoryAdapter:
_doCountImpl and _doDeleteImpl to use ConditionID.resolveIDValue for ID resolution.parameters ?? namedParameters) to internal implementations.Dependency updates:
vm_service: ^15.0.2 → ^15.1.0Fixed referenceTable and referenceColumn assignment in unique constraint SQL entries to allow nullable references.
SQLGenerator:
referenceTable and referenceColumn assignment in unique constraint SQL entries to allow nullable references.generateAddUniqueConstraintAlterTableSQL and generateAddEnumConstraintAlterTableSQL to use normalized column names with double underscores for consistency.Dependency updates:
async_extension: ^1.2.22reflection_factory: ^2.7.5swiss_knife: ^3.3.14meta: ^1.18.2hotreloader: ^4.4.0googleapis_auth: ^2.3.0build_runner: ^2.13.1test: ^1.31.0…bones_api_utils_fast_checksum.dart instead of deprecated Adler32 and Crc32 classes.
Added bones_api_utils_fast_checksum.dart:
getAdler32Uint8List, getAdler32Hex, getCrc32Uint8List, and getCrc32Hex for Adler-32 and CRC-32 checksums as byte arrays and hex strings.getAdler32 and getCrc32 from archive package for checksum calculation.WeakEtag class (bones_api_base.dart):
WeakEtag.adler32 and WeakEtag.crc32 factories to use getAdler32Hex and getCrc32Hex from bones_api_utils_fast_checksum.dart instead of deprecated Adler32 and Crc32 classes.bones_api.dart:
bones_api_utils_fast_checksum.dart utility.Dependencies:
async_extension from ^1.2.20 to ^1.2.21.swiss_knife from ^3.3.3 to ^3.3.5.archive from ^4.0.7 to ^4.0.9.build_runner from ^2.10.5 to ^2.11.1.ensureInitialized: added onError handler to then call to route errors to _onInitializationError.
Initializable mixin:
ensureInitialized: added onError handler to then call to route errors to _onInitializationError.executeInitializedCallback:
onError handler to then call on async initialization result to throw InitializationError with stack trace._FutureExtension:
toCompleter: added onError handler to then to complete completer with error and stack trace if not completed.Added nullable field indexName to represent the name of the index if one exists.
TableFieldReference:
indexName to represent the name of the index if one exists.Added new class TableRelationshipReferenceEntityTyped extending TableRelationshipReference:
sourceFieldEntityType and targetFieldEntityType fields of type TypeInfo.copyWithEntityTypes method to create typed copies.TableRelationshipReference:
sourceRelationshipFieldIndex and targetRelationshipFieldIndex.copyWithEntityTypes method returning TableRelationshipReferenceEntityTyped.EntityHandler:
getFieldsListEntityTypes method to return a map of fields that are list entities or references with their TypeInfo.SQLDialect:
foreignKeyCreatesImplicitIndex boolean flag with default true.createIndexIfNotExists to indicate support for IF NOT EXISTS in CREATE INDEX (default true).CreateIndexSQL:
buildSQL method to conditionally include IF NOT EXISTS only if dialect supports it.DBPostgreSQLAdapter:
foreignKeyCreatesImplicitIndex flag to PostgreSQL dialect set to false._findAllTableFieldsReferences query to include foreign key index name (fk_index_name) by joining with pg_index and pg_class.indexName in TableFieldReference instances from query result.sourceRelationshipFieldIndex and targetRelationshipFieldIndex from indexName.DBMySQLAdapter:
createIndexIfNotExists to false in MySQL dialect capabilities.DBSQLAdapter:
parseConfigDBGenerateTablesAndCheckTables: changed return type from List<bool> to a record with named fields (generateTables, checkTables).extractTableSQLs: updated regex to also match CREATE INDEX statements in addition to CREATE and ALTER TABLE._populateTablesFromSQLsImpl: fixed error handling for CREATE INDEX statements when the SQL dialect does not support IF NOT EXISTS.
_checkDBTableSchemeReferenceField to return TableRelationshipReferenceEntityTyped with entity types._DBTableCheck class:
missingReferenceIndexes and missingRelationshipReferenceIndexes._DBRelationshipTableColumn subclass of _DBTableColumn to represent relationship table columns with relationship table name.generateAddColumnAlterTableSQL.Dependency updates:
async_extension: ^1.2.19 → ^1.2.20meta: ^1.18.0 → ^1.18.1Added override for addSlice to handle partial chunk addition and update _inputLength accordingly.
GZipSink:
addSlice to handle partial chunk addition and update _inputLength accordingly.addSlice to call _gzipSink.close() when isLast is true and full chunk is added.BytesSink:
addSlice to use new addPart method for partial chunk addition.BytesBuffer:
addPart method to add a slice of bytes from a given offset and length, resizing buffer if needed.add method to delegate to addPart.addPart.async_extension: ^1.2.18 -> ^1.2.19
mapDataTypeToDartType: added support for PostgreSQL types smallint and smallserial mapping to int.
DBPostgreSQLAdapter:
mapDataTypeToDartType: added support for PostgreSQL types smallint and smallserial mapping to int.Added calls to _forceLogFlushMessages() before throwing InitializationError in:
Initializable mixin:
_forceLogFlushMessages() before throwing InitializationError in:
_checkDependency_setInitializedDependenciesCompleters_onInitializationError_checkAllDependenciesOk_finalizeInitializationcheckInitializedexecuteInitializedLogging:
_forceLogFlushMessages() function to call logging.Logger.root.forceFlushMessages().Logger extension:
forceFlushMessages() method to invoke LoggerHandler.forceFlushMessages().LoggerHandler abstract class:
forceFlushMessages() method.LoggerHandlerGeneric implementation:
forceFlushMessages() returning false.LoggerHandlerIO implementation:
forceFlushMessages() to flush the print message queue immediately if not empty.Updated all Map.unmodifiable usages to explicitly specify type arguments, e.g. Map .unmodifiable.
EntityHandler:
Map.unmodifiable usages to explicitly specify type arguments, e.g. Map<String, TypeInfo>.unmodifiable.fieldsWithEntityReference, fieldsWithEntityReferenceList, fieldsEntityAnnotations, fieldsWithType, getFieldsTypes, getFieldsEnumTypes, getFieldsEntityTypes, and constructors to use typed unmodifiable maps.Dependency updates:
meta: ^1.18.0keyToSQL: added check to throw ConditionEncodingError if keys is empty.
ConditionSQLEncoder:
keyToSQL: added check to throw ConditionEncodingError if keys is empty.keyFieldReferenceToSQL to recursively resolve multi-level key references by walking keys and resolving intermediate tables and relationships._resolveReferenceField and _resolveFinalField to modularize reference resolution logic.DBSQLAdapter:
_JoinEntry typedef to represent SQL JOIN fragments with explicit alias dependencies (defs and refs).List<_JoinEntry> to perform dependency-aware sorting of JOINs ensuring referenced aliases are resolved before use._JoinEntry with defined and referenced aliases.Dependencies:
async_extension from ^1.2.17 to ^1.2.18.build_runner from ^2.10.4 to ^2.10.5.Updated reflection_factory dependency from ^2.7.2 to ^2.7.3.
reflection_factory dependency from ^2.7.2 to ^2.7.3.Improved error messages in instantiation methods to include a list of instantiator function keys.
DBAdapter:
Dependencies:
async_extension from ^1.2.15 to ^1.2.17.test from ^1.28.0 to ^1.29.0._resolvePayloadFromString: Improved JSON payload parsing:
APIServer:
_resolvePayloadFromString: Improved JSON payload parsing:
null for empty bodies.statistics: ^1.2.1
One column per quarter.