github.com/itzmeanjan/ette

I noticed sometimes during chain reorganization previous entries from DB needs to be removed and new ones are required to be inserted, during that due to not using cascading removal of transactions/ events from respective DB tables for removed block header(s), conflict used to rise.

This release includes only one PR #64 , which attempts to address this problem.

What's new ?

• Fast, lock-free, concurrent-safe job queue implementation, powering ( non-duplicated ) Block Processing
• Only using Redis for Pub/Sub, all other usage such as SortedSet/ List removed, using 👆 queue for those
• Improved code organisation as part of better house keeping

Changes 📝

• Automatic clean up of historical delivery data by independent worker [ Generally used for determining whether user has crossed allowed rate limit or not ]
• More graphQL APIs
• Improved API documentation [ Check readme please ]
• Updated how API keys are generated for users, now considering latest UNIX timestamp also to make it harder to guess
• HTTP GET API for querying how many websocket sessions being handled by ette deployment
• Better logging with how many read/ write ops performed on underlying socket connection, during life time of one websocket connection

Changes

• Updated chain reorganisation protection & reconsidering reorg-ed blocks
• Improved pubsub for blocks, transactions, events publishing
• Single consumer go-routine for each of three root topics i.e. {"block", "transaction", "event"}, serving on single websocket connection
• Each client needs to create single websocket connection to ette & they can now subscribe to N many topics
• Improved query response time by optimising how DB indices being used for responding back

Changes

• Improved retry queue manager implementation, attempts to process failed blocks in delayed mode ✅
• Improved unfinalised block queue implementation, as soon as length reaches what it should be starts processing finalised blocks 🥳
• Revamped missing block finder implementation, now performing block number range based queries, leading to improved performance ⚡️
• Case insensitive matching of addresses during subscription request evaluation, non checksummed ethereum address supported 🤩

Changes

• Always attempt to publish block data 🚀
• Even if its processing fails in mid in certain iteration, when taken from retry queue attempt to publish it ✅
• Better console logging 🤖
• Improved performance by using uuid as primary key of delivery_history table, instead of bigserial 🦾

Changes

• Taking snapshot of existing data store [ Concurrent ]
• Restoring data from snapshot into DB [ Concurrent ]
• Improved logging with time taken to process each block
• Improved documentation with instructions for production grade deployment & management using systemd
• Improved real-time subscription mode, not attempting to store anything in DB, when running in EtteMode = 2

Changes

• All requests being made to /v1/graphql to be scanned by rate limiter, using APIKey provided
• When successfully data can be delivered in response, only then book keeping to be done
• 👆 to be used while deciding whether request can be entertained or not, when next time request from same client to be received

If you're running ette on production, you should definitely update to this version.

Changes

Now ette can help you in fetching certain event log given block number/ hash & log index in block. Both REST & GraphQL APIs support it.

type Query {
    eventByBlockHashAndLogIndex(hash: String!, index: String!): Event!
    eventByBlockNumberAndLogIndex(number: String!, index: String!): Event!
}

In response you'll receive 👇

type Event {
  origin: String!
  index: String!
  topics: [String!]!
  data: String!
  txHash: String!
  blockHash: String!
}

And as it's also available using GraphQL API, you get to choose what are specific fields you want to receive in response.

Changes ⚡️

• For addressing Chain Reorganisation issue, implemented Redis backed delayed block processing queue 🥳
• From now on in .env file ( under BlockConfirmations field ), it can be set, how many block confirmations are required before finally persisting blocks in DB
• If nothing is provided, 0 will be considered, but highly not recommended to use it this way
• Allowing password less connection to Redis instance [ Not recommended, at least in production ]
• Workers will always attempt to keep non-final block count in queue as same as BlockConfirmations set in config
• Fixed issue in checking API key rate limit, before accepting request from client
• Max Block number range & time range, that can be used while performing range based queries, can be specified in config file