All notable changes to this project will be documented in this file.
- Ability to specify option for compression when publish data
- Add on_first_sync callback
- Add validate_types to model interface
- Fix of all types taking type cast into account
- DateTime can be String now
- Add check of data scalar types in order to provide readable error text
- Updated
rabbit_messaginggem version to1.7.0 - Use kwargs in Rabbit.publish calls in order to bypass inconsistent hash interpolations in our existing product codebases due to new requirements in
rabbit_messaginggem.
- Removed
tainboxgem from dependencies rabbit_messaginggem version is now locked to>= 1.6.3
- Delete table_name and schema_name as required for TableSync::Publishing::Raw
- Delete table and schema as required for TableSync::Instrument#notify
- Add custom version attributes
- Set table_name and schema_name as required for TableSync::Publishing::Raw
rabbit_messaginggem version is locked on~> 1.1.0in order to provide a way to keep "up-to-date" the actual rabbit_messaging gem version on our projects with table_sync update;
- Remove ruby 3.0 from ci workflow
- Add ruby 3.3 to ci workflow
- Add expiration time for cache used in debouncing sync
- Update dependencies
- Fix rubocop issue
- Drop ruby 2.7 support
- Remove ruby 2.7 from ci workflow
- Required fields allow accepting empty collections.
- Required fields validation for the Raw and Batch publishers.
- Fix sort in receiving (#79).
- Send all original attributes for
deleteevents instead of just PK.
rabbit_messaginggem version is locked on~> 0.13in order to provide a way to keep "up-to-date" the actual rabbit_messaging gem version on our projects with table_sync update;
- Instead of using
object_classforTableSync::Publishing::Raw, usemodel_name, which acceptsStringinstead ofClass. - Can manually disable notifications by setting
TableSync.notify = false. TableSync::Publishing::Rawaccepts two additional optional arguments:table_nameandschema, this attributes only used to fill information for the notification.
Debouncercalculates propernext_sync_time
Fixed bug when routing key is nil.
- Fixed bug: skip publish when object is new and event is destroy for ActiveRecord
- fixed docs
- A lot of specs for all the refactoring.
- Docs
- 100% coverage
- Heavy refactoring of Publisher and BatchPublisher. All code is separated in different modules and classes.
- Job callables are now called:
- single_publishing_job_class_callable
- batch_publishing_job_class_callable
- Now there are three main classes for messaging:
- TableSync::Publishing::Single - sends one row with initialization
- TableSync::Publishing::Batch - sends batch of rows with initialization
- TableSync::Publishing::Raw - sends raw data without checks
Separate classes for publishing, object data, Rabbit params, debounce, serialization.
-
Jobs are not constrained by being ActiveJob anymore. Just need to have #perform_at method
-
Changed some method names towards consistency:
- attrs_for_routing_key -> attributes_for_routing_key
- attrs_for_metadata -> attributes_for_headers
-
Moved TableSync setup into separate classes.
-
Changed ORMAdapters.
-
Destroyed objects are initialized. Now custom attributes for destruction will be called on instances.
- Obj.table_sync_destroy_attributes() -> Obj#attributes_for_destroy
- Event constants are now kept in one place.
- Plugin Errors
- Provide current fired event to wrap receiver. You'll be able to get it with
wrap_receiving(event:, **rest) {}as usual fordata, target_keys, version_key - Update rails dependencies with patch version
- documentation
- update gems
- Fix
deleteevents being broken when either#attrs_for_routing_keyor#attrs_for_metadatawas defined on a model.
- Instead of original attributes (default raw model attributes), use published attributes (as defined by
#attributes_for_syncor.table_sync_destroy_attributes) forTableSync.routing_key_callableandTableSync.routing_metadata_callable. - Send all original attributes for
deleteevents instead of just PK.
- potential data corruption with batches
- bug with sorting data in handler, it was bad idea to use
.hashreplaced to.to_s
- No changes. Just stabilization release.
- bug with sorting data in handler
- dead locks in receiving module (see:
spec/receiving/handler_spec.rb#avoid dead locks)
- move
TableSync::Instrument.notifyfrom models to the handler - fire
TableSync::Instrument.notifyafter commit insted of in transaction
- config inheritance
- TableSync::Plugins
- option
except to_modelin receive method- TableSync::Utils::InterfaceChecker
- .rubocop.yml
- documentation
- modules hierarchy (split receiving and publishing)
TableSync::Publisher->TableSync::Publishing::PublisherTableSync::BatchPublisher->TableSync::Publishing::BatchPublisherTableSync::ReceivingHandler->TableSync::Receiving::Handler - made data batches processing as native
- implemented callbacks as options
- implemented
wrap_receivingas option - type checking in options
before_commit on: event, &block->before_update(&block)orbefore_destroy(&block)after_commit on: event, &block->after_commit_on_update(&block)orafter_commit_on_destroy(&block)- changed parameters in some options:
add
raw_datacurrent_row->row... see documents for details
- TableSync::Config::CallbackRegistry
- TableSync::EventActions
- TableSync::EventActions::DataWrapper
- config option
on_destroy - config option
partitions - config option
first_sync_time_key
- ruby 2.7 in Travis
- Gemfile.lock
- some fixes to get rid of warnings for ruby 2.7 (implicit conversion of hashes into kwargs will be dropped)
- TableSync.sync now explicitly expects klass and kwargs (it converts them into hash)
- TableSync::Instrument.notify now explicitly expects kwargs and delegates them further as kwargs
- ruby 2.3, 2.4 from Travis
- Introduce Plugin ecosystem (TableSync::Plugins);
- Updated docs for batch_publishing to fully reflect changes in 2.1.0
- TableSync::BatchPublisher: custom headers;
- TableSync::BatchPublisher: custom events;
- Slight changes to specs
- Sequel publishing hooks: checking for
:destroyevents inside:if/:unlesspredicates
- TableSync::BatchPublisher: incorrect
attrs_for_metadatadefinition (typo in method name);
- Wrapping interface around receiving logic (
wrap_receiving);
- The
default_valuesoption no longer overrides original values.
- Payload for existing ActiveSupport adapter contains both
tableandschemafields now.
- Receiving: inconsistent events raises
TableSync::UnprovidedEventTargetKeysError(events that include only some of the target keys (or none))
- convert symbolic values to strings in hashes to support older versions of activejob
- add notifications
debounce_timeoption for publishing
on_destroyreturn value for manipulation withafter_commitcallback;
on_destroy- defines a custom logic and behavior fordestroyevent;