v0.5.0

Breaking Changes

• The Minimum Supported Rust Version (MSRV) has been changed to 1.70.

• All features in the bonsaidb crate have been updated to support the new
  features in Rust 1.60. Instead of needing separate flags for
  client-websockets and server-websockets, a single feature flag
  websockets is now able to work in conjunction with the client/server
  features to ensure everything works correctly.

  The net change is fewer feature flags. If you find that a feature flag is
  missing on upgrade, try removing "local-", "client-", or "server-" from the
  beginning of the feature and recompiling.

• The Key implementation for Option<T> has changed. If you wish to preserve
  backwards compatibility, wrap the key type with OptionKeyV1.

  The old behavior was broken in two ways:

  • The length of the key was improperly always reported as a constant length.
    This only affects code that was using composite keys.
  • When using a type that can encode itself as 0 bytes, there was no way to
    distinguish between a 0-length contained value and None. Thus, Some("")
    would panic as a precaution. However, this type of lurking panic in an
    esoteric edge case is exactly the behavior a database should never exhibit.

• The Key implementation for tuples (e.g., (T1, T2)) has changed. If you
  wish to preserve backwards compatibility, wrap the key type with TupleKeyV1.

  The old behavior had an incorrect sort order for the generated bytes. Instead
  of sorting only by the values themselves, the lengths of the variable-length
  Key types were taking precedence over the values. For more information, see
  issue #240.

  This change also deprecates encode_composite_key()/decode_composite_key()
  in favor of two new types: CompositeKeyEncoder and CompositeKeyDecoder.

• Collection's get/list/list_with_prefix/insert/overwrite and View's
  with_key_range/with_key_prefix have all been updated to accept borrowed
  representations of keys. For example, when a Collection's PrimaryKey is
  String, &str can now be supplied to these functions.

  For most users, adding & in front of the argument will generally fix the
  compiler error upon upgrade.

• DocumentId is no longer Copy, and now can support up to 64kb of data.
  Previously, the type was limited to 64 total bytes of stack space.

• [Async]StorageConnection::schematic() have been moved to its own trait,
  HasSchema.

• [Async]StorageConnection::authenticate() no longer takes a username or user
  id directly. This has been moved to the Authentication type.

• AuthenticationMethod has moved from bonsaidb::core::permissions::bonsai to
  bonsaidb::core::connection

• ApiName has been moved from bonsaidb::core::schema to
  bonsaidb::core::api.

• Various error variants that were simply String representations have been
  consoldiated into bonsaidb::core::Error::Other.

• bonsaidb::server::Backend now takes a &self parameter in all functions
  except configure(). Implementing Default for your Backend implementor
  will allow all existing code to continue working.

• Client::effective_permissions() is no longer async.

• CustomServer::connected_clients() is no longer async.

• CustomServer::broadcast() is no longer async.

• CustomServer::listen_on now takes impl Into<BonsaiListenConfig> instead of
  just a u16 parameter specifying the port. BonsaiListenConfig implements
  From<u16> to minimize code breakage.

• The command-line Serve type has had its listen-on field changed to a
  SocketAddr type to reflect the support for BonsaiListenConfig. The full
  address and port must be specified when providing a listen-on argument now.

• Client has been renamed to AsyncClient, and a new type, BlockingClient
  has been added. This splits the Client into two separate parts: an async
  version and a blocking version. This makes it less likely to accidentally call
  a blocking method in an async context.

  Client::send_api_request_async has been renamed to send_api_request.

  This change has also been introduced to RemoteDatabase and
  RemoteSubscriber: both async and blocking versions are available.

• #254 Key::from_ord_bytes has had its &'k [u8] parameter changed to a new type
  with an additional lifetime: ByteSource<'k, 'e>. This new type allows
  from_ord_bytes to be called with an owned Vec<u8>, a Key-lifetime bound
  byte slice (&'k [u8]), or an ephemeral byte slice (&'e [u8]).

  This change allows code paths that can pass an owned Vec<u8> in for decoding
  to not require allocations in some cases -- for example, when using
  Vec<u8>::from_ord_bytes.

  This change also means that Key implementors should expect to be called with
  any of the three variants of ByteSource, because BonsaiDb's internal code
  only passes borrowed slices in some code paths.

  Thank you to @asonix for the request and help on implementation!

• KeyEncoding::describe is a new function that allows a key encoder to
  document the data contained within the encoded representation. The goal of
  this is to allow tools to generically operate on key types used within
  BonsaiDb. This function is automatically implemented when using the Key
  derive.

  The new KeyDescription type uses this function to create a nested
  representation of the contained information.

  Types that utilize a custom encoding format can use KeyDescription::Other to
  uniquely identify the key.

• Schematic has had several methods changed to impl Iterator of the original
  type being returned to avoid extra unnecessary allocations. These methods are:

  • Schematic::views_in_collection
  • Schematic::eager_views_in_collection
  • Schematic::collections

• StorageConnection::list_available_schemas/AsyncStorageConnection::list_available_schemas
  now return a Vec<SchemaSummary>. The SchemaSummary type contains
  additional information such as what collections and views each schema
  contains.

• bonsaidb::cli::Command now flattens the server command rather than
  in-lining the admin command. This introduces additional top-level commands
  that were previously hidden underneath the server command.

• ConnectedClient::all_sessions() is a new function that returns all of the
  active sessions for the given client.

• View schemas are now partially deriveble, and have had their map/reduce
  functionality split into their own traits. The following notes all apply to
  this refactoring:

  • ViewSchema::map() and ViewSchema::reduce() have been moved to a new
    trait: MapReduce.
  • CollectionViewSchema has been removed, and the map() and reduce()
    functions have been moved to a new trait: CollectionMapReduce.
  • ViewSchema::unique() and ViewSchema::lazy() have been replaced with
    ViewSchema::update_policy(), which returns a new enum: ViewUpdatePolicy.
    This enum contains three variants: Lazy, Eager, and Unique, allowing the same
    options that the previous two methods supported without any ambiguities.
  • ViewSchema is now implementable/derivable for all views, regardless of
    whether the map/reduce functionality utilizes SerializedCollections or
    not.

• ViewSchema::MappedKey<'doc> is a new associated type with a generic
  associated lifetime that enables map()/reduce() to utilize borrowed data
  for the view's key type. This has caused the map() and reduce() functions
  to have new lifetime annotations added. These changes are part of an effort to
  support more flows where borrowing data is possible to minimize allocations
  while indexing and querying views.

  For all existing users, setting this associated type to the view's Key type
  will work, or pasting this associated type definition in:

  type MappedKey<'doc> = <Self::View as View>::Key

• natural_id support in the Collection derive macro has been changed. The
  attribute now expects an expression rather than a closure. The expression can
  reference self. This was done to avoid the required type annotations that
  the closure approach required.

  Alternatively, #[natural_id] can be annotated directly on a field to have it
  become the natural id automatically.

• bonsaidb::server::api::Handler has had its generic arguments order reversed,
  which allows the type to specify a default Backend of NoBackend.

• View query APIs now return CollectionMap types instead of Map types. The
  change allows for the Collection::PrimaryKey type to be used instead of
  DocumentId. The affected APIs are:

  • bonsaidb::core::connection::View::query()
  • bonsaidb::core::connection::View::query_with_docs()
  • bonsaidb::core::connection::AsyncView::query()
  • bonsaidb::core::connection::AsyncView::query_with_docs()
  • bonsaidb::core::connection::LowLevelConnection::query()
  • bonsaidb::core::connection::LowLevelConnection::query_with_docs()
  • bonsaidb::core::connection::AsyncLowLevelConnection::query()
  • bonsaidb::core::connection::AsyncLowLevelConnection::query_with_docs()
  • MappedDocuments: Both mappings and documents have had their types
    updated.
  • MappedSerialiedSocuments::deserialized()

Deprecated

• bonsaidb::core::connection::ViewMappings as been moved to
  bonsaidb::core::schema::view::ViewMappings. A deprecated re-export has been
  provided to minimize code breakage when upgrading.

Added

• #239 Key can now be derived on enums and structs, allowing an easier way
  to use composite keys. The primary-keys example has been updated to use the
  derive macro instead of a tuple for the composite key.

• Key is now implemented for Result<T,E>, where T and E both implement Key
  and KeyEncoding using the same error type.

• Key is now implemented for Cow<'a, str>.

• Key is now implemented for isize and usize. This is implemented using
  variable integer encoding, allowing for proper cross-architecture behavior.
  When trying to decode a value that is too large for the given target
  architecture, an error will be returned.

• Key is now implemented for NonZeroU* and NonZeroI* types, wrapping the
  inner type's encoding functionality and adding extra checks for 0 values. For
  signed types, next_value() will skip 0.

• bonsaidb::core::key::KeyFormat is a new transmog::Format that can be used
  as a SerializedCollection::Format associated type. This implements
  serialization using the Key trait rather than Serde.

  When deriving the Collection trait, specifying serialization = Key will
  use this format.

• ViewSchema can now be derived.

• bonsaidb_core::Error::is_unique_key_error() is a convenience function to
  quickly check if an error is a result of a unique key violation from a
  specific view.

• bonsaidb_core::Error::conflicting_document() is a convenience function to
  return the conflicting document's header if the error is a conflict from a
  specific collection.

• Operation::Check is a new operation that can be performed during a
  transaction. It checks whether a document exists in a given collection, and
  optionally can verify that a revision matches the currently stored revision.

  These constructor functions provide more ergonomic ways to create this variant:

  • Operation::check_document_id_exists(CollectionName,DocumentId)
  • Operation::check_document_exists<Collection>(Collection::PrimaryKey)
  • Operation::check_document_is_current(&impl HasHeader)

• Storage now acquires an exclusive lock to ensure multiple instances are not
  able to be opened at the same time. This lock is held across all locations the
  database is accessed, including background threads.

• bonsaidb-files is a new crate that enables storing large files in BonsaiDb,
  and includes abstractions for reading/writing files using common
  traits:

  • std::io::Read
  • std::io::Seek
  • std::io::Write
  • tokio::io::AsyncRead
  • tokio::io::AsyncSeek
  • tokio::io::AsyncWrite
  • Iterator<Item = std::io::Result<Vec<u8>>>
  • futures::Stream<Item = std::io::Result<Vec<u8>>>

  This crate can be added directly to your project, or if you're using the
  omnibus crate, feature files will enable this crate at bonsaidb::files.

• SerializedCollection::push_all[_async]() is a new function that accepts an
  iterator of document contents to push into the database using a single
  transaction. It returns the created collection documents if successful. If any
  errors occur, no documents will be inserted.

• ViewSchema::lazy()/CollectionViewSchema::lazy() are provided functions
  that return true by default. This preserves the existing behavior for
  map/reduce views -- they are only updated when queried, and as dictated by the
  AccessPolicy. By returning false, a view can become eagerly updated, which
  means that the views are fully updated by the time each transaction completes.

  This was how unique views already functioned, but now users who have workflows
  where an eagerly-updated view will be more efficient than a lazy view can
  opt-in to this behavior.

• bonsaidb::server::cli::Command has a new function, execute_on which
  accepts an already constructed server instance.

• ServerDatabase, AnyDatabase, and AnyConnection now all implement Clone
  and Debug.

• DefaultPermissions now implements From<Vec<Statement>> and
  From<Statement>, enabling simpler usage when using default_permissions()
  and authenticated_permissions().

• [Async]StorageConnection::authenticate_with_password are new functions
  providing a simpler interface for authenticating with a username/user id and
  password.

• [Async]StorageConnection::authenticate_with_token is a new function that
  allows authenticating with a previously created AuthenticationToken.

  Token authentication is an optional feature, enabled with the
  token-authentication feature. This feature is currently powered by
  BLAKE3, but has been written to be
  able to support multiple algorithms.

  An AuthenticationToken is a randomly generated id, a private token, and an
  identity that are used to perform a cryptographically secure verification
  during authentication without transferring the private token to the server.
  Upon successful authentication, the identity is assumed on the newly returned
  connection. This supports both assuming users and roles.

• The Api trait can now be derived.

• SerializedView has two new methods: entries and entries_async, which
  enable a "Type-first" query pattern. These two statements produce the same
  results:

  let mappings = MyView::entries(&db).query()?;
  let mappings = db.view::<MyView>().query()?;

  These new APIs are provided purely for style preference considerations.

• LimitedResolutionDuration now has a function checked_add() supports
  iter::Sum into Option<LimitedResolutionDuration<T>> or
  LimitedResolutionDuration<T>. The option-wrapped version does a checked
  operation, and the other will panic if the result of the operation is not
  representable by the LimitedResolutionDuration.

• BonsaiListenConfig is a new structure that controls the settings of the
  BonsaiDb network protocol server socket. This structure currently allows
  specifying the specific SocketAddr to listen on and whether the
  SO_REUSEADDR flag should be specified on the underlying socket.

• CollectionDocument now implements Serialize and/or Deserialize if both
  PrimaryKey and Contents are Serialize and/or Deserialize, respectively.

• Backend::client_session_ended is a new function that is invoked any time a
  connected client's session is ending.

• VarInt<T> is a new type that implements Key using the ordered-varint
  crate. This allows using types such as VarInt<u64> instead of u64 to
  reduce the number of bytes encoded keys consume on average.

• SerializedCollection now has insert_in_transaction(),
  push_in_transaction(), and overwrite_in_transaction() which are new
  helpers that help writing transactional code easier by creating and pushing
  the Operations in one step.

• CollectionDocument<T> now has update_in_transaction() and
  delete_in_transaction() which are new helpers that help writing
  transactional code easier by creating and pushing the Operations in one
  step.

• Client's builder now has two additional settings: request_timeout and
  connect_timeout. If not specified, both timeouts are 60 seconds. Thank you
  to @phantie for requesting these settings in #296.

• CollectionDocument::refresh()/CollectionDocument::refresh_async() are new
  methods that reload the document from the database.

• ConnectedClient::connected() is a new function that returns the current
  state of whether the client is still connected. Because ConnectedClient is
  able to be cloned and sent between threads, this allows other threads to
  notice when clients have disconnected without needing to be notified via the
  Backend.

Changed

• bonsaidb::cli::Command::Server now calls CommandLine::open_server() rather
  than constructing the server directly. This allows CommandLine implementors to
  use open_server as a location to launch extra services upon server startup.
• bonsaidb::cli::Args and bonsaidb::cli::Command::execute now accept a token
  id or a username, allowing for commands to be executed with authentication if
  the features are enabled.
• bonsaidb::local::cli::Command now offers the ability to create a user and
  set a user's password.
• bonsaidb::local::cli::admin is a newly exposed module that allows some basic
  user management. This set of commands is also available on
  bonsaidb::cli::Command through the Admin variant, allowing for both local
  and remote administration.
• Header, CollectionHeader, and Revision now all implement Hash.
• Database names are no longer case insensitive. This was implemented poorly,
  and in hindsight, the feature was removed in favor of strictness. Because the
  database name is used as a directory name on-disk, care must be taken on
  case-insensitive filesystems to not attempt to create two databases with
  different casing.
• TimedArgonParameters now guarantees that the minimum parameters chosen will
  meet the OWASP recommendations. Manual configuration still is allowed to set
  exact parameters.

Fixed

• Unique views/eager views now are properly updated when an overwrite operation
  is performed that resulted in a new document being created.
• Argon2 ram configuration is now correctly applied. Previously, the memory cost
  was being supplied in bytes, but the underlying API was expecting kilobytes.
• When a View's version has changed, the view is now fully cleaned before
  reindexing. Previously, BonsaiDb was lazily cleaning up the entries, which led
  to slower reindexing and disk bloat.
• Fixed the source field of mappings returned from a View query when a
  document was updated but emitted the same key. Previously the value was
  correctly updated, but the source's revision was not updated.
• When querying a View with AccessPolicy::NoUpdate, the integrity scanner is
  now checked and waited upon before allowing access to the view.
• When querying a View with AccessPolicy::UpdateAfter, the update task is no
  longer blocked until complete.
• CustomServer::listen_for_shutdown() now listens for
  CustomServer::shutdown() in addition to operating system signals.
• Client will no longer return a Session from a previous connection. Because
  the Client automatically reconnects, the Sessions are no longer
  authenticated.
• CustomServer::shutdown now fully shuts down the server by forcefully
  disconnecting clients after the optional grace period has elapsed.
  Additionally, QUIC-connected workers are sent the proper disconnection
  notification.
• The Key generic parameter to Operation::overwrite_serialized is now used
  in the function definition. Because previously C::PrimaryKey was hard-coded,
  any existing code should work as long as the correct Key type was provided.
  This fix also allows for types like str to be used instead of String with
  this function.
• Limited case insensitivity was implemented incorrectly yielding inconsistent
  results.
• CustomServer::listen_on no longer will return an error if an incoming
  connection fails during the TLS or QUIC handshake. Thank you to @phantie for
  reporting this in #296.