Envelope Storage

Starting with version 3.3.0, Xbus provides a more efficient way to store the envelopes in the database. This new storage is not enabled by default.

A typical post-3.3.0 migration configuration would be:

storage:
  envelope:
    impl: sql-ng
    fallback: sql
    cache:
      enabled: true
      max-size: 1G

Primary Storage

Setting the primary storage, ie the storage for new envelopes, is done with the option storage.envelope.impl. It can take one of the following values:

  • sql (default). The historical envelope storage. It stores the envelope content in different tables, allowing an access to the envelope inner data.

    The tables used are:

    • envelope : store the envelope metadata

    • event: store the message metadata (id, sum…)

    • item: store data chunks of the messages

    This storage has some performances issues, as it has to split then rebuild the fragments all the time.

  • sql-ng. This storage stores directly the serialized envelope fragments, allowing a much faster read and write.

    It uses the following tables:

    • envelope_storage stores the envelope metadata

    • envelope_storage_data stores the fragments

Fallback storage

As there is no way to migrate envelopes from a storage to another, a mechanism of fallback storage is provided.

It allows to specify a secondary storage, which will be used for reading envelopes that are not found in the primary storage.

The setting is storage.envelope.fallback, and the possible values are the same as storage.envelope.impl.

Cache

A cache can be enabled on top of the chosen storage. Two options are available:

storage.envelope.cache.enabled

Enable the cache

storage.envelope.cache.max-size

(default 1073741824 (ie 1GiB)) Set the cache size, in bytes.

Metrics

When xbusd metrics are enabled, new metrics are available:

  • xbus_storage_calls, a summary with the following dynamic labels:

    • name: The storage name (“cache”, “default”…)

    • implementation: The storage implementation (“cache”, “sql”, “sql-ng”)

    • function: The function. Possible values are:

      • ReadEnvelope: Read a fragment

      • StoreEnvelope: Store a fragment