Xbus filegateway

Provides actors that act as gateway to/from a filesystem. The filesystem can be local or remote.

Services

The provided actor services are:

file-gateway-emitter

Observe a pattern and send any matching file to the bus, then delete it.

Settings:

  • source: The source url/pattern to watch

  • source.<option>: Extra option for the source filesystem

  • message-type: The message type to send to the bus

  • post-action: What to do with the file after emitting it. Can be:

    • "delete" (default): delete the file

    • "rename <template>": rename the file to the given name. <template> is a golang text/template template string, with the following variables available:

      • {{ .Full }}: the complete source file path

      • {{ .Dir }}: the source file directory

      • {{ .Base }}: the source file base name (name without the directory)

      • {{ .Ext }}: the source file extension, leading dot included

      • {{ .WithoutExt }}: the source file base name with the extension removed

      • {{ .EnvelopeID }}: the emitted envelope id

      Warning

      Make sure the target filename does _not_ match the source pattern or you will get an infinite loop of envelopes sent to the bus.

Example:

service:
  type: file-gateway-emitter
  settings:
    source: scp://root:password@somehost/factures/*.pdf
    source.host-pubkey: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIL+BET9m3jCj5yhI1cRrHJF8HDG5ZMnCl/vI07sY+Oyr somehost
    source.polling-interval: 1m
    message-type: printed-invoice

file-gateway-consumer

Write the received files to a target directory.

Settings:

  • target: The target directory uri

  • target.<option>: Extra option for the target filesystem

Example:

service:
  type: file-gateway-consumer
  settings:
    target: file:///exports/
    target.prefix: /var/xbus-data/

Filesystems

A filesystem url is a standard url, in which the query params are ignored. It can contain user/password, a host, a port number.

Common options

  • polling-interval - for polling-based implementations, the interval between polls. Supported units are “h”, “m”, and “s”. The default is “10s”.

file:// - local filesystem

The url host must be empty.

Options:

  • prefix: The base path in which to work

scp:// - ssh/scp remote filesystem

The url host must be set.

Password and private key authentication is supported.

Options:

  • basedir: The base path in which to work. Default: “.”

  • private-key: The private-key to use for authentication, as a PEM encoded string.

  • host-pubkey: The remote host public key, in the authorized key format. Mandatory.

sftp:// - sftp remote filesystem

The url host must be set.

Password and private key authentication is supported.

Options:

  • basedir: The base path in which to work. Default: “.”

  • private-key: The private-key to use for authentication, as a PEM encoded string.

  • host-pubkey: The remote host public key, in the authorized key format. Mandatory.

  • maxconn (default 2): The maximum number of concurrent connections to the target host.

  • keep-alive (default “1m”): How long to keep a connection opened after last using it.

ftp:// - ftp remote filesystem

The url host must be set

A username and a password must be set.

Options:

  • maxconn (default 2): The maximum number of concurrent connections to the target host.

  • flavor (default “default”): Specify the kind of ftp server. Can be “default” or “vsftpd”.

  • basedir (default “/”): The base path. If the emitter does not work, try “”.

  • tempdir (default “”): A local path for storing temporary files. If empty, use the system default directory for temporary files.

  • maxmembufsize (default 16777216 - 16MiB): How much of a file to store in memory before swithing to a temporary file.

  • maxattempts (default 5): How many times to re-attempt an operation when getting a network timeout error.

ftps:// - ftps remote filesystem

Same options as ftp://, plus the following:

  • host-cert (required): the pem encoded public key of the remote server.

  • tls-mode: can be “explicit” (default) or “implicit”. The value must match the server tls mode.

  • tls-servername: The server name according to the certificate, if different from the URL host.

gcs:// - Google Cloud Storage

In a GCS url, the ‘host’ part is actually the bucket name. For example: gcs://BUCKET/, or gcs://BUCKET/PATH.

Additional options are:

  • endpoint: The API endpoint, if not the official google api. Should be a complete URL, for example: http://localhost:8080/storage/v1/.

  • credentials: The google credentials, as encoded JSON. If equal to ‘noauth’, the default credentials will not be used.