Xbus filegateway
================

Provides actors that act as gateway to/from a filesystem.
The filesystem can be local or remote.

Services
--------

The provided actor services are:

``file-gateway-emitter``
''''''''''''''''''''''''

Observe a pattern and send any matching file to the bus, then delete it.

Settings:

- ``source``: The source url/pattern to watch
- ``source.<option>``: Extra option for the source filesystem
- ``message-type``: The message type to send to the bus
- ``post-action``: What to do with the file after emitting it. Can be:

   - ``"delete"`` (default): delete the file
   - ``"rename <template>"``: rename the file to the given name. ``<template>``
     is a golang text/template template string, with the following variables
     available:

     - ``{{ .Full }}``: the complete source file path
     - ``{{ .Dir }}``: the source file directory
     - ``{{ .Base }}``: the source file base name (name without the directory)
     - ``{{ .Ext }}``: the source file extension, leading dot included
     - ``{{ .WithoutExt }}``: the source file base name with the extension removed
     - ``{{ .EnvelopeID }}``: the emitted envelope id

     .. warning::

         Make sure the target filename does _not_ match the source pattern or
         you will get an infinite loop of envelopes sent to the bus.

Example:

.. code-block:: yaml

   service:
     type: file-gateway-emitter
     settings:
       source: scp://root:password@somehost/factures/*.pdf
       source.host-pubkey: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIL+BET9m3jCj5yhI1cRrHJF8HDG5ZMnCl/vI07sY+Oyr somehost
       source.polling-interval: 1m
       message-type: printed-invoice

``file-gateway-consumer``
'''''''''''''''''''''''''

Write the received files to a target directory.

Settings:

- ``target``: The target directory uri
- ``target.<option>``: Extra option for the target filesystem

Example:

.. code-block:: yaml

   service:
     type: file-gateway-consumer
     settings:
       target: file:///exports/
       target.prefix: /var/xbus-data/


Filesystems
-----------

A filesystem url is a standard url, in which the query params are ignored.
It can contain user/password, a host, a port number.

Common options
''''''''''''''

- ``polling-interval`` - for polling-based implementations, the interval
  between polls. Supported units are "h", "m", and "s".
  The default is "10s".


``file://`` - local filesystem
''''''''''''''''''''''''''''''

The url host must be empty.

Options:

- ``prefix``: The base path in which to work

``scp://`` - ssh/scp remote filesystem
''''''''''''''''''''''''''''''''''''''

The url host must be set.

Password and private key authentication is supported.

Options:

- ``basedir``: The base path in which to work. Default: "."
- ``private-key``: The private-key to use for authentication, as a PEM encoded string.
- ``host-pubkey``: The remote host public key, in the authorized key format. Mandatory.

``sftp://`` - sftp remote filesystem
''''''''''''''''''''''''''''''''''''

The url host must be set.

Password and private key authentication is supported.

Options:

- ``basedir``: The base path in which to work. Default: "."
- ``private-key``: The private-key to use for authentication, as a PEM encoded string.
- ``host-pubkey``: The remote host public key, in the authorized key format. Mandatory.
- ``maxconn`` (default 2): The maximum number of concurrent connections to the target host.
- ``keep-alive`` (default "1m"): How long to keep a connection opened after last using it.


``ftp://`` - ftp remote filesystem
''''''''''''''''''''''''''''''''''

The url host must be set

A username and a password must be set.

Options:

- ``maxconn`` (default 2): The maximum number of concurrent connections to the
  target host.
- ``flavor`` (default "default"): Specify the kind of ftp server.
  Can be "default" or "vsftpd".
- ``basedir`` (default "/"): The base path. If the emitter does not work, try "".
- ``tempdir`` (default ""): A local path for storing temporary files. If empty, use
  the system default directory for temporary files.
- ``maxmembufsize`` (default ``16777216`` - 16MiB): How much of a file to store
  in memory before swithing to a temporary file.
- ``maxattempts`` (default ``5``): How many times to re-attempt an operation when
  getting a network timeout error.


``ftps://`` - ftps remote filesystem
''''''''''''''''''''''''''''''''''''

Same options as ``ftp://``, plus the following:

- ``host-cert`` (required): the pem encoded public key of the remote server.
- ``tls-mode``: can be "explicit" (default) or "implicit". The value must match
  the server tls mode.
- ``tls-servername``: The server name according to the certificate, if different
  from the URL host.

``gcs://`` - Google Cloud Storage
'''''''''''''''''''''''''''''''''

In a GCS url, the 'host' part is actually the bucket name. For example:
``gcs://BUCKET/``, or ``gcs://BUCKET/PATH``.

Additional options are:

- ``endpoint``: The API endpoint, if not the official google api. Should be a
  complete URL, for example: ``http://localhost:8080/storage/v1/``.
- ``credentials``: The google credentials, as encoded JSON. If equal to 'noauth',
  the default credentials will not be used.