Skip to main content

Sharing data source configuration

For larger datasets, Motif can move your query processing from the browser to a dedicated pool of workers. We can configure our orchestration layer to regularly ingest your dataset - for many customers, every 24 hours is sufficient. Motif can ingest data from a broad array of external sources.

Each source has specific instructions for configuring external access, but all share some common steps. Generally, it involves granting IAM permissions to our or provisioning a dedicated service account with credentials that we can use to authenticate to access your warehouse or bucket.

Collect Credentials:​

Depending on your source, we will need the appropriate credentials provided to us.

For example, to ingest from S3, we need the following information:

  • Bucket Name
  • AWS Access Key ID
  • Secret Access Key
  • File extension

See the relevant data source page for more details:

Map Core Event Dimensions:​

Motif expects one table of events (rows) and dimensions (columns). The following column names are needed to configure Motif to interpret your dataset into sequences and should be provided along with the credentials in the next step.

Note: You can test which fields to use with a subset of your data by loading a subset of your data in Local mode in Motif.

Actor​

Column used for grouping events into separate sequences. No need to break actor sequences into “sessions” - you will be able to do it with a query later in Motif.

Time​

Column used for ordering events within each sequence.

Event Name​

Column used as event name in Motif UI.

Share Credentials & Mapping:​

  • The information above can be shared with us through the secure mechanism of your choice.

  • Note: We are currently working on a dedicated solution for customer-managed dataset & credential management, but this will likely not be ready until Q3 2024

  • A simple option which works well for most customers and with our own internal credential management is to share this information via OnePassword. See this documentation for more details.

  • For example, to create and share the above information with us via OnePassword, you would create the following custom API credential, and share it like so:

motif's orchestration service

  • Once we have this information, we will configure our system to begin regular ingestion of your data into Motif’s cloud. Once your import is configured, those in your organization who have been granted Motif accounts will be able to load an up-to-date replica of your dataset as easily as any other dataset from the top left dropdown menu.