Skip to main content

ETL Pipeline Deployment Workflows

  1. Export the Prompt studio project as a tool

  2. Now that you've exported a Prompt studio project, let's create a Workflow that we'll launch as an Task pipeline. First off, let's create a new empty Workflow:

    • From the side navigation menu, choose Workflows 🞂 New Workflow. This should bring up the new Workflow dialog: img Create New Workflow
    • For Workflow Name enter any name for your workflow.
    • For Description enter some detail about the type of workflow
    • Click on Create Workflow. This should land you in an empty workflow in the Workflow Studio, which allows you to build sophisticated workflows.

  3. Let's configure the workflow. img Create New Workflow

    1. Input Configuration: To build an ETL Pipeline, choose File System from this list and click on the settings button near the dropdown and configure the input file system
    2. Output Configuration: To build an ETL Pipeline, choose Database from this list and click on the settings button near the dropdown and configure the database system
    3. Tools Pane: this pane lists all built in and user created custom Tools available in the platform. Prompt Studio projects you export are made available as Tools in this pane from where you an drop and drop them into the Workflow Chain to construct Workflows you want.
    4. Workflow Chain: Search for your prompt studio exported project under Tools, and drag and drop it in the workflow container(in between Input and output configuration)
    5. Tool Settings Pane: This is where settings for various tools are made available. You'll need to select a Tool from the Workflow Chain in order to change the settings of Tools from here.Below is the detailed screenshot for point no.5 img Create New Workflow
    6. Deploy As ETL Pipeline: use this button to deploy workflows as unstructured data ETL Pipelines.