Skip to main content

Set Up Task Pipeline

This guide shows you how to create and configure a Task Pipeline in Unstract.

Prerequisites​

  1. Export a Prompt Studio project as a tool
  2. Configure your source filesystem connector
  3. Configure your destination filesystem connector (can be the same or different from source)

Creating the Task Pipeline​

  1. Navigate to Workflows in the side navigation
  2. Create a new workflow following the Workflow Setup Guide
  3. Configure your workflow:
    • Source: Select your filesystem connector
    • Destination: Select your filesystem connector (for output)
    • Tool: Select your exported Prompt Studio project
  4. Click Deploy as TASK
tip

The source and destination can use the same filesystem connector, but ensure the output folder is different from the input folders to avoid circular processing.

Option 2: Direct Task Creation​

  1. Navigate to Task Pipelines in the side navigation
  2. Click New Task Pipeline
  3. Select an existing workflow or create a new one
  4. Configure Task-specific settings (see below)

Task-Specific Configuration​

Scheduling​

Set up automated execution with cron scheduling:

  1. For Display Name, enter a descriptive name for your Task pipeline
  2. In Cron Schedule, specify when the Task should trigger (in UTC time)
    • Example: 0 3 * * * (runs daily at 3:00 AM UTC)

Task Deployment Configuration

Deployment Confirmation​

Once deployed, the Task pipeline will automatically trigger at the configured time.

Managing Your Task Pipeline​

Manual Execution​

Trigger your Task pipeline on demand:

  • Task Pipeline → Actions → Manual Sync

Monitoring Execution​

View execution logs and status:

  • Task Pipeline → Actions → View Logs

Or navigate to: MANAGE → Logs → Task Sessions

File Processing Behaviour

Once a file is processed in an ETL pipeline, it is tracked to avoid reprocessing. If you wish to reprocess the files, you can clear the processed file history.

  • Clear Processed File History: ETL Pipeline → Actions → Clear Processed File History

Notifications​

Set up alerts for your Task pipeline:

Task Pipelines → Actions → Setup Notifications