Skip to main content

File History

File History keeps track of files that have been processed by your workflows. It helps prevent duplicate processing and lets you see the status of each file.

What File History Does​

  • Tracks processed files: Records every file that goes through your workflow
  • Prevents duplicates: Successfully processed files are automatically skipped in future runs
  • Shows errors: Displays which files failed and why
  • Enables reprocessing: Delete records to process files again

Accessing File History​

From ETL Pipeline, Task Pipeline, or Workflow​

  1. Navigate to ETL Pipelines, Task Pipelines, or Workflows page
  2. Find your pipeline or workflow in the list
  3. Click the Actions menu (three dots)
  4. Select View File History

Accessing File History from Actions Menu

Understanding the File History Table​

File History Table

Table Columns​

ColumnDescription
File PathThe location of the processed file
StatusWhether processing succeeded or failed
Total AttemptsNumber of times the file was processed (e.g., "1 / 3" means 1 attempt out of 3 maximum)
Last AttemptWhen the file was last processed
ErrorError message if processing failed
ActionDelete button to remove the record

Status Values​

StatusColorMeaning
CompletedGreenFile processed successfully. Won't be processed again
ErrorRedProcessing failed. Will retry automatically (up to 3 times)

Filtering Files​

Use the filters at the top of the File History table to find specific files:

FilterWhat It Does
File PathSearch by file name or path
StatusShow only Completed or Error files
Min AttemptsShow files processed at least N times
Max AttemptsShow files processed at most N times

After setting filters, click Apply to update the results. Click Reset to clear all filters.

Clearing File History​

Clear file history when you want to reprocess files that were already completed.

Clear All Files​

Quick way to clear everything:

  1. Go to your pipeline's Actions menu
  2. Select Clear Processed File History
  3. All records for that pipeline are removed

Delete Individual Files​

  1. Open View File History
  2. Find the file in the table
  3. Click the Delete button in the Action column
  4. Confirm the deletion

Delete Multiple Files​

  1. Open View File History
  2. Check the boxes next to files you want to delete
  3. Click Delete Selected
  4. Confirm the deletion

Delete Files Matching Filters​

  1. Apply filters to show only the files you want to delete
  2. Click Clear with Filters
  3. Review the count and confirm

Common Questions​

Why isn't my file being processed?​

If a file was already processed successfully, it won't run again. To reprocess it:

  1. Open View File History
  2. Find the file
  3. Delete its record
  4. Run your pipeline again

What happens when a file fails?​

Files with errors will automatically retry on the next pipeline run, up to 3 times total. After 3 failed attempts, the file stops retrying.

How do I reprocess a file that failed 3 times?​

  1. Check the Error column in File History to understand what went wrong
  2. Review the Execution Logs for more details (navigate to MANAGE → Logs in the left panel)
  3. Fix the issue with the file if possible
  4. Open View File History and delete the record
  5. Run your pipeline again

How do I reprocess all files?​

Use Clear Processed File History from the Actions menu to remove all records, then run your pipeline.

Tips​

  • Check errors first: Look at the Error column to understand why files failed
  • Use filters: When you have many files, filters help you find specific ones quickly
  • Copy paths easily: Click the copy icon next to file paths to copy them
  • Refresh for updates: Click the Refresh button to see the latest data