Filecatalyst Workload Automation ~upd~ (2024)

#!/bin/bash # workload_processor.sh # Step 1: Compress files tar -czf /data/prepared/batch1.tar.gz /data/raw/*.csv fta-cli --server fc.example.com --port 11001 --username auto_user --put /data/prepared/batch1.tar.gz --target /incoming/ Step 3: Verify success (check exit code) if [ $? -eq 0 ]; then echo "Transfer successful, triggering downstream API" curl -X POST https://processing.api/start -d '"file":"batch1.tar.gz"' else echo "Transfer failed" >> /var/log/fc_errors.log fi Method B: Hotfolders – Best for Simple, Event-Driven Workloads Configure hotfolder.properties to watch a directory. Any file dropped is automatically transferred.

# Retry up to 3 times RETRIES=3 for i in $(seq 1 $RETRIES); do fta-cli --put critical_file.dat --target /incoming/ && break || sleep 10 done | Tool | Integration Method | |------|--------------------| | Apache Airflow | Use BashOperator with fta-cli or SimpleHttpOperator for REST API | | Jenkins | Execute shell script step calling fta-cli | | Rundeck | Create a job step: "Command" → fta-cli ... | | Control-M | FileCatalyst provides a Control-M plugin (File Transfer Hub) | | Apache NiFi | Use ExecuteProcess processor to call fta-cli | filecatalyst workload automation

For native workload automation features (dependency management, SLA tracking, visual pipelines), you would typically wrap FileCatalyst commands into a dedicated workload automation platform like , using FileCatalyst as the file movement plugin. # Retry up to 3 times RETRIES=3 for

success = run_fta(f, "/incoming/", "fc-server.company.com", "auto", "secret") if success: logging.info(f"Success: f") # Post-processing: log to database subprocess.run(["psql", "-c", f"INSERT INTO transfers VALUES('f', 'original_hash')"]) else: logging.error(f"Failed: f") time.sleep(30) # Backoff before retry if == " main ": main() Summary Table: Choosing an Automation Method | Requirement | Recommended Method | |-------------|--------------------| | Simple directory watching | Hotfolder | | Scripted, scheduled transfers | CLI + cron/systemd timer | | Complex workflow with multiple steps | CLI + Bash/Python logic | | Integration with Airflow/Jenkins | REST API or BashOperator | | Central management of many transfers | REST API + custom dashboard | f"INSERT INTO transfers VALUES('f'

fta-cli --server hostname --port 21 --username user --password pass \ --put /local/file.txt --target /remote/destination/

headers = "X-API-Key": API_KEY resp = requests.post(f"API_BASE/transfer", json=payload, headers=headers) transfer_id = resp.json()["id"]