Documentation

Custom Pipelines let teams turn repeatable marketing operations into AI-built, production-ready workflows. Instead of manually running one-off scripts, stitching tools together, or asking engineering to build every custom automation, users can describe the workflow to Improvado Agent and get a reusable pipeline that can be run on demand or scheduled.

At a business level, this is useful when a process is too specific for a standard product feature, but too recurring or valuable to keep doing manually. Examples include cross-platform audience syncs, recurring data quality checks, campaign launch validations, reverse ETL into CRM or ad platforms, long-running backfills, budget pacing checks, and custom reporting jobs.

What you can do

• Create custom Python workflows with AI Agent.
• Run pipelines manually when you need an immediate result.
• Schedule recurring runs with cron-based automation.
• Connect pipelines to Improvado data sources and destinations using saved workspace connections.
• Use durable state for cursors, bookmarks, and incremental processing.
• Track every run with status, timing, output, errors, and activity timeline.
• Publish artifacts such as reports, extracts, or debug files for later download.
• Pause, resume, edit, or archive pipelines from the Custom Pipelines page.

When to use it

Use Custom Pipelines for workflows that combine data movement, transformation, validation, and external actions. A good fit is any process where the team can clearly describe the inputs, business rules, and expected output, but the workflow is too custom for a fixed UI wizard.

Typical use cases:

• Backfill 12 months of spend or conversion data with resumable daily batches.
• Sync audiences from a warehouse or CDP into Meta Ads, Google Ads, TikTok, or LinkedIn.
• Check marketing tables for freshness, missing data, or broken naming rules on a schedule.
• Push attribution or lifecycle metrics back into HubSpot, Salesforce, or other business tools.
• Validate campaign briefs, UTMs, landing pages, and tracking before launch.
• Generate recurring reports or files and publish them as pipeline artifacts.

How it works

A pipeline is created through Improvado Agent. The Agent writes the workflow, configures the required connections, and saves the pipeline in the workspace. Depending on the pipeline type, it can run either as a subprocess pipeline or as a Temporal workflow pipeline.

Temporal workflow pipelines are designed for long-running, multi-step, retryable workflows. Temporal gives the workflow durable execution, retries, scheduling, activity timelines, fan-out/fan-in patterns, and safer recovery from worker restarts.

Credentials are not hardcoded in pipeline code. Connections are declared at the pipeline level and resolved securely at runtime. The workflow receives tenant-scoped access to the selected data sources, destinations, Improvado storage, and temporary file or artifact storage.

How to use

1. Open Custom Pipelines.
2. Click New Pipeline.
3. Describe the workflow to Improvado Agent: what data to use, what should happen, how often it should run, and what output you expect.
4. Review the generated pipeline overview, configuration, connections, and schedule.
5. Run the pipeline manually for the first test.
6. Check Run History for status, timeline, output, errors, and generated artifacts.
7. Use Edit with AI to adjust logic, parameters, schedule, or connections.
8. Pause or resume the pipeline when you need to temporarily stop or restart scheduled execution.

What users see

The Custom Pipelines list shows all available pipelines in the workspace, including status, category, schedule, last run, creator, and pipeline type. Temporal pipelines are marked with a Temporal badge.

Each pipeline has a detail page with:

• Overview: summary, status, schedule, category, and high-level configuration.
• Configuration: visual workflow graph, step details, and source code.
• Run History: recent executions, live status, duration, timeline, output, logs, metrics, and errors.

Good prompt examples

“Create a daily data quality pipeline that checks yesterday’s Meta Ads and Google Ads spend tables for missing rows, stale data, and zero-spend anomalies. Send a summary artifact with the failed checks.”

“Build a weekly reverse ETL pipeline that reads high-LTV customers from Improvado Storage and pushes audience updates to Meta Ads and Google Ads.”

“Create a campaign launch validation pipeline that reads a Google Sheet with campaign briefs, checks naming conventions and landing page tracking, then returns a report with pass/fail results.”