Introduction

Purpose

This policy establishes the guidelines and procedures for handling the migration and deprecation of integrations and data extraction processes at Improvado. It ensures that all migrations, whether due to external API deprecation or internal system upgrades, are managed in a transparent, efficient manner that minimizes disruption to client operations while maintaining data integrity and service quality.

Scope

This policy applies to all data integrations, extraction orders, and related processes managed by Improvado, including those affected by third-party API changes and internal system migrations. It covers the procedures for notification, migration implementation, data preservation, and decommissioning of deprecated systems or processes.

Objectives

• Establish clear processes for managing integration migrations and deprecations
• Ensure timely and transparent communication with clients about upcoming changes
• Minimize service disruptions during migration periods
• Maintain data integrity and accessibility throughout transition periods
• Define consistent timelines and procedures for deprecating outdated systems
• Provide guidelines for data retention during and after migrations

Definitions and Terminology

To ensure clarity and compliance, the following terms are used throughout this policy:

• Data provider - A third-party platform or service that is the source of data extracted by Improvado.
• API (Application Programming Interface) - A set of protocols that allows different software systems to communicate with each other.
• Extraction order - A configured process that extracts data from a specific source according to defined parameters.
• Deprecated API - An API version that has been marked for removal by its provider.
• Migration - The process of moving from one system, platform, or API version to another.
• Data table - A structured set of data stored in a database.

Procedure

Migration Scenarios

There are two main scenarios in which we need to migrate client extraction orders:

External API Deprecation

When a data provider deprecates a specific API version on their side:

• Example: Reddit deprecated all v2 endpoints. Our older reports were using those v2 endpoints, so we created new reports based on the v3 API.
• We continuously and automatically monitor upcoming breaking changes from providers and update our integrations in a timely manner. Every 60 days, we automatically scan the API documentation of each platform and use AI to determine the latest available version and the sunset dates of the current one. If we detect any changes that may affect our clients, we always notify them via email.

Internal Cross-Service Migration

When Improvado needs to migrate from one internal system to another:

• Example: Migration from our legacy extraction system to a new one.
• These migrations are typically planned well in advance as part of system improvement initiatives.

Both scenarios require creating new reports, which in turn require creating new extraction orders. Since new extracts use new data tables, we ask clients to update their BI tools to point from the old tables to the new ones.

Migration Process

Notification and Planning

• Before any migration or deprecation, we notify users via email at least 14 days in advance when possible.
• Some providers announce changes only a few days before they take effect, in which case we notify clients as soon as we become aware of the changes.
• The notification includes:
  • Reason for the migration
  • Timeline for implementation
  • Actions required by clients
  • Impacts on existing data and reports
  • Contact information for support during the transition

Implementation

During migration/deprecation, we follow these steps:

1. Create new extraction orders for the new reports - only for accounts that were used in the old extracts.
2. Pause the old extraction orders.
3. Provide support to clients as they update their BI tools to reference the new data tables.
4. Monitor the new extraction orders to ensure proper functionality.

Decommissioning

• 30 days after the migration (after the new orders are created), we delete the old extraction orders.
• The specific deletion date is always mentioned in the notification email.
• Old data in existing data tables remains intact - clients can still view it in Data Preview or re-load it if needed.
• We only remove the extraction orders to keep things clean and avoid confusion, as those orders would fail upon resuming anyway.

Data Management

Data Preservation

• Historical data in existing tables is preserved and remains accessible even after extraction orders are deleted.
• Clients can access this data through the Data Preview feature or by re-loading it from Improvado's storage.

Data Continuity

• Upon request, we can join historical data from the old tables into the new ones.
• In most cases, we expect clients to handle this integration on their side according to their specific requirements.

Data Table Management

• New extraction orders create new data tables with updated schemas when necessary.
• Clients are responsible for updating their BI tools and reports to reference these new tables.
• Documentation on new table schemas is provided when significant changes occur.

Communication Protocol

Client Notifications

• Email is the primary channel for notifying clients about upcoming migrations and deprecations.
• Notifications include clear timelines, actions required, and support contacts.
• For major migrations affecting many clients, additional communication channels may be used (e.g., in-app notifications, account manager outreach).

Feedback and Support

• Dedicated support channels are established during major migrations to address client questions and issues.
• Feedback collected during migrations is used to improve future migration processes.

Emergency Procedures

Urgent API Deprecations

• In cases where data providers announce immediate or short-notice API deprecations (less than 14 days), an expedited migration process is initiated.
• The Engineering team prioritizes developing solutions for these urgent migrations.
• Clients are notified as soon as possible with the abbreviated timeline and any immediate actions required.

Migration Rollback

• If significant issues are discovered during a migration, a rollback procedure may be initiated.
• The decision to roll back is made by the Engineering team in consultation with Product Management.
• Clients are notified promptly of any rollback and its implications.

Monitoring and Metrics

Improvado monitors the following metrics related to migrations and deprecations:

• API deprecation detection time vs. provider announcement time
• Advance notice provided to clients (target: minimum 14 days)
• Migration completion rate within planned timeframes
• Client support requests during migration periods
• System performance during and after migrations

‍