Resource Migration

1. Overview of Migration Function

Applicable Scenarios

Scenario 1: Test to Production Migration: When strictly separating test and production, resources that have been modified in the test environment need to be migrated to production to reduce the frequency of changes in production, ensure consistency between the two environments, and improve migration efficiency.

Scenario 2: Headquarters (/Head Office) Template Synchronization to Branches: When headquarters wants to synchronize data standards or dashboard templates to branches, the migration function can be used to synchronize headquarters templates to other environments.

Scenario 3: OEM Vendors Initialize Dashboards in Customer Environments: After integrating Guanyuan BI into other B-end platforms, the migration function can be used to initialize dashboards or migrate industry templates, and continuously update them.

Function Introduction

Online migration is suitable for customers whose test and production networks are interconnected. By configuring in Management Center - Operation & Maintenance Management - Resource Migration and clicking migrate on the corresponding resource page, test resources can be synchronized to production. See [Online Migration](4-Online Migration.md) for details.

Offline migration is suitable for customers whose test and production networks are isolated. Export the resource package in the test environment's Management Center - Operation & Maintenance Management - Resource Migration, then import it in the same path in the production environment to synchronize test resources to production.

2. Migration Logic

Terminology

• Export Settings - Export Upstream Resources: This will include the upstream resources of the selected resource, such as exporting a dashboard will include upstream data accounts, datasets, ETL. You can choose not to import some of them during import.

• Associated Resources: Resources not displayed on the resource lineage page, such as a dashboard's associated resources will show specific cards, and a dataset's associated resources will show global parameters, custom drivers, etc.

Resource Migration Logic Diagram

When import selects ID change

When import selects ID unchanged

3. Usage Restrictions

Various restrictions are reflected in best practices - step 1: pre-migration instructions and step 4: post-import configuration checklist.

• Supported for administrator use. If ordinary users need to use it, relevant RBAC permissions must be granted through custom roles;

• Resource permissions, dataset row/column permissions, and desensitization templates for various resources will not be migrated during migration;

• To ensure data account security, migration of data account passwords is not supported;

• To avoid performance issues and long migration times, data migration is not supported;

• Some special configurations and resources may not be covered by migration and need to be rechecked, such as custom fonts and component templates.

4. Best Practices

Function Entry

Management Center > Resource Management > Resource Migration.

Step 1. Pre-migration Instructions

Version: It is recommended to use the same version.

Feature switch: Please ensure that the functions of both environments are consistent.

If ordinary users need to perform migration, you can set as follows:

Administrator Configuration

To allow ordinary users to perform migration, configure functional permissions for the ordinary user role under [Management Center - User Management - Role Configuration];

Step 2. Export the Appropriate Resource Package

Supports exporting dashboards, data screens, smart attribution pages, mobile light applications, desktop applications, ETL, and datasets that you have permission for.

Application package name: Supports up to 20 characters. You can use unique identifiers such as date or content for easy subsequent queries;

Export settings:

[Export Upstream Resources]: It is recommended to check this for the first migration, unless the relevant upstream resources already exist in the target environment; for subsequent migrations, it is generally not checked;

Resource selection: Select the resources to be exported for the corresponding resource type.

After exporting, click the migration record tab to download the resource package. If there are many resources, there may be a waiting time.

Note: Support for card datasets was added after version V6.5.0.

Step 3. Import the Resource Package

Resource import: Only zip package files exported by the system are supported;

Resource parsing:

• Whether the resource ID changes

  • Select Yes: The imported resource ID changes, commonly used in the following scenarios: 1) One-time import with no subsequent iteration required; 2) The target environment requires the resource in multiple domains;
  • Select No: The imported resource ID does not change, commonly used in the following scenarios: 1) After importing into the target environment, migration needs to be iterated continuously; 2) It is expected to ensure the uniqueness of the resource in the target environment.

• Parsing list

  • Before import, display resource type, resource name, resource ID before import, import method. You can choose to import or not, but please confirm that the upstream resources not imported already exist in the target environment with the same ID, otherwise the import may fail;

• After import, the imported resource ID, path, and import status will be displayed. Please keep this page until Step 3 is completed.

If there is an error during import, please adjust according to the error message and re-import.

• If you selected ID unchanged, you can directly re-import the adjusted resource package;

• If you selected ID changed, please clean up the previously successfully generated resources.

Step 4. Post-import Configuration Checklist

Please check resource issues in the order of this list. You can click [Imported Resource ID] on the post-import list page to jump directly to the corresponding resource. Data accounts need to be searched manually.

Resource Type	Items to Check and Possible Issues
Data Account	For password confidentiality, data account passwords need to be re-entered and user permissions added.
Dataset	Due to the possible large size of datasets and performance risks, data migration is not performed. For direct or extracted datasets, run updates manually and check update configuration, row/column permission templates, data desensitization templates, and add user permissions. For file datasets, replace the dataset manually and check row/column permission templates, data desensitization templates, and add user permissions. For fill-in datasets and FTP datasets, check whether the fill-in function works properly. Currently unsupported dataset types include third-party datasets (Erwei Huo, Xiaoshouyi, Wenjuanwang, WeChat Official Account, Sanqianke, Jinshuju), rolling plan datasets, and account datasets.
ETL	Run ETL manually and check update configuration and add user permissions.
Dashboard	Check quick queries and add user permissions; if the dashboard uses some special resources, check them, such as indoor maps, custom icon icons, user attributes, custom fonts, and component templates.
Data Screen	Check user permissions;
Light Application	Check user permissions;
Desktop Application	Check user permissions.

Possible Situations When Import Fails

Resource Type When Import Fails	Error Description	Error Example
Whole Package	Feature switch is off, please contact Guanyuan CSM to enable this feature.
Whole Package	Exceeds license limit, please contact Guanyuan business.	Error message: [message] = This application contains data screens, and the number of data screens currently exceeds the limit.
Card	When ID changes, the import path is invalid. You can modify the import path and re-import.	Attachment file path /guandata-store/images/bb021cf8dd1004239939ede7.jpg is invalid.
ETL	ETL input dataset is not in the import package.	Input dataset not matched.
ETL	Datasets with the same name are not allowed in the same directory. Please rename the dataset in the export environment before importing.	There is a file with the same name in the output dataset directory. Please modify.
Dataset	If a dataset with the same ID has been deleted in the target environment, please use ID change import or copy a dataset in the export environment and replace it.	The current dataset has been deleted in the target environment and cannot be imported again.
Dataset	The target environment is multi-domain, and there is a dataset with the same ID in another domain, so it cannot be imported. Please use ID change import or copy a dataset in the export environment and replace it.	There is a dataset with the same ID in another domain, and the dataset cannot be imported again into the current domain.
Dataset	The dataset has a deleted field with the same ID. Please use ID change import or copy a dataset in the export environment and replace it.	Duplicate entry 't991094c76a394229b308950-a23d2757620fc47b7a6e0534' for key 'ds_fd'
Data Account	Data account name is unique, but ID is inconsistent. Please rename the data account in the export environment.	Data account name is duplicated.