Empower your data journey with EazyDI's intuitive pipeline creation. Choose connectors, map fields, schedule seamlessly. Elevate your data integration experience—simple, powerful, and effective
1. Enter Mapping Integration Name
Start Building Your Pipeline: Begin by providing a distinctive name for identification. Click the pencil icon, and an input box will appear for entering the integration pipeline name.
2. Configure Source Connection
Step 1: Select Source Connector
Browse the list and choose a source connector that aligns with your business requirements.
Step 2: Connect to Source
If a connection exists, click "Connect" to access the Source Configuration page.
If no connection, create one:
Redirected to the Source Configuration page. A pop-up screen with connection attributed is displayed
Enter connection attributes.
Test and validate the connection.
Create the connection.
Step 3: Choose Connection Name
Select an existing connection name or create a new one on-the-fly by clicking "Add Connection."
Step 4: Define Connection Objects
Depending on the connector type, specify connection objects:
Database (SQL): Tables
NoSQL Database: Documents
Application Services: Modules (e.g., accounts, users, campaigns)
Excel: Workbook and sheet names
File system: File names
Social Media: Ad metrics (campaigns, Ads, Adsets)
Step 5: Configure Connection Parameters (Custom Connectors)
Custom application connection might have some additional attributes . Specify connection parameters as required.
Step 6: Proceed
Click "Next" button to advance to the next stage of pipeline configuration.
The user will be directed to the Target Connection page
3. Configure Target Connection
Step 1: Select Target Connector
Browse the list and choose a target connector that aligns with your business requirements.
Step 2: Connect to Target
If a connection exists, click "Connect" to access the Target Configuration page.
If no connection, create one:
Redirected to the Target Configuration page. A pop-up screen with connection attributed is displayed
Enter connection attributes.
Test and validate the connection.
Create the connection.
Step 3: Choose Connection Name
Select an existing connection name or create a new one on-the-fly by clicking "Add Connection."
Step 4: Define Connection Objects
Depending on the connector type, specify connection objects:
Database (SQL): Tables
NoSQL Database: Documents
Application Services: Modules (e.g., accounts, users, campaigns)
Excel: Workbook and sheet names
File system: File names
Step 5: Configure Connection Parameters (Custom Connectors)
Custom application connection might have some additional attributes . Specify connection parameters as required.
Step 6: Proceed
Click "Next" to advance to the next stage of pipeline configuration.
The user will be directed to the Field Mapping Connection page
4. Mapping Source Attributes to Target Attributes
This screen provides the flexibility to map unstructured source attributes to structured target attributes. The attributes are referred to as fields here in this application
Upon selecting the Connection objects, the application fetches the metadata (attributes) of the connector object, presenting it on the field mapping page.
On the left side of the screen, a list of source metadata fields is displayed, while the right side of the table showcases the target metadata fields. This interface allows you to efficiently map raw source data to structured target data.
The features available on this screen are
Auto Match: This is the default view as well – it acts as a name pair match (source fields will be mapped if it matches the same target field name)
Drag and Drop: You can drag a source field to the target field that you want to map to
Clear Mapping: It clears the existing mapping (i.e.: removes everything from the Mapped Field/Expression area
Search box: Using this you can find the field name
Delete a mapping: You can remove a mapping by clicking on the trash icon
FX: On click of a pop-up screen will open. Here you can add simple operations on your target data like :
Removing white spaces using Trim() , LTRIM(), RTRIM()
combine data of 2 source field to one target field using Concat(), OR
Set the desired date-format using To_Date()
Convert the other data types using TO_Float(), To_BigInt(), To_Char, To_Decimal(), To_Integer()
5. Configure Pipeline Scheduling
Access Scheduling Configuration: Navigate to the designated screen to configure scheduling for your pipeline based on business requirements
Features:
Scheduling Options: The screen offers users the flexibility to set up scheduling in two ways:
Multiple Runs: Choose this option if you want your pipeline to run at specified intervals.
Manual Run: Opt for this option if you prefer a one-time manual execution of the pipeline.
Radio Buttons: Utilize the provided radio buttons to easily toggle between the two scheduling options. Select the appropriate button based on your scheduling preference.
Details for Setting up Manual Run or Multiple Runs for Pipeline:
i) Manually run pipeline without Schedule
If the user wants to just save the pipeline and manually trigger the pipeline, you can choose this option.
Click and select the radio button, Without Schedule.
It will minimize the scheduling configuration section of the screen.
You can click on Next button to browse to Summary screen
ii) Run pipeline to run on given schedule
The user can choose this option to schedule the pipeline to run multiple time at a given schedule as per business requirements
You will have to first enter the following details :
Schedule Name: Provide the schedule name
Description: Short description for the schedule for future reference
You can choose from the following Scheduling Options
Start time: Select the start time
Time Zone: Select the Timezone from the available list
Repeats: This dropdown box allows user to select the frequency at which you want the pipeline to run:
Hourly
Daily
Monthly
Every N hours
Every N minutes
DoesNotRepeat
After selecting the scheduling options, you can click on Next button to browse to Summary screen
5. Summary and Save Mapping pipeline
This screen provides an overview of the pipeline created or edited by the user, showcasing key details.
Summary: The page displays a concise overview of connection details and field mapping settings configured for pipeline construction.
Back Button: Clicking this button redirects the user to the scheduling configuration page.
Save Button: Once the user has verified all pipeline details, clicking the save button will store the information and present a success message pop-up.
Upon clicking the "Done" button within the success message box, the application will navigate to the Manage Mapping page.