Data Integration Jobs
The following sections provide information about the parameters and attributes of jobs that work with Data Integration platforms and services.
AWS Glue Job
The following table describes AWS Glue job attributes.
Attribute |
Description |
---|---|
Connection profile |
Defines the connection profile for the job. Rules:
|
Glue Job Name |
Defines the AWS Glue job name that you want to run. After you create an AWS Glue pipeline, it is saved as a jobname and you can run it. |
Glue Job Arguments |
Determines whether to add arguments to the AWS Glue job. |
Arguments |
Defines the AWS Glue job runtime parameters Format: {"--myArg1": "myVal1", "--myArg2": "myVal2"} |
Status Polling Frequency |
(Optional) Determines the number of seconds to wait before checking the status of the job between intervals. Default: 15 |
Azure Data Factory Job
Azure Data Factory is a cloud-based ETL and data integration service that allows you to create data-driven workflows to automate the movement and transformation of data.
The following table describes the Azure Data Factory job type attributes.
Attribute |
Description |
---|---|
Connection profile |
Defines the connection profile for the job. Rules:
|
Resource Group Name |
Determines the Azure Resource Group that is associated with a specific data factory. A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. |
Data Factory Name |
Determines the name of the Azure Data Factory that contains the pipeline you want to run. |
Pipeline Name |
Determines which data pipeline runs when the Control-M job is executed. |
Parameters |
Defines specific parameters in JSON format that are passed when the Data Pipeline runs. Format: {"var1":"value1", "var2":"value2"} |
Status Polling Frequency |
(Optional) Determines the number of seconds to wait before checking the status of the job between intervals. Default: 15 |
Azure Synapse Job
Azure Synapse Analytics enables you to perform data integration and big data analytics.
The following table describes Azure Synapse job parameters:
Attribute |
Description |
---|---|
Connection profile |
Determines which connection profile is used to connect to the Azure Synapse workspace. |
Pipeline Name |
Defines the name of a pipeline that you defined in your Azure Synapse workspace. |
Parameters |
Defines pipeline parameters to override when the job runs, defined in JSON format as pairs of name and value. Format: {"param1":"value1", "param2":"value2"} For no parameters, specify {}. |
Status Polling Interval |
(Optional) Defines the number of seconds to wait before checking the status of the job. Default: 20 seconds |
Boomi Job
The Boomi job enables you to integrate Boomi processes with your existing Control-M workflows. The following table describes Boomi job parameters:
Parameter |
Description |
---|---|
Connection profile |
Determines which connection profile is used to connect to the Boomi endpoint. |
Atom Name |
Defines the name of a Boomi Atom associated with the Boomi process. |
Process Name |
Defines the name of a Boomi process associated with the Boomi Atom. |
Status Polling Frequency |
Determines the number of seconds to wait before checking the status of the job between intervals. Default: 20 seconds |
Tolerance |
Defines the number of API call retries during the status check phase. If the API call that checks the status fails due to the Boomi limitation of a maximum of 5 calls per second, it will retry again according to the number in the Tolerance field. Default: 3 times |
Informatica Job
The Informatica job enables you to automate an Informatica workflow or tasks within the workflow, and define the parameters to pass to the workflow.
The following table describes the Informatica job type attributes.
Attribute |
Description |
---|---|
Connection Profile |
Defines the connection profile for the job. Rules:
Variable Name: %%INF-ACCOUNT |
Repository Folder |
Defines the Repository Folder containing the workflow that you want to run. Variable Name: %%INF-REP_FOLDER |
Workflow |
Defines the workflow that you want to run in Control-M for Informatica. Variable Name: %%INF-WORKFLOW |
Instance Name |
Defines the specific instance of the workflow that you want to run. Variable Name: %%INF-INSTANCE_NAME |
OS profile |
Enables you to specify an OS profile when running or rerunning an Informatica job. |
Run Options |
Defines options regarding the workflow task hierarchy. |
Depth |
Determines the number of levels within the workflow task hierarchy that is used to select workflow tasks. Default: 10 Variable Name: %%INF- DEPTH |
Run |
Determines whether to run the whole workflow, start from a specific task, or run a single task as follows:
|
Parameters |
Determines an array of parameters that is passed to the workflow. Each parameter is comprised of the following:
|
Include workflow events log in Job output |
Determines whether to include the workflow event log in the job outputA tab in the job properties pane in the Monitoring domain that shows the output of a job, which indicates whether a job ended OK, and used, for example, with jobs that check file location. |
Include detailed error log for failed sessions |
Determines whether to include a detailed error log for a workflow that failed. |
Get session statistics and log |
Determines whether to retrieve session statistics and log messages. |
Action on rerun |
Determines which operation is executed when the workflow is in a suspended status, as follows:
|
Workflow parameters file |
Defines the path and name of the workflow parameters file. Variable Name: %%INF-WORKFLOW_PARAMETERS_FILE |
Informatica Cloud Services Job
Informatica Cloud Services jobs enable you to automate your Informatica workflows for multi-cloud and on-premises data integration through the Control-M environment.
The following table describes Informatica Cloud Services Job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Defines the connection profile for the job. Rules:
Variable Name: %%INF-ACCOUNT |
Task Type |
Determines one of the following task types to run on Informatica Cloud:
|
Task Name |
Defines the name of the task to execute on Informatica Cloud. This parameter is not relevant for a taskflow. |
TaskFlow URL |
Defines the Service URL of the taskflow to execute on Informatica Cloud. In Informatica Data Integration, you can obtain this Service URL through the Properties Detail option of the taskflow. |
Input Fields |
Defines input fields for a taskflow. Format: input1=value1&input2=value2&input3=value3 |
Call Back URL |
(Optional) Defines a publicly available URL to which to post the job status. |
Verification Poll Interval |
Defines the number of seconds between polls for job status verification. |
IBM InfoSphere DataStage Job
The following table describes the IBM InfoSphere DataStage job type attributes.
Attribute |
Description |
---|---|
Connection Profile |
Defines the connection profile for the job. Rules:
Variable Name: %%DataStage-ACCOUNT For more information about creating a local connection profile for this job, see Creating a connection profile. |
Project |
Defines the Control-M for IBM InfoSphere DataStage project name. Variable Name: %%DataStage-PROJECT |
DataStage Job |
Defines the Control-M for IBM InfoSphere DataStage job name. Variable Name: %%DataStage-JOB_NAME |
Job Invocation ID |
Defines the Control-M for IBM InfoSphere DataStage job invocation ID. Variable Name: %%DataStage-JOB_INVOCATION_ID |
Parameters Type |
Determines from where Control-M retrieves the parameters, with options as follows:
|
Parameters |
Displays the parameters and their values from the DataStage Job when Server or Server and File is selected from Parameters Type. |
Parameters File |
Defines the IBM InfoSphere DataStage parameter file. Variable Name: %%DataStage-PARAMS_FILE |
More Options |
Opens more options. |
Limits |
Defines limits on the job. |
Stop stages after <value> Rows |
Defines the maximum number of rows that the job can contain. Control-M stops the stages after the maximum is reached. Variable Name: %%DataStage-MAX_ROWS |
Abort job after <value> Warnings |
Defines the maximum number of warnings about the job. Control-M aborts the job after the maximum is reached. Variable Name: %%DataStage-MAX_WARNINGS |
Job Output |
Determines the type of information that goes in the output as follows:
|
Run Options |
Defines run options for the job. |
Run in restart mode |
Runs the Control-M for IBM InfoSphere DataStage job in restart mode. Variable Name: %%DataStage-RESTART_SEQUENCE |
Reset job before run |
Resets the Control-M for IBM InfoSphere DataStage job before the job runs. Variable Name: %%DataStage-RESET_JOB |
Talend Data Management Job
The Talend Data Management Job enables the integration of data management and data integration tasks or plans from Talend with your existing Control-M workflows.
The following table describes Talend Data Management job attributes.
Attribute |
Description |
---|---|
Connection profile |
Defines the connection profile for the job. Rules:
|
Task/Plan Execution |
Determines one of the following options for execution in Talend:
|
Task Name /
|
Defines the name of the Talend task or plan to execute, as defined in the Tasks and Plans page in the Talend Management Console. |
Parameters |
(For a task) Defines specific parameters to pass when the Talend job runs, defined as JSON pairs of parameter name and value. All parameter names must contain the parameter_ prefix. Format: {"parameter_param1":"value1", "parameter_param2":"value2"} For no parameters, specify {}. |
Log Level |
(For a task) Determines one of the following levels of detail in log messages for the triggered task in the Talend Management Console:
|
Bring logs to output |
(For a task) Determines whether to show Talend log messages in the job output. Values: checked|unchecked Default: unchecked |
Task Polling Intervals / Plan Polling Intervals |
Determines the number of seconds to wait before checking the status of the triggered task or plan. Default: 10 second |