Ab Initio Data Engineer

Veriipro; Berkeley Heights (NJ); On-site; USD 120,000 - 150,000; Full time

Job summary

Veriipro is seeking an experienced ETL Architect to design and maintain complex Ab Initio workflows. You will develop end-to-end data integration across RDBMS, mainframe, flat files, and streaming platforms, and optimize performance through partitioning and parallelism.

In this role you will implement CDC, manage metadata in EME, and mentor junior developers, collaborating with DBAs and business teams to translate requirements into scalable data solutions.

Qualifications

• Bachelor's degree required in CS/Engineering or related field.
• 7+ years of ETL development experience including 5+ years with Ab Initio.
• Strong experience with Ab Initio GDE, Co>Op, EME, Conduct>It, Data Profiler and Continuous Flows.
• Strong SQL skills and advanced Unix/Linux shell scripting.
• Solid understanding of data warehousing concepts: dimensional modeling, SCD, star and snowflake schemas.
• Experience designing and implementing Kafka-based data pipelines.
• Strong debugging, troubleshooting and performance tuning skills for large-volume data processing.

Responsibilities

• Design, develop, and maintain complex ETL workflows and graphs using Ab Initio GDE.
• Architect end-to-end data integration solutions across multiple source and target systems including RDBMS, mainframe, flat files, message queues, and streaming platforms.
• Optimize Ab Initio graph performance through parallelism, partitioning techniques, layout optimization, and checkpointing strategies.
• Develop and manage solutions using Ab Initio components including Co>Op, EME, Conduct>It, Data Profiler, and Continuous Flows.
• Implement CDC and near-real-time data processing pipelines.
• Manage metadata, lineage, and data governance activities within EME.
• Troubleshoot production issues, perform root-cause analysis, and implement permanent fixes.
• Define development standards, review ETL code, and enforce Ab Initio best practices.
• Mentor junior and mid-level Ab Initio developers.
• Collaborate with data architects, DBAs, and business teams to translate requirements into technical solutions.

Skills

ETL development

SQL

Unix/Linux shell scripting

Data warehousing concepts

Debugging and performance tuning

Parallel processing concepts

Kafka data pipelines

Dimensional modeling

Education

Bachelor’s degree in Computer Science, Engineering, or related technical field

Tools

Ab Initio GDE

Co>Operating System

EME

Conduct>It

Data Profiler

Continuous Flows

Kafka

Job description

Required Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related technical field.
• 7+ years of ETL development experience with 5+ years of hands-on experience in Ab Initio development.
• Strong expertise with:Ab Initio GDE (Graphical Development Environment),Co>Operating System (Co>Op),Enterprise Meta>Environment (EME),Conduct>It,Data Profiler and Continuous Flows.
• Strong understanding of parallel processing concepts, partitioning strategies, and Multi-File System (MFS).
• Advanced SQL skills with experience in relational databases, especially DB2.
• Strong experience with Unix/Linux shell scripting.
• Solid understanding of data warehousing concepts:Dimensional modeling,Slowly Changing Dimensions (SCD),Star and Snowflake schemas.
• Experience designing and implementing Kafka-based data pipelines.
• Strong debugging, troubleshooting, and performance tuning skills for large-volume data processing.

Preferred Qualifications

• Experience with mainframe data integration:COBOL copybooks,VSAM,Flat files
• Exposure to Kubernetes or OpenShift environments.
• Familiarity with Hadoop/Spark ecosystems and hybrid ETL architectures.
• Experience with API-based integrations and streaming data solutions.
• Prior experience as a technical lead or in client-facing architecture roles.

Roles and Responsibilities

• Design, develop, and maintain complex ETL workflows and graphs using Ab Initio GDE.
• Architect end-to-end data integration solutions across multiple source and target systems including:RDBMS,Mainframe systems,Flat files,Message queues,Streaming platforms
• Optimize Ab Initio graph performance through:Parallelism,Partitioning techniques (Hash, Broadcast, Range),Layout optimization,Checkpointing strategies
• Develop and manage solutions using Ab Initio components including Co>Op, EME, Conduct>It, Data Profiler, and Continuous Flows.
• Implement CDC (Change Data Capture) and near-real-time data processing pipelines.
• Manage metadata, lineage, and data governance activities within EME.
• Troubleshoot production issues, perform root-cause analysis, and implement permanent fixes.
• Define development standards, review ETL code, and enforce Ab Initio best practices.
• Mentor junior and mid-level Ab Initio developers.
• Collaborate with data architects, DBAs, and business teams to translate requirements into technical solutions.