ETL AB initio Developer

Cloud Hybrid Technologies, LLC; Jersey City (NJ); On-site; USD 90,000 - 130,000; Full time

Job summary

Cloud Hybrid Technologies, LLC is seeking a skilled professional with expertise in Python, Java, and Abinitio to build scalable data integration solutions. The role involves creating and managing efficient processing pipelines utilizing technologies like Apache Spark and AWS services.

The ideal candidate will have a strong understanding of distributed computing principles and experience with data workflows. Cloud Hybrid values diversity and is committed to equal opportunity employment.

Qualifications

• Experience in building data integration solutions using the AB initio ETL.
• Creating efficient and scalable data processing pipelines using Apache spark.
• Proficient in understanding distributed computing principles.
• Implementing, scheduling, and monitoring complex data workflows.
• Good understanding of cloud technology, with strong technical experience.

Responsibilities

• Collaborate with the data engineering team to improve data integration pipelines.
• Identify, diagnose, and resolve issues within airflow workflows.
• Optimize DAGs for performance, scalability, and resource efficiency.

Skills

Python

Java

Ansi SQL

Abinitio

Spark

Tools

AWS

Job description

Mandatory Skills: Python, Java, Ansi SQL, Abinitio, Spark

• Experience in building data integration solutions using the AB initio ETL .
• Creating efficient and scalable data processing pipelines and applications using Apache spark
• Proficient in understanding distributed computing principles.
• Creating ,optimizing, and maintaining directed acyclic graphs(DAGs) in Python to define and orchestrate data pipelines and automated tasks.
• Implementing , scheduling, and monitoring complex data workflows, ensuring timely and accurate data processing.
• Identifying, diagnosing, and resolving issues within airflow workflows, and optimizing DAGs for performance, scalability, and resource efficiency.
• Good understanding of cloud technology. Must have strong technical experience in Desing Mapping Specifications HLD LLD.
• Must have the ability to relate to both business and technical members of the team and possess excellent communication skills.
• Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, EMR and integrate with our internal Archival Service Platform for efficient data purging and Life Cycle management.
• Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
• Building data integration solutions using the Ab initio ETL .
• Creating efficient and scalable data processing pipelines and applications using Apache spark.
• Implementing, scheduling, and monitoring complex data workflows, ensuring timely and accurate data processing.
• Implement and manage agents for monitoring, logging, and automation within aws environments

Cloud Hybrid is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. Cloud Hybrid will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will Cloud Hybrid require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract.