Data Engineer

TechDigital Group; Holmdel Township (NJ); On-site; USD 130,000 - 180,000; Full time

Job summary

TechDigital Group seeks an experienced data engineer to transform conceptual designs into scalable data pipelines and provide timely production support. Requirements include 10+ years IT, 7+ years ETL/ELT, Databricks/Spark, Python, SQL, UNIX shell, and experience with GitHub/Jenkins/Informatica.

Finance/insurance domain knowledge is a plus. Collaborate across teams to ensure robust, reliable data solutions.

Qualifications

• 10+ years of IT experience with 7+ years as ETL/ELT developer or data engineer.
• Databricks data engineering experience; Spark certification preferred.
• Knowledge of GitHub, Jenkins, and Informatica.
• Finance or Insurance industry knowledge is a plus.
• Expert in AWS Databricks, Python, Spark.
• Strong SQL and UNIX shell scripting experience.
• Performance tuning of data pipelines and issue resolution.
• Production support for applications and incident triage.

Responsibilities

• Transform conceptual designs into reliable, scalable data pipelines.
• Collaborate with Development teams, technology partners, and stakeholders to deliver timely support.
• Maintain quality and resolution standards to meet business needs.

Skills

Analytical thinking

Problem solving

Root cause analysis

Troubleshooting

Stakeholder communication

Tools

Databricks

Spark

Python

SQL

UNIX shell

GitHub

Jenkins

Informatica

AWS

Job description

who is a self-starter, fueled by collaboration, able to transform conceptual designs into reliable, scalable processes that meet or exceed customer needs. A person should be very good at analytical, problem solving and troubleshooting data pipelines root cause analysis and resolutions. Working with various Development team, Technology partners and business stakeholders to deliver unified and timely support, while upholding quality and resolution standards to fulfil the operational needs of our business stakeholders.

Person should have

• 10+ Years of IT experience, with 7+ years of experience as an ETL/ELT developer/data engineer.
• 3+ years of Databricks data engineering experience (Databricks Data Engineer and/or Spark Developer certification preferred)
• Knowledge in GitHub, Jenkins, Informatica
• Working knowledge of the Finance or Insurance industry
• Expert in AWS Databricks, Python, Spark
• Strong SQL, and UNIX Shell scripting experience
• Performance tuning data pipelines, track record communicating issues & follow-up through issue resolution.
• Production Support for applications, monitor production jobs, triage issues to appropriate team & resolve any issues in a timely manner.