Data Engineer: Cloud Pipelines, Lakehouse & CI/CD

Pro Mach Inc; Kentucky; Hybrid; USD 95,000 - 130,000; Full time

Job summary

ProMach is seeking a Data Engineer to design, build, and maintain data flows between corporate IT systems and the data lake, leveraging Databricks and Delta Lake. You will implement robust data pipelines and collaborate closely with analysts to deliver actionable insights across cloud platforms.

The role requires 3-5 years in data engineering, strong SQL and Python skills, and experience with CI/CD, data warehousing, and data governance.

Qualifications

• 3-5 years of experience in data engineering or a similar role.
• Expertise with Databricks and the Databricks ecosystem (Delta Lake, SQL, Runtime).
• Proficiency in SQL, Python, and Apache Spark.
• Experience with cloud platforms (Azure, AWS, or GCP).
• Strong data modeling, ETL/ELT, and data warehousing knowledge.
• Experience with CI/CD tools and practices.
• Excellent problem-solving and communication skills, plus eagerness to learn.

Responsibilities

• Data Pipeline Development: design, build, and maintain scalable data pipelines using Databricks, Spark, and Delta Lake.
• ETL/ELT Processes: develop robust workflows for data extraction, transformation, and loading; monitor scheduled jobs.
• Data Architecture and Modeling: contribute to data models for data lakes and warehouses; advise on platforms.
• Cloud Integration: integrate data solutions with Azure, AWS, or GCP.
• Data Quality and Governance: implement data quality checks and governance best practices.
• Automation: implement and maintain CI/CD pipelines to automate deployment.
• Collaboration: work with analysts to define data requirements and deliver insights.

Skills

Databricks

Delta Lake

SQL

Python

CI/CD

Data Modeling

Cloud Platforms (Azure/AWS/GCP)

ETL/ELT

Data Warehousing

Education

Bachelor’s Degree in Information Systems, Computer Science, or related field

Tools

Databricks

Azure

AWS

GCP

Job description

ProMach is seeking a Data Engineer to design, build, and maintain data flows between corporate IT systems and the data lake, leveraging Databricks and Delta Lake. You will implement robust data pipelines and collaborate closely with analysts to deliver actionable insights across cloud platforms.

The role requires 3-5 years in data engineering, strong SQL and Python skills, and experience with CI/CD, data warehousing, and data governance.