Data Engineer II

Jobtailor; Providence (RI); On-site; USD 90,000 - 140,000; Full time

Job summary

Jobtailor in Providence, RI is seeking a data engineer to design, develop, and maintain scalable ELT/ETL pipelines using Databricks, Spark, SQL, and Python to support analytics and AI use cases.

You will build reliable datasets, implement Delta Lake storage, collaborate with stakeholders, and help modernize cloud data platforms on AWS, ensuring quality and performance.

Qualifications

• Bachelor's degree or equivalent practical experience in a related field.
• 3+ years of Data Engineering or related technical work.
• Strong SQL development skills.
• Strong Python programming experience.
• Experience with Spark and Databricks for data pipelines.
• Experience building modern ETL/ELT pipelines.
• Knowledge of data modeling and warehousing concepts.
• Experience with cloud-based data platforms, preferably AWS.
• Familiarity with Delta Lake or similar storage tech.
• Experience with Git and collaborative development.

Responsibilities

• Design, develop, and maintain scalable ELT/ETL pipelines.
• Build reliable datasets for analytics, reporting, and AI use cases.
• Develop data transformations using Delta Lake and Lakehouse patterns.
• Integrate data from various sources including APIs and databases.
• Collaborate with stakeholders to understand data requirements.
• Monitor, troubleshoot, and optimize production pipelines.
• Participate in migrating legacy processes to cloud-native architectures.
• Improve data quality through validation and automation.
• Contribute to engineering best practices through code reviews and docs.
• Support continuous improvement of the enterprise data platform.

Skills

SQL

Python

Databricks

Apache Spark

ETL/ELT

Data modeling

AWS

Delta Lake

Git

Collaboration

Education

Bachelor's degree in Computer Science / Information Systems / Engineering or equivalent

Tools

Databricks

Apache Spark

Delta Lake

Git

AWS

SQL

Job description

Responsibilities

• Design, develop, and maintain scalable ELT/ETL pipelines using Databricks, Apache Spark, SQL, and Python.
• Build and maintain reliable, high-quality datasets that support enterprise analytics, operational reporting, and AI use cases.
• Develop data transformations using Delta Lake and Lakehouse architecture best practices.
• Integrate data from enterprise applications, APIs, databases, files, and other internal and external sources.
• Collaborate with business stakeholders, analysts, and product teams to understand data requirements and deliver scalable solutions.
• Monitor, troubleshoot, and optimize production data pipelines for reliability, scalability, and performance.
• Participate in migrating and modernizing legacy data processes to cloud-native architectures.
• Improve data quality through validation, testing, monitoring, and automation.
• Contribute to engineering best practices through code reviews, documentation, and knowledge sharing.
• Support continuous improvement of the enterprise data platform by identifying opportunities for automation, optimization, and simplification.

Requirements

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience.
• 3+ years of experience in Data Engineering or a related technical role.
• Strong SQL development skills.
• Strong Python programming experience.
• Experience building data pipelines using Apache Spark and Databricks (or similar distributed data processing platforms).
• Experience developing modern ETL/ELT pipelines.
• Understanding of data modeling and data warehousing concepts.
• Experience working with cloud-based data platforms, preferably AWS.
• Familiarity with Delta Lake or similar modern data storage technologies.
• Experience using Git and collaborative software development practices.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration skills.

Core Competencies

Proficient in designing and developing scalable ELT/ETL pipelines using Databricks, Apache Spark, SQL, and Python, with a strong focus on data quality and optimization. Experienced in collaborating with stakeholders to deliver data solutions that support enterprise analytics and operational reporting.