Senior Data Engineer

Long Finch Technologies

Atlanta (GA)

On-site

USD 120,000 - 170,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Long Finch Technologies in Atlanta, GA seeks a Senior Data Engineer to design and scale ETL pipelines, leveraging Databricks, PySpark, and Spark. You will lead data integration across cloud platforms, ensure data quality, and collaborate with analytics teams to enable BI and ML initiatives.

You will mentor engineers, implement governance, and optimize streaming and batch processing in AWS/GCP environments. Seniority requires strong SQL and data modeling, plus Agile experience.

Qualifications

  • Hands-on Databricks experience with Delta Lake, Delta Live Tables, Unity Catalog.
  • Proficient in Python, PySpark, Spark SQL and Apache Spark.
  • Strong ETL, data pipelines design, and data transformation experience.
  • Experience with Hadoop, Kafka, Hive, HDFS, and Sqoop.
  • SQL, data modeling and data warehousing expertise.
  • Experience with batch and real-time processing using Kafka and Spark Streaming.
  • Cloud experience with AWS or Google Cloud Platform.
  • Knowledge of data quality, validation, cleansing, and governance.
  • Experience in Agile environments (Scrum, Kanban, SAFe).

Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL solutions.
  • Build data ingestion and transformation frameworks for structured and unstructured data.
  • Develop and optimize batch and streaming data processing solutions.
  • Implement data quality, validation, and governance standards.
  • Integrate heterogeneous data sources into enterprise data platforms.
  • Support analytics, reporting, BI, and machine learning initiatives.
  • Collaborate with Data Scientists, BI teams, and business stakeholders.
  • Lead technical discussions, mentor junior engineers, and drive data engineering best practices.

Skills

Databricks
Python
PySpark
Apache Spark
ETL development
Data pipelines
Kafka
Hadoop
Hive
HDFS
SQL
Data warehousing
Spark Streaming
Cloud: AWS/GCP
Agile (Scrum/Kanban/SAFe)

Tools

Delta Lake
Delta Live Tables
Unity Catalog
Spark SQL
Databricks
Hadoop
Kafka

Job description

Job Description

We are looking for an experienced Senior Data Engineer with strong expertise in Databricks, Python, PySpark, Apache Spark, and large-scale data engineering. The ideal candidate will be responsible for designing and developing scalable ETL pipelines, implementing data processing solutions, managing data ingestion frameworks, and optimizing cloud-based data platforms. The role also involves technical leadership, data governance, and collaboration with analytics and business intelligence teams.

Required Technical Skills
  • Strong hands-on experience with Databricks, Delta Lake, Delta Live Tables, and Unity Catalog.
  • Proficiency in Python, PySpark, Spark SQL, and Apache Spark.
  • Strong experience in ETL development, data pipeline design, and data transformation.
  • Knowledge of Hadoop, Kafka, Hive, HDFS, and Sqoop.
  • Expertise in SQL, data modeling, and data warehousing.
  • Experience with batch and real-time data processing using Kafka and Spark Streaming.
  • Cloud experience with AWS and/or Google Cloud Platform.
  • Strong understanding of data quality, validation, cleansing, and governance.
  • Experience working in Agile environments (Scrum, Kanban, SAFe).
Preferred Skills
  • Scala / Scala-Spark, IBM DataStage, TIBCO.
  • AWS S3, Glue, Lambda, Kinesis/MSK.
  • Tableau, Power BI, Spotfire, OAC.
  • Teradata, DB2, BigQuery, Mainframe.
  • Git, data governance, and retention modeling.
Roles & Responsibilities
  • Design, develop, and maintain scalable data pipelines and ETL solutions.
  • Build data ingestion and transformation frameworks for structured and unstructured data.
  • Develop and optimize batch and streaming data processing solutions.
  • Implement data quality, validation, and governance standards.
  • Integrate heterogeneous data sources into enterprise data platforms.
  • Support analytics, reporting, BI, and machine learning initiatives.
  • Collaborate with Data Scientists, BI teams, and business stakeholders.
  • Lead technical discussions, mentor junior engineers, and drive data engineering best practices.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

Veriipro • Atlanta (GA)

On-site
USD 140,000 - 180,000
Senior Data Engineer
Senior Data Engineer

TechDigital Group • Chicago (IL)

On-site
USD 120,000 - 150,000
Senior Data Engineer
Senior Data Engineer

Bitful • United States

Remote
USD 120,000 - 180,000
Senior Databricks Data Engineer
Senior Databricks Data Engineer

Virtue America • New York (NY)

On-site
USD 140,000 - 190,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 160,000
Data Engineer
Data Engineer

GSquared Group • Atlanta (GA)

On-site
USD 120,000 - 160,000
Data Engineer ETL
Data Engineer ETL

Compunnel Inc. • Durham (NC)

On-site
USD 100,000 - 130,000
Data Engineer
Data Engineer

TechDigital Group • Burbank (CA)

On-site
USD 120,000 - 150,000
Senior Data Engineer
Senior Data Engineer

9nexus • United States

Remote
USD 120,000 - 190,000
Senior Consultant (Databricks)
Senior Consultant (Databricks)

Unison Group • San Francisco (CA)

On-site
USD 180,000 - 250,000