Data Engineer

SFE; Atlanta (GA); On-site; USD 110,000 - 150,000; Full time

Job summary

SFE in Atlanta, GA is seeking a Data Engineer to design, build, and maintain scalable data pipelines and ETL solutions for analytics.

You will work with Databricks, PySpark, SQL, and cloud platforms (AWS/GCP) to ingest, cleanse, and model data for BI and ML initiatives.

Collaborate with data scientists and BI teams, mentor juniors, and drive best practices across projects.

Qualifications

• Must have hands-on experience with building scalable data pipelines and ETL solutions.
• Strong SQL querying and data modeling skills for large-scale datasets.
• Experience with Databricks, PySpark, and Spark ecosystems is required.
• Cloud experience on AWS or GCP and data warehousing skills are essential.
• Knowledge of streaming data (Kafka, Spark Streaming) is a plus.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL solutions.
• Build and optimize data ingestion frameworks for structured and unstructured data.
• Implement data quality, validation, governance, and standardization processes.
• Develop data infrastructure integrating multiple heterogeneous data sources.
• Support analytics, reporting, ML, and BI initiatives.
• Collaborate with Data Scientists and BI teams to prepare data for analytics.
• Develop scalable solutions for Big Data processing and streaming workloads.
• Analyze data and provide insights to business stakeholders.
• Gather requirements and translate into technical solutions.
• Lead and mentor junior data engineers.
• Collaborate across multiple projects as the Data Engineering SME.
• Drive adoption of Data Engineering best practices and standards.

Skills

Databricks

Python/PySpark

Spark SQL

ETL/Data Pipelines

Big Data stack

SQL/Data Modeling

Data Ingestion/ETL

Cloud AWS/GCP

Data Warehousing

Streaming with Kafka

Agile Scrum/SAFe

Tools

Scala/Scala-Spark

ETL Tools: DataStage/Tibco

AWS: S3/Glue/Lambda/Kinesis/MSK

BI Tools: Tableau/Power BI/Spotfire/OAC

Databases: Teradata/DB2/BigQuery/Mainframe

Git

Data governance/retention modeling

Job description

Role: Data Engineer
Location: Atlanta, GA
Term: Full Time

Job Description

Must Have Technical/Functional Skills

Must Have Skills:

• Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog)
• Python and PySpark
• Spark SQL and Apache Spark
• ETL/Data Pipeline Development
• Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop
• Strong SQL querying and data modeling
• Data ingestion, transformation, cleansing, and validation
• Cloud platforms (AWS/GCP)
• Data warehouse and large-scale data processing experience
• Experience with streaming data using Kafka and Spark Streaming
• Agile methodologies (Scrum, Kanban, SAFe)

Preferred Skills:

• Scala / Scala-Spark
• IBM DataStage, Tibco, or similar ETL tools
• AWS services: S3, Glue, Lambda, Kinesis/MSK
• BI Tools: Tableau, Power BI, Spotfire, OAC
• Databases: Teradata, DB2, BigQuery, Mainframe
• Git version control
• Data governance and retention modeling

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL solutions.
• Build and optimize data ingestion frameworks for structured and unstructured data.
• Implement data quality, validation, standardization, and governance processes.
• Develop data infrastructure integrating multiple heterogeneous data sources.
• Support analytics, reporting, machine learning, and business intelligence initiatives.
• Work with Data Scientists and BI teams to prepare and curate data for analytics.
• Develop scalable solutions for Big Data processing and streaming workloads.
• Analyze data and provide actionable insights and recommendations to business stakeholders.
• Gather requirements from business teams and translate them into technical solutions.
• Lead and mentor junior data engineers.
• Collaborate across multiple projects as the Data Engineering SME.
• Drive adoption of Data Engineering best practices and standards.