Data Engineer - Real-Time Pipelines, Atlanta Onsite

Speria; Dunwoody (GA); On-site; USD 90,000 - 130,000; Full time

Job summary

Speria MTech in Atlanta, GA, is seeking a Data Engineer to design, build, and maintain scalable data pipelines that support machine learning, optimization, and operational reporting. You will transform data from sensors and enterprise systems into high-quality datasets using batch and near real-time processing.

The ideal candidate has 2–4 years of data engineering experience, hands-on Spark and Azure/Databricks experience, and a passion for data quality, cost efficiency, and collaboration with

Qualifications

• Bachelor’s degree in CS, engineering, or related field.
• 2–4 years of data engineering or data systems experience.
• Experience with Apache Spark and large-scale data processing.
• Experience building batch pipelines in production.
• Experience with cloud platforms (Azure, Databricks).
• SQL and Apache Spark expertise.

Responsibilities

• Design and build batch and near real-time data pipelines.
• Develop transformation workflows for high-quality datasets.
• Ensure data quality, integrity, and reliability.
• Collaborate with MLEs for feature data readiness.
• Implement data monitoring and anomaly detection.
• Support synthetic data for testing and simulation.
• Optimize pipelines to remove redundancy and reduce costs.
• Improve performance, scalability, and efficiency.
• Maintain pipeline reliability and documentation.

Skills

Batch data pipelines

SQL

Apache Spark

Data quality monitoring

Collaboration with ML engineers

Education

Bachelor’s degree in Computer Science, Engineering, or related field

Tools

Azure

Databricks

Event Hub

Cosmos DB

Functions

Job description

Speria MTech in Atlanta, GA, is seeking a Data Engineer to design, build, and maintain scalable data pipelines that support machine learning, optimization, and operational reporting. You will transform data from sensors and enterprise systems into high-quality datasets using batch and near real-time processing.

The ideal candidate has 2–4 years of data engineering experience, hands-on Spark and Azure/Databricks experience, and a passion for data quality, cost efficiency, and collaboration with