Data Engineer — Databricks + AWS Pipelines & Genies

6AM City, LLC; North Carolina; On-site; USD 170,492,000 - 230,665,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) match

Job summary

6AM City, LLC in Raleigh, NC is seeking a Data Engineer to design and optimize large-scale data pipelines onsite. The role emphasizes Databricks, AWS, PySpark, and pharma datasets, delivering scalable products for sales insights and forecasting.

You will collaborate with Commercial Analytics, Data Science, IT, and business teams to translate requirements into robust data solutions while ensuring governance and security standards.

Qualifications

• 5–10+ years of experience in Data Engineering or Big Data Analytics.
• Strong hands-on experience with Databricks, Genie, Delta Lake, and MLflow.
• Proficiency in PySpark, Python, SQL, and distributed computing.
• Deep experience with AWS services (S3, Glue, Lambda, EMR, Redshift, IAM).
• Experience with Life Sciences commercial datasets.
• Knowledge of data modeling, ETL/ELT, and cloud architecture.
• Familiarity with CI/CD, Git, and DevOps.
• Ability to build scalable, high-performance data pipelines.

Responsibilities

• Build, optimize, and maintain data pipelines using Databricks, Delta Lake, and Genie-powered workflows.
• Implement LLM-based automation and query generation with Databricks Genie for data exploration and self-service.
• Develop notebooks, jobs, workflows, and ML-ready datasets within Databricks.
• Design and manage data ingestion, storage, and processing using AWS services.
• Ensure data quality, lineage, governance, and compliance with industry standards.

Skills

Databricks

Genie

Delta Lake

MLflow

PySpark

Python

SQL

AWS

Life Sciences data

Data modeling

ETL/ELT

CI/CD

Git

DevOps

Automation

Big data analytics

Education

No Education Requirements

Tools

Databricks Unity Catalog

Tableau

Power BI

Qlik

Lakehouse architecture

Job description

6AM City, LLC in Raleigh, NC is seeking a Data Engineer to design and optimize large-scale data pipelines onsite. The role emphasizes Databricks, AWS, PySpark, and pharma datasets, delivering scalable products for sales insights and forecasting.

You will collaborate with Commercial Analytics, Data Science, IT, and business teams to translate requirements into robust data solutions while ensuring governance and security standards.