Data Engineer (AWS, Databricks)

Career Connections, LLC.

Nebraska

On-site

USD 120,000 - 180,000

Full time

28 hours ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Qualifications

Bachelor's degree or equivalent hands-on experience preferred.
Databricks on AWS with end-to-end pipeline development and production support.
5+ years of professional data engineering experience.
Experience delivering data products enabling KPI reporting, forecasting, and ML.
Advanced SQL and Python/PySpark with production-grade code for large datasets.
Strong knowledge of Spark, Delta Lake, partitioning, schema evolution, and data modeling.
Hands-on experience with AWS data services (S3, IAM, KMS) and secure Databricks workloads.
Experience with APIs, files, databases, SaaS connectors, CDC, and orchestration frameworks.
Git-based development, CI/CD, data testing, code review, and IaC practices.
Data quality, observability, cataloging, lineage, and access-control for data products.

Responsibilities

Design, deliver, and operate production-grade data solutions in Databricks on AWS.
Architect and implement data pipelines, ensure performance and cost efficiency.
Develop and maintain end-to-end data products supporting analytics and ML.
Collaborate on KPI reporting, forecasting, and profitability analyses.
Implement data quality, observability, and lineage across data products.

Skills

Advanced SQL

Python/PySpark

Education

Bachelor's degree or equivalent

Tools

Databricks on AWS

Spark

Delta Lake

AWS S3

IAM / KMS

Job description

New fulltime position open for a seasoned Data Engineer in the Lancaster, PA area!

This is an onsite role and not a remote based position.

Qualified applicants should currently be based in the greater Lancaster, PA area OR willing to relocate to this area.

Strong Databricks experience is required, along with advanced SQL skills.
US Citizenship or Permanent Resident status in the United States is a requirement for this position. Client is unable to transfer any visas including H1-B, F-1, C visas, etc.

Skills and background required:

Bachelor's Degree preferred, or equivalent practical hands-on experience.

A proven track record designing, delivering, and operating production-grade solutions in Databricks on AWS, including responsibility for architecture implementation, pipeline development, deployment, support, performance, and cost.

>5 years of professional experience in data engineering, data platform engineering, analytics engineering, or similar.

Proven experience delivering multiple end-to-end projects where engineered data products enabled sophisticated analysis - such as cross-system KPI reporting, forecasting, optimization, profitability analysis, anomaly detection, predictive modeling, scenario analysis, or AI/ML.

Advanced SQL and strong Python/PySpark skills, including experience developing maintainable, tested, production-quality code for large and complex datasets.

Strong knowledge of Spark, Delta Lake or comparable lakehouse patterns, dimensional and analytical data modeling, partitioning, schema evolution, data contracts, and performance tuning.

Hands-on experience with core AWS data-platform capabilities such as S3, IAM, KMS, VPC networking, CloudWatch, Secrets Manager, and related services used to secure, integrate, and operate Databricks workloads.

Experience building batch and incremental pipelines using APIs, files, databases, SaaS connectors, change data capture, and at least one orchestration framework or managed workflow capability.

Experience with Git-based development, CI/CD, automated data testing, code review, environment management, and infrastructure-as-code practices.

Experience implementing data quality, observability, cataloging, lineage, access control, and operational support for business-critical data products.