Data Engineer

Career Connections, LLC; Lancaster (Lancaster County); On-site; USD 100,000 - 160,000; Full time

Job summary

Career Connections, LLC seeks a seasoned Data Engineer in the Lancaster, PA area for an onsite role. Strong Databricks experience is required, with advanced SQL and Python/PySpark skills.

Responsibilities include designing, delivering, and operating production-grade data solutions, building scalable pipelines, and ensuring data quality and observability across the data platform. Relocation to the Lancaster region is a plus.

Qualifications

• Bachelor's degree or equivalent practical hands-on experience.
• 5+ years in data engineering or related fields.
• Experience delivering production-grade data solutions on Databricks (AWS).
• Advanced SQL and Python/PySpark for large datasets.
• Knowledge of Spark, Delta Lake, and data modeling; performance tuning.
• Experience with AWS data-services (S3, IAM, etc.) and CI/CD practices.

Responsibilities

• Design, deliver, and operate production-grade data solutions on Databricks/AWS.
• Build batch and incremental data pipelines using APIs, files, and databases.
• Ensure data quality, observability, cataloging, and data contracts.
• Collaborate with teams to enable analytics, forecasting, and ML use cases.

Skills

Databricks

Advanced SQL

Python/PySpark

Delta Lake

Spark

AWS services

CI/CD

Data quality

Observability

Education

Bachelor's degree or equivalent

Tools

Git

Terraform

Job description

Date:

October 15, 2026

New fulltime position open for a seasoned Data Engineer in the Lancaster, PA area!

This is an onsite role andnota remote based position.

Qualified applicants should currently be based in the greater Lancaster, PA areaORwilling to relocate to this area.

Strong Databricks experience is required, along with advanced SQL skills.

Skills and background required:

• Bachelor's Degree preferred, or equivalent practical hands-on experience.
• A proven track record designing, delivering, and operating production-grade solutions inDatabricks on AWS, including responsibility for architecture implementation, pipeline development, deployment, support, performance, and cost.
• >5 years of professional experience in data engineering, data platform engineering, analytics engineering, or similar.
• Proven experience delivering multiple end-to-end projects where engineered data products enabled sophisticated analysis - such as cross-system KPI reporting, forecasting, optimization, profitability analysis, anomaly detection, predictive modeling, scenario analysis, or AI/ML.
• Advanced SQLand strong Python/PySpark skills, including experience developing maintainable, tested, production-quality code for large and complex datasets.
• Strong knowledge of Spark, Delta Lake or comparable lakehouse patterns, dimensional and analytical data modeling, partitioning, schema evolution, data contracts, and performance tuning.
• Hands-on experience with core AWS data-platform capabilities such as S3, IAM, KMS, VPC networking, CloudWatch, Secrets Manager, and related services used to secure, integrate, and operate Databricks workloads.
• Experience building batch and incremental pipelines using APIs, files, databases, SaaS connectors, change data capture, and at least one orchestration framework or managed workflow capability.
• Experience with Git-based development, CI/CD, automated data testing, code review, environment management, and infrastructure-as-code practices.
• Experience implementing data quality, observability, cataloging, lineage, access control, and operational support for business-critical data products.