AWS Databricks Cloud Data Engineer (Lakehouse)

Credence Management Solutions, LLC; McLean (VA); Presencial; USD 155.000 - 185.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Health care plan (Medical, Dental &amp

Retirement Plan (401k, IRA)

Life Insurance (Basic, Voluntary &

Paid Time Off (Vacation, Sick & PH

Family Leave (Maternity, Paternity)

Training & Development

Wellness Resources

Descripción de la vacante

Credence Management Solutions, LLC is seeking an AWS Databricks Engineer to design and optimize scalable cloud-based data solutions. You will build robust data pipelines, implement modern lakehouse architectures, and collaborate across engineering, data science, and stakeholders to advance federal missions.

You will work on data governance, DevOps automation, and secure data practices while delivering high-performance capabilities. Strong AWS and Databricks expertise are essential for success.

Formación

• U.S. citizenship with the ability to obtain and maintain a Secret clearance is required.
• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, Engineering, or a related technical field.
• Hands-on experience in data engineering, cloud engineering, or related technical roles.
• Hands-on experience building and maintaining production data solutions using Databricks, Apache Spark, and PySpark.
• Strong experience with AWS data services such as S3, Glue, RDS, Redshift, Lambda, IAM, and CloudWatch.
• Strong understanding of ETL/ELT processes, data lake and lakehouse architectures, data modeling, and large-scale data processing.
• Hands-on experience with Delta Lake, SQL, Python, and Spark-based data processing.
• Experience developing, scheduling, monitoring, and troubleshooting production data pipelines and workflows.
• Experience with Git and CI/CD platforms such as GitLab, GitHub Actions, Jenkins, or AWS CodePipeline.
• Understanding of cloud security, identity and access management, encryption, and secure data-handling practices.
• Strong communication skills and a client-oriented mindset with the ability to collaborate across technical and non-technical teams.

Responsabilidades

• Databricks Engineering: Design, build, and optimize scalable data pipelines and processing workflows using Databricks, Spark, PySpark, and Delta Lake.
• Data Lakehouse Architecture: Design and implement modern lakehouse solutions with Databricks and AWS, applying Bronze/Silver/Gold layers.
• AWS Data Integration: Build ETL/ELT pipelines across S3, Glue, RDS, Redshift, Lambda, IAM, and related services.
• Data Transformation & Optimization: Develop data transformation workflows using SQL, Python, and PySpark; optimize Spark workloads.
• Data Governance & Quality: Implement data quality, cataloging, lineage, governance, and access-control practices.
• Cloud Automation & DevOps: Automate deployments using IaC, CI/CD, and AWS-native tools.
• Collaborative Engineering: Work with data engineers, software engineers, data scientists, and stakeholders.
• Technical Rigor & Documentation: Write clean, well-documented code and support reviews, testing, and design docs.

Conocimientos

Databricks

AWS data services

ETL/ELT processes

Delta Lake

Spark (PySpark)

Git & CI/CD

Cloud security & IAM

Communication skills

Client-facing mindset

Data modeling

Educación

Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, Engineering, or a related technical field

Herramientas

Terraform

GitHub Actions

Jenkins

AWS CodePipeline

Databricks Unity Catalog

Delta Live Tables

Descripción del empleo

Credence Management Solutions, LLC is seeking an AWS Databricks Engineer to design and optimize scalable cloud-based data solutions. You will build robust data pipelines, implement modern lakehouse architectures, and collaborate across engineering, data science, and stakeholders to advance federal missions.

You will work on data governance, DevOps automation, and secure data practices while delivering high-performance capabilities. Strong AWS and Databricks expertise are essential for success.