Databricks Data Engineer — Lakehouse & Governance Expert

UNAVAILABLE; McLean (VA); Presencial; USD 120 000 - 170 000; Tempo integral

Resumo da oferta

Unknown is seeking a seasoned Data Engineer to design, build, and migrate enterprise-grade data platforms, lakes, and lakehouses using Databricks. The role emphasizes data governance, cataloging with Unity Catalog, and collaboration with teams to deliver scalable data solutions across AWS/Azure/GCP.

You will apply PySpark/Python, SQL, and cloud technologies to support governance and data processing while participating in an Agile development lifecycle.

Qualificações

• Ability to obtain and maintain a U.S. government security clearance.
• Bachelor's degree in computer science, information systems, engineering, or a related technical discipline.
• 2–4 years of industry experience developing production software and solving complex technical problems.
• 2–4 years of direct experience in Data Engineering, including Databricks, Spark, Delta Lake, SQL, and data pipelines.
• Hands‑on experience implementing and working with Databricks Unity Catalog, data cataloging, governance, and permissions.
• Experience with data governance practices within Databricks data lake or lakehouse environments.
• Advanced SQL knowledge and experience with relational databases (T‑SQL, PostgreSQL, MySQL).
• Experience with large, structured and unstructured datasets; ability to modernize pipelines with Databricks.
• Experience with cloud platforms (AWS, Azure, GCP) and data lakehouse architectures.

Responsabilidades

• Design, develop, and support migration of data lake and lakehouse environments using Databricks for performance, reliability, and scalability.
• Implement and support Databricks Unity Catalog to enable centralized data governance and access control.
• Assess and understand existing ETL jobs, workflows, data marts, BI tools, reports, and data dependencies.
• Design, develop, and optimize scalable data pipelines and processing solutions using Databricks.
• Develop and support database, data warehouse, data mart, data lake, and lakehouse solutions in cloud environments (AWS/Azure/GCP).
• Apply hands-on expertise with Databricks, Unity Catalog, SQL, PySpark/Python, and cloud technologies to governance solutions.
• Support an Agile software development lifecycle.
• Contribute to the AI & Data Exploitation Practice growth.
• Collaborate with project teams including developers and data scientists building dashboards, reports, analytics, and ML models.

Conhecimentos

Communication skills

Customer service

Problem solving

Agile experience

Formação académica

Bachelor's degree in computer science/in information systems/engineering or related technical discipline

Ferramentas

Databricks

Apache Spark

Delta Lake

SQL (T-SQL / PostgreSQL / MySQL)

Databricks Unity Catalog

Databricks Workflows

Airflow

AWS/Azure/GCP cloud services

Python ( PySpark )

Java

Scala

Iceberg

Informatica EDC / Collibra / Alation / Purview / DataZone

Descrição da oferta de emprego

Unknown is seeking a seasoned Data Engineer to design, build, and migrate enterprise-grade data platforms, lakes, and lakehouses using Databricks. The role emphasizes data governance, cataloging with Unity Catalog, and collaboration with teams to deliver scalable data solutions across AWS/Azure/GCP.

You will apply PySpark/Python, SQL, and cloud technologies to support governance and data processing while participating in an Agile development lifecycle.