Onsite Data Engineering Lead & PM - Azure Databricks

Estrada Consulting, Inc; Clearfield (UT); On-site; USD 90,000 - 124,000; Full time

Job summary

Estrada Consulting, Inc. seeks a hands-on Data Engineering Project Manager and Technical Lead to run a six-month on-site engagement at a university research center in Clearfield, Utah.

You will plan work with center staff, set priorities against the data dictionary, lead weekly partner meetings, and perform core engineering on pipelines and the data model while ensuring the center can own the system when we depart.

Qualifications

• Three years or more of data engineering experience.
• Experience leading technical projects: planning, weekly status with the client, priorities and acceptance of deliverables.
• Experience with data ingestion, data pipelines, data modeling and monitoring.
• Cloud experience (Azure, AWS or GCP), including IaaS and PaaS services.
• Data security, permission management and secure networking experience.
• U.S. citizen able to work onsite in Clearfield, Utah, full time.

Responsibilities

• Plan and track the work: high-priority data sources in the first three months, then the remaining sources.
• Lead weekly partner meetings, report progress and risks, and get deliverables reviewed and accepted.
• Work with researchers and staff to design extraction for each machine and refine existing extraction scripts.
• Set the technical direction for staging, the production data model, orchestration and alerting on Databricks.
• Own security and permissions in the Azure Government environment.
• Train and mentor student interns, and lead the handover: training sessions, recordings and documentation.

Skills

Data engineering

Project leadership

Data ingestion

Data pipelines

Data modeling

Cloud platforms

Data security

Onsite work

Stakeholder management

Tools

Databricks

Apache Spark

Azure Government

Azure Data Factory

Azure Blob Storage

Security & IAM

Job description

Estrada Consulting, Inc. seeks a hands-on Data Engineering Project Manager and Technical Lead to run a six-month on-site engagement at a university research center in Clearfield, Utah.

You will plan work with center staff, set priorities against the data dictionary, lead weekly partner meetings, and perform core engineering on pipelines and the data model while ensuring the center can own the system when we depart.