Data Engineer – ADF / Databricks / Snowflake

TALENT Software Services; Dallas (TX); On-site; USD 83,000 - 124,000; Full time

Job summary

TALENT Software Services seeks a Data Engineer with 10+ years of experience for a 6-month engagement in Dallas, TX. You will design and implement scalable data pipelines using ADF, Databricks, and Snowflake, while working with healthcare and SAP SCM data.

Proficiency in SQL, Python, and BI tools like Tableau and Power BI is required, along with experience in CI/CD, Jira, and Agile development. The role emphasizes data visualization and delivery of actionable insights.

Qualifications

• 10+ years of experience as a Data Engineer.
• Hands-on with ADF, Databricks, and Snowflake.
• Healthcare data experience and SAP SCM domain knowledge.
• CI/CD pipelines, GitHub, and deployment best practices.
• Databricks Workflows and Apache Airflow exposure.
• Agile/Scrum with Jira for sprint planning and backlog management.
• ETL migration experience from DataStage to Azure.
• AI-assisted development and automation tools experience.
• Ability to drive data visualization and insights.

Responsibilities

• Build scalable data pipelines with Databricks.
• Develop data models using Delta Lake and Spark.
• Create SQL/Python data transformations and optimizations.
• Visualize data with Tableau and Power BI to derive insights.
• Work with healthcare datasets and SAP SCM data standards.
• Implement CI/CD, GitHub workflows, and deployment best practices.
• Utilize Databricks Workflows and Airflow for orchestration.
• Collaborate in Agile teams using Jira for planning.
• Assist in ETL migrations to Azure Data Factory.
• Leverage AI-assisted tools for code generation and optimization.

Skills

Databricks

SQL

Python

Tableau

Power BI

Healthcare data analysis

Clinical data

Agile/Scrum

Jira

Airflow

CI/CD

GitHub

Data visualization

Tools

Azure Data Factory

Databricks

Snowflake

ETL/ELT

DataStage

Azure

Databricks Workflows

Airflow

GitHub

Tableau

Power BI

Job description

• Duration: 6 months
• GBaMS ReqID: 10926596
• Experience Required: 10+ years
• Essential Skills: Data Engineer - ADF, Databricks, and Snowflake

Job Details

• Job Title: Data Engineer - ADF / Databricks / Snowflake
• Location: Dallas, TX
• Duration: 6 months
• GBaMS ReqID: 10926596
• Experience Required: 10+ years
• Essential Skills: Data Engineer - ADF, Databricks, and Snowflake
• Keyword: Digital: Google Data Engineering

Key Skill Set

• Strong hands-on experience with Databricks for building scalable data pipelines.
• Strong knowledge of:
  • Delta Lake
  • Apache Spark
  • Databricks
• Advanced proficiency in:
  • SQL
  • Python
• Experience with SQL and Python for:
  • Data transformation
  • Data optimization
  • Data automation
• Experience with data visualization tools, including:
  • Tableau
  • Power BI
• Ability to deliver actionable business insights through data visualization.
• Solid understanding of healthcare data models.
• Experience working with:
  • Healthcare claims datasets
  • Clinical datasets
  • Healthcare data standards
  • Domain-specific compliance requirements
• Strong SAP Supply Chain Management domain expertise.
• Expertise in:
  • CI/CD pipelines
  • GitHub
  • Version control
  • Deployment best practices
• Experience with workflow orchestration tools, including:
  • Databricks Workflows
  • Apache Airflow
• Familiarity with Agile development methodologies.
• Experience using Jira for:
  • Sprint planning
  • Task tracking
  • Backlog management
• Exposure to ETL migration, specifically migrating DataStage jobs to Azure.
• Practical experience leveraging AI-assisted development tools for:
  • Code generation
  • Code optimization
  • Documentation
  • Developer productivity
• Ability to use AI tools to accelerate development and delivery.

Technical Skills

• Data Engineering: Azure Data Factory, Databricks, Snowflake, ETL/ELT
• Big Data: Apache Spark, Delta Lake
• Programming: Python, SQL
• Visualization: Tableau, Power BI
• Cloud: Microsoft Azure
• DevOps: CI/CD, GitHub, Version Control
• Orchestration: Databricks Workflows, Airflow
• ETL Migration: DataStage to Azure
• Enterprise Systems: SAP
• Project Management: Jira, Agile/Scrum
• AI: AI-Assisted Development, Code Generation, Code Optimization

Domain Skills

• Healthcare Data
• Claims Data
• Clinical Data
• Healthcare Compliance
• SAP Supply Chain Management
• Enterprise Data Engineering

Preferred Skills

• Advanced Databricks and Delta Lake development.
• Azure Data Factory pipeline development.
• Snowflake data engineering.
• Healthcare analytics and data platforms.
• SAP Supply Chain data integration.
• DataStage-to-Azure migration experience.
• AI-enabled software development and automation.