Data Engineer

Jobtailor; Illinois; On-site; USD 140,000 - 180,000; Full time

Job summary

Jobtailor is seeking a senior data engineer in Illinois to design and implement advanced data ingestion pipelines, lakehouse architectures, and analytics solutions. You will work with Azure Databricks, Delta Lake, and orchestration tools to deliver scalable data platforms and insights.

Responsibilities include collaborating with clients, leading design sessions, and guiding governance and security practices while driving measurable business impact.

Qualifications

• Bachelor’s degree in computer science or related field.
• 6+ years of experience in data engineering, data platform development, or cloud data solutions.
• 3+ years of hands-on experience with Azure Databricks, Apache Spark, or similar distributed data processing technologies.
• Expertise with Microsoft Azure infrastructure and data resources including Fabric, Azure Data Factory, Synapse Data Analytics, Power BI, Azure SQL, Azure Cosmos DB, and Azure Database for PostgreSQL.
• Expertise with Databricks including Unity Catalog, data warehousing, data sharing, and Mosaic AI.
• DevOps for data, GitHub, automated testing, and containers such as AKS, Docker, and registries.
• Excellent communication skills and ability to explain concepts clearly to teammates and customers.
• Ability to quickly learn new concepts and technologies.
• Preferred: experience working with clients or cross-functional teams in consulting or professional services.
• Preferred: experience building data agents and Fabric Data Agents.
• Preferred: experience with data management, governance, security, and industry security requirements.
• Preferred: broad experience with data/reporting tools, architectures, cloud vendors, and data/AI concepts beyond Microsoft.

Responsibilities

• Work directly with clients to understand business goals, data challenges, and technical requirements
• Lead or support discovery sessions, requirements workshops, architecture discussions, and solution reviews
• Develop and optimize batch and streaming data ingestion pipelines from enterprise applications, databases, APIs, and file-based sources
• Implement medallion/lakehouse architectures, dimensional models, and data transformation workflows for analytics and reporting
• Engineer solutions using PySpark, Spark SQL, SQL, Python, Delta Lake, and orchestration tools within Azure and Databricks
• Recommend best practices for data modeling, governance, lineage, monitoring, DevOps, and security
• Contribute to solution architecture
• Deliver high-quality modern data solutions that drive measurable business value
• Facilitate design sessions with client stakeholders and present recommendations to leadership teams

Skills

Data Engineering

Azure Databricks

PySpark

Data Governance

DevOps for Data

Data Modeling

Data Architecture

SQL

Python

Spark SQL

Delta Lake

Data Ingestion Pipelines

Education

Bachelor’s Degree in Computer Science or Related Field

Tools

Microsoft Azure

Azure Data Factory

Synapse Data Analytics

Power BI

Azure SQL

Azure Cosmos DB

Azure Database for PostgreSQL

GitHub

Docker

AKS

Job description

• Work directly with clients to understand business goals, data challenges, and technical requirements
• Lead or support discovery sessions, requirements workshops, architecture discussions, and solution reviews
• Develop and optimize batch and streaming data ingestion pipelines from enterprise applications, databases, APIs, and file-based sources
• Implement medallion/lakehouse architectures, dimensional models, and data transformation workflows for analytics and reporting
• Engineer solutions using PySpark, Spark SQL, SQL, Python, Delta Lake, and orchestration tools within Azure and Databricks
• Recommend best practices for data modeling, governance, lineage, monitoring, DevOps, and security
• Contribute to solution architecture
• Deliver high-quality modern data solutions that drive measurable business value
• Facilitate design sessions with client stakeholders and present recommendations to leadership teams

Requirements

• Bachelor’s degree in computer science or related field
• 6+ years of experience in data engineering, data platform development, or cloud data solutions
• 3+ years of hands-on experience with Azure Databricks, Apache Spark, or similar distributed data processing technologies
• Expertise with Microsoft Azure infrastructure and data resources, including Fabric, Azure Data Factory, Synapse Data Analytics, Power BI, Azure SQL, Azure Cosmos DB, and Azure Database for PostgreSQL
• Expertise with Databricks, including enterprise-level strategy and architecture with Unity Catalog, data warehousing, data sharing, and Mosaic AI
• DevOps for data, GitHub, automated testing, and working with containers such as AKS, Docker, and registries
• Excellent communication skills and ability to explain concepts clearly to teammates and customers
• Ability to quickly learn new concepts and technologies
• Preferred: experience working directly with clients, business stakeholders, or cross-functional teams in consulting or professional services
• Preferred: experience building data agents, including NLQ, Databricks Genie, and Fabric Data Agents
• Preferred: experience with data management, governance, security, master data management, and industry security requirements
• Preferred: broad experience with data/reporting tools, architectures, cloud vendors, and data/AI concepts other than Microsoft

Core Competencies

Demonstrates expertise in data engineering and cloud data solutions, with a strong focus on Azure Databricks, data ingestion pipelines, and data architecture. Proven ability to communicate effectively with clients and stakeholders while delivering high-quality data solutions that drive business value.

Highest-signal resume keywords

• Data Engineering
• Azure Databricks
• PySpark
• Data Governance
• DevOps for Data

ATS Optimization Keywords

Hard Skills

• Data Ingestion Pipelines
• SQL
• Python
• Spark SQL
• Delta Lake
• Data Transformation Workflows
• Data Modeling
• Data Architecture
• Data Warehousing
• Data Sharing

Soft Skills

• Excellent Communication Skills
• Ability to Explain Concepts Clearly
• Quick Learner

Certifications & Qualifications

• Bachelor’s Degree in Computer Science or Related Field

Industry Keywords

• Data Management
• Data Governance
• Master Data Management
• Industry Security Requirements
• Cloud Data Solutions
• Data/AI Concepts

Tools & Technologies

• Microsoft Azure
• Azure Data Factory
• Synapse Data Analytics
• Power BI
• Azure SQL
• Azure Cosmos DB
• Azure Database for PostgreSQL
• GitHub
• Docker
• AKS