Senior Data Engineer

Illinois LLC

Chicago (IL)

Hybrid

USD 102,000 - 151,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Competitive benefits
Flexible schedules
Paid vacation
Paid sick leave
Paid personal days
Paid holidays
Parental leave
Pension plan
Deferred compensation
Pre-tax programs
Tuition reimbursement

Job summary

Illinois LLC in Chicago is seeking a Senior Data Engineer to design and operate enterprise-grade data platforms in a hybrid work setting. You will lead large-scale data pipelines and governance to support analytics, regulatory reporting, and operations.

The role offers senior technical leadership across cross-functional squads, with ongoing education and cross-team collaboration to establish enterprise standards and reliability.

Qualifications

  • Four (4) years of college in CS/CE/IS/Data Eng or related field.
  • Five (5) years professional IT experience in data roles.
  • Five (5) years designing enterprise-scale data pipelines and ETL/ELT patterns.
  • Five (5) years with distributed data processing and Spark optimization.
  • Five (5) years with lakehouse architecture design across platforms.
  • Four (4) years engineering reusable data access layers.

Responsibilities

  • Lead design, development, and optimization of enterprise data pipelines and scalable platforms.
  • Design and optimize ETL/ELT pipelines and analytics data models.
  • Govern enterprise data lake architecture with high availability and observability.
  • Implement CI/CD automation for data workloads.
  • Enforce role-based access control for data platforms.
  • Engineer data workloads across relational and non-relational platforms.
  • Provide senior technical leadership across cross-functional squads.
  • Maintain knowledge through ongoing education and conferences.
  • Perform other duties as required.

Skills

Data engineering
ETL/ELT design
Spark optimization
Data governance
CI/CD automation
Data pipelines
SQL performance tuning
Distributed data processing
Cross-functional leadership

Education

Four-year college degree in CS/CE/IS/Data Eng

Tools

Microsoft SQL Server
Azure SQL Database
DB2
Databricks
Apache Spark
Azure Synapse
Snowflake
Teradata Vantage
Cosmos DB
MongoDB
Redis
GitHub Actions
Terraform
Azure Bicep
Databricks Unity Catalog

Job description

Design and operate enterprise-grade data platforms for the Department of Innovation and Technology (DoIT) within a hybrid work setting in Chicago. As a Senior Data Engineer, you will lead the creation and optimization of large-scale data pipelines and distributed processing systems that support analytics, operations, and regulatory reporting, while strengthening governance, reliability, and visibility across the state’s data estate.

In this role, you will serve as a senior technical contributor and advisory partner across cross-functional data delivery squads, helping establish enterprise data engineering standards. You will also have the opportunity to grow your skills through ongoing education and professional development, including online training, meetings, seminars, and conferences.

Responsibilities
  • Under administrative direction, perform highly complex professional and advisory functions for DoIT, including leading the design, development, and optimization of enterprise data pipelines and scalable data platforms.
  • Design, build, and optimize enterprise ETL and ELT pipelines, distributed processing workloads, and semantic data models for analytics, business intelligence, and operational reporting.
  • Lead technical governance for enterprise data lake architecture, with focus on high availability, data quality, cost-efficient storage tiering, and end-to-end observability across the enterprise data asset inventory.
  • Design, implement, and govern CI/CD automation frameworks for data engineering workloads to support code promotion, environment parity, and production deployment that align with enterprise reliability and security standards.
  • Implement, govern, and continuously improve role-based access control for data platforms, ensuring data access policy is enforced technically, auditable, and aligned with DoIT security standards and applicable state and federal data governance regulations.
  • Engineer and optimize data workloads across relational and non-relational platforms and emerging technologies, including large-scale RDBMS and systems such as document databases, key value stores, and graph databases.
  • Provide senior technical leadership in cross-functional squads, representing enterprise data engineering standards in feature development and advising vendor partners and executive stakeholders.
  • Maintain current knowledge through continuing education, training sessions, seminars, and conferences.
  • Perform other duties as required or assigned within the scope of the position.
Requirements
  • Knowledge, skill, and mental development equivalent to four (4) years of college with coursework in Computer Science, Computer Engineering, Information Systems, Data Engineering, Software Engineering, Management Information Science, or a closely related field.
  • Five (5) years of professional experience in a related Information Technology field, including data engineering, software engineering, database administration, business intelligence development, data architecture, systems integration, or closely related work.
  • Five (5) years designing, building, and optimizing enterprise-scale data pipelines, fault-tolerant ETL/ELT patterns, and distributed data processing systems in a high-volume near-time production environment, including hands‑on SQL performance tuning and execution plan analysis across platforms such as Microsoft SQL Server, Azure SQL Database, DB2, or equivalent.
  • Five (5) years with distributed data processing and Spark execution engine optimization, including tuning large-scale batch, micro‑batch, and streaming workloads, with proficiency in execution plans, shuffle and partition strategy, memory management, and cluster autoscaling across platforms such as Databricks, Apache Spark, Azure Synapse, Teradata Vantage, Snowflake, Amazon EMR, Starburst, or Cloudera.
  • Five (5) years with enterprise data platform and lakehouse architecture design, including multi‑zone lake and lakehouse patterns, open table format optimization, schema evolution, and integration of storage, compute, and catalog layers across platforms such as Microsoft Fabric, Databricks Unity Catalog, Snowflake, or equivalent.
  • Four (4) years engineering reusable data access layers and gateway patterns that abstract source system complexity, reduce redundant data movement, and provide consistent governed interfaces for downstream consumers.
Technologies
  • SQL, ETL, ELT, CI/CD, DataOps, Spark
  • Microsoft SQL Server, Azure SQL Database, DB2
  • MongoDB, Couchbase, Redis, DynamoDB
  • Neo4j, Cosmos DB Gremlin API
  • Databricks, Apache Spark, Azure Synapse, Teradata Vantage, Snowflake, Amazon EMR, Starburst, Cloudera
  • Microsoft Fabric, Databricks Unity Catalog
  • Terraform, Azure Bicep, Databricks Asset Bundles, GitHub Actions
Benefits
  • Competitive group insurance benefits including health, life, dental, and vision.
  • Flexible work schedules when available and dependent upon position.
  • 10 to 25 days of paid vacation annually (10 days for first year of state employment).
  • 12 days of paid sick time annually, carry over year to year.
  • 3 paid personal business days per year.
  • 13 to 14 paid holidays per year depending on election years.
  • 12 weeks of paid parental leave.
  • Pension plan through the State Employees Retirement System.
  • Deferred Compensation Program, voluntary supplemental retirement plan.
  • Optional pre‑tax programs: Medical Care Assistance Plan (MCAP) and Dependent Care Assistant Plan (DCAP).
  • Tuition Reimbursement Program and Federal Public Service Loan Forgiveness Program eligibility.
  • For more information on State of Illinois benefits: https://www2.illinois.gov/cms/benefits/Pages/default.aspx
Job Details
  • Location: Chicago, IL (hybrid); 1 to 5 days per week in office depending on operational needs
  • Salary: Anticipated starting salary $9,130 per month; full range $9,130-$13,573
  • Work Hours: 8:30 AM to 5:00 PM, Monday through Friday (hours may vary)
  • Job Type: Salaried; Full Time
  • Number of Vacancies: 1
  • Union position: AFSCME (see collective bargaining agreement/labor contract provisions)
  • Agency: Department of Innovation and Technology
  • Headquarter Location: 555 W Monroe St, Chicago, Illinois, 60661
  • Agency Contact: Lorna.A.Bock@illinois.gov

Note on sponsorship: The State of Illinois does not provide sponsorship for visa status (such as H-1B) and cannot provide OPT extensions under STEM-designated degrees. Applicants must be currently authorized to work in the United States on a full‑time basis.

Reasonable accommodation: The State of Illinois is committed to working with and providing reasonable accommodations for people with disabilities. To request an accommodation during the hiring process, contact CMS.DisabilityResCen@illinois.gov or call (217) 524-7514.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Predictive Analytics and Data Scientist (Information Systems Analyst II, Opt. A)
Predictive Analytics and Data Scientist (Information Systems Analyst II, Opt. A)

Illinois Department of Innovation & Technology (DoIT) • Springfield (IL)

On-site
USD 90,000 - 130,000
Health insurance
Vacation days
Sick time
+4
Information Services Specialist II, Option A (Salesforce Release Technician)
Information Services Specialist II, Option A (Salesforce Release Technician)

State of Illinois • Springfield (IL)

On-site
USD 72,000 - 104,000
Data Engineer (Principal Consultant #1858)
Data Engineer (Principal Consultant #1858)

Illinois State Board of Education • Illinois

On-site
USD 64,322 - 109,467
Data Engineer (Principal Consultant #1086, 1858)
Data Engineer (Principal Consultant #1086, 1858)

Illinois Education Job Bank • Chicago (IL)

On-site
USD 64,000 - 109,000
Chief Compliance Officer (SPSA Opt. 1)
Chief Compliance Officer (SPSA Opt. 1)

Illinois Department of Innovation & Technology (DoIT) • Springfield (IL)

On-site
USD 130,000 - 140,000
Competitive Group Insurance benefits
Flexible work schedules
Paid vacation and sick leave
+2
DATA MANAGEMENT UNIT TECHNICIAN (ENGINEERING TECHNICIAN III)
DATA MANAGEMENT UNIT TECHNICIAN (ENGINEERING TECHNICIAN III)

Illinois LLC • United States

Hybrid
USD 46,000 - 80,000
Flexible work schedules (hybrid)
Health, Life, Vision, and Dental Ins.
Pension Plan
+6
CONTRACTUAL WORKER Data Strategy Advisor Job Details | State of Illinois
CONTRACTUAL WORKER Data Strategy Advisor Job Details | State of Illinois

Illinois Department of Children & Family Services • Chicago (IL)

Hybrid
USD 123,000 - 145,000
Hybrid work
Health insurance
Pension plan
+1
RECRUITMENT SOURCING SPECIALIST
RECRUITMENT SOURCING SPECIALIST

Illinois LLC • Springfield (IL)

Hybrid
USD 68,000 - 98,000
Flexible schedule
Pension plan
Paid time off
Chief Information Officer - IDOL
Chief Information Officer - IDOL

Illinois Department of Innovation & Technology (DoIT) • Springfield (IL)

On-site
USD 170,000 - 230,000
Health insurance
Dental & Vision plans
Pension plan
+5
DATA SCIENCE LEAD
DATA SCIENCE LEAD

City of Chicago Department of Law • Chicago (IL)

On-site
USD 120,000 - 180,000