Senior Data Engineer: Build Scalable Data Pipelines

Compunnel, Inc.; Lisle (IL); On-site; USD 100,000 - 130,000; Full time

Job summary

A data solutions company in Lisle seeks a Data Engineer to collaborate with teams on building scalable data solutions. You will design robust data pipelines and ensure data quality while leveraging Google Cloud services. Candidates should have significant experience in data engineering, proficiency in SQL and Python, along with a Bachelor's degree in a related field. The role offers opportunities for mentorship and working in an agile environment.

Qualifications

• 6+ years of experience in data engineering, including data platforms and analytics.
• Excellent problem-solving and critical-thinking abilities.
• Experience working in an agile development environment.

Responsibilities

• Collaborate with teams to gather and understand requirements.
• Design, develop, and maintain data pipelines.
• Optimize data pipelines for performance and reliability.
• Implement CI/CD pipelines for data products.
• Ensure data quality and document pipeline design.

Skills

SQL

Python

Data ingestion technologies (GCP Pub/Sub, Kafka, Spark)

Data modeling and semantics

CI/CD pipeline implementation

Airflow for workflow management

Query tuning

Agile development experience

Education

Bachelor's degree in Computer Science or related field

Master's degree preferred

Tools

Google Cloud services (BigQuery, Composer, GCS, DataStream, Dataflow)

Job description

A data solutions company in Lisle seeks a Data Engineer to collaborate with teams on building scalable data solutions. You will design robust data pipelines and ensure data quality while leveraging Google Cloud services. Candidates should have significant experience in data engineering, proficiency in SQL and Python, along with a Bachelor's degree in a related field. The role offers opportunities for mentorship and working in an agile environment.