Associate Data Engineer: Build Scalable Data Pipelines

IBM; Chicago (IL); On-site; USD 90,000 - 130,000; Full time

Job summary

IBM's Associate Data Engineer program invites university hires to harness data to uncover stories and patterns. You will contribute to data gathering, storage, and both batch and real-time processing, collaborating with diverse teams to decide suitable data systems and the data required for insightful analysis.

As an Associate, you'll translate business needs with the Architect and Client, design scalable cloud-based architectures, optimize data pipelines, and create dashboards and reports for

Qualifications

• Ability to apply statistical and ML techniques.
• Basic understanding of Cloud platforms (AWS, Azure).
• Proficiency in Java, Python, and Scala to build data pipelines.
• Experience with ETL/ELT tools for data prep.
• Experience with Linux, SQL, Spark, Hadoop, and Java.
• Experience with data warehousing design and analytics pipelines.
• Willingness to travel up to 100% based on project requirements.

Responsibilities

• Design scalable data architecture for cloud environments.
• Optimize data pipelines for performance.
• Work with ETL/ELT ingestion pipelines.
• Collect and analyze data to identify trends for insights.
• Collaborate with cross-functional teams to translate requirements.
• Present findings to technical and non-technical stakeholders.
• Ensure data integrity through cleaning and preprocessing.
• Explain business requirements to solution teams.
• Contribute to data gathering, storage, and real-time processing.

Skills

Statistics & ML basics

Cloud fundamentals

Programming: Java, Python, Scala

ETL/ELT pipelines

SQL & Data querying

Apache Spark / Hadoop

Travel willingness

Education

Bachelor's Degree

Tools

dbt

Snowflake

BigQuery

Airflow

CI/CD basics

Job description

IBM's Associate Data Engineer program invites university hires to harness data to uncover stories and patterns. You will contribute to data gathering, storage, and both batch and real-time processing, collaborating with diverse teams to decide suitable data systems and the data required for insightful analysis.

As an Associate, you'll translate business needs with the Architect and Client, design scalable cloud-based architectures, optimize data pipelines, and create dashboards and reports for