Data Engineer

Stellantis; Auburn Hills (MI); In loco; USD 120.000 - 180.000; Tempo pieno

Descrizione del lavoro

Stellantis is seeking a Senior Data Engineer to design, build, and optimize robust data pipelines for massive datasets in batch and real-time modes. You will operate at the intersection of software engineering and data science to ensure scalable and reliable data architectures that follow industry best practices.

Responsibilities include designing Spark-based pipelines, building distributed systems, and managing cloud infrastructure with Terraform.

Competenze

• Bachelor's degree in Computer Science, Engineering, Mathematics, or related technical discipline.
• Minimum 5 years in data engineering and SDLC.
• At least 4 years building/maintaining production data applications with relational and columnar stores.
• At least 4 years working with AWS cloud services.
• Proficiency in Python, Java, or Scala.
• Experience with Spark, BigQuery, AWS S3, Azure, etc.
• Familiarity with Kafka, Flink, and event-driven architectures.
• Experience with Airflow, Luigi, Azure Data Factory, etc.
• Experience designing real-time pipelines and data quality validation.
• Strong SQL for data transformation and performance tuning.
• Terraform for infrastructure and GitHub Actions for CI/CD.
• Knowledge of Clean Code practices in data engineering.
• Experience with Flink for low-latency stream processing.
• Python scripting for automation.

Mansioni

• Design and implement complex data processing pipelines using Apache Spark.
• Build scalable, distributed systems for high-throughput data streams and batch processing.
• Manage cloud infrastructure using Terraform.
• Streamline development workflows with GitHub Actions for testing and deployment.
• Maintain code quality through unit/integration testing and reviews.
• Collaborate with stakeholders to translate business requirements into technical specs.

Conoscenze

Apache Spark

Python

Java/Scala

AWS Cloud

SQL Proficiency

Kafka / Flink

Airflow

Terraform

DevOps CI/CD

Formazione

Bachelors degree in Computer Science or related field

Strumenti

Databricks

GitHub Actions

S3 / AWS tooling

Descrizione del lavoro

The AI & Data Analytics Team is looking for a Senior Data Engineer to join our team. In this role, you will be responsible for designing, building, and optimizing robust data pipelines that process massive datasets in both batch and real-time. You will work at the intersection of software engineering and data science, ensuring that our data architecture is scalable, reliable, and follows industry best practices.

Priorities can change in a fast-paced environment like ours, so this role includes, but is not limited to, the following responsibilities:

• Pipeline Development: Design and implement complex data processing pipelines using Apache Spark.

• Architectural Leadership: Build scalable, distributed systems that handle high-throughput data streams and large-scale batch processing.

• Infrastructure as Code: Manage and provision cloud infrastructure using Terraform.

• CI/CD & Automation: Streamline development workflows by implementing and maintaining GitHub Actions for automated testing and deployment.

• Code Quality: Uphold rigorous software engineering standards, including comprehensive unit/integration testing, code reviews, and maintainable documentation.

• Collaboration: Work closely with stakeholders to translate business requirements into technical specifications.

Basic Qualifications:

• Bachelors degree in Computer Science, Engineering, Mathematics, or a related technical discipline
• A minimum of 5 years of experience in the data engineering and software development life cycle. Including:
  • A minimum of 4 years of hands-on experience in building and maintaining production data applications, current experience in both relational and columnar data stores.
  • A minimum of 4 years of hands-on experience working with AWS cloud services
• Comprehensive experience with one or more programming languages such as Python, Java, or Rust
• Comprehensive experience working with Big Data platforms (i.e., Spark, Google Big Query, Azure, AWS S3, etc.)
• Familiarity with time series database, data streaming applications, event driven architectures, Kafka, Flink, and more
• Experience with workflow management engines (i.e., Airflow, Luigi, Azure Data Factory, etc.)
• Experience with designing and implementing real-time pipelines
• Experience with data quality and validation
• Experience with API design
• Distributed Computing: Deep expertise in Apache Spark (Core, SQL, and Structured Streaming).
• Programming Mastery: Strong proficiency in Scala or Java. You should be comfortable building production-grade applications in a JVM-based environment.
• SQL Proficiency: Advanced knowledge of SQL for data transformation, analysis, and performance tuning.
• DevOps & Tools: Hands-on experience with Terraform for infrastructure management and GitHub Actions for CI/CD pipelines.
• Software Engineering Foundation: Solid understanding of data structures, algorithms, and design patterns. Experience applying "Clean Code" principles to data engineering.
• Stream Processing: Experience with Apache Flink for low-latency stream processing.
• Scripting: Proficiency in Python for automation, data analysis, or scripting.
• Cloud Platforms: Experience with AWS, Azure, or GCP data services (e.g., EMR, Glue, Databricks).
• Data Modeling: Familiarity with dimensional modeling, Lakehouse architectures (Delta Lake, Iceberg), or NoSQL databases.

Preferred Qualifications:

• Comprehensive knowledge of relational database concepts, including data architecture, operational data stores, Interface processes, multidimensional modeling, master data management, and data manipulation
• Expert knowledge and experience with custom ETL design, implementation and maintenance
• Comprehensive experience designing, implementing, and iterating data pipelines using Big Data technologies
• Certification in AWS or other cloud providers
• Experience with Databricks notebook workflows
• Experience with Terraform

The AI & Data Analytics Team is looking for a Senior Data Engineer to join our team. In this role, you will be responsible for designing, building, and optimizing robust data pipelines that process massive datasets in both batch and real-time. You will work at the intersection of software engineering and data science, ensuring that our data architecture is scalable, reliable, and follows industry best practices.

Priorities can change in a fast-paced environment like ours, so this role includes, but is not limited to, the following responsibilities:

• Pipeline Development: Design and implement complex data processing pipelines using Apache Spark.

• Architectural Leadership: Build scalable, distributed systems that handle high-throughput data streams and large-scale batch processing.

• Infrastructure as Code: Manage and provision cloud infrastructure using Terraform.

• CI/CD & Automation: Streamline development workflows by implementing and maintaining GitHub Actions for automated testing and deployment.

• Code Quality: Uphold rigorous software engineering standards, including comprehensive unit/integration testing, code reviews, and maintainable documentation.

• Collaboration: Work closely with stakeholders to translate business requirements into technical specifications.

At Stellantis, we assess candidates based on qualifications, merit, and business needs. We welcome applications from all people without regard to sex, age, ethnicity, nationality, religion, sexual orientation, disability, or any characteristic protected by law. We believe that diverse teams reflect our identity as a global company, enabling us to better address the evolving needs of our customers and care for our future.