Staff Data Engineer

General Motors; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking a Staff Data Engineer to join their Marketing Applied Sciences team. This pivotal role focuses on designing and maintaining robust data pipelines and architectures to support innovative marketing strategies. The ideal candidate will leverage their extensive experience in data engineering, collaborate with diverse teams, and ensure high-quality data is available for strategic insights. If you are passionate about data and eager to drive impactful solutions in a dynamic environment, this is the perfect opportunity for you to make a significant contribution to the future of marketing analytics.

Qualifications

• 7+ years of hands-on experience in data engineering.
• Strong understanding of ETL processes and data pipeline management.
• Experience with big data frameworks and cloud platforms.

Responsibilities

• Design and maintain data architectures and pipelines for marketing strategies.
• Collaborate with cross-functional teams to ensure data quality.
• Document best practices and optimize data processes.

Skills

Data ETL processes

Apache Hadoop

Apache Spark

Apache Kafka

Python

Java

Scala

Database administration

Cloud platforms

Data governance principles

Education

Bachelor's degree in Computer Science

Master's degree or PhD in related field

Tools

Docker

Kubernetes

MySQL

PostgreSQL

NoSQL databases

Job description

This role is based remotely but if you live within a 50-mile radius of an office [Atlanta, Austin, Detroit, Warren, Milford or Mountain View], you are expected to report to that location three times a week, at minimum.

The Role

The Marketing Applied Sciences team is responsible for developing analytics-driven solutions to help GM Marketing Organizations achieve their business goals. The Staff Data Engineer role is responsible for designing, developing and maintaining data pipelines, databases and data infrastructure to enable the efficient data collection, storage and analysis in support of marketing strategies. This role requires collaboration with data scientists, data infrastructure architects and data consumption stakeholders to ensure the availability of high-quality data for insights and decision making is available and will deliver value to the GM strategic vision for the future and meet prioritized business needs. This role drives to deliver innovative solutions with technical fidelity and thrives to work effectively across a cross-functional data and stakeholder ecosystem.

What You’ll Do

1. 7+ years of hands-on experience.
2. Bachelor's degree (or equivalent work experience) in Computer Science, Data Science, Software Engineering, or a related field.
3. Design, construct, install and maintain data architectures, including database and large-scale processing systems.
4. Develop and maintain ETL (Extract, Transform, Load) processes to collect, cleanse and transform data from various sources inclusive of cloud.
5. Design and implement data pipelines to collect, process and transfer data from various sources to storage systems (data warehouses, data lakes, etc).
6. Implement security measures to protect sensitive data and ensure compliance with data privacy regulations.
7. Build data solutions that ensure data quality, integrity and security through data validation, monitoring, and compliance with data governance policies.
8. Administer and optimize databases for performance and scalability.
9. Maintain Master Data, Metadata, Data Management Repositories, Logical Data Models, and Data Standards.
10. Troubleshoot and resolve data-related issues affecting data quality fidelity.
11. Document data architectures, processes and best practices for knowledge sharing across the GM data engineering community.
12. Participate in the evaluation and selection of data related tools and technologies.
13. Collaborate across other engineering functions within EDAI, Marketing Technology, and Software & Services.

Your Skills & Abilities (Required Qualifications)

1. Master's degree or PHD in equivalent area.
2. Strong understanding and ability to provide mentorship in the areas of data ETL processes and tools for designing and managing data pipelines.
3. Proficient with big data frameworks and tools like Apache Hadoop, Apache Spark, or Apache Kafka for processing and analyzing large datasets.
4. Hands on experience with data serialization formats like JSON, Parquet and XML.
5. Consistently models and leads in best practices and optimization for scripting skills in languages like Python, Java, Scala, etc for automation and data processing.
6. Proficient with database administration and performance tuning for databases like MySQL, PostgresSQL or NoSQL databases.
7. Proficient with containerization (e.g., Docker) and orchestration platforms (e.g., Kubernetes) for managing data applications.
8. Experience with cloud platforms and data services for data storage and processing.
9. Consistently designs solutions and build data solutions that are highly automated, performant, with quality checks that provide data consistency and accuracy outcomes.
10. Experienced at actively managing large-scale data engineering projects, including planning, resource allocation, risk management, and ensuring successful project delivery and adjust style for all delivery methods (ie: Waterfall, Agile, POD, etc).
11. Understands data governance principles, data privacy regulations, and experience implementing security measures to protect data.
12. Able to integrate data engineering pipelines with machine learning models and platforms.
13. Strong problem-solving skills to identify and resolve complex data engineering issues efficiently.
14. Ability to work effectively in cross-functional teams, collaborate with data scientists, analysts, and stakeholders to deliver data solutions.
15. Ability to lead and mentor junior data engineers, providing guidance and support in complex data engineering projects.
16. Influential communication skills to effectively convey technical concepts to non-technical stakeholders and document data engineering processes.
17. Models a mindset of continuous learning, staying updated with the latest advancements in data engineering technologies, and a drive for innovation.