Senior Cloud Data Engineer — IoT & Real-Time Streams

General Motors Company; Nacogdoches (TX); Hybrid; USD 125,000 - 169,000; Full time

Job summary

General Motors is seeking a senior Data Engineer to join the Energy Data Engineering team in a hybrid role based near Austin. The candidate will design, implement, and optimize cloud-based data warehouse solutions, including an ODS, with real-time streaming pipelines using Flink, Pulsar, and Kafka.

The role requires 7+ years of software development, strong Java skills, GraphQL API design, and experience with PostgreSQL, Databricks, Azure, and CI/CD.

Qualifications

• 7+ years of professional software development experience.
• Strong proficiency in Java and object-oriented design.
• Experience building production-grade backend services and distributed systems.
• Hands-on experience with streaming technologies, preferably Apache Flink, Apache Pulsar, and Apache Kafka.
• Strong understanding of event-driven architecture, messaging patterns, data partitioning, consumer groups, delivery semantics, and stream processing.
• Experience designing and developing GraphQL APIs.
• Strong experience with PostgreSQL, including schema design, SQL, performance tuning, indexing, and transaction management.
• Experience with Databricks and data engineering or analytics platforms.
• Experience developing and deploying applications in Microsoft Azure.
• Practical CI/CD experience with automated testing, packaging, deployment, and release management.
• Experience with source control and collaborative development workflows, such as Git and pull requests.
• Strong troubleshooting, communication, and technical documentation skills.

Responsibilities

• Design, develop, test, and maintain scalable Java services and distributed applications.
• Build and optimize real-time data streaming pipelines using Apache Flink, Apache Pulsar, and Apache Kafka.
• Develop reliable event-driven services with strong attention to throughput, latency, resiliency, ordering, replay, and fault tolerance.
• Design and implement an Operational Data Store (ODS) layer using GraphQL and PostgreSQL.
• Develop GraphQL APIs and services that provide secure, efficient, and well-structured access to operational data.
• Design relational schemas, queries, indexes, and data-access patterns for PostgreSQL.
• Integrate streaming and operational data services with Databricks and cloud-based data platforms.
• Deploy and operate applications in Microsoft Azure using cloud-native architecture and services.
• Implement automated build, test, deployment, and release processes through CI/CD pipelines.
• Apply software engineering best practices, including code reviews, automated testing, observability, documentation, and secure development.
• Troubleshoot complex issues across application, streaming, database, and cloud environments.
• Participate in technical design discussions and contribute to architecture decisions.
• Mentor other engineers and help establish engineering standards and reusable patterns.
• Use generative AI tools and effective prompting techniques to accelerate development, troubleshoot issues, create documentation, and improve engineering productivity while applying appropriate security and data-handling practices.

Skills

Java

Distributed systems

Streaming technologies

Event-driven architecture

GraphQL APIs

PostgreSQL

Azure

CI/CD

Git

Troubleshooting

Tools

Databricks

Apache Flink

Apache Pulsar

Apache Kafka

Job description

General Motors is seeking a senior Data Engineer to join the Energy Data Engineering team in a hybrid role based near Austin. The candidate will design, implement, and optimize cloud-based data warehouse solutions, including an ODS, with real-time streaming pipelines using Flink, Pulsar, and Kafka.

The role requires 7+ years of software development, strong Java skills, GraphQL API design, and experience with PostgreSQL, Databricks, Azure, and CI/CD.