Senior Data Engineer - Cloud Data Pipelines & Observability

Capital One National Association; Chicago (IL); On-site; USD 179,000 - 205,000; Full time

Job summary

Capital One is seeking a Data Engineer to design, develop, and operate cloud-first data solutions across Agile teams. You’ll work with Python, Spark, and SQL, leveraging Databricks, Snowflake, and other ecosystems to support data-driven decision making for millions of customers.

Join a collaborative environment that emphasizes scalable architectures, data security, and continuous learning. This role includes mentorship, architectural guidance, and a focus on performance and reliability.

Qualifications

• BS degree in Computer Science or related quantitative field.
• 4+ years of experience in application development.
• 2+ years of experience in distributed data.
• 2+ years of experience with SQL.
• 2+ years of experience with one programming language (Python, Java, or Scala).
• 2+ years of experience in data pipeline design and development.
• 1+ year of experience in data modeling and end-to-end data solutions using relational and non-relational databases.

Responsibilities

• Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
• Influence a team of developers, data analysts and data scientists with deep experience in ML, distributed microservices, lakehouse architecture, and full-stack systems
• Utilize programming languages such as Python and Spark, along with open-source relational and NoSQL databases, and cloud-based data warehousing platforms including Databricks and Snowflake
• Share passion for trends in data, experimenting with new technologies, and mentoring other data community members
• Collaborate with product managers and software engineers to deliver cloud-first data solutions for millions of Americans
• Independently design, build and deliver world-class cloud data solutions with little supervision
• Architect data engineering design patterns to ensure code quality and reusability
• Serve as ambassador for the data engineering team to internal and external stakeholders
• Design and build data pipelines focusing on scalability and operational efficiency
• Implement data security standards, including encryption and fine-grained access control

Skills

Python

SQL

Spark

Data modeling

Education

Bachelor's Degree in Computer Science or related quantitative field

Tools

Databricks

Snowflake

Redshift

EMR

Glue

Airflow

Dagster

MongoDB

Cassandra

DynamoDB

Job description

Capital One is seeking a Data Engineer to design, develop, and operate cloud-first data solutions across Agile teams. You’ll work with Python, Spark, and SQL, leveraging Databricks, Snowflake, and other ecosystems to support data-driven decision making for millions of customers.

Join a collaborative environment that emphasizes scalable architectures, data security, and continuous learning. This role includes mentorship, architectural guidance, and a focus on performance and reliability.