Senior Data Engineer - AI-Driven Data Platforms

Visual Lease; San Diego (CA); On-site; USD 146,000 - 190,000; Full time

Benefits offered by this job

Comprehensive healthcare coverage

401(K) retirement plan with matching

Employee stock purchase plan

Paid time off

Tuition reimbursement

On-site fitness center

Job summary

CoStar Group is seeking a Senior Data Engineer to design and oversee dimensional data modeling and cloud data platform structures in San Diego, CA. You will build scalable pipelines with Databricks, Snowflake, and AWS services, integrating AI/ML into data workflows and ensuring data governance across domains.

The role requires 6+ years of hands-on experience in data engineering, strong Python/PySpark/SQL skills, and a track record of delivering enterprise data solutions.

Qualifications

• Bachelor's degree in a related field from an accredited institution.
• 6+ years of hands-on software development experience.
• Strong data engineering skills with Python, PySpark, and SQL.
• Experience building enterprise data platforms and data warehouses.
• Knowledge of cloud data services and data governance concepts.

Responsibilities

• Design and oversee dimensional modeling and data platform structures.
• Develop secure data warehouses and data lakehouses; implement Medallion Architecture.
• Build scalable ETL pipelines using Databricks, Snowflake, and AWS services.
• Optimize Spark jobs and data processing workflows; ensure data quality and performance.
• Integrate ML models into data pipelines and leverage Databricks ML / AWS ML.
• Lead data governance, lineage, testing, and monitoring frameworks.

Skills

Python

PySpark

SQL

Object-Oriented Programming

Dimensional modeling

Education

Bachelor's degree

Tools

Databricks

Snowflake

AWS

Power BI

DynamoDB

S3

Kafka

Azure

Job description

CoStar Group is seeking a Senior Data Engineer to design and oversee dimensional data modeling and cloud data platform structures in San Diego, CA. You will build scalable pipelines with Databricks, Snowflake, and AWS services, integrating AI/ML into data workflows and ensuring data governance across domains.

The role requires 6+ years of hands-on experience in data engineering, strong Python/PySpark/SQL skills, and a track record of delivering enterprise data solutions.