Data Engineer

dorle-controls-llc; Columbus (IN); On-site; USD 90,000 - 120,000; Full time

Job summary

dorle-controls-llc is looking for a Data Engineer in Columbus, IN. The role involves building high-performance data pipelines and understanding business processes for analytics.

Ideal candidates will have 5+ years of experience, strong programming skills in Spark and SQL, and familiarity with data lake and data warehouse concepts. Collaboration with analysts and stakeholders is crucial for success in this position.

Qualifications

• 5+ years of hands-on data engineering experience.
• Strong programming skills in Spark, Scala, and Python.
• Strong SQL skills, especially in aggregations and large joins.

Responsibilities

• Develop, optimize, and productionize Spark pipelines.
• Write complex aggregation logic for business KPIs.
• Build and maintain multi-layer Data Lake architectures.
• Design dimensional models and optimize table structures.

Skills

Spark

Scala

Python

SQL

Data Lake concepts

Data Warehouse concepts

Tools

Databricks

Snowflake

Azure Data Factory

Synapse

Job description

Title: Data Engineer

Location: Columbus, IN.

About the Role
We are hiring a Data Engineer with strong hands‑on experience in building
high‑performance data pipelines for a heavy data analytics project. The candidate must beexcellent at writing complex aggregations, understanding business processes and analyticalrequirements, and designing scalable data lake and data warehouse solutions. Experienceacross multiple data platforms (Databricks, Snowflake, Azure Data Factory, Synapse, etc.) isa strong advantage.

Key Responsibilities
1. Data Pipeline & ETL/ELT DevelopmentDevelop, optimize, and productionize Spark (PySpark/Scala) pipelines.
Ingest, transform, cleanse, and aggregate large datasets from varied sources.
Implement scalable ETL/ELT logic for batch and near-real-time pipelines.
Apply best practices in partitioning, caching, Delta Lake optimization, and performancetuning.
2. Heavy Data Analytics & Business UnderstandingWrite complex aggregation logic (window functions, rollups, grouping sets, analytical
functions).
Understand business KPIs, metrics, and analytical use cases.
Translate business needs into technical transformations and data models.
Validate data outputs against business logic and analytics expectations.
Collaborate with analysts on calculations: weekly/monthly aggregates, trend lines,performance metrics, dimensional rollups.
Ensure accuracy, consistency, and traceability of business-critical metrics.
3. Data Lake EngineeringBuild and maintain multi-layer Data Lake architectures (Bronze/Silver/Gold).
Work with Parquet, Delta Lake, ORC, and columnar storage formats.
Implement schema evolution, auditing, and metadata strategies.
4. Data Warehouse EngineeringDesign dimensional models: Star Schema and Snowflake Schema.
Build fact and dimension tables supporting analytics and reporting.
Optimize table structures, keys, and partitioning strategies.
5. Databricks (Added Advantage)Develop notebooks/jobs using PySpark/Scala.
Manage clusters, workflows, and Delta Live Tables.
Implement best practices for performance and cost efficiency.
6. SQL EngineeringStrong command of SQL for aggregations, analytical functions, joins,profiling, andvalidation.
Write and optimize complex queries supporting dashboards, metrics, and reports.
7. Cloud Data PlatformsAzure: Data Factory, Synapse Analytics, ADLS Gen2, Azure Functions (optional).
Snowflake: Virtual Warehouses, Snowpipe, Streams & Tasks, performance tuning.
8. Data Quality & DocumentationValidate transformation logic against business rules.
Document data flows, transformation rules, aggregation logic, and data
dictionary/metadata.
Work with QA and analysts to ensure outputs match business expectations.

Required Qualifications
5+ years of hands‑on data engineering experience.
Strong programming skills: Spark, Scala, Python.
Strong SQL skills (aggregations, analytical functions, large joins).
Experience with Data Lake and Data Warehouse concepts.
Experience with Spark-based processing (delta optimization, shuffle tuning, partitioning).
Experience with at least one cloud data ecosystem (Azure/AWS/GCP).

Preferred Skills
Experience with Databricks (highly desirable).
Experience with Snowflake or modern cloud DWH.
Experience with ADF/Synapse/Airflow/dbt for orchestration.
Knowledge of CI/CD for data pipelines.
Experience with large-scale data analytics environments.

Soft Skills
Strong understanding of business logic behind analytics outputs.
Ability to translate business metrics into technical transformations.
Strong problem-solving and debugging skills.
Good communication and cross-team collaboration.