Senior Data Engineer

Ideas2IT Technologies

Chennai

Hybrid

INR 1,000,000 - 1,800,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Ideas2IT Technologies in Chennai is seeking a Data Engineer to design and build robust data pipelines on AWS or Azure, and optimize ETL for large-scale data. You will work with PySpark, Pandas, and cloud data services, write Python or Scala, and collaborate with analytics teams to translate data needs into scalable infrastructure.

The role emphasizes architecture input, hands-on PoCs, and participation in hackathons, with a culture that values capability over tenure.

Qualifications

  • Strong SQL and experience with MySQL, PostgreSQL, or SQL Server.
  • Hands-on with cloud data warehouses: Redshift, Snowflake, Azure Synapse.
  • Experience in AWS or Azure and building large-scale data pipelines.
  • Proficiency with PySpark, Pandas, Polars; Python or Scala fluency.
  • Ability to handle large, messy data from multiple sources.

Responsibilities

  • Design and build robust data pipelines and architectures on AWS or Azure.
  • Develop and optimize ETL processes handling large-scale data.
  • Work with distributed processing using PySpark and Pandas.
  • Use cloud services like Glue, Lakeformation, EMR, Synapse, Datalake, Fabric.
  • Model and warehouse data to feed BI tools for analytics.
  • Write clean Python or Scala to automate and integrate.
  • Collaborate with analytics and business teams to translate data needs.
  • Contribute to tech stack decisions.

Skills

SQL mastery
RDBMS
Cloud data warehouses
Cloud platforms
ETL pipelines
PySpark
Pandas/Polars
Python/Scala
Data wrangling
Cloud data services

Tools

Docker
Kubernetes

Job description

What you'll actually do as a Data Engineer with us.

  • Design and build robust data pipelines and architectures on AWS or Azure
  • Develop and optimize ETL processes that handle large-scale structured and unstructured data from multiple sources
  • Work with distributed processing tools like PySpark and Pandas to move and transform data at scale
  • Use cloud services hands-on: Glue, Lakeformation, EMR for AWS Datafactory, Synapse, Datalake, Fabric on Azure
  • Model and warehouse data that feeds BI tools and supports real analytical decisions
  • Write clean Python or Scala to automate, integrate, and build
  • Collaborate with analytics and business teams to translate data needs into infrastructure that works
  • Contribute to tech stack decisions: your opinion on architecture matters here, not just your execution.
Must-haves:
  • Strong SQL and hands-on experience with relational databases: MySQL, PostgreSQL, or SQL Server
  • Strong hands-on experience with Cloud data warehouses: AWS Redshift, Snowflake, Azure Synapse Analytics
  • Experience in at least one cloud environment: AWS or Azure
  • Proven experience building large-scale data pipelines and ETL processes
  • Proficiency with distributed data processing: PySpark, Pandas, Polars
  • Fluency in Python and/or Scala
  • Experience with cloud data services (Glue, Lakeformation, EMR for AWS Datafactory, Synapse, Datalake, Fabric on Azure)
  • Ability to handle messy, large-scale data from internal and third-party sources and make it usable
Nice-to-haves:
  • Experience with data visualization tools: Power BI or Tableau
  • Familiarity with orchestration and workflow management tools
  • Familiarity with Data quality, testing and observability tools
  • Familiarity with Docker and Kubernetes

What that looks like day-to-day:
  • Autonomy to contribute to architecture and tech stack choices, not just implement them
  • Access to technical PoCs if there's a technology you want to explore
  • Internal Shark Tanks and Hackathons if you want to take an idea further
  • A culture that weights capability over tenure
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Questhiring • Gurugram District

On-site
INR 1,400,000 - 2,400,000
Senior Data Engineer
Senior Data Engineer

Backbase • Hyderabad

On-site
INR 4,000,000 - 7,000,000
Senior Data Engineer
Senior Data Engineer

Lifesight • Bengaluru

On-site
INR 2,800,000 - 4,200,000
Senior Data Engineer
Senior Data Engineer

AppSierra • India

On-site
INR 3,500,000 - 5,200,000
Data Engineer
Data Engineer

PW (PhysicsWallah) • Dadri

On-site
INR 900,000 - 1,300,000
Data Engineer
Data Engineer

DataPhi • Pune District

On-site
INR 1,200,000 - 1,800,000
Senior Data Engineer
Senior Data Engineer

Workatbackbase • Hyderabad

On-site
INR 900,000 - 1,400,000
Lead Data Engineer
Lead Data Engineer

Sahaj Software • Pune District

On-site
INR 2,500,000 - 4,000,000
Data Engineer / Data Analyst
Data Engineer / Data Analyst

Remote Software Solutions Pvt. Ltd. • India

On-site
INR 700,000 - 1,100,000
Data Engineer
Data Engineer

Valuenode Private Limited • India

Remote
INR 1,200,000 - 1,800,000