Job Description
Job Summary:
We are seeking a highly experienced Senior Data Engineer with 12+ years of hands-on experience in data engineering and data pipeline development. The ideal candidate will have strong expertise in Databricks, DBT, AWS, and Apache Spark, and a proven track record of working with modern data platforms in large-scale enterprise environments.
This role requires a professional who is adept at building scalable data pipelines, transforming data using DBT, working extensively with AWS cloud infrastructure, and developing in a Spark/Databricks environment. Strong SQL query writing skills are essential.
- Mandatory Skills & Experience:
- 10+ years of experience in data engineering or relevant fields.
- Expert-level knowledge of Databricks – including notebooks, jobs, Delta Lake, and workspace management.
- Strong hands-on experience with DBT (Data Build Tool) – including writing models, tests, and documentation.
- Proficiency in AWS services – including S3, Glue, Redshift, Lambda, IAM, etc.
- Strong experience with Apache Spark – both batch and streaming data processing.
- Strong experience writing efficient SQL queries for data transformation and analysis.
- Proven ability to work independently in a remote environment.
- Solid understanding of data modeling, ETL/ELT processes, and modern data architecture.
- Experience in version control (Git), CI/CD pipelines, and Agile methodologies.
- Preferred Qualifications:
- AWS Certification (e.g., AWS Data Analytics, AWS Solutions Architect)
- Experience with orchestration tools like Airflow or AWS Step Functions.
- Familiarity with DevOps practices in a data environment.
- Exposure to data quality frameworks and testing strategies.
- Responsibilities:
- Design, build, and optimize robust and scalable data pipelines using Databricks and DBT on AWS.
- Develop, deploy, and maintain ELT/ETL processes to transform raw data into actionable insights.
- Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver data solutions.
- Ensure data integrity, quality, and governance across all data flows.
- Continuously monitor and improve performance, reliability, and cost-efficiency of data pipelines.
- Write and maintain high-quality documentation.
Apply Now
Apply Now