Azure Data Engineer (Senior)
Tenarai (formerly Infogain) · IT Services & Consulting
- Noida / Pune / Bangalore / Mumbai / Hyderabad / Chennai / Gurugram / Kochi, India
- On-site
- Posted today
- Data & AI
- Full-Time
About the job
<h4><strong><span style="color:rgb(0, 0, 0);">Job Description:</span></strong></h4><p><span style="color:rgba(0, 0, 0, 0.87);">We are looking for experienced </span><strong><span style="color:rgba(0, 0, 0, 0.87);">Databricks Data Engineers</span></strong><span style="color:rgba(0, 0, 0, 0.87);"> to support the migration of SAP BW/EDW workloads to the Databricks Lakehouse platform.</span></p><h3><strong><span style="color:rgba(0, 0, 0, 0.87);">Key Responsibilities</span></strong></h3><ul><li><p><span style="color:rgba(0, 0, 0, 0.87);">Develop data pipelines and transformation logic using </span><strong><span style="color:rgba(0, 0, 0, 0.87);">Databricks and SQL</span></strong><span style="color:rgba(0, 0, 0, 0.87);">.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Build and maintain </span><strong><span style="color:rgba(0, 0, 0, 0.87);">bronze, silver and gold</span></strong><span style="color:rgba(0, 0, 0, 0.87);"> layer datasets.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Re-engineer existing SAP BW/EDW transformation logic into Databricks.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Implement source-to-target mappings and data transformation rules.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Work with SAP BW consultants to understand BW objects, transformations and business logic.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Develop scalable and reusable data engineering frameworks.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Support data validation, reconciliation, SIT and UAT.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Troubleshoot data quality and pipeline performance issues.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Work with Azure Data Factory and other Azure services as required.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Support production deployment and operational activities.</span></p></li></ul><h3><strong><span style="color:rgba(0, 0, 0, 0.87);">Required Skills</span></strong></h3><ul><li><p><span style="color:rgba(0, 0, 0, 0.87);">Strong </span><strong><span style="color:rgba(0, 0, 0, 0.87);">Databricks</span></strong><span style="color:rgba(0, 0, 0, 0.87);"> experience.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Strong </span><strong><span style="color:rgba(0, 0, 0, 0.87);">SQL</span></strong><span style="color:rgba(0, 0, 0, 0.87);"> skills.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Hands-on experience with </span><strong><span style="color:rgba(0, 0, 0, 0.87);">PySpark</span></strong><span style="color:rgba(0, 0, 0, 0.87);">.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Experience developing ETL/ELT data pipelines.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Good understanding of Lakehouse and </span><strong><span style="color:rgba(0, 0, 0, 0.87);">Medallion Architecture</span></strong><span style="color:rgba(0, 0, 0, 0.87);">.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Experience with data modelling and transformation.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Experience with data quality, reconciliation and performance optimisation.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Good understanding of Azure data services.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Strong communication and collaboration skills.</span></p></li></ul><h3><strong><span style="color:rgba(0, 0, 0, 0.87);">Good to Have</span></strong></h3><ul><li><p><span style="color:rgba(0, 0, 0, 0.87);">SAP BW / BW/4HANA knowledge.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">SAP S/4HANA / CDS Views.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Azure Data Factory / Synapse.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Power BI.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Unity Catalog.</span></p></li><li><p><span style="color:rgba(0, 0, 0, 0.87);">Experience migrating EDW/BW workloads to Databricks.</span></p></li></ul><p><strong><span style="color:rgba(0, 0, 0, 0.87);">Key Profile:</span></strong><span style="color:rgba(0, 0, 0, 0.87);"><br></span><strong><span style="color:rgba(0, 0, 0, 0.87);">Databricks + SQL + PySpark + ETL/ELT + Lakehouse + Data Migration</span></strong></p>