Associate III - Data Engineering

Job Description: Senior Software Engineer - Databricks Role: Senior Software Engineer - Databricks / Data Engineering Experience: 5+ years overall, with relevant data engineering experience Role Summary We are looking for a hands-on Senior Software Engineer with strong expertise in Apache Spark, Databricks, SQL, and Python. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data-processing solutions and should have excellent analytical and troubleshooting abilities. Key Responsibilities • Develop scalable data pipelines and processing solutions using Databricks and Apache Spark. • Build data transformations and processing logic using PySpark/Python and SQL. • Write complex and optimized SQL queries for data processing and analysis. • Develop and maintain ETL/ELT pipelines. • Analyze large datasets and troubleshoot data-quality or processing issues. • Optimize Spark workloads and improve pipeline performance. • Perform unit testing, debugging, and production issue resolution. • Participate in technical design discussions and code reviews. • Work closely with technical leads, architects, analysts, and other engineering teams. • Follow engineering standards for maintainable, reusable, and production-ready code. Mandatory Skills • Apache Spark / PySpark: Mandatory • Databricks: Strong hands-on experience • SQL: Strong, with expertise in complex queries and data analysis • Python: Strong programming skills • Strong knowledge of ETL/ELT and data-engineering concepts • Strong analytical and problem-solving skills • Ability to troubleshoot data and application issues independently • Experience processing large datasets using distributed computing frameworks Good to Have • Delta Lake • Azure/AWS cloud platforms • Databricks workflows/job orchestration • Git and CI/CD • Data modeling and warehousing concepts • Agile/Scrum delivery experience