Lead I - Data Engineering

. Design, develop, test, and deploy scalable Big Data solutions on AWS Cloud. 2. Build and maintain batch and streaming data pipelines using Scala, Python, Spark, and PySpark. 3. Analyze, process, and transform large volumes of structured and unstructured data. 4. Develop and support distributed data processing frameworks and data ingestion platforms. 5. Integrate new data sources and tools into existing big data ecosystems. 6. Optimize performance of Spark, Hadoop, EMR, and distributed computing workloads. 7. Design and implement cloud-native APIs and microservices supporting data products and operational platforms. 8. Collaborate with product owners, architects, QA teams, and business stakeholders throughout the software development lifecycle. 9. Automate data workflows, deployments, monitoring, and operational processes. 10. Evaluate emerging Big Data, AI, and cloud technologies and prototype innovative solutions. 11. Ensure adherence to security, governance, reliability, and software engineering best practices. Required Qualifications 1. Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or equivalent experience. 2. 5-8 years of experience in software engineering and Big Data application development. 3. Strong hands-on experience with: Scala Python Apache Spark / PySpark Hadoop ecosystem (HDFS, MapReduce, Hive) Kafka or similar event-streaming technologies 4. Strong AWS experience with services such as: EMR S3 EC2 ECS/EKS Airflow (MWAA preferred) Step Functions API Gateway Lambda DynamoDB and/or Aurora/RDS 5. Experience building and supporting large-scale batch and API-based data processing solutions in AWS. 6. Strong Linux and shell scripting skills. 7. Experience with data ingestion, transformation, data modeling, schema design, and data lifecycle management. 8. Strong understanding of performance tuning for Spark, Hadoop, EMR, and distributed systems. 9.. Experience with Agile/Scrum development methodologies. 10. Understanding of secure software development practices and cloud security fundamentals. 11. Strong analytical, troubleshooting, and problem-solving skills. 12. Excellent written and verbal communication skills. Preferred Qualifications 1. Experience with Generative AI, AI-assisted development tools (GitHub Copilot, Amazon Q, Cursor, etc.), and AI-driven automation. 2. Understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Vector Databases, and AI/ML solution integration. 3. Experience building AI-enabled data platforms and data pipelines supporting ML workloads. 4. Experience with DevOps and CI/CD tools such as Git, GitHub, GitLab, Bitbucket, Jenkins, Maven, Gradle, and Artifactory. 5. Experience with containerization technologies such as Docker and Kubernetes. 6. Knowledge of automated testing frameworks and data quality validation tools. 7. Experience working in enterprise-scale cloud environments. 8. AWS certification(s) and/or Databricks certification(s) are a plus.