QA Architect - R01571456
Brillio · IT Services & Consulting
- Bangalore, Karnataka, India
- On-site
- Posted today
- Data & AI
- Employee
About the job
QA Architect
Job requirements
Experience Range: With at least 7 years of quality assurance experience, including substantial hands-on work with data science and machine learning testing frameworks Key Responsibilities:
Design and implement automated testing strategies for AI and data science outputs, ensuring accuracy and reliability across models and pipelines
Develop and maintain robust evaluation and validation frameworks for backend and frontend components, leveraging statistical and machine learning techniques
Collaborate with data scientists and engineers to define test cases, hypotheses, and statistical metrics for model assessment and improvement
Integrate advanced statistical tests such as T-Test, Z-Test, and regression analyses into automated QA workflows to validate model performance
Utilize tools like Great Expectations, Evidently AI, and specific machine learning frameworks (TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, MXNet) to monitor, track, and report on model drift, anomalies, and forecast accuracy
Optimize testing processes for scalability and efficiency using Python, PySpark, R, and related technologies in large-scale data environments
Configure and manage testing infrastructure using platforms such as KubeFlow and BentoML to streamline deployment and evaluation cycles
Troubleshoot and resolve issues in automated testing pipelines, driving continuous improvement and high-quality deliverables
Required Skills:
Advanced proficiency in Python and PySpark for test automation and statistical analysis
Expertise in statistical testing methods including Hypothesis Testing, T-Test, Z-Test, and Regression (Linear, Logistic)
Strong experience with machine learning frameworks such as TensorFlow, PyTorch, Sci-Kit Learn, CNTK, Keras, and MXNet
Hands-on knowledge of Great Expectations and Evidently AI for data validation and monitoring
Proficiency in SAS and SPSS for statistical computing and analysis
Deep understanding of probabilistic graph models and classification algorithms including Decision Trees and SVM
Experience with forecasting techniques including Exponential Smoothing, ARIMA, and ARIMAX
Familiarity with distance metrics such as Hamming, Euclidean, and Manhattan Distance
Advanced skills in R and R Studio for statistical modeling and QA scripting
Experience configuring testing platforms such as KubeFlow and BentoML
Preferred Skills:
Experience automating evaluation pipelines for AI/ML in production environments
Expertise in integrating QA processes with CI/CD workflows and cloud-native architectures
Knowledge of emerging ML testing tools and frameworks beyond industry standards
Ability to develop custom statistical metrics for model evaluation
Experience with QA automation for distributed systems at scale
Desired Qualifications:
Bachelor's degree in Computer Science, Data Science, Statistics, Mathematics, or a quantitative discipline
Certification in Quality Assurance, Data Science, or Machine Learning (e.g., ISTQB Advanced Test Analyst, TensorFlow Developer Certificate)
Certification in statistical analysis tools or platforms (e.g., SAS Certified Specialist, SPSS Certification)