Title: Test Analyst - ETL QA Engineer
Location: Remote (Baltimore, MD)
Duration: Contract
Mandatory Skills: Pytest Framework exp
(build Pytest framework from the scratch)
Need current project experience with Pytest.
We are seeking a skilled and detail-oriented Databricks ETL QA Engineer to join our data engineering team. The ideal candidate will have a strong background in QA processes, data validation, and testing of ETL pipelines, with specific experience in Databricks and big data platforms.
Key Responsibilities:
• Collaborate with Data Engineers and Analysts to understand business requirements and ETL processes.
• Design, develop, and execute comprehensive test cases, test plans, and test scripts for Databricks ETL pipelines.
• Perform data validation, data quality checks, and ensure data integrity across multiple systems.
• Verify unity catalog implementation and build data governance scenarios and test them
• Automate testing processes for large datasets using tools like PySpark, SQL, and Databricks notebooks.
• Work on performance testing and optimization of ETL jobs.
• Perform API testing by identifying the endpoints, request methods, and expected responses and build test scenarios and test data to cover all possible scenarios
• Identify, document, and track defects and drive them to resolution.
• Contribute to continuous improvement of QA processes and best practices.
• Ensure compliance with data governance and security policies.
• Build and maintain the end to end test management suite by leveraging the right test management tools
Required Skills:
• Strong experience in ETL testing, data validation, and QA methodologies.
• Hands-on experience with Databricks, Spark (PySpark/Scala), and big data platforms.
• Proficiency in SQL for data analysis and validation.
• Familiarity with Delta Lake, Parquet, and other data formats.
• Experience with API testing and automation
• Experience with test automation frameworks and scripting.
• Strong understanding of data warehousing concepts and ETL workflows.
• Experience with cloud platforms like AWS
• Familiarity with version control tools (e.g., Git).
Preferred Skills:
• Knowledge of CI/CD pipelines and DevOps practices.
• Experience with Python or other scripting languages.
• Familiarity with BI tools (e.g., Power BI, Tableau) for data validation reporting.
• Experience with Agile/Scrum methodologies.
Qualifications:
• Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.
• 10+ years of relevant experience in ETL QA/testing roles.
Job description:
· Pytest Framework – Hands-on knowledge is Mandatory
· Python Programming
· Unit Testing
· Integration Testing
· Test Automation
· Selenium (if applicable)
· API Testing
· Data-Driven Testing
· Behavior-Driven Development (BDD)
· Continuous Integration/Continuous Deployment (CI/CD)
· Test Case Design
· Test Execution
· Debugging and Troubleshooting
· Code Coverage Analysis
· Mocking and Fixtures in Pytest
Job Type: Contract
Pay: $36.71 - $44.21 per hour
Work Location: Remote
Apply Now
Apply Now