As we accelerate the development of autonomous AI agents and expand across Web, Mobile, API/Backend, and AI/LLM surfaces, product complexity and release frequency are scaling rapidly. To maintain high quality standards without slowing development velocity, we are moving beyond traditional QA toward a modern, engineering-driven quality architecture.
Addressing non-deterministic AI/LLM outputs and microservice consistency requires embedding quality early in the development lifecycle (“Shift-Left”) and building automated testing pipelines. We are seeking a Senior QA Engineer (SET) to drive our test automation architecture, establish quality gates, and foster a culture of quality ownership across our global engineering organization.
Technical Environment
- ENS Development Common Stack
- Programming Languages: C#, Python, TypeScript
- Backend Frameworks: ASP.NET Core, FastAPI
- Frontend Framework: Angular
- Infrastructure: Microsoft Azure
- Databases: SQL Server, Cosmos DB
- Communication: Slack, Google Meet, Jira
- Documentation: Azure Wiki, Confluence
- AI Tools: Claude Code, Devin, Gemini
- QA Specific Tools
- Web E2E Automation: Playwright, Cypress
- API Testing: Postman, REST-assured, or code-based API testing tools
- CI/CD: Azure Pipelines
- Primary Development Languages: TypeScript, C#
Highlights of the Role
- Global Cross-Cultural Team: Work primarily in English alongside cross-cultural engineering teams based in Tokyo and EMEA.
- Architectural Impact: Take full ownership of shaping our test automation foundation, CI/CD integration, and quality governance rather than simply executing test cases.
- Cutting-Edge AI/LLM Testing: Gain rare hands-on experience defining evaluation frameworks and automated regression testing for non-deterministic generative AI features.
Responsibilities
- Lead our quality engineering strategy across Web, Mobile, API/Backend, and AI/LLM features—from architecture design of test automation frameworks to CI/CD pipeline integration.
- Cross-Platform Test Strategy: Define test plans across Web (Frontend), Mobile (iOS/Android), API/Backend microservices, and AI/ML features while leading “Shift-Left” practices from the design phase to identify defects early.
- Test Automation Architecture: Architect, build, and maintain scalable test automation frameworks (Playwright, Cypress, Appium, REST, etc.), optimizing execution speed and eliminating test flakiness.
- CI/CD Quality Gates: Integrate automated test suites into CI/CD pipelines (Azure Pipelines, GitHub Actions, CircleCI, etc.) to automate release verification and support high-frequency deployments.
- AI/LLM Quality Evaluation: Establish methodologies and automated metrics for evaluating LLM outputs, including regression testing for prompt updates, edge-case validation, and safety assessments.
- Exploratory Testing & Triage: Perform advanced exploratory testing to uncover subtle edge cases, drive root-cause analysis, and streamline defect prioritization with engineering teams.
- Technical Leadership: Track and visualize quality metrics (coverage, defect rates, escaped bugs) and partner with engineering leadership to embed quality standards across teams.
Requirements
- 5+ years of software testing experience, including 3+ years architecting test automation frameworks.
- Strong hands-on experience with Web E2E automation using Playwright or Cypress.
- Experience with API testing (Postman, REST, or code-based API testing tools).
- Proven track record of running automated test suites within CI/CD pipelines (GitHub Actions, CircleCI, Jenkins, Azure Pipelines, etc.).
- Business-level English proficiency (primary language for daily technical operations).
- Conversational Japanese proficiency (JLPT N3 equivalent or above).
Nice to haves
While not specifically required, tell us if you have any of the following.
- Experience testing AI/ML or LLM-powered features (including non-deterministic output evaluation).
- Experience with performance, load, or security testing.
- Background in B2B SaaS, multi-sided platforms, or marketplace products.
- Software testing certifications (e.g., ISTQB).
- Practical experience with mobile test automation using Appium (iOS and Android).
- Strong programming proficiency in TypeScript or C#.
- Experience collaborating or negotiating with external stakeholders or vendors.
- Team leadership, management, or vendor management experience.
Compensation
¥7,000,000 ~ ¥12,000,000 annually.