Note: The job is a remote job and is open to candidates in USA. Microsoft is seeking a highly experienced Experimentation Technical Program Manager to own and evolve a mature, high-velocity web experimentation program for one of the most iconic video game franchises in the world. In this fully remote role, you will drive the full experiment lifecycle and champion a culture of data-driven decision-making, translating experiment results into actionable recommendations that influence the live website experience.
Responsibilities
- Own the end-to-end experimentation program and operating rhythm, guiding teams through the full experiment lifecycle from idea and design through build, launch, analysis, and decision
- Manage the experimentation pipeline across intake, review, development, and analysis stages, ensuring the process remains healthy, efficient, and durable
- Maintain the experimentation backlog and drive stakeholder communication, including cross-functional coordination and status visibility across all relevant teams
- Help product teams build clear hypotheses, success metrics, and learning goals to improve decision quality and reduce downstream churn
- Improve the integrity of analysis artifacts such as experiment scorecards by incorporating statistical power so results can be interpreted with high confidence
- Promote experimentation principles that optimize for trustworthiness, velocity, and democratization across the organization
- Coach product managers, designers, and engineers on experimentation best practices and reduce friction so teams can increasingly self-serve
- Partner with product leads to identify high-leverage experimentation opportunities and embed experimentation into delivery roadmaps
- Build and maintain shared program assets including playbooks, dashboards, and onboarding materials to standardize how experimentation is run
- Publish regular experimentation updates sharing results, in-flight work, recommendations, and key learnings with broad stakeholder groups
- Create and present quarterly recaps to socialize learnings, wins, and measurable impact
- Ensure learnings are documented and reusable over time through repeated insights libraries and institutional memory resources
- Pull, review, and summarize data to help drive the product roadmap based on experimentation outcomes
- Stay attuned to what is happening across web properties to identify potential changes or features worth testing
- Work directly with Product Owners to review winning experiment variants and behaviors in order to get successful changes implemented on the live website
Skills
- 4+ years of hands-on web-based experimentation experience including A/B testing and iterative feature testing on live web platforms
- 4+ years of customer-focused data analysis experience, ideally with a strong foundation in statistics and statistical analysis
- 6+ years of cross-functional technical program management experience supporting web or other online products
- Bachelor's degree in a quantitative field such as Statistics, Mathematics, Computer Science, Data Science, or a related discipline, or equivalent practical experience
- Proven track record managing experimentation programs at scale on a web-based platform
- Experience partnering with product and engineering teams to embed experimentation into delivery roadmaps
- Deep expertise in web-based A/B and multivariate experimentation methodologies and best practices
- Strong analytical and statistical skills including hypothesis testing, statistical significance, confidence intervals, and power analysis
- Proficiency in customer-focused data analysis, including pulling, reviewing, and summarizing data to inform product decisions
- Experience with Adobe tools such as Adobe Experience Manager or similar web content and experimentation platforms
- Solid understanding of general website operations and web product delivery
- Excellent cross-functional program management skills with the ability to coordinate across Product, Engineering, Design, and Analytics teams
- Strong written and verbal communication skills with the ability to present complex findings clearly to both technical and non-technical audiences
Benefits
- Medical, dental & vision
- Hospital plans
- 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
- Life Insurance (Company paid Basic Life and AD&D as well as voluntary Life & AD&D for the employee and dependents)
- Company paid short and long-term disability
- Health & Dependent Care Spending Accounts (HSA & DCFSA)
- Employee Assistance Program
- Time Off/Leave(PTO, Allegis Group Paid Family Leave, Parental Leave
Company Overview