Note: The job is a remote job and is open to candidates in USA. Smartsheet has been empowering teams to manage work seamlessly for over 20 years, and they are seeking a Senior Software Engineer II to architect AI-driven solutions that transform work execution at enterprise scale. The role involves designing and building production-grade AI agents, collaborating with cross-functional teams, and mentoring other engineers in AI development best practices.
Responsibilities
- Design and build production-grade LLM-powered agents and workflows within Smartsheet, including architecture decisions on system design, data pipelines, and deployment strategies
- Develop and optimize prompts, RAG pipelines, and agent reasoning patterns; build evaluation frameworks to measure accuracy, hallucination rates, and performance across model versions
- Implement and manage end-to-end AI systems on cloud infrastructure (AWS, GCP, or Azure), including monitoring, optimization, and incident response
- Collaborate with Engineering, Product, and cross-functional teams to translate business requirements into technical AI solutions that drive real impact
- Mentor and guide other engineers on applied ML best practices, modern AI development patterns, and production considerations
- Establish best practices and standards for AI development, model governance, and responsible AI within the organization
- Stay current with the rapidly evolving AI/LLM landscape and evaluate emerging tools, frameworks, and techniques for Smartsheet's needs
Skills
- 8+ years of software engineering experience, with at least 2 years working directly with LLMs in production
- Deep, hands-on experience with prompt engineering and context engineering, you understand how model behavior changes with framing, structure, and input design
- Strong working knowledge of RAG architectures: chunking strategies, embedding models, retrieval evaluation, and failure diagnosis
- Experience building or extending LLM evaluation frameworks, you have designed scorers, worked with golden datasets, and thought carefully about what good looks like
- Fluency in agent system design, you don't need to own the architecture, but you can engage as a peer on architectural tradeoffs that affect quality
- Strong Python skills; comfortable working in data-heavy environments (Databricks, Delta tables, or equivalent)
- Ability to communicate complex quality findings (written and verbal) to both technical and non-technical stakeholders, you can explain what's broke, why it matters, and what needs to happen next without losing the room
- Strong cross-functional judgment, you know when to escalate, when to resolve independently, and how to build credibility across engineering, product, and AI platform teams
- A bias for clarity in ambiguous situations, when failure modes are murky and trade-offs are real, you bring structure and a clear point of view rather than waiting for consensus
- Legally eligible to work in the U.S. on an ongoing basis
- BS or MS in Computer Science, a related field, or equivalent industry experience
- Experience with MLflow or similar experiment tracking platforms
- Familiarity with CI-integrated evaluation pipelines
- Experience with multi-agent orchestration frameworks
- Prior work in an Applied AI or LLMOps function within a product company
Benefits
- Employer subsidized medical/vision and dental coverage for full-time employees
- 401k Match to help you save for your future (50% of your contribution up to the first 6% of your eligible pay)
- Monthly stipend to support your work and productivity
- Flexible Time Away Program, plus Sick Time Off
- US employees are automatically covered under Smartsheet-sponsored life insurance, short-term, and long-term disability plans
- US employees receive 12 paid holidays per year
- Up to 24 weeks of Parental Leave
- Personal paid Volunteer Day to support our community
- Opportunities for professional growth and development including access to Udemy online courses
- Company Funded Perks, including a counseling membership, local retail discounts, and your own personal Smartsheet account
- Teleworking options from any registered location in the U.S. (role specific)
Company Overview