Quality Engineering Lead – Agent Evaluation & AI Platforms

Make your mark at Comcast -- a Fortune 30 global media and technology company. From the connectivity and platforms we provide, to the content and experiences we create, we reach hundreds of millions of customers, viewers, and guests worldwide. Become part of our award-winning technology team that turns big ideas into cutting-edge products, platforms, and solutions that our customers love. We create space to innovate, and we recognize, reward, and invest in your ideas, while ensuring you can proudly bring your authentic self to the workplace. Join us. You’ll do the best work of your career right here at Comcast. (In most cases, Comcast prefers to have employees on-site collaborating unless the team has been designated as virtual due to the nature of their work. If a position is listed with both office locations and virtual offerings, Comcast may be willing to consider candidates who live greater than 100 miles from the office for the remote option.)<p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:left"><b>Job Summary</b></p>We are seeking seasoned Quality Lead (SDET 4) to orchestrate our enterprise release cycles and lead the quality strategy for our next-generation AI initiatives. This role is designed for a strategic technical leader who combines deep Comcast domain expertise across all 4 Lines of Business (LOBs) and Device Provisioning with specialized experience in Agentic AI validations.<br><br>You will lead the quality process from the front: running scrums, managing the end-to-end defect lifecycle, conducting structural code reviews, and serving as the principal architect for our Agent Evaluation strategies within Google ADK environments.<br><br>The AI Agentic Team designs, builds, and runs intelligent agents end to end. We own agent logic, workflows, integrations, production deployment, and ongoing performance. By combining build and operations, we deliver agentic systems that are reliable, secure, and scalable in real-world use.<p style="text-align:inherit"></p><p style="text-align:left"><b>Job Description</b></p><p><b>High-Impact Leadership & Process Ownership</b></p><ul><li><p><b>Agile Governance:</b> Run daily Scrums and stand-ups, optimizing team velocity, managing task assignments, and ensuring delivery milestones are met on time.</p></li><li><p><b>Defect Triaging:</b> Lead rigorous defect triaging meetings. You will be the primary driver tracking high-priority issues, coordinating directly with development teams, and pushing defects to rapid closure.</p></li><li><p><b>Requirement & Architecture Inception:</b> Embed yourself in upstream requirement meetings. You will challenge assumptions, identify missing edge cases, and establish the technical "Definition of Done."</p></li><li><p><b>Business & UAT Orchestration:</b> Act as the strategic partner to Business teams, leading <b>User Acceptance Testing (UAT)</b> and translating complex technical deployments into business-ready validation.</p></li></ul><p></p><p><b>AI Governance & Agent Evaluation Strategy</b></p><ul><li><p><b>Agentic AI Expertise:</b> Bring at least <b>2+ years of hands-on experience</b> in Agentic AI validations, navigating the non-deterministic nature of large language models and intelligent workflows.</p></li><li><p><b>Evaluation Frameworks:</b> Drive the strategy for Agent Evaluation techniques, ensuring the team effectively implements automated verification patterns (such as LLM-as-a-judge) within Google ADK environments.</p></li><li><p><b>Prompt Engineering Oversight:</b> Apply deep knowledge of Prompt Engineering to validate agent behavior, test fallback mechanisms, and ensure intent optimization.</p></li><li><p><b>Quality Governance & Code Reviews:</b> Conduct technical Code Reviews for the team's automation repositories. Ensure that Python or TypeScript test scripts targeting the AI layers are modular, efficient, and scale effectively.</p></li></ul><p></p><p><b>Comcast LOB, Provisioning & Quality Governance</b></p><ul><li><p><b>Full LOB Coverage:</b> Apply deep operational knowledge across all <b>4 Comcast LOBs</b> (Xfinity Video, High-Speed Data, Digital Voice, and Xfinity Mobile) to ensure cross-functional workflows are flawlessly validated.</p></li><li><p><b>Device Provisioning Master:</b> Leverage an intricate understanding of backend Device Provisioning pipelines, activation flows, and equipment states to isolate complex environments and system-level issues.</p></li><li><p><b>Automation Oversight & Code Reviews:</b> Maintain strong technical oversight of the team's codebase. You will possess a solid understanding of test automation methodologies and basic knowledge of testing frameworks to conduct structural <b>Code Reviews</b>, ensuring the automation scripts meet quality, maintainability, and scalability standards.</p></li><li><p><b>E2E Strategy & Mentorship:</b> Guide junior engineers in designing comprehensive Test Plans and Strategies. You will define <i>what</i> to test and <i>how</i> to test it, ensuring total coverage of complex subscriber journeys.</p></li></ul><p></p><p><b>Execution, Observability & Reporting</b></p><ul><li><p><b>Technical Troubleshooting:</b> Navigate Kibana logs and system metrics independently to perform deep-dive root cause analysis on failures before handing them off to development.</p></li><li><p><b>Custom JIRA Architecture:</b> Build and manage advanced JIRA dashboards to visualize real-time quality trends, escape-defect rates, and overall sprint health for leadership.</p></li><li><p><b>Executive Reporting:</b> <span>Prepare and present Daily and Weekly Status Reports to stakeholders, providing absolute transparency on risks and release readiness</span></p></li></ul><p></p><p><b>Required Experience & Skill Set</b></p><ul><li><p><b>Leadership Track:</b> 10+ years of experience in Software Quality Engineering, with a seasoned background in managing large-scale delivery and leading functional testing strategies.</p></li><li><p><b>AI Specialization:</b> 2+ years of specific experience with <b>Agentic AI validations</b>, <b>Prompt Engineering</b>, and <b>Agent Eval techniques</b> (specifically with <b>Google ADK or similar models)</b>.</p></li><li><p><b>Comcast Domain Mastery:</b> Deep, demonstrable knowledge of <b>all 4 Comcast LOBs</b> and their underlying technical architectures. (asIS)</p></li><li><p><b>Provisioning Infrastructure:</b> Comprehensive understanding of the functional lifecycle for device provisioning, including complex billing system (ACSR and AMDOCS) handoffs and activation workflows.</p></li><li><p><b>Automation & Framework Literacy:</b> Strong functional understanding of automation frameworks (e.g., Python, TypeScript), providing the technical insight necessary to oversee code quality and ensure automated suites align with business requirements.</p></li><li><p><b>Process Tooling:</b> Power-user proficiency in JIRA for advanced dashboarding and Kibana for deep-dive log analytics to drive data-driven quality decisions.</p></li></ul><p><br><b>Disclaimer:</b> This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.</p><p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:left"><b>Skills</b></p>Adaptability, Agentic AI, AI Agent Orchestration, AI Platforms, Critical Thinking, Test Automation<p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:left"><b>Compensation</b></p>Primary Location Pay Range: $135,683.35 - $203,525.03<p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:inherit"></p>Comcast intends to offer the selected candidate base pay within this range, dependent on job-related, non-discriminatory factors such as experience. The application window is 30 days from the date job is posted, unless the number of applicants requires it to close sooner or later.<p style="text-align:inherit"></p><p style="text-align:inherit"></p><p>Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work.    Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus.  Additionally, Comcast provides best-in-class Benefits to eligible employees.  We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most.  That’s why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality – to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the <a href="https://jobs.comcast.com/benefits" target="_blank" rel="noopener noreferrer">compensation and benefits summary</a> on our careers site for more details.</p><p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:inherit"></p><p style="text-align:left"><b>Education</b></p>Bachelor's Degree<p style="text-align:inherit"></p><p style="text-align:inherit"></p>While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.<p style="text-align:inherit"></p><p style="text-align:left"><b>Certifications</b> (if applicable)</p><p></p><p><b>Relevant Work Experience</b></p>7-10 Years<p style="text-align:inherit"></p><p style="text-align:inherit"></p>Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.

Back to blog

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...