Healthcare Specialist, Rubric Design

SME Careers, a fast-growing AI data services company and subsidiary of SuperAnnotate that delivers training data to many of the world's largest AI companies and foundation-model labs, is hiring Healthcare Subject Matter Experts for hourly, remote contract work at $100/hr. Open to applicants in the United States.

You will use your clinical or professional expertise to write challenging, expert-level healthcare questions, evaluate AI-generated clinical content, and design rubrics that define what an accurate, complete and professionally appropriate answer should contain. That includes spotting where frontier AI models give incomplete, inaccurate, poorly reasoned, overconfident or clinically inadequate answers; defining essential clinical concepts, reasoning steps, caveats, acceptable alternatives, significant omissions and critical errors; critiquing rubrics written by other healthcare professionals; and producing reference answers, critiques and gold-standard content for RL and SFT workflows.

Requirements: a professional degree, license, credential or recognized qualification in medicine, dentistry, pharmacy, nursing or another healthcare discipline; minimum C1 English; 3+ years of professional or clinical experience (senior and specialist-level professionals strongly preferred); demonstrated expertise in a clearly defined specialty or subspecialty; the ability to write challenging clinical questions and detailed evaluation rubrics, and to tell minor omissions from errors that materially affect clinical quality or safety; comfort reviewing other professionals' rubrics; reliable, self-directed, consistent written work. Strongly preferred: experience with medical education, examination writing, competency assessment, clinical guidelines, teaching, peer review or quality assurance. Preferred: prior AI evaluation, RLHF, SFT, data annotation, model benchmarking or rubric-based scoring experience.

Details and apply link: https://aigigscout.com/jobs/sannotate/healthcare-specialist-rubric-design-healthrubric-26-511?utm_source=jobboardsearch&utm_medium=jobboard&utm_campaign=aigigscout_growth

Disclosure: posted by AI Gig Scout, an independent listing site (not SME Careers). The apply link may be a referral link.