Senior Psychometrician

POSITION OVERVIEW NBME's Psychometrics and Data Analysis (PADA) unit is seeking an experienced Senior Psychometrician to join its team. We are particularly interested in candidates with experience or a demonstrated interest in using artificial intelligence and emerging analytical approaches to support operational assessment processes. Candidates should submit a cover letter with their application describing their interest in the position and how their experience aligns with the role. The Senior Psychometrician will apply advanced measurement expertise to support NBME's operational assessment programs, research initiatives, and continued advancement of psychometric practices. This position will contribute across six primary areas: Lead and contribute to applied research focused on enhancing examination design, psychometric methodologies, and analytical procedures. Plan, conduct, and evaluate psychometric analyses supporting operational examination programs, including scoring, equating, scaling, and score reporting activities. Apply psychometric and measurement expertise to develop and evaluate responsible uses of artificial intelligence, machine learning, and emerging analytical methodologies in operational assessment processes. Advance the educational measurement field through professional presentations, publications, and research aligned with both individual and organizational research priorities. Assess current psychometric processes and methodologies to identify opportunities, including appropriate uses of AI and automation, to improve the accuracy, efficiency, and effectiveness of examination processing. Develop and evaluate data visualizations and graphical displays that clearly communicate assessment information through stakeholder reports, examinee feedback, and interactive data tools. The Senior Psychometrician will work collaboratively across NBME with professionals in areas such as data analysis, test development, information technology, and program management. The position also involves external communication and collaboration with clients and subject matter experts representing a broad range of medical and health care disciplines. This role has been designated as primarily remote, with occasional onsite attendance required for team collaboration, meetings, or other business needs. We are prioritizing applicants within a 50-mile radius of our Philadelphia office, while also considering qualified candidates in other states where NBME is authorized to employ staff. Please note that onsite interviews and onboarding at our Philadelphia office may be required for this role. If applicable, advance notice will be provided to support planning. At NBME, we continue to innovate and improve how we fulfill the evolving needs of the health care community. This commitment starts and ends with the people at NBME. By recruiting and empowering talented individuals from various disciplines and backgrounds, which includes professionals with diverse life experiences, abilities, and perspectives, NBME can take a well-informed, robust approach to advancing medical education and assessment for years to come. QUALIFICATIONS Skills and Abilities Strong foundation in psychometric theory and its application to operational assessment, including expertise in item response theory (IRT), classical test theory (CTT), statistical methods, test development practices, and research design. Advanced knowledge of measurement methodologies such as equating, scaling, generalizability theory, Bayesian methods, and/or standard setting. Demonstrated ability to translate complex assessment results into clear and meaningful graphical displays that support appropriate interpretation, decision-making, and action. Knowledge of or experience with natural language processing, process-data analytics, formative assessment methodologies, and related emerging approaches. Strong technical and computational skills, including experience using statistical and psychometric software and programming tools such as calibration software, SAS, Python, and/or R. Excellent written and verbal communication skills, with the ability to explain technical measurement concepts to audiences with varying levels of psychometric expertise. Strong analytical and problem-solving skills, including the ability to identify data, methodological, and operational issues and develop appropriate solutions. Excellent interpersonal and collaboration skills, with demonstrated success working with colleagues and stakeholders from diverse professional, technical, and educational backgrounds. Experience A minimum of 5 years of professional experience in a Psychometrician, Measurement Scientist, or comparable applied measurement role is required for consideration. Experience supporting operational testing or assessment programs is strongly preferred. Education A doctoral degree in Educational Measurement, Psychological Measurement, Statistics, or a closely related quantitative field is required. ABOUT NBME NBME is a not-for-profit organization that specializes in the creation of assessments and learning tools for physicians and health professionals. Our mission is to advance assessment of these professionals to achieve optimal care for all, supporting the development of a highly effective, diverse and compassionate health care workforce. Founded in 1915, NBME develops and manages the United States Medical Licensing Examination with the Federation of State Medical Boards. We offer a comprehensive portfolio of assessment products for every stage of the medical school journey and provide assessment services to various health profession organizations. We are dedicated to advancing innovative assessment approaches through research, collaboration and contributions to the medical education and assessment communities. In 2024, we expanded our assessment capabilities to include simulation through the acquisition of MedVR Education. Learn more on NBME's website. NBME'S COMMUNITY COLLABORATIONS AND CONTRIBUTIONS NBME believes that a key path to meeting our mission is collaborating with and supporting our communities. Our Community Collaborations and Contributions programs provide resources and recognition to medical educators and researchers advancing the field of assessment throughout their careers. In addition, we invest in fee assistance, scholarships and pathway programs to aid learners and help support increased representation in health care. Headquartered in Philadelphia, NBME also gives to local organizations that advance health equity and access and positively impact social determinants of health. The compensation we are offering ranges from $138,688 to $173,360. Please note that the offered rate for this position typically aligns with the minimum to midrange of this position, but it can vary based on the successful candidate's qualifications and experience, department budget, and an internal equity review. The NBME offers competitive salaries, excellent benefits, and a rewarding work environment. Excellent Benefits include: Healthcare, Dental, Prescription, and Vision plans; 401(k) w/match, Tuition Reimbursement Plan, Commuter Benefit: Public Transit or Parking options. Remote Friendly Workplace. NBME is an Equal Opportunity Employer. We will consider all qualified applicants for employment without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability. Apply now How would you like to verify your identity? Continue with phone Verify with a one-time text code Continue with email Verify with a one-time code or apply with