Psychometrics Lead

Position Description Are you excited by the power of data, measurement, and continuous improvement? Do you thrive on solving complex challenges and turning insights into action? As Psychometrics Lead, you'll have the opportunity to lead innovative assessment initiatives, collaborate with passionate experts, and drive meaningful change across the health professions sector. Bring your expertise, curiosity, and leadership to a role where your work truly matters, and your impact will be felt every day. As Psychometrics Lead at Touchstone Institute, you will play a pivotal role in advancing fair, meaningful, and defensible assessments that support excellence in health professions education and regulation. This is an exciting opportunity to combine psychometric expertise, innovative research, and data-driven decision-making to influence how knowledge, skills, and competencies are measured across diverse assessment programs. In this role, you will lead the design, validation, evaluation, and continuous improvement of high-quality assessment tools that accurately measure learner and test taker performance. Your work will directly contribute to the integrity, reliability, and fairness of assessments used to inform important educational and regulatory decisions. As a trusted lead, you will collaborate closely with program teams, subject matter experts, and stakeholders across the organization. You will translate complex psychometric and assessment data into clear, actionable insights, support evidence-based quality assurance initiatives, and help strengthen assessment practices across Touchstone Institute. Join us in shaping the future of health professions assessment and making a lasting difference in the communities we serve. Key Accountabilities Assessment Development & Validation Reporting to the Manager, Psychometrics, and working closely with the Senior Director, Assessment and Senior Director of Program Design & Implementation and relevant SMEs, develops and operationalizes clinical and written examinations and assessments that are deemed appropriate and psychometrically defensible Drives innovation in assessment and learning design, ensuring the development of evidence-based, valid, and impactful evaluation practices Translates competency frameworks and stakeholder input into comprehensive examination and assessment blueprints that support meaningful and equitable evaluation Designs and facilitates engaging workshops and collaborative sessions with SMEs to support examination and assessment development Leads and validates examination item and case development processes, ensuring alignment with psychometric principles, industry standards, and best practices Establishes and continuously enhances standardized approaches to candidate assessment, scoring, and performance evaluation Champions adherence to professional standards to ensure exams and assessments comply with industry standards (e.g. AERA, APA, NCME guidelines) Serves as a trusted advisorto the Assessment Oversight Committee, providing technical expertise and evidence-based recommendations. Prepares and presents analyses, reports, and briefing materials that support governance, policy development, and informed decision-making on assessment strategy and quality Quality Assurance and Quality Improvement Leads the ongoing review of examinations, scoring systems and methodologies to ensure ongoing psychometric integrity Partners with cross-functional teams to transform assessment data into actionable insights that drive organizational learning, program effectiveness, and continuous improvement Fosters a culture to collaborate with cross-functional teams inside the organization to turn data into ideas/insights for organizational growth Maintains documentation/records of QA activities, audit findings, and corrective actions to support transparency and accountability Data Analysis & Interpretation Conducts advanced psychometric and statistical analyses to evaluate assessment quality and generate insights that inform program design, candidate outcomes, and organizational decision-making Integrates research, quality assurance findings, and performance data to strengthen measurement strategies and demonstrate assessment effectiveness Translates complex data into clear, compelling reports, presentations, and recommendations for internal and external stakeholders. Develops opportunities to integrate research and quality assurance reports to inform and support strategies and demonstrate effectiveness in measurement Supports building networks and accessing expert advisors Position Requirements Knowledge Advanced knowledge of exam theory, learning theory, and assessment methodologies Strong expertise of measurement theory and methods Knowledge of competency-based assessment principles, particularly within health professions education and regulation Demonstrated expertise in examination development and test analysis Good judgment, critical thinking skills, and close attention to detail applied in this field Skills Strong time management skills and ability to prioritize multiple assignments and work demands Strong analytical and report writing skills and the ability to translate statistical information for clients and regulatory bodies Excellent verbal and written communication skills Strong conceptual, analytical and problem-solving skills Advanced problem-solving and critical-thinking abilities, with a track record of identifying challenges and delivering practical solutions Strong interpersonal and collaboration skills to work with others to coordinate information Strong data visualization skills, and comfort applying them in new and innovative ways Strong leadership skills: experience managing a team of analyst staff is an asset Proficiency in the use of word processing, and presentation software (e.g. Word, Excel, MS Project and PowerPoint) Abilities Ability to deal with emergent issues in a professional and timely manner Ability to work independently or as part of a team, with a range of clients at all organizational levels Demonstrated ability to process multiple types of information, perform multiple tasks simultaneously, and prioritize Qualifications Master’s degree in Statistics, Biostatistics, Quantitative Psychology, Developmental Psychology and Education, or other equivalent degrees At least 5 years of experience in an educational or testing environment Related educational assessment and measurement experience as well as experience in test development and analysis Job level: 6 Closing date: Monday, September 14, 2026 Notes for Candidates This is an existing vacancy within the organization This is a contractual role of 24 months Touchstone Institute does not use artificial intelligence (AI) tools as part of the applicant screening process. Applications will be reviewed by members of our recruitment and hiring teams to ensure a fair and thorough assessment. The salary range considers many factors including, but not limited to experience, education, and training, etc. It is not typical for the salary to be offered near the top of the range, it is dependent on numerous factors. Touchstone Institute is dedicated to employment equity and fostering a diverse work environment. We encourage applications from Indigenous Peoples, racialized persons, persons with disabilities, and other equity-seeking groups. We believe that a diverse team is essential to understanding and serving the evolving needs of the healthcare sector. We follow a hybrid work model with two designated in-office days: Tuesdays and Wednesdays. Successful candidates must be available to work on-site during these days at 145 Wellington St W Suite 600, Toronto, ON M5J 1H8 with or without accommodation. All applications should be submitted through our application tracking system via Bamboo at: https://tsin.bamboohr.com/careers/112 Touchstone Institute is an equal-opportunity employer that is committed to fostering a diverse work environment and exercises accessible employment practices. In accordance with the Ontario Human Rights Code, Accessibility for Ontarians with Disabilities Act (AODA) and Touchstone Institute’s policies, a request for accommodation will be accepted in all aspects of our hiring process. If you anticipate needing any type of accommodation during the recruitment process, please email hr@tsin.ca in advance of your appointment.