We worked with the Occupational Therapy Board of New Zealand (OTBNZ) to help validate their best practice model through the analysis of eight years’ worth of textual data from their members.
Ensuring the safety of occupational therapy practice
There are more than 3,000 kaiwhakahaora ngangahau (occupational therapists) with practising certificates in New Zealand. A condition of each therapist’s certification is that they need to submit an electronic portfolio every two years, detailing their work and how it met the OTBNZ’s five core competencies. The core competency model of best practice was first developed by US nurse Patricia Benner, whose ‘From novice to expert’ concept explains how nurses (and other healthcare professionals) develop competencies over time from both education and experience on the job. This concept was so influential that other healthcare sectors picked it up as a way of enhancing competence and ensuring safety from the ground up – in other words, using both theory and practitioners’ experience to form a picture of best practice.
Investigating the core competencies model
This year, the OTBNZ wanted to investigate whether assessing their members against core competencies was the best way to measure the quality and safety of their practice. We created a thematic analysis on the thousands of documents provided by practitioners, to create a comprehensive map of the topics found within their portfolios. The thematic analysis used word embeddings, an approach used by large language models to identify concepts in text. The map allowed the Board to quickly and efficiently identify all the themes their members wrote about, from little-known topics to the most popular ones.
As a result of this analysis, OTBNZ was able to validate the efficacy of the competency framework and start work on a new code of ethics. OTBNZ is planning to use this analysis as a base for measuring best practice, with a view to a re-assessment in two years’ time.
Project team
We were really, really pleased with Dragonfly—they’re very clever people, but they’re also great to work with. They ask all the right questions, led wonderful workshops with our team, and the Dragonfly project manager, Emma, was fantastic.
When we started looking for who could deliver this work, it was really important to us to engage with a group of professionals with high integrity, who were ethically sound and understood our responsiveness to Te Tiriti. Dragonfly ticked all of those boxes.
Rhondda Knox
Chief Executive and Registrar
Occupational Therapy Board of New Zealand .