The expectations regarding learning design and delivery encompassed:

• delivery methods and learning activities
• management of time zone differences
• learning hours
• monitoring learner progress and achievement.

Delivery methods

Among programmes monitored by NZQA, three TEOs used a resource-based delivery model (Murray and Alkema, 2021) whereby most learning activities were asynchronous and made full use of the TEO’s Learning Management System (LMS) functionality.

The majority of TEOs – the four remaining programmes and all eight training schemes – relied on a teacher-based model (Murray and Alkema, 2021), with synchronous learning activities facilitated primarily through videoconferencing. In most cases, the TEO’s LMS was available alongside, but served merely as a document repository.   

Moodle was the most widely employed LMS, followed by Canvas. Others included Google Classroom, Blackboard, Schoology and in-house systems. Zoom was the most popular videoconferencing tool, used by 10 TEOs.

All delivery models have their advantages and disadvantages. However, the teacher-based model was clearly not an informed choice for most TEOs but rather an emergency response to an unprecedented situation. NZQA encourages TEOs to reflect on the fit of their model to the subject matter, intended learning outcomes, NZQF level and, most importantly, specific needs of their target learner groups.     

Learning activities

Fourteen out of 15 TEOs met NZQA’s expectation – i.e. they provided intentional and measurable learning activities that were effective in supporting online learners based offshore to achieve the intended learning outcomes.

Synchronous activities included real-time presentations, pair and group work, role plays, whole class discussions, practice exercises with teacher feedback, educational games and demonstrations. Asynchronous activities included recorded lectures, quizzes, LMS forum discussions, reflections, guided reading with in-built (automated) or facilitator feedback, case scenarios, video-based exercises and other subject-specific activities. 

NZQA commends TEO staff for their resilience and creativity in adapting face-to-face learning activities for online delivery. For example, in a naturopathy programme, offshore learners were unable to access a herbarium, so the lecturer filmed plants in their own backyard and on daily walks.

Other examples of good practice included:

• activities that are explicitly structured
• guidance to learners on how to manage their weekly schedule – which activities to complete and approximately how long to spend on each activity
• model answers (exemplars from past learner work)
• teachers being present and proactive on LMS forums, and providing encouraging feedback on learner posts.

Challenges included:  

• lengthy synchronous sessions – learners found it difficult to concentrate through a full day (6–8 hours) in front of the camera
• not recording live lectures and tutorials – learners with temporary access issues missed out on parts of the content
• activities impossible to convert into an online mode, such as teaching practicums in education programmes and clinical practicums in health programmes.

NZQA recommends TEOs employ a balanced combination of synchronous and asynchronous learning activities to facilitate achievement of the learning outcomes as well as provide flexibility of access for online learners based offshore.

Management of time zone differences

Twelve out of 15 TEOs met NZQA’s expectation – i.e. they demonstrated that their management of contact hours and time zone differences met offshore learner needs.

For the few TEOs offering mainly asynchronous online learning, no change was necessary as offshore learners were engaging in the learning activities at a time convenient for them. This is the most flexible and scalable method of delivery.

The vast majority of TEOs scheduled online live classes in the afternoon New Zealand time, allowing learners in different countries, predominantly in Asia, to participate and interact with other learners and the teacher during daylight hours in their local time zone. This required adjustment to timetables and contracted hours, as teaching staff in New Zealand were working as late as 10pm.   

For TEOs delivering mainly through teacher-led live videoconferencing, challenges arose when:

• learners were spread out across multiple, disparate time zones – the only option in this case was negotiating contact times with each learner individually
• online classes for offshore learners were scheduled at the same time as for onshore learners – this meant the offshore learners had to attend during the night.

NZQA advises TEOs to avoid the above two arrangements as these are unsustainable and disadvantage offshore learners. TEOs can market their offshore educational offering to learner groups in similar time zones. The TEOs aiming to attract learners globally should develop asynchronous learning activities and move toward resource-based delivery (Murray and Alkema, 2021).       

Learning hours     

The expectation regarding learning hours was applied to programmes only. Seven out of seven TEOs met the minimum expectation, i.e. the learning hours they delivered offshore online matched those approved by NZQA.

There was one case of minor under-delivery and one case of over-delivery.

NZQA recommends TEOs conduct periodic audits to ensure that the number of hours delivered in each programme component is the same as found in the NZQA-approved course descriptor.

Monitoring learner progress and achievement

All 15 TEOs met NZQA’s expectation – i.e. they demonstrated that learner progress and achievement is monitored and comparable with onshore delivery.  

There were a variety of ways in which TEOs monitored offshore learner progress:

• teachers checking LMS access reports – resources viewed and activities completed by learners, login times, weekly logged-in hours
• teachers and/or dedicated support staff following up with learners absent from compulsory online classes or not submitting assessments
• maintaining regular contact with offshore learners through emails, text messages, ad hoc video-calls and planned one-on-one catch-ups
• observing learner participation and contributions on LMS discussion forums and during synchronous online classes
• recording and analysing learner performance on formative tests and summative assessments
• weekly review of progress against each learner’s individual learning plan.

Data on the comparability of offshore and onshore learner achievement was inconclusive as the number of learners who were studying or had completed their programme or training scheme online offshore was relatively small. Nevertheless, for programmes, comparison of the final course grades indicates that offshore learner achievement is on par with onshore learners. For training schemes, although offshore learners may not acquire English language skills at the same rate as onshore learners, some TEOs presented evidence in the form of pre- and post-completion test scores to confirm that most offshore learners do make tangible progress and improve their English.

NZQA acknowledges TEO staff for monitoring and fostering learner progress during 2020 and beyond. NZQA recommends that TEOs continue to collect and analyse progress and achievement data for learners studying online offshore to enable a more systematic comparison between offshore and New Zealand-based cohorts.  

NZQA’s expectations regarding assessment and moderation aimed to gauge how well TEOs managed to:

• ensure authenticity of offshore learner work
• design assessment tasks appropriate for the online context
• moderate assessments completed online by learners offshore.

NZQA evaluated and commented on systems for ensuring authenticity of learner work for all 15 TEOs regardless of whether the validation report related to a training scheme or a programme. The minimum expectations to do with assessment task design and moderation were applied to programmes only (seven TEOs).   

Authenticity of learner work

All 15 TEOs met NZQA’s expectation – i.e. they had an effective system for ensuring the authenticity of learner work.

Nearly all TEOs explained their academic integrity policy and expectations to all new learners, including those studying online offshore, during orientation. Further guidance and reminders were provided in several ways, for example in the Student Code of Conduct, programme handbooks, course outlines, assessment briefs, or on LMS pages. Some TEOs required learners to tick or sign an authenticity declaration at the point of submitting written work for marking, or immediately before sitting online tests and examinations.

Most TEOs provided guidance regarding plagiarism but few included an explanation of contract cheating in their information for learners. Most frequently guidance came in the form of written resources. Three TEOs also offered interactive online workshops or modules on academic integrity facilitated by Learning Advisors. Two TEOs made these sessions compulsory for all learners to complete, which NZQA considers good practice.

Seven out of seven TEOs delivering programmes online to learners offshore used text-matching software such as Turnitin, SafeAssign, Unicheck or VeriCite. Three out of eight TEOs delivering training schemes online to learners offshore used such software. The latter is not a concern because in the context of English language learning, class sizes are small, so the teacher is familiar with each learner’s writing style. Also, a lot of assessments are formative or conducted in real time with the teacher supervising remotely via video.  

NZQA advises TEOs to provide training to academic staff so they recognise the warning signs of cheating, and proactively investigate and deal with any suspected cases.      

Assessment tasks

Three out of seven TEOs met NZQA’s expectation, i.e. had assessment tasks purposefully designed or modified for online delivery. Two TEOs were using existing tasks from their pre-COVID-19 face-to-face programme but NZQA deemed these appropriate to administer online. The other two TEOs were using their pre-COVID-19 assessment tasks unchanged but NZQA found these not fit for purpose.   

Examples of assessment tasks appropriately designed for online delivery included:

• open-book written assignments – essays, reports, reflection journals
• oral presentations – submitted as video recordings
• role-plays – assessed by the teacher in real time
• making artefacts – submitted as photographs with written explanation or demonstrated live on camera.

Other examples of good practice noted by NZQA evaluators were:

• including a mix of different assessment tasks in each programme component
• making participation on LMS discussion forums compulsory and allotting up to 10% of the total marks to participation
• breaking up large project-type assessments into weekly milestones with work-in-progress submissions – this helped to motivate learners and ensure authenticity
• tweaking assessment instructions to allow offshore learners to use their local context rather than a New Zealand context where doing so was compatible with the learning outcomes.

Key issues preventing TEOs from meeting the minimum expectation were:

• assessment tasks that did not give learners the opportunity to demonstrate achievement of all the learning outcomes being assessed
• group assessment tasks that did not allow each learner to demonstrate their individual achievement of the learning outcomes
• assessment tasks that were unsuitable for online delivery because they required work placement or the physical presence of the learner and assessor for safety reasons.

TEO performance was variable in terms of modifying assessment tasks to ensure they are appropriate for online delivery to learners offshore. NZQA urges TEOs to review all assessments in relevant programmes for suitability to this context, amend or redevelop tasks where necessary, and internally pre-assessment moderate those assessment tasks that have been changed.

Moderation

Five out of seven TEOs met NZQA’s expectation, i.e. demonstrated that their system for moderation of assessments that learners complete offshore is as effective as onshore moderation.

NZQA considered two types of evidence when judging the quality of TEO moderation practice, the samples of assessed offshore learner work and internal moderation reports.

TEOs whose moderation systems were effective provided samples of learner work which:

• met the learning outcomes
• were at the required NZQF level
• contained written assessor feedback on the strengths and points for improvement which was consistent with the mark awarded.

The internal moderation reports from these TEOs showed that either all assessments completed by offshore learners were being post-assessment moderated or that there was a deliberate sampling rationale (for example, include at least one offshore learner sample within each course or curriculum area being moderated). The moderation reports included detailed comments on the quality of the work and verification that the learner had achieved the learning outcomes.

The TEOs whose moderation systems were deemed ineffective presented learner samples that did not meet the learning outcomes or NZQF level requirements. Some moderation of offshore learner work was occurring, but a sampling target or approach was not considered. Internal moderators did not confirm whether the learner had achieved the learning outcomes or not.  

All 15 TEOs met NZQA’s expectation, i.e. ensured that staff received ongoing professional development in facilitating online courses, and/or designing and developing learning resources appropriate for online delivery including to learners offshore.

All monitored TEOs upskilled their teaching staff in the use of technology, particularly the chosen LMS and videoconferencing tools. The March 2020 lockdown spurred the sector on to quick, decisive action.

Most TEOs also provided professional development in facilitating online learning such as:

• internal tutorials led by an e-learning specialist
• one-on-one coaching sessions for individual teachers as needed
• external webinars through TEC, Ako Aotearoa and subject-specific professional associations.

Examples of successful initiatives noted by NZQA evaluators included:

• identifying or appointing LMS champions or online learning design champions among staff to assist others
• using a ‘community of inquiry’ approach and establishing staff working groups to research technology options and good practice in online course facilitation
• setting up an online collaboration space for teachers to reflect and share ideas.  

Seven TEOs delivering training schemes and one delivering a programme provided no evidence of staff professional development in relation to developing learning resources appropriate for online delivery. TEOs delivering training schemes mainly used PDF excerpts from general English or English for Academic Purposes textbooks as opposed to designing their own resources for online teaching.

NZQA encourages TEOs to continue offering targeted professional development opportunities to staff who are involved in online delivery including to learners offshore. These opportunities should move beyond familiarising staff with LMS and other technology tools, toward building capability in online pedagogy, instructional design and assessment.         

NZQA’s expectations regarding student information and support encompassed:

• systems for general, academic and technical support
• learner experience and satisfaction.

Support systems

All 15 TEOs met the expectation, i.e. had systems for general, academic, and technical support that are effective in catering for all learners.

All TEOs provided comprehensive information to offshore learners regarding how to use the online tools – LMS and/or videoconferencing software. Many TEOs reported that learners were conversant with the tools and generally needed little technical assistance. Offshore learners also received clear information on what support services were available and how to access these.

Teachers were generally the main source of academic support. Additionally, a number of TEOs employed learning advisors to provide on-demand guidance with study skills, academic writing, preparing for assessments and referencing. One TEO ran a ‘global study centre’ rostering qualified teachers from its affiliated schools around the world to answer online queries, making academic support available 24/7 outside of class times.

Examples of good practice included:      

• timely, relevant communications to offshore learners using a range of channels
• knowledge of which channel offshore learners find easiest, for example WeChat versus LMS messaging versus email
• all LMS pages displaying ‘virtual office hours’ when the teacher is available
• health and welfare checks
• virtual drop-in clinics for study and wellbeing support
• one-on-one meetings with learners on request
• reference materials organised for easy access (such as the ‘need to know’ features of the LMS, guidance on specific assessment types, tips for successful online learning).

NZQA is confident that TEOs are providing strong general, academic and technical support to offshore learners.

Learner experience and satisfaction

Thirteen out of 15 TEOs met NZQA’s expectation, i.e. demonstrated that the offshore learner experience and satisfaction were comparable with onshore delivery.

Fourteen TEOs gathered data by surveying learners and comparing survey results between the offshore and onshore learner groups. For some training schemes no concurrent onshore delivery was occurring, so comparisons were made to previous intakes which had completed their studies in 2019.

In addition to surveys, TEOs collected information on learner experience and satisfaction at individual progress reviews, during live classes, and from learner emails and posts on the LMS (refer to Monitoring learner progress and achievement above). One TEO provided a powerful example of feedback from a group of offshore learners captured on video.  

Overall, offshore learners were positive about their study experience and felt that online they were able to learn either more or at least as much as they did in face-to-face classes. Many commented on the high quality and increased volume of academic and pastoral support received from their institution.

There were few problematic areas noted by NZQA evaluators:

• practical learning activities and assessments were not equivalent to an in-person experience, received lower satisfaction ratings from offshore learners, and some learners did not find these activities adequate
• two TEOs did not disaggregate survey responses from onshore and offshore groups making it impossible to make comparisons.   

NZQA is confident that, except for practical learning activities and assessments, the online learning experience and satisfaction of offshore learners are comparable with onshore provision. NZQA recognises that subject areas involving practicums may not be suitable for exclusively online delivery and require blended approaches instead.

1. Provide a balanced mix of synchronous and asynchronous online learning activities that fulfil the needs of the specific group(s) of offshore learners.
2. Ensure online learning activities are well structured, support the achievement of learning outcomes and set out weekly completion targets for learners.
3. Avoid scheduling live classes for offshore learners at the same time as for New Zealand-based learners. Be mindful of time zone differences so that offshore learners are not disadvantaged, and are able to fully engage with the content, other learners and teacher.
4. Record lectures and tutorials, and ensure learners have ongoing access to the recordings.

1. Review all assessment tasks for suitability to the online, offshore context. Amend or redevelop assessment tasks where needed, especially tasks that are practical in nature.
2. Offer online workshops on academic integrity and make these compulsory for offshore learners to complete.
3. Establish an appropriate sampling target for post-assessment moderation of online assessments completed by learners offshore.

1. Focus staff professional development efforts on designing learning resources and assessments suitable for online delivery to learners offshore.

1. Maintain regular contact with offshore learners. Use a variety of communication channels and establish which channel your group of offshore learners prefer.
2. Collect offshore learner achievement data and compare it with the achievement of New Zealand-based cohorts.
3. Monitor levels of learner satisfaction with the online learning experience by gathering feedback and surveying offshore and onshore groups separately.