The current programme monitoring priorities are:

• Business and Management focused programme from levels 5-9, including Hospitality Management and Healthcare Management programmes
• Information Technology focused programmes at levels 5-9
• programmes deliveref offshore and online
• Level 7 diplomas

NZQA is also monitoring the effectiveness of internal moderation systems, and programme review processes across a range of TEOs and programme subject areas.

NZQA moderators are contracted based on their subject area expertise, assessment and moderation experience and educational qualifications.

NZQA has a robust conflict of interest process to ensure moderators are neutral.

NZQA contacts the person listed as the TEO’s Quality Assurance contact on the NZQA website. The TEO can nominate another staff member to be the contact person if they wish.

This person will be NZQA’s contact for confirming the programme monitoring activity and, as required, confirming the agenda, supplying documents, and liaising with the monitors.

Documentation

NZQA requests documentation associated with programme delivery and quality management. See the Appendix to the notification letter for a list of what documentation NZQA needs.

If the monitoring activity has a moderation component, NZQA also requests academic results so it can select marked learner work for verification of assessment decisions.

NZQA typically selects assessments with the heaviest course weighting and assessments conducted later in the course, if possible.

The agenda

When the monitoring activity involves interviews with TEO staff, these usually include academic staff, programme management and learners.

The TEO can adjust the timings or rearrange the order of interviews and any class observations for minimal disruption.

Interviews

Who might be interviewed?

• Quality and academic manager
• Programme leaders responsible for the programme(s)
• Academic records/administration staff
• Key academic staff members for the programme(s) (e.g. programme delivery, teaching staff, assessors)
• Learners currently enrolled on the programme(s).

What will NZQA ask them about?

Questions for academic manager, quality manager, programme leader, programme developer, and academic staff members may cover:

• programme length and structure
• changes to term/semester/block delivery
• timetables (staff/internal and published)
• approved learning hours
• programme regulations
• integration of practical and work-based components
• rules for award of qualification
• moderation systems
• review of programme performance and self-assessment.

Questions for academic registrar and administration staff may cover:

• English language requirements for international students
• credit recognition and transfer
• results and certification.

NZQA may also select a sample of student enrolment files to check admission processes are being applied appropriately. The monitors bring this list with them on the day.

Questions for academic staff members may cover:

• qualifications
• professional development
• assessment materials
• moderation practice
• plagiarism prevention, detection and management
• assessment methodology.

Questions for support staff may cover:

• support services
• educational resources
• pastoral care.

Questions for current learners may cover:

• learning experience
• assessment experience
• programme requirements and expectations.

Recording the interviews

The monitors ask permission to record staff interview sessions before the discussion starts. TEOs can request copies of the recordings. The monitors do not record interviews with learners.

NZQA uses the recordings only to assist in writing the programme monitoring report and destroys the recording six months after finalising the report.

Monitors observing TEO classes

ZFor onsite visits, monitors may ask to sit in on a class. The monitors are checking that the activity reflects the programme documentation and the information that the TEO publishes about the programme.

For example, that learners are attending as required, English is the medium of instruction, and qualified staff are leading the teaching activity.

NZQA provides the TEO with a draft programme monitoring report, including a summary of moderation results.

If there is any delay NZQA will notify the TEO.

TEO and NZQA follow-up actions

If NZQA has concerns with the integrity of qualifications awarded to learners, it will decide if these concerns require immediate remedial actions or specific actions in a short timeframe.

If the report identifies areas for improvement, NZQA may agree an action plan with the TEO. NZQA follows up with the TEO to ensure the action plan is effective. Where the issues are significant, the NZQA Risk Team will also be involved.

Where the issues identified are relatively minor, NZQA expects the TEO to address any requirements or recommendations without further follow-up by NZQA.

Student data and learner work held by NZQA

NZQA holds all submitted materials, including learner work samples and enrolment data, securely and only uses it for completing the programme monitoring report.

NZQA holds copies of learner materials on file for two years (i.e. to finalise the report and allow any follow-up). Then the records are destroyed.

Learner IDs (not names) are used instead of names in the moderation reports.

NZQA summarises programme monitoring outcomes in a report with sections relating to one or more of the relevant criteria. 

The programme monitoring report recognises and acknowledges effective delivery, progress and achievement. NZQA also acknowledges development during the programme’s first delivery and the TEO’s efforts in continuous improvement.

In most monitoring report templates, NZQA categorises findings under four headings:

1. Commendations
2. Issues
3. Requirements
4. Recommendations

Commendations reinforce good practice, whereas issues indicate there is insufficient evidence for one or more of the criteria to maintain approval or accreditation.

Requirements reflect the issues identified in the report and must be actioned by the TEO.  NZQA follows up with the TEO to ensure the requirements are addressed.

TEOs do not have to implement recommendations in the report but NZQA encourages TEOs to do so.

Based on the report findings, NZQA proposes an overall rating to answer therelevant programme monitoring KEQ.

For example:

If the overall outcome is ‘meets some’ or ‘does not meet’ programme criteria, the report will indicate how serious NZQA considers the findings to be.

The primary audience for the programme monitoring report is the TEO’s management and programme leadership. It is not intended for the learners.

The following Quality Assurance Division (QAD) business units view the programme monitoring report in both draft and final states:

• Approvals and Accreditation
• Consistency
• External Evaluation and Review
• Quality Assurance Māori
• Risk.

Programme monitoring reports are not published on the NZQA website but feed into other quality assurance processes, including Approvals and Accreditation and External Evaluation and Review.

NZQA asks the TEO to check the programme monitoring report and point out what they believe to be any errors of fact. This may include disagreeing with the moderation findings.

Where applicable, the TEO may include in its response an outline of what it will do to address any concerns or areas for improvement raised in the programme monitoring report.

NZQA evaluates the TEO response and issues the TEO with a final programme monitoring report, including any amendments.

Where the report identifies serious quality assurance concerns, NZQA may consider putting any new approvals on hold until the identified issues are resolved.

NZQA moderation reports include feedback on whether there is sufficient evidence that:

• the learner has achieved the learning outcome(s)
• the learner work is authentic
• assessment materials allow learners the opportunity to achieve the learning outcome(s).

How to read moderation reports

Section 1: Assessment details

NZQA populates the moderation report template with key data from the TEO’s approved programme, including the level and credits of the components, the associated learning outcomes, the assessment type, conditions and weighting.

Section 2: Summary of moderation outcomes

This section states the overall outcome for the moderation of that component and includes:

• the ‘agreement rate’, which is the moderator’s agreement with the assessor decisions on the achievement of learning outcomes; and
• a brief description of the main issues (if any).

Section 2 also includes a moderation outcome summary for assessor decisions and assessment materials (assessment tasks and marking guidance). Moderators select one of three options for assessor decisions and one of three options for assessment materials:

Section 3: Verification of assessor decisions

This section includes the moderator’s comments on the learner work and the moderator’s judgement about whether the learning outcomes were achieved.

Moderators note any issues in order of significance in relation to achievement of the learning outcomes. Moderators may also comment on depth of learner responses in relation to NZQF Level, and authenticity of learner work as appropriate. 

Section 4: Assessment tasks and marking guidance

Moderators check that the assessment tasks provide the opportunity for learners to achieve the learning outcomes, at the stated NZQF level.

Moderators also check that marking guidance supports fair and consistent assessment, and clearly states minimum evidence expectations for achievement of each learning outcome.

Section 5: Learner work examples

Moderators may select excerpts of learner work that support their judgement and to exemplify the issues outlined in Section 3.