# Assessment Report

## Level 3 Statistics 2021

### Part A: Commentary

Candidates need to ensure that they complete as much as they can in each question to achieve. Candidates who clearly showed how they reached their answers were more likely to achieve Merit or Excellence grades than those who only wrote down their answers. Candidates should write full answers supported by calculations.

A common issue was candidates rounding answers prematurely, which resulted in inaccurate/incorrect answers. Candidates should ensure that they do not round their work to less than 4 decimal places, until their final answer. Probabilities may be very small. Candidates should ensure that they are familiar with working with small numbers, particularly those in scientific notation.

Candidates need to be familiar with instructions such as “investigate”, “compare” and “justify”. Responses to such questions should include statistical calculations and sufficient working should be shown to indicate the candidate’s thinking and reasoning. Final responses need to be linked back to the question or statement that has been given.

Candidates who were not able to identify cues in the question as to what they were being expected to answer provided incomplete responses to questions. Graph drawing skills continue to be poor, and it was observed that many candidates drew rough sketches rather than accurate graphs with scales on both axes and key parameters identified. Scales on graphs need to be correct and even. Candidates who keep their answers as fractions must order them correctly on their axes: 1/15 is not bigger than 1/12. Probability skills beyond finding the probability of a simple event are necessary to gain grades at higher levels, e.g. combined events involving ‘and’, ‘or’ conditional and relative probabilities.

Candidates should not expect that questions in the examination will be like past examinations, as questions can be about any context and in any form.

Part B: Report on standards

#### 91584:  Evaluate statistically based reports

##### Examinations

The examination included three questions, each in 4 parts, of which candidates were required to respond to all three. Candidates were provided with a resource booklet and a question-and-answer booklet. The resource booklet contained three reports, one for each question. The reports were set in real-life contexts.

The questions covered the requirements of the 2021 assessment specification, which were to answer questions about statistically based reports. The questions required the candidate to evaluate claims or conclusions made in the report, including identifying and discussing potential sources of error associated with statistical studies, calculating and interpreting margins of error and considering study design and the type of inference.

Note that candidates should recall and use the “rules of thumb” based on
for margin of error.

##### Observations

Candidates are required to assess the quality of reports using statistical methods indicated by the question, whether it is to do with the design of the study, or to identify potential issues with aspects reported in the study. Candidates need to read the report and the questions carefully, and then consider what is being asked of them, perhaps by highlighting or underlining key words, that allows them to focus better their responses. In many cases, candidates provided a great answer that did not answer the specific question at hand, but rather another question elsewhere.

Candidates should avoid using generic, learned answers, for example “farmers generally do not need the internet”, without considering the context of the report or providing necessary explanation to relate their observations back to the statistical reports.

Candidates should not say that there is a need to ‘eliminate bias’. Where bias is being discussed, the term to use is “reduce” or words to that effect.

It is important for candidates to understand the effect of sample size on the margin of error.

After calculating comparison confidence intervals, candidates need to comment on the claim by describing the confidence interval in context, and appropriately discuss the underlying population. For example, if the confidence interval was [11.4%, 16.6%], then candidates needed to interpret this correctly by saying something like “I am pretty sure the true proportion of NZ children born in 2009 and 2010 who sometimes or often speak simple words of Te Reo Maori is somewhere between 11.4% and 16.6% lower than the proportion who rarely or never speak Te Reo Maori” before answering the claim. The claim must also be in context and separate from the judgment. For example, "as this confidence interval is entirely positive, there is evidence to support that a lower proportion of NZ children born in 2009 and 2010 sometimes or often speak simple words of Te Reo Maori than rarely or never."

Candidates need to be able to identify, describe and discuss both experimental and observational studies and apply that knowledge. Further, it is a good idea that mention what type of study the report is, even if it is not clear from the question that it needs to be identified. Candidates must also realise that a causal claim can be inferred from an experimental study, but not from an observational study. Students, when discussing the features or results of an experimental or observational study, should be sure to include empirical evidence contained within the report to back up any statements made.

Candidates who were awarded Achievement commonly:

• commented on key features by referring to statistical evidence provided in the reports without including specific details
• calculated the margin of error (MOE) and described why the MOE is required
• identified the explanatory and response variables in an experimental study
• correctly located information in a table of figures
• had a sound understanding of self-selection or convenience sampling, and supported this reasoning with enough context to explain how the sampling frame did not adequately reflect characteristics of the population of interest
• calculated a confidence interval correctly.

Candidates whose work was assessed as Not Achieved commonly:

• did not refer to the statistical nature of the reports
• did not identify the correct sample size to use with the rule of thumb formula to calculate the MOE or describe why it is used
• did not identify the explanatory variables in an experiment
• did not use the appropriate MOE to calculate the relevant confidence interval for a comparison confidence interval
• did not define or correctly use statistical terms and statistical terminology
• did not identify random allocation as a key idea in experimental design
• incorrectly focussed on the size of samples to discredit studies.

Candidates who were awarded Achievement with Merit commonly:

• calculated a comparison confidence interval, but could not correctly interpret it within the context, or justify why a claim was true
• calculated a single confidence interval, and interpreted it in context
• demonstrated sound understanding of self-selection or convenience sampling, and supported this reasoning with enough context
• identified an issue with a survey, and related it to representativeness with a specific population
• analysed experimental data and supported their findings with relevant contextual information.

Candidates who were awarded Achievement with Excellence commonly:

• described an issue with study design and directly linked it to the strength of a claim in context
• calculated a comparison confidence interval and interpreted it in context, justifying a statistical claim using correct statistical language, including identifying the population
• used and correctly applied statistical language
• recognised that causal claims can be made from experimental studies, and that these causal claims can be strengthened where the experiment has been well designed, and where necessary re-randomization tests are undertaken
• acknowledged that for randomised experiments, causal statements can be made, however, generalisations or links between variables can only extend to the participants involved in the experiment.

#### 91585:  Apply probability concepts in solving problems

##### Examinations

The examination included three questions, each in 5 parts, of which candidates were required to respond to all three.  Candidates were provided with a question-and-answer booklet.

The questions covered the requirements of the 2021 assessment specification which were to calculate probabilities from formulae, a probability distribution table or graph, tables of counts or proportions, simulation results, or from written information.  Familiarity with use of Venn diagrams, probability trees and two-way tables of counts was required.

It was necessary for candidates to clearly show the method they have used to calculate probabilities, and state any assumptions made.

##### Observations

Many candidates need to practise determining what a question is asking them to do.  They need to attempt at all three questions, rather than focussing on only one or two. Candidates should write full responses, in context, and explain their decisions when answering questions, supporting their responses with calculations.

Specifically, candidates require more preparation in setting up and solving linear equations, usually from tree diagrams and they need further clarification work on interpreting the likelihood ratio (times as likely vs times more likely). More practice is required in analysing situations that may or may not involve conditional probabilities. The ability of students to describe how to use a simulation to plan continues to improve, but still needs more work. It is useful for students to be able to recall tests for complementary, mutually exclusive, and independent events.

Candidates who were awarded Achievement commonly:

• assumed independence of events in order to calculate the probability of a combined event
• used contextual aspects of the chance situation being modelled in their answers
• used two-way tables appropriately to model situations
• used probability trees appropriately to model situations
• calculated and compared risks
• used straightforward probability methods to solve problems
• used clear and correct probability statements as part of their working.

Candidates whose work was assessed as Not Achieved commonly:

• provided written responses only, with few supporting probability calculations
• worked with whole numbers, rather than probabilities
• did not interpret the probability information provided in the text
• did not select an appropriate probability tool or representation to solve a problem
• did not calculate a risk or interpret a risk in context
• demonstrated a lack of understanding of chance variation.

Candidates who were awarded Achievement with Merit commonly:

• selected and combined appropriate probability methods to solve problems
• related probability concepts to the context of the chance situation being modelled
• justified the assumption of independence to calculate a combined/joint probability
• justified a claim using statistical statements and/or calculations
• selected and used an appropriate test for independent events.

Candidates who were awarded Achievement with Excellence commonly:

• used partitioning and other effective strategies for solving problems involving two or more events
• developed a good chain of reasoning based on the properties of the probabilities of combined and conditional events
• demonstrated familiarity with using the results of a simulation to assess whether there is evidence against chance acting alone
• communicated the strategy used to solve a problem
• integrated contextual and statistical knowledge.

#### 91586:  Apply probability distributions in solving problems

##### Examinations

The examination included three questions, each in 5 or 6 parts, of which candidates were required to respond to all three.  Candidates were provided with a question-and-answer booklet.

The questions covered the requirements of the 2021 assessment specification which were to clearly identify the probability distribution applied in solving each problem, stating assumptions made and to calculate probabilities from probability distributions presented as formulae, tables or graphs of data, simulation results or written information.

Candidates needed to be familiar with, and justify the use of, the normal, Poisson, binomial, uniform, and triangular distributions and they were expected to understand the calculation of the mean and standard deviation of a random variable.

##### Observations

The paper highlighted candidate confusion and misconceptions around a number of terms and concepts: in particular independence. Many candidates confused independence of an event with the probability of the event.

Candidates need to be able to identify the random variable being discussed in a question, and to explain how the conditions of a given probability distribution model are appropriate or inappropriate to the given context of that random variable. Being able to identify the random variable of a given situation is a key skill. Some candidates were unclear about the conditions of each probability distribution model and, when discussing a particular probability distribution, frequently confused its conditions with those of other probability distributions. Some candidates confused describing features of a distribution shown graphically with the conditions of the theoretical distribution model.

When drawing graphs, candidates must ensure that they clearly show scales on both sets of axes, particularly those on the y-axis. To gain Merit and Excellence grades, candidates must be familiar with all the probability distributions, and be able to articulate the conditions of each probability distribution model. They need to be able to compare the features of observed data when displayed graphically with those of a theoretical model, and decide on the suitability of the model. Their responses should be detailed and supported with evidence from the graph, probability calculations and reference to the context of the problem. For Excellence, candidates need to be able to explain statistical concepts such as independence, randomness, expected values, variation, and standard deviation. They also need to be able to explain how these concepts are seen in data displayed in tables, graphs and/or the ‘real life’ context of the question.

A significant number of candidates were not able to identify which distribution to use when answering a question. Many candidates appeared to be familiar with only one distribution, e.g. normal, and applied this distribution to all questions. Candidates need to be familiar with all the probability distributions and their conditions and features.

Probabilities can often be quite small, and candidates need to be able to handle calculations with small numbers in scientific notation when answering questions. Understanding of what a probability means in context is also important, e.g. events with probabilities of 0.0023 and 0.0000012 are both highly unlikely, and so it is not appropriate to say that the event with 0.0022 probability is more likely than the other. When and how to round numbers is important when answering questions. Numbers should not be rounded to less than four decimal places during calculations. Fractions that result in recurring decimals are not uncommon, and candidates need to understand what these are, how their calculator handles them and how to round them appropriately: 0.7 is not an appropriate rounding of a calculation involving 2/3.

Understanding of probability concepts such as variation, variance, and randomness, and of statistical parameters such as expected value, mean, median, mode and standard deviation, as well as the ability to find these parameters is important. Emphasis in this standard is on deciding the appropriateness of a probability distribution model to data. This data could be in a variety of forms, including tables and graphs. Candidates need to understand that observed data is not usually a perfect fit to a probability distribution model. This is particularly important when discussing the suitability of using a particular probability model to that data. In this examination several candidates believed that because the mean, median and mode were not exactly the same the model would not be appropriate, when it was.

Candidates who were awarded Achievement commonly:

• selected appropriate probability distribution models
• calculated simple probabilities using probability distribution models
• understood terms such as ‘at least’ and ‘more than’
• identified the correct parameters needed to solve a probability distribution problem
• accurately drew a rectangular distribution graph
• accurately drew the graph of a triangular distribution.

Candidates whose work was assessed as Not Achieved commonly:

• applied an inappropriate distribution model to a problem
• did not draw the graph of a rectangular or a triangular distribution
• did not calculate a given probability for a binomial, poisson, normal or triangular distribution
• did not calculate probabilities from a table showing the probability distribution of a random variable and provided calculation errors or rounded prematurely
• misunderstood the difference between a condition of a distribution model and the features of a graph
• did not describe the conditions of a distribution in context.

Candidates who were awarded Achievement with Merit commonly:

• completed multi-step problems across a range of distributions
• identified the random variable for a given situation and explained whether the given probability distribution model was appropriate for modelling that variable
• understood assumptions made when using probability distribution models and were able to discuss whether these assumptions were likely to be valid in the context of a problem
• calculated conditional probabilities and combined probabilities of two events involving ‘or’ and ‘and’
• provided a correct graph of probability distribution models
• communicated their thinking using appropriate statements and calculations, e.g. stating the probability distribution model and parameters, correctly using probability notation, providing calculations for any general statements they made and linking these to the problem they were investigating.

Candidates who were awarded Achievement with Excellence commonly:

• showed a depth of understanding across a range of distributions, appropriately linking statistical and contextual information
• discussed the appropriateness (or inappropriateness) of a probability distribution model by considering features of the probability distribution, statistical evidence and/or the context of the situation
• understood and devised a strategy to solve multi-step probability distribution problems
• investigated claims involving relative probabilities
• justified the suitability of a theoretical distribution model for a given observed distribution by comparing the features of the model with that of the graph of the observed distribution
• explained what the expected value and standard deviation were, and how these related to probabilities in a distribution table for a random variable
• understood that the graph of a distribution of a sample may not perfectly reflect a normal distribution, and that differences in the observed sample distribution mode may not reflect those of the population.

### Mathematics and Statistics subject page

Previous years' reports

2019 (PDF, 121KB)

2018 (PDF, 139KB)

2017 (PDF, 52KB)

2016 (PDF, 228KB)