Evaluating Project Impact and Iteration

Evaluating project impact requires systematic collection and analysis of multiple data sources to judge whether the PBL unit achieved intended learner outcomes, developed skills and dispositions, and produced sustainable changes in classroom practice. Use a mixed-methods, evidence‑based approach: triangulate learner attainment, learner and stakeholder feedback, and observational data to form a reliable judgement and to guide iteration.

---

Key impact indicators (classroom and school level)

1. Cognitive outcomes

   • Mastery of CAPS knowledge and specific LO/assessment standards
   • Performance on summative tasks and assessments mapped to CAPS

2. Skills and dispositions (21st-century skills)

   • Inquiry and problem-solving
   • Collaboration and communication
   • Self-regulation, persistence, creativity, real-world application

3. Engagement and behaviour

   • Attendance, participation rates, time-on-task
   • Evidence of learner agency and ownership

4. Equity and inclusion

   • Differential outcomes for learners with diverse needs
   • Accessibility of tasks and participation for all learners

5. Transfer and sustainability

   • Use of learned skills in other subjects or community contexts
   • Adoption of PBL practices by other teachers/grades

6. Implementation fidelity and practicality

   • Degree to which planned scaffolds, assessments and timelines were followed
   • Teacher and support capacity; resource constraints

---

Data sources and instruments

Use these sources together to triangulate findings.

• Summative assessment results

  • Project artefacts, test scores, and rubric grades
  • Pre‑ and post‑tests for targeted knowledge/skills

• Formative assessment records

  • Ongoing checklists, exit tickets, progress logs

• Learner evidence portfolios

  • Drafts, reflective journals, multimedia artefacts, presentations

• Observational data

  • Structured classroom observation protocols (focusing on collaboration, inquiry, teacher scaffolding)
  • Video recordings for replay and moderation

• Learner voice and feedback

  • Short surveys, focus groups, reflective prompts, self-assessment rubrics

• Stakeholder feedback

  • Teacher team reflections, parent/community partner feedback, subject advisor notes

• Attendance and participation records

  • Registers, group participation logs

• Moderation records

  • Internal and cross-school moderation notes verifying standards

Sample mapping (indicator → method → frequency)

• Mastery of CAPS content → summative rubric + pre/post-test → end of project (+ baseline)
• Collaboration skills → observation checklist + peer assessment → mid and end
• Learner agency → learner reflection survey → mid and end
• Differential outcomes → disaggregated grades by learner groups → end

---

Designing instruments and rubrics

• Use clear, CAPS-aligned criteria written in learner-friendly language.
• Include separate strands for knowledge, process skills, and dispositions.
• Provide exemplars for performance levels (1–4 or 1–7 depending on CAPS practice).
• Ensure reliability: moderate rubrics with colleagues and use anchor samples.

Example rubric criteria (summary)

Criterion	Descriptor (Excellent)	Evidence sources
CAPS knowledge	Accurate, complete application of key concepts	Final report, test
Inquiry & problem-solving	Defines problem, collects relevant data, draws supported conclusions	Process logs, presentation
Collaboration	Shares tasks, resolves conflict, builds on ideas	Observation, peer assessment
Reflection	Identifies growth, next steps, and learning behaviours	Reflective journal

---

Analysis and triangulation

1. Prepare data set: compile scores, observation notes, survey responses, portfolio items.
2. Disaggregate by learner groups: gender, prior attainment bands, learners with barriers to learning.
3. Triangulate:
   • Confirm cognitive gains with both test scores and project artefacts.
   • Validate skill development with observation + learner self/peer assessment.
   • Corroborate engagement levels with attendance and participant feedback.
4. Identify patterns: strengths, common errors, groups underperforming, unexpected outcomes.
5. Judge impact level:
   • High impact: clear gains across multiple measures for most learners.
   • Moderate: gains in some domains, uneven by group.
   • Low: limited or no measurable improvements; issues in fidelity or design.

---

Reporting for CAPS and school records

• Document evidence that maps directly to CAPS LOs and assessment standards.
• Use a short impact statement per project:
  • Learning objectives, assessment methods, % learners meeting standard, examples of learner work, recommended next steps.
• Submit moderated results where required (internal moderation, cluster moderation, subject advisor).
• Keep portfolios and anonymised data for verification and auditing.

Suggested report headings:

1. Project title and grades involved
2. CAPS outcomes targeted
3. Assessment framework and instruments used
4. Summary of quantitative outcomes (disaggregated)
5. Qualitative evidence highlights (quotes, artefacts, photos)
6. Moderation process and sign-offs
7. Recommendations and iteration plan

---

Iterating project design: step-by-step

1. Convene reflection meeting (teacher team + relevant stakeholders)

   • Review data summary and triangulated findings.
   • Ask: What worked? What didn’t? For whom?

2. Prioritise issues to address

   • Safety and logistics first, then learning design elements.
   • Focus on high-impact, feasible changes (scaffolding, assessment criteria, group formation).

3. Revisit learning objectives and success criteria

   • Tighten alignment to CAPS where necessary.
   • Make success criteria measurable and observable.

4. Adjust learning pathway and scaffolds

   • Add formative checkpoints, mini-lessons, modelling sessions.
   • Differentiate tasks (choice boards, tiered tasks) for diverse learners.

5. Redesign assessment

   • Simplify or clarify rubric descriptors.
   • Introduce interim assessments to catch misconceptions earlier.

6. Reconfigure collaboration structures

   • Change group size, roles, accountability measures, rotation of roles.

7. Enhance documentation and feedback loops

   • Standardise progress logs, require short learner reflections at specified points.
   • Build in peer and teacher feedforward moments.

8. Test changes on small scale (pilot iteration)

   • Implement revised elements with one class or group before whole-school rollout.

9. Monitor and collect targeted data

   • Focus on indicators relevant to the changes you made.
   • Use short-cycle evaluation (weekly checkpoint surveys, two-week formative checks).

10. Institutionalise successful changes

    
    

• Share findings with SMT and subject teams.
• Update project plan templates, resources and policy where necessary.

---

Practical tools and checklists

Quick teacher checklist for deciding whether to iterate

• Did most learners meet the CAPS-aligned success criteria?
• Were targeted skills observable in classroom behaviour and artefacts?
• Were outcomes equitable across learner groups?
• Was the workload and timing manageable?
• Did community partners contribute as planned?
• Is there clear evidence for the specific problem to fix?

Action-plan template (one-page)

• Issue identified:
• Evidence (source and excerpt):
• Desired change:
• Action steps (who, what, by when):
• Success indicator(s):
• Resources needed:
• Monitoring plan (data sources and dates):

---

Moderation and quality assurance

• Internal moderation: subject teams moderate samples against rubrics; record adjustments.
• Cross-school moderation: where possible, share exemplars with cluster colleagues for calibration.
• Use video or artefact portfolios for moderation to build shared standards.
• Keep an audit trail: stamped/moderated samples, meeting minutes, and signed moderation forms.

---

Timeline for review

• During project: formative checks weekly; mid-project review (adjust scaffolding).
• End of project: summative evaluation and moderation within 2 weeks of completion.
• Post-project reflection: termly review to plan iteration and share practice with peers.
• Follow-up: monitor transfer of skills into subsequent term (select sample tasks).

---

Ethical and inclusive practice

• Obtain consent for recordings/photographs; anonymise when reporting.
• Use inclusive assessment practices and reasonable accommodations for learners with barriers.
• Treat learner feedback confidentially and address sensitive issues responsibly.

---

Summary: Effective evaluation merges CAPS-aligned summative evidence with formative records, learner voice and observation. Triangulate data, disaggregate by learner groups, moderate findings, and use a structured, incremental iteration cycle — test changes on a small scale, monitor targeted indicators, then institutionalise successful improvements. This approach ensures PBL remains rigorous, equitable and continuously improving within the Senior Phase context.