Monday, 27 August 2012

Kirsty Wier talks with us about using the Ako Aotearoa Impact Evaluation Framework

Mike and I met with Kirsty, Research Manager for Ako Aotearo, last Wednesday to discuss her experiences using their Impact Evaluation Framework. Kirsty had just come from spending several days at OLT in Sydney where she was working with Tilly and others to refine and develop the framework for the Australian Higher Education context.

Since our meeting with Tilly earlier in the week, this meeting provided a great chance to better understand how the Framework is currently used in practice and what are some key things to consider if we do choose to adopt it or elements of it in our evaluation process.

Some key characteristics/outcomes of the use of the Framework:
  • Provides legitimacy for project teams to actively influence others
  • Keeps the project focus on the learner
  • Provides opportunities for structured reflections (impact journal)
  • Enables funding body to see where intervention is needed or would be of value
  • Data collected can inform future funding models and requirements through developing better understandings of what constitutes a quality project
  • Expands the scope of relationships between funding agency and project teams
  • Using the framework is time consuming and it can be difficult where the project has had little impact which has not been directly related to the quality of the project

An important part of our discussion centred on answering the following questions about the impact on learning and teaching outlined in the Evaluation Framework. This was due to the Network’s distance or large degree of separation from the experience of learners and consequently, about our concern in addressing such questions.

Kirsty explained that whilst there may be quite a distance between project activities and the learning experience in many projects, such at NATA, it is important to attempt to answer this question to the best of the project's ability. For, in addressing this question it forces the project to really clarify how it fits within the learning and teaching sphere where the learner experience as being critical. This could be done through mapping the different steps by which the project's activities hope to influence the learner experience or if feasible, reflecting with teachers about changes in their teaching or reviewing student surveys for example. However, Kirsty did make it clear that it is important to keep true to the project and not lose sight of the nature and objectives of the project itself. 

In reflecting upon this question we felt that whilst quite abstracted from the learner experience, we could map how we felt the project aims to influence change in learning and teaching . Moreover, we felt we could ask association members in the research survey how they felt their involvement in professional associations may have influenced their teaching. How what we are doing seeks to impact learner outcomes could also be a key question for the executive focus groups and leader interviews. Going through these processes would also help us keep the learner in mind whilst keeping true to our project.

Ako Aotearoa currently uses the framework to assess projects as they progress 6 months, one year and two years post completion. However, they are now looking at trying to embed this framework in upcoming projects so that it helps project teams identify what impact would look like and develop strategies to achieve that at earlier stages in the project life-cycle. Thus, by adopting such an approach ourselves, we may be able to provide real experiences and feedback that aids in the design of such an approach.

Wednesday, 22 August 2012

Follow us on twitter...

We now have a twitter account where we seek to share key trends and ideas relevant to the Tertiary education today and into the future @NATAonthenet

Tuesday, 21 August 2012

Opportunities for Network Members to help with new Impact Evaluation Process being piloted by OLT

Yesterday, Mike and I had a productive meeting with Tilly Hinton from OLT to discuss OLT’s new pilot of the Ako Aotearoa Impact Evaluation Framework. Tilly has been seconded to refine and implement the pilot of the Impact Evaluation approach developed by Ako Aotearoa in the Australian context. Initially, all grants projects which have been completed for 6 months, 1 year or 2 years by October this year, will be invited to participate in the program. Whilst this includes 63 projects, funding for the pilot is limited so only as many as can be resourced will actually be involved.

Participating projects will be sent a pre-filled version of the framework for additional comment and detail two weeks prior to a discussion between project members, Tilly (or alternative OLT representative) as well as a discipline expert drawn from the Networking Projects. These conversations will run for 1 to 2 hours and be done predominantly in person, although other approaches will be trialled to meet the needs of diverse project teams. The first round of conversations will occur between November and December 2012, with the second half conducted in the January to March period in 2013.

This approach aims to build relationships between the OLT (funding body) and the project teams, provide a guide to the impact of the provision of funding and help facilitate cross-project benchmarking. Whilst this tends to support the idea of a ‘one best recipe’ approach to developing grants, the importance of funding more experimental and diverse projects was also recognised as being of high value.

As part of Tilly’s secondment, she is also conducting a range of workshops related to the Good Practice Reports from November 2012 to March 2013. This offers us a great opportunity to draw upon Tilly’s experience in hosting these workshops when designing our own dissemination webinars. In addition, it provides an opportunity to cross-promote the workshops with a captive audience.

Whilst NATA will not be directly involved in the pilot, Tilly has very generously agreed to attend our Face-to-Face meeting in October to engage in discussions of evaluation and how we may adapt the framework in our evaluation and reflective practices.

As mentioned above, Tilly is looking for Network members to participate in the 1-2hr conversations and it would be greatly appreciated if you may be in contact with her if you are interested in being involved, indicating your areas of expertise and interest.

Kind Regards, Natasha

Monday, 20 August 2012

The challenges and opportunities for professional societies in higher education in Australasia: A PEST analysis

Click here to read Doherty, Steel & Parrish's article exploring the challenges and opportunities faced by professional societies in Australasian higher education.

Monday, 6 August 2012


Three new HERDSA Guides were launched at the Hobart Conference in early July, which can be ordered online through the HERDSA website

1. Peer Observation Partnerships in Higher Education (2nd edition) (2012) by Maureen Bell

 In this Second Edition of her popular HERDSA Guide to peer observation
 of teaching Maureen Bell provides a comprehensive and practical guide
 for three different models of Peer Observation Partnerships:
 self-directed; guided; and coordinated. This is a practical guide with
 a scholarly base and is written to support colleagues working together
 informally for their own professional development; educational
 developers supporting partnerships and Deans and Heads of Department
 implementing faculty or departmental programs.

2. Using Stories in Teaching (2012) by Frances Miley, Amy Griffin, Barbara Cram, Robert Kennelly, Coralie McCormack, Andrew Read

 Using Stories in Teaching is a scholarly and practical guide to assist
 teachers in higher education. The authors outline the benefits of
 storytelling and how it fits within the broader category of narrative.
 The Guide covers practical aspects of using stories in teaching
 including where to find stories, how to incorporate storytelling into
 teaching and which types of stories might be suitable for different teaching purposes.

3. Effective Feedback for Student Learning in Higher Education (2012) by Iris Vardi

 Ensuring your students get good quality feedback that they can use is
 one of the most powerful ways to truly make a difference to their
 learning and satisfaction. This HERDSA Guide will show you how to plan
 for and provide feedback to students in a time effective way that helps
 them improve their learning and performance. Based on the latest
 research and models of feedback, this Guide is full of practical
 suggestions, insights and techniques.  It begins by examining the role
 that feedback plays in in the educational environment and then provides
 an overview and synthesis of the literature, providing practical lists
 on what improves and impedes student performance and confidence.


We have recently added a variety of resources to our Delicious site on MOOCs. Views supporting and criticising this increasingly popular form of 'education' are presented.

Also watch "Disruptive Innovation in Higher Education" a TED talk that highlights the importance of asking the right questions in order to reform Higher Education into the Future.