AERA professional Development Training and Extended Courses Proposal

Evaluating Quality in Doing and Writing Action Research in Schools, Neighbourhoods and Communities

Abstract

This interactive course provides opportunities for beginning and experienced action researchers, studying and teaching on higher degree programmes, to explore issues of quality and validity in designing, doing, and writing action research. Specifically, it focuses on assessing quality in the three related areas of what counts as (a) quality practice; (b) quality research; (c) a quality report. Participants will explore the kinds of values-based criteria and standards of judgement appropriate for judging quality in action research and the validity of research claims, and how these can be communicated effectively through written and multimedia reports. The course itself will take the form of doing action research, as participants ask, How do I~we improve my~our practice?' (where ~' indicates transformational potential), and show the dynamic relationships between their values and research-based practices. Using a range of data gathering techniques, including video, to communicate the experience of the course, they will be able to test the validity of their claims to achieving quality in practice-based research, and explain the transformational potentials of their living theories of educational influence for sustainable local and global wellbeing. Participants will be encouraged to explore how the most advanced social theories of the day can be integrated within the living theories of individuals, working collectively.

Proposal

Rationale for the course

The idea of self-study action research is well established globally as a powerful means of professional education, with potential transformational influence for sustainable communities of practice at school, neighbourhood and global levels. What is not so well established are the means of making judgements about the quality of the research and the practice in which it is grounded, or of the accounts produced to offer descriptions and explanations for the research, in the form of the living theories that practitioners produce as they research their practices. Establishing this quality has important implications for claiming validity for the research claims, which can be further strengthened through the production of high quality research accounts. The course therefore focuses on how quality can be judged in practice-based research and the reports it generates. The course will explore the potentials of transforming the values that underpin the practice into relationally-dynamic standards of judgement that can be used to assess quality and the validity of research claims and research accounts. It will also explore ideas to do with individuals' transformational educational influences through the production of their living theories of educational practice as their contributions to a critically-oriented public knowledge base (Snow 2001).

Goals

This proposal is for a one-day course that will enable 50 participants, working collaboratively, to:

· explore the potentials of asking critical questions about their practice as they ask, How do I~we improve my~our practice?' (Whitehead 1989);

· produce their own living theories of practice to show and explain how they have done so;

· engage in their action enquiries, through their experience of the course, and test their emerging claims to knowledge against the critical feedback of other course participants who are similarly engaged;

· explore ideas to do with their potentials for educational influence in their own learning, and the learning of colleagues, with transformational implications for the education of the social formations in which they live and work;

· investigate what counts as a quality report, as they produce their individual and collaborative accounts of the experience of the course, and submit them to the critical evaluation of others, focusing especially on the procedures they have developed to enable them to claim quality for their practice and research, validity for their research claims, and communicability for their research accounts;

· use the internet to bring their learning back to their own contexts, and to develop neighbourhood and global networks of educational influence.

Learning outcomes

By the end of the course, therefore, participants will be able to:

• design and conduct a systematic and methodologically rigorous action enquiry;
• explain how they will judge the quality of the practice-base of the enquiry;
• explain how they will judge the quality of the research;
• explain their understanding of the relationships of influence between themselves and their communities;
• know how to produce a report that will be judged as of educational quality;
• know how to use their learning from this course in the development of communities of enquiry in their own contexts.

Theoretical frameworks

The organization for this course is grounded in key theoretical frameworks. Overarching frameworks draw on the work of Whitehead (1989) and Whitehead & McNiff (2006), about the living nature of educational enquiry; the work of McNiff (2007) about the generative transformational nature of educational relationships; and the work of Biesta (2006) on educational responsibility. These broad frameworks embed secondary frameworks, which are themselves in a dynamic transformational relationship in the generation of living educational theories:

• ideas about the immanent and genetically-endowed capacity for human growth, drawing on the generative transformational work of Chomsky (1986);
• ideas about the interrelationship of all things, drawing on the transformational evolutionary work of Bateson (1972), and the ideas of Buber (1937), Fromm (1956) and Tillich (1973), as well as on the environmental philosophy of Zimmerman et al. (2001);
• ideas of the inevitable capacity for human influence, drawing on the ideas of Foucault (1980) and Said (1994);
• ideas about the capacity of humans for making choices about the exercise of their influence, drawing on the work of Berlin (1998);
• ideas about how the production of living educational theories can influence the education of social formations (Whitehead, 2004) for sustainable global wellbeing.

Procedures

The course will take the form of an interactive workshop, with group and pair work, personal and small group presentations, mini-lectures, mini-writing and reading sessions, demonstrations using video, role-play, and performance. Participants will be invited to engage interactively throughout the day in investigating their understanding of their practice as they explore the potentials of conducting an action enquiry, that focuses on the production of their claim to know their practice as they ask, How do I~we improve what I~we are doing?' Especially they will focus on demonstrating and testing the validity of their claims to knowledge, through the production of evidence whose quality they can test on the spot against the critical feedback of their peers, acting as critical friends and validation groups. They will experiment with different forms of data gathering, including video recording, and negotiate values-based standards for generating evidence and judging its quality in relation to their knowledge claims. They will also experiment with different forms of reporting, including the production of video narratives, and justify their choice of form of representation and its content as demonstrating communicative adequacy. They will test the quality of their research and their form of communication through mini-presentations to other participants acting as validation groups and peer referees.

Assessing participants' prerequisite skills and knowledge

Participants need a working knowledge of the principles and practices of action research. They need to be committed to engaging in participative forms of enquiry, a willingness to experiment with innovative forms of thinking, an openness to the risk of making their experimental thinking public and testing their provisional claims to knowledge, and a general capacity for open forms of thinking and the development of new epistemologies. The action reflection procedure we use includes a process of adaptation to different backgrounds and needs. We have experience of responding to groups that include beginning researchers, individuals and groups on masters enquiries, and individuals and groups exploring doctoral research programmes.

Materials

Course materials will be made available in the form of

• a course booklet (see below)
• a notebook to be used as a reflective diary
• handouts and worksheets
• books, including relevant excerpts from the authors' latest book Doing and Writing Your Action Research (London, Sage Publications)
• writing materials
• large charts for recording impressions and decisions
• tape recorders for data gathering
• video recorders for videotaping elements of the workshop, for instant replay and possible subsequent modification
• video recorder and digital projector (live access to the internet) for showing currently available resources

Information and materials will be made available on the websites www.actionresearch.net and www.jeanmcniff.com in advance of the course, and potential participants alerted through communication via e-lists and established self-study action research networks.

Course syllabus

The course syllabus will provide a systematic framework for the day, as participants work through the following critical questions:

• What is my~our concern?
• Why am I~are we concerned?
• How do I~we gather data to show the situation as it is and as it unfolds?
• What can I~we do about it (what options are available)? What will I~we do about it (what action is planned)?
• How do I~we ensure that any conclusions I~we come to are reasonably fair and accurate?
• How do I~we account for my~our educational influences in learning?
• How do I~we modify my~our ideas and practices in the light of the evaluation?

Activities and timings for the day

Activities will be organised according to the plan for the day (see Appendix 1). We will prepare a course booklet for participants, which will include the day's action plan. We place the action plan in this course outline for comprehensiveness, and to show the kind of participant-friendly materials we will distribute at the beginning of the course.

Instructional staff

The course will be organized and conducted by Jean McNiff and Jack Whitehead. A summary of their capacity and expertise follows.

Jean McNiff

Jean is an independent researcher, working with schools, agencies, and higher education institutions in international contexts. She has wide practical experience of conducting workshops and seminars in how to do and evaluate action research, as well as in teaching creative academic writing for publication. She has worked in Canada, Iceland, India, Ireland, Israel, Northern Ireland, Palestine, the People's Republic of China, Italy, South Africa, and the United Kingdom. She is currently active in Ireland, where she supports the workplace enquiries of teachers working with the National Centre for Guidance in Education, and the doctoral enquiries of practitioners at the University of Limerick, where she is adjunct professor; in Iceland, where she works with higher education personnel in supporting teacher professional education programmes for higher degree accreditation; in South Africa, where she supports the masters degree programmes of thirteen teachers in a township near Cape Town, as well as supporting the action enquiries of a group of ten higher education practitioners at the Nelson Mandela Metropolitan University (where she is a research associate) as they support the higher degree studies of teachers; and in the United Kingdom, where she is a part-time professor of educational research at St Mary's University College, supporting the masters and doctoral enquiries of ten members of staff. These practical activities are in turn supported by her writings. She produces her own educational writing that offers a systematic account of her theorizing as she seeks systematically to improve her work and learning, as well as text books, written collaboratively with Jack Whitehead. These textbooks are used widely on higher degree courses around the world, and have proven influential in disseminating ideas and establishing the legitimacy of self-study forms of educational action research for demonstrating accountability in personal and social practices. She is active in delivering workshops and lectures around the world.

Jack Whitehead

Jack's educational research programme at the University of Bath has focused on the development of the living educational theories of individuals and the development of appropriate action research methods for enquiries of the kind, How do I improve what I am doing?' His Presidential Address to the British Educational Research Association was on research-based professionalism in education and this is consistent with the form and content of the workshop. He has organised similar workshops in Japan, China, Ireland, Canada, the US, the UK and in South Africa. He pioneered the use of visual narratives in research degrees at the University of Bath. He is a former distinguished scholar in residence of Westminster College Utah and a visiting professor at China's Experimental Centre for Educational Action Research in Foreign Language Teaching at Ningxia Teachers University. His award winning web-site http://www.actionresearch.net is an international resource for practitioner-researchers. It includes examples of the beginners, masters and doctoral accounts he has supervised at the University of Bath. He is the convenor of the 2007-8 British Educational Research Association Practitioner-Researcher SIG e-seminar on What standards of judgement do we use in evaluating the quality of the educational knowledge and educational theories we are creating as practitioner-researchers? He is a founder member of the AERA Self-Study of Teacher Education Practices SIG and of the BERA Practitioner-researcher SIG.

References

Bateson, G. (1972) Steps to an Ecology of Mind. New York, Dutton.

Berlin, I. (1998) The Proper Study of Mankind: an Anthology of Essays. London, Pimlico.

Biesta, G. J. J. (2006) Beyond Learning; Democratic Education for a Human Future. Boulder, Paradigm Publishers.

Buber, M. (1937) I and Thou. Edinburgh, Clark.

Chomsky, N. (1986) Knowledge of Language: Its Origin, Nature and Use. New York, Praeger.

Foucault, M. (1980) Truth and Power' in C. Gordon (ed.) Power/Knowledge: Selected Interviews and Other Writings, 1972–1977. Brighton, Harvester.

Fromm, E. (1956) The Art of Loving. New York, Harper & Row.

McNiff, J. (2007) My story is my living educational theory' in D.J. Clandinin (ed.) Handbook of Narrative Inquiry: Mapping a Methodology. Thousand Oaks, Sage, pp 308–329.

Said, E. (1994) Beginning: Intention and Method. London, Granta.

Snow, C. (2001) Knowing what we know: children, teachers, researchers' in Educational Researcher, 30 (7): 3–9. Presidential Address to the American Educational Research Association Annual Meeting, Seattle.

Tillich, P. (1973) The Courage To Be. London; Fontana.

Whitehead, J. (1989) Creating a Living Educational Theory from Questions of the Kind, "How do I improve my practice?"', Cambridge Journal of Education, 19 (1): 137–53.

Whitehead, J. (2004) What counts as Evidence in the Self-Studies of Teacher Education Practices?' in J. J. Loughran, M. L. Hamilton, V. K. LaBoskey and T. Russell (eds) International Handbook of Self-Study of Teaching and Teacher Education Practices. Dordrecht, Kluwer Academic Publishers.

Whitehead, J. and McNiff, J. (2006) Action Research: Living Theory. London, Sage.

Zimmerman, M. E., Callicott, J. B., Sessions, G., Warren, K. H. and Clark, J. (2001) Environmental Philosophy: from Animal Rights to Radical Ecology (third edition). Upper Saddle River, NJ, Prentice-Hall.

Appendix 1 Plan for the day

Part 1 (Morning) 8.30am–12.30am

Part I of the workshop will focus on how to do action research to generate new claims to knowledge, and making decisions about judging the quality of those claims and validating them, prior to putting them in the public domain.

Part II (Afternoon) 1.30pm–4.30pm

Part II will focus on making decisions about how to write reports about those claims, and how to judge the quality of those reports.

PART I Evaluating quality in educational action research

8am–8.30am Introduction to the day

Welcome!

What the course is about

Activity

Please write down in your reflective diary what you hope to get out of this workshop. Tell your neighbour. Say why it is important that you get this. Report back to the whole group.

Think about this

Evaluating something means making a judgement about its worth. We make prospective judgements about the quality of a coat before we purchase it, and retrospective judgements about the quality of a television programme when we have watched it. These judgements enable us to decide whether or not we will commit to the thing, i.e. purchase the coat, or comment favourably on the programme.

We ask:

• How worthwhile is this?
• How valuable is it?
• What do I value about it?

8.30am–9.00am Exercise

• Form pairs.
• Think of a significant event that you have been involved in during recent times. Tell your colleagues what you valued about it.
• Imagine you are writing a report, and want to communicate the same ideas using the words, My key values in this event were ...' You might use words such as honesty', integrity', trust', and kindness'. Write these values in your reflective diary.
• Report back to the whole group.

9.00am–10.30am Doing action research (Refer to Worksheet 1)

For this exercise, think of the values involved in action research. This means engaging in the experience of doing action research.

Please do the following:

• Break into groups of four or five. Think of something on your desk that you have to attend to. Ignore the other hundred things there, and think of just this one, which is of pressing concern.
• Say what the issue, or the concern, is.
• Say why it is an issue or concern. What values underpin the concern? Are you living them or denying them in your practice?
• Find a way of showing the concern to your colleagues. For example, draw a picture to show the concern. Use your reflective diary for this.
• Ask your colleagues for advice about what you can do. Choose one option for action.
• Imagine that you have implemented the action. Go forward in time.
• Now draw another picture to show the situation as it has changed.
• Say how you will assess whether or not the situation has improved. Have you been able to live your values more fully in your practice?

Please respond in your reflective diary to these questions:

• What is the most significant thing I have done so far?
• What is the most significant thing I have learnt so far?
• What is important about my learning?

10.30am–11.00am (BREAK)

11.00am Reconvene

11.00am–11.45pm Group discussion/presentations

Now tell the group the story of the research you have just done. Explain to your small group how you can make judgements about (1) the quality of the practice, (2) the quality of the research. Use these questions as prompts:

• Did the practice show an improvement? How do you make judgements about any improvement in practice you may have observed?
• Did the practice show an improvement in learning? How do you make judgements about the quality of the learning?
• How can you show how your learning has influenced someone else's learning? How do you make judgements about the quality of that influence?

Think about this

• We have identified our values as what guides our practices (our standards of practice), so we can think of evaluating the quality of the practice in terms of the extent to which those values were realised. Did we realise our values of, for example, honesty, independent enquiry, efficient timekeeping, courtesy to others?

• We have identified our values as what guides our research (our standards of judgement), so we can think of evaluating the quality of the research in terms of the extent to which those values were realised. Did we realise our values of, for example, methodological rigour, systematic enquiry, collaborative working, validating a claim to knowledge through the production of authenticated evidence?

• Can we discern that our values come to act as our living standards of practice and judgement as they emerge in our practice?

• To what extent do the values of practice and the values of research overlap?

We shall ask for two individuals to talk to the whole group about the research project they have just undertaken. The whole group will act as a validation panel, to make judgements about the quality of the practice, the research, and the account.

11.45am–12.30pm Whole group discussion

Explaining the significance of what we have done

Points for discussion

• Would you say that this exercise was worthwhile? Why? What have we learnt?

• What might be the significance of this kind of learning for continuing professional teacher development? Can we offer new ways of evaluating teacher performance, in the context of the continuing professional education of teachers?

• If we expect teachers to evaluate their own practices, what does that say about what we need to do as higher education practitioners?

12.30pm–1.30pm BREAK FOR LUNCH

Part II Writing quality in educational action research

1.30pm Reconvene

1.30pm–2.15pm Reflective activity

Please remind your neighbour of what you did in Part I, and what you learned.

Please write a brief report on what you did and what you learned (about half a side of office paper). Please use your reflective diary to do this. Use the following points to guide your writing.

Action research reports are distinguished mainly by the following features

• They are written for a reader. The author takes the responsibility for not assuming that the reader already knows what the author is thinking. The author leads the reader carefully through the account.

• They are explanatory research reports, that is, they give the reasons for undertaking the research, as well as the goals for the research. They do not offer only descriptions, although descriptions of actions and learning are a key part of the report.

• Because they are research reports, they articulate a claim to knowledge. This means first identifying and articulating a research question, showing how data was gathered and how evidence was generated by searching the data for instances that show how the research question is addressed. Evidence is then extracted from the data, and is used as the grounds against which to test the validity of the claim to knowledge.

• They explain the processes of the research, in relation to the criteria for doing research: does it show methodological rigour, systematic enquiry, the initial identification of a research issue and the generation of evidence; i.e. do they show the processes of establishing the validity of the research claim?

• They adopt a narrative form, in which the I' tells the research story. The story is generative and transformational: each part unfolds into the next part, and each part contains the other parts within itself.

• They demonstrate communicative adequacy, that is, they write for a reader. Therefore they communicate ideas in ways that are comprehensible, honest, sincere and appropriate to the social and cultural context.

• They talk explicitly about the generation of knowledge, and they show the significance of the practice and the research for new forms of practice and new forms of theory. Therefore they speak about the potential implications of the research for how research can inform practice, and practice can inform research. They articulate the significance of this approach for continuing professional teacher development.

Please discuss the ideas above with your neighbour. Do you agree with what you have read? What further points can you add?

2.15pm–3.00pm Exercise

Now, please read Worksheet 2. Working in twos, tell you neighbour which of the two accounts is, in your opinion, a better quality research report. Why? What standards have you used to judge the quality of the report?

How do you judge the quality of the communicative adequacy of the report? Think about Habermas's (1976) four criteria for judging communicative validity. He says that a speech (or writing) act needs to demonstrate the following qualities:

• It is understandable, that is, the writer must find a way of communicating their ideas so that the writer and the reader can understand each other.
• It must be sincere, so the writer needs to communicate the story in a way that the reader can share the knowledge of the writer.
• It must be truthful, so that the reader can believe the writer.
• It must be communicated appropriately, that is, it must written in a way that takes into account the socio-cultural circumstances of the present situation, so that both writer and reader can see that the report makes sense within an existing context.

Does the report you are reading stand up to these validity tests?

Six pairs of volunteers to report back to the whole group.

3.00pm–3.30pm Break

3.30pm–4.00pm Whole group discussion

Some dilemmas and problematics

Can we consider these issues:

• What kind of pedagogical practices are appropriate to supporting quality research writing?
• What kind of editorial contexts are we working in? Will they support the publication and dissemination of action research reports?
• How can we influence the situation regarding publication? Catherine Snow (2001) spoke about creating a knowledge base of works by teachers for teachers. How would we judge the quality of this knowledge base? What kind of reports would qualify?
• What else?

4.00pm–4.30pm Evaluation: self evaluation, course evaluation

Evaluating the quality of your research-based practice during this workshop

Please tell your neighbour:

• What did you do during the workshop?
• What did you learn during the workshop?
• How do you evaluate the quality of your practice during the workshop?
• How do you evaluate the quality of your research during the workshop?
• What are the links between your practice and your research?
• So, what claims to knowledge are you prepared to make now?
• What kind of evidence can you produce to show how those claims have been come about?
• How will do you test the validity of those claims?
• How will you use the knowledge you have generated during this workshop?

Conclusion

Where do we go from here? How will we ensure that we remain in touch as a support group? How will we disseminate our accounts of practice?

Farewell!

Worksheet 1

An Action Research Planner

• What is my concern?
• Why am I concerned?
• What experiences can I describe to show why I am concerned?
• What can I do about it?
• What will I do about it?
• What kind of data will I gather to show the situation as it unfolds?
• How do I ensure that any conclusions I come to are reasonably fair and accurate?
• How do I explain the significance of my research?
• How do I modify my concerns, ideas and practices in the light of my evaluation?

Worksheet 2

Account 1

I began my research project in April 2007, and continued until September 2007. I was investigating how I could raise examinations scores in maths in my primary school. I decided to work with ten children from Class AB who were achieving the lowest scores in progress tests.

I gave them a pre-test to see what their level of ability was. Their scores indicated that they needed additional help in areas of basic numeracy, so I set about teaching a programme that would enable them to learn key skills. I focused primarily on the areas of addition, subtraction, multiplication and division. The programme lasted for six weeks, after which I gave them a post-test. The post-test showed that their scores had improved. I therefore decided to continue with the programme, and set periodic tests.

Account 2

I began my research project in April 2007, and continued until September 2007. I was investigating how I could find ways of raising examinations scores in maths in my primary school. Mathematical literacy is a curriculum priority area, along with reading literacy, both of which are key skills in enabling primary school children to make a head start in education, and I needed to find ways of helping children to improve their capacity in these areas. I decided to work with ten children from Class AB who were achieving the lowest scores in progress tests because I believe that it is my responsibility to ensure that all children achieve their maximum educational potential in school. I wanted to find ways of enabling them to achieve to the best of their ability.

I gave them a pre-test to see what their level of ability was. Their scores indicated that they needed additional help in areas of basic numeracy, so I set about teaching a programme that would enable them to learn key skills. Becoming proficient in key skills would, I reasoned, give them a basis for the development of more refined skills as their learning progressed. I focused primarily on the areas of addition, subtraction, multiplication and division, basic computation skills that are essential for mathematical literacy. The programme lasted for six weeks, after which I gave them a post-test, to see whether they had improved their level of attainment. The post-test showed that their scores had improved, so I reasoned that my intervention was successful in achieving my pedagogical values. I therefore decided to continue with the programme, and set periodic tests to assess learning progress.