Questioning Systems of Deformative Assessment
In a series of new blog posts we want to shine a light on what it means to offer An Education for Global Citizenship. We will explore debates, emerging findings from the research literature and above all share how we at IAG are meeting the challenges and opportunities presented by both this the Fourth Industrial Revolution and the reality of an unknown future.
In this blog we want to argue that state maintained schools within England be brave, reject a purely GCSE driven curriculum with its deformative effects upon learning and adopt an approach which enables all to engage with the tenets of the IB Middle Years Programme. We argue that the MYP as a summative assessment framework provides the most effective means of providing an education for liberation.
Questions and themes relating to low and high stakes assessment practices and reforms are addressed throughout the chapters of Berry, R., & Adamson, B., Eds., (2011) Assessment Reform in Education, Dordrecht, Netherlands: Springer. Featured authors such as my colleague Eleanore Hargreaves (UCL IOE) and Mary James (University of Cambridge) address, respectively, issues concerning the role of teacher feedback within the assessment and learning process and the manner in which assessment is interpreted and applied with variance throughout the UK (with Scotland a lantern in the modern gloom of high stakes assessment).
The most striking article is that presented in Chapter 11 (pp.155-165) through which David Scott (UCL IOE) addresses the topic, Assessment Reform: High-Stakes testing and Knowing the Contents of Other Minds.
David uses Foucault (1979) as a means to enter into an exposition of the false beliefs and dualisms at the heart of high stake assessment beliefs and practices. David uses the often impenetrable philosophy of Critical Realism (of Roy Baskar and Margaret Archer) to explore assessment thus enabling overlooked aspects such as causality ‘backwash’.
David recognises the existence of two forms of knowledge; Ka and Kb. ‘Ka represents those knowledge sets, skills, and dispositional states of a person, collectively known as capacities.’ and ‘Kb represents those knowledge sets, skills and dispositional states which allow this person to do well in tests, and, in particular, high-stakes tests.’. A separation of knowledge enables David to explore how assessment favours, incorrectly, one (Kb) over the other and how the resultant assessment systems feedback to shape deformatively Ka knowledge.
If an education system introduces high-stakes testing, that is, testing in which there are significant rewards attached to success in the test both for the individual and the institution in which she works, then there are two consequences. The first is that Kb becomes the dominant form of knowledge in the curriculum and the second is that Ka over time is transformed so that it becomes more like Kb, that is, is has more of its characteristics. (p.155)
Furthermore, conflating these different forms of knowledge into one is common practice, evidenced by approaches taken globally to high-stakes testing (see PISA) and evident within the foundation of false beliefs of which many assessments are built upon.
Testers commonly conflate Ka and Kb, and in doing so make a number of false assumptions about knowledge and its assessment, with the consequence that these two forms of knowledge become indistinguishable in the minds of policy-makers, educational practitioners, students and other stakeholder. (ibid)
David identifies 8 false beliefs at the heart of high-stakes assessment. However we feel that these beliefs also rest within day-to-day assessment practices and as such impact upon attempts to use assessment to both measure and activate effective learning (for example widely applied AfL practices).
A person has a knowledge, skill or dispositional set, which is configured in a particular way (a grammar), and it is this configuration, perceived to be uniform across peoples, which is directly assessed when someone is tested. But the presented and observable answer is only the tip of a knowledge iceberg which itself could only be identified through logical inference and or retroductive analysis. This grammar is organised into elements, there are relations between those elements, and each element can be scaled, so a person may have more or less of that element, which can then be investigated/tested. But when the assessment only looks for a pre-determined element is fails to recognise that a number of elements may be called upon and enacted thus the assessment fails to recognise a broader and nuanced Ka knowledge. In the use of a knowledge-set, or in the performance of a skill, or in the application of a disposition within an assessment context, no internal transformation takes place.Testing a person’s knowledge, skills and aptitudes has no wash back effects on either Ka, the original knowledge construct, or Kb, the internally transformed knowledge set ready for testing. But Ka and Kb are both activated and both influence and shape each other in a transformative loop. Completing an assessment of Kb requires Ka which itself is transformed by interacting externally and internally with Kb (a process which can be aligned with thinking around Assessment as Learning). Micro (person level) and Macro (system level) ‘washback effects’ are well documented and observable. How often has teaching changed or the curriculum altered in response to a change in or the results of an assessment regime? Such an effect can be deformative altering an approach which then subverts Ka knowledge development. The process of testing works in a unidirectional linear fashion. But washback and an individuals place within a society recognises bidirectionality of all and every interaction. Thus a test of X at a single moment in time can not accurately record ones capacity for X from that point forward. Different types of knowledge can be tested using the same algorithmic process. Where do I begin with critiquing this state of affairs? The performance on the test represents to a greater or lesser extent what the testee can do or show, rather than there being a qualitative difference between the performance on the test and the construct, skill, or disposition of the testee. Capacity in terms of an original construct can not be assessed when the testee must alter that construct to conform to the requirements of the set test. reframing to fit the test rather than actually being tested on ones true capacity. Thus an assessment must recognise and reflect the individuals constructed knowledge and to do so over time. Real world learning and authentic assessment strategies seem wholly relevant here. A test can be constructed which is culture-free or free of those issues which disadvantage some types of learners at the expense of others. But of course all tests are shaped by and informed by a prevalent culture. Assessment systems do not exist within a closed system.The very fact that all summative tests in England are written and are written in English reflects a culture and by doing so disadvantages those why may possess, for example, a well formed mathematical capacity but not one for the complexities of exam paper English. In addition chosen references presented within exams may deviate significantly to the reference frames used by individuals to construct their Ka knowledge.
False belief 4 is extremely powerful as a causal mechanism shaping our world and can be linked back to both Foucault and Freire’s writings. It also enhances my own views about the purpose of education for Education and the goal of enabling a state of Liberated Learning.
Cognitive psychologists and test constructors implicitly adopt a computational or symbol-processing view of mind. Learning…,and the assessment of this learning…, are understood as inputting coded unambiguous information about the world, which is then sorted, stored, retrieved and managed in the same way that a computer processes data… [this is] an impoverished view of the role interpretation plays in learning. (p.160)
David highlights that the application of the above view to testing, assessment and its inevitable washback upon teaching and curriculum fosters a behaviourist approach to learning. In addition the viewpoint suggests a number of dualisms separating out language from reality, mind from body and the individual from society. The dualism between mind and body is of particular interest. Such a separation locates learning and cognition solely in the mind, with this mind physically separate from body, and by default separate from the social world around that body. Such a view sees learning as a passive process and as such assessment as a passive process of isolating and knowing the contents of anothers’ mind. But
Knowledge is not understood as a passive body of facts about the environment but as an interactive process of reconstructing meanings.
Thus assessments should enable interaction and reconstruction of knowledge rather than passive regurgitation. Again authentic assessment is that which is real world, problem passed and interactive assessment.
So where does this leave us? For us at IAG, what David offers is a means of looking again at assessment both in their summative and formative forms. What it also highlights is the deformative power of assessment systems and how they, built largely on incorrect assumptions, can have a ‘washback effect’ upon the structures of education, curriculum, pedagogy and ultimately on a learners experience of their own Education.
The reality is that students must adhere to systems of high-stakes summative assessments. Students sit exams at (arguably indiscriminate) points in their lives and we as informed educators and school designers can make a choice about what form this summative assessment takes. In England when students, aged 16, sit what is possibly the most significant of the high-stakes assessments, a ‘choice’ of sorts exists. Students can sit the iGCSE, BTEC and GCSE forms of assessment. The vast majority sit the GCSE, an assessment regime which is in perpetual flux of design, redesign, tweeks, changes and of questionable validity. Many schools are slaves to the GCSE, as measures of school effectiveness (Ebacc, Progress 8 et.,al.) are determined by student and school performance within this mode of assessment. Thus the tail wags the dog.
Innovative approaches to teaching, learning and curriculum design are needed and wanted by many within the system but while a ‘choice’ is made for us and a certain type of assessment imposed then the Kb knowledge needed to do well in this form of assessment, due to its weighted significance, will shape the Ka knowledge we are able to activate in the five years preceding this high-stakes performance. But we have a choice. Some high-stakes and summative assessment systems seem to have considered many of David’s observations. The International Baccalaureate in both its Middle Years Programme (assessed age 16/17) and the Diploma (assessed 18/19) frameworks offers an alternative assessment model which recognises an individuals Ka development over time and recognises the need to assess applied knowledge within real world contexts; authentic assessment. If such a system of assessment were to be adopted the ends would enable the means to be created empowering schools, teachers and learners to focus on Ka knowledge sets, skills, and dispositional states without the need for Kb.
The IB has been traditionally the preserve of private and international schools with some state schools replacing A Level assessments with the Diploma. We are suggesting that state maintained schools within England be brave, reject GCSE’s and their deformative effect upon learning and adopt the IB MYP as a summative assessment framework which provides a more effective means of providing an education for liberation. The IB is not perfect but what assessment framework is? Until we have greater autonomy, as is present in Scandinavian countries for example, to design and administer our own end of phase assessments, then we must look for a system which is more in line with effective learning and in line with the world we wish to shape today and tomorrow.
An Education for Global Citizenship – the IAG Curriculum
This is why we at the International Academy of Greenwich are developing a Curriculum which is highly personalised, authentic and actively promotes the importance of language. A Curriculum driven by valuable Ka rather than exam bound Kb knowledge. A Curriculum that enables all learners to develop as a Global Citizens and as a lifelong-lifewide learner. Through a model of cross-subject enquiries guided by essential real-world questions, learners will be presented with countless opportunities to engage deeply with the knowledge, attitudes and skills of a wide range of academic disciplines while seeing its direct relevance to the world around them.
Our choice of enquiries are designed around real-world issues, enabling students to make valuable connections between their studies and the world around them.Our unique Professional Pathways curriculum at Key Stage 4 seeks to prepare students for further study, the world of professional work and a lifetime of learning.Our approach is designed to enable all students to master individual subjects, to develop the ‘future-proof’ literacies, competencies and attributes of a ‘future-ready’ learner and Global Citizen.
The Curriculum is being designed to promote the development of not just cognitive, intrapersonal and interpersonal competencies but also to ensure academic success. To be mindful of false beliefs and to directly address them. Through this approach we seek to develop knowledge producers and not just knowledge consumers, thus we seek to build personal empowerment for all our learners. Literacy and Literacies, such as Mathematical, Cultural and Digital are the keys to academic progression in all subjects and we will work tirelessly to ensure all our pupils have a mastery of these skills. We recognise these crucial literacies to be:
Mathematical literacy (numeracy);
Digital (ICT) literacy;
and Cultural literacy.
Opportunities for the mastery of these literacies will exist discreetly across enquiries but will also be explicitly addressed across the structures of our Curriculum.
We are designing our Curriculum to ensure that students achieve high standards in core subject areas to facilitate success at key transition points (for example at age 13, 16 and 19). Our emerging ‘IAG Standards’ will be woven throughout the curriculum and alongside our curriculum design principles our curriculum will provide students with opportunities to make progress within the crucial knowledge, understanding, skills and attributes associated with a range of subject disciplines. In addition to the development of academic competencies our Curriculum will also enable learners to develop the attributes of a lifelong-lifewide and ‘future-ready’ learner. Such features, reflecting the International Baccalaureate Learner Profile, we call the Global Citizenship Profile.
Our Curriculum is being built in recognition of best practice within the domains of teaching, learning and education, by the ongoing work of both the World Economic Forum and the OECD, and most significantly by the rigorous framework of the International Baccalaureate Middle Years Programme (MYP).
We are adopting the MYP as a facilitating Curriculum framework, in recognition of the arguments above and because, as a framework, it:
encourages international-mindedness, starting with a foundation in their own language and culture;
promotes a positive attitude to learning by challenging students to solve problems, show creativity and resourcefulness, and participate actively in their communities at an ever increasing scale;
reflects real life by providing a framework that allows students to see the connections among the subjects themselves, and between the subjects and real issues;
supports the development of communication skills to encourage enquiry, understanding and language acquisition, and to allow student reflection and expression to shape an individual’s journey across their education;
emphasises, through the learner profile, the development of the Global Citizen – physically, intellectually, emotionally and ethically;
and is exciting, innovative, meeting students’ personalised needs, satisfying all National and many International Curriculum requirements and providing a supportive framework for all national qualifications and accreditation, including GCSE.