It’s a more difficult question than you might think.
There’s a maxim in the education world that only subjects that are tested are funded. Thus an imperative for arts education champions to get the arts included in required standardized tests. In a STEM world, the arts don’t exist.
But how do you make standardized tests for the arts? Multiple choice questions might measure knowledge but do they measure creativity or arts skill? And when you ask students to make art, things get sticky when you try to grade with a number. Famously:
“when the National Assessment of Education Progress, or NAEP, included an arts test in 1997, it required students to create real works of art in addition to answering standard multiple-choice questions. (That year’s test famously led to semitractor-trailers full of student-created clay bunnies; since then, efforts have been made to digitize work in photos or videos.)”
We live in a culture that seems to feel that unless we can empirically measure something, it’s not worth much. And it’s not just enough to measure, the measure must be translatable into a standardized number of some sort. Sarah Butrymowisz reports in The Atlantic that:
Coming up with a uniform and efficient way to measure a subject that’s all about creativity is difficult. In its arts tests, Florida has incorporated multiple-choice and short-answer questions that are easy to score efficiently. New Hampshire and Michigan are trying something more ambitious: devising tasks that require a student to submit a finished piece of artwork or perform a piece of music. These tests are time-intensive to administer and grade, however, and the results are difficult to translate into a single numeric score.
The push to find the best way to test the arts is coming from arts educators themselves in many instances. They hope to foster not only student improvement, but also a sense that the arts are as valuable to curriculum and society as such long-tested subjects as math and reading.
“It’s very important for arts to be seen as a subject that can be and should be tested,” said Frank Philip, an arts-assessment consultant. “It’s a parity thing.”
Got it. The arts want to be considered as legit as science or math or reading. So the search for a measure that means something. But maybe it doesn’t exist.
We tend to measure success in the creative world by how many copies of a book were sold, how many music tracks were bought, how many tickets of a play or concert were purchased. These sales numbers ultimately define the “worth” of the art (and before anyone writes in to point out the difference between the value of art and entertainment, I want to add that the arts also measure success by how many – and who – pays attention, so it’s a popularity score too, just on a different scale).
We’re undergoing a realignment of measurement right now. Popularity isn’t worth what it used to be (if it ever was). If a video can get a billion views on YouTube, what’s that worth? A billion is an astonishing number, but only if it produces some tangible return. Once upon a time, to sell a million records meant something because it translated into cash and fame. Your career as an artist likely was unsustainable if you couldn’t find an audience to sell to.
Today a billion views might not result in much financial return, and fame has been so devalued its worth is suspect too. Over on Slipped Disc this week, Norman Lebrecht dinged classical performers who had a lot of Facebook likes (relatively speaking) and had seemed to have bought those likes. So if you appear to have lots of fans, you must be popular, and that translates into what exactly? Web companies have been struggling for some time to define “engagement” rather than raw audience since page views and clicks have become so devalued as to be almost meaningless. The arts are all about engagement. But measuring it? No one has yet figured out a standardized number to quantify a person’s engagement with a Beethoven symphony. Now, is that a problem or does it say something something fundamentally important about the nature of art?
This question is something I’ve been considering for a while now. We seem biased to admit only quantifiable measures, and maybe this is related to how commodified many of the arts have become, but what this further emphasizes is the passive experience of art from the outside rather than actual creativity and personal engagement. From inside the art or engagement process the idea of flow and other psychological and aesthetic qualities become important. Not only are they resistant to quantification they are themselves measures from a different scale. This is a similar type confusion to the one I described in that essay on intrinsic value. Means and ends are both values, but each has its own peculiar function.
It might be useful to consider the Sorites paradoxes to help untangle why quantifying the arts is problematic. The Sorites paradoxes are types of confusion where we have two sets of measure that can be applied to the same thing but which fail to negotiate on each other’s terms.
For instance, you can count the number of hairs on a head and you can judge a person bald. If you took away one hair from a person with a full mane you would not make them bald. Another won’t do the trick, and you can keep going. At some point the person will obviously qualify as bald, but you can’t measure with precision which hair removed did the trick.
The paradox is that there is no such thing as a precisely quantified division between bald and not-bald. For all the counting we can do the precise quantification escapes us. And yet we are not confused calling a person bald or not regardless of whether we have accurately counted their hair. The problem is that we are using two separate measures, one that counts hair and one that makes a broad visual judgment, and while they seem to refer to the same thing they are measuring on significantly different and only apparently overlapping scales.
Our problem in measuring the arts is that all the really interesting things have nothing to do with quantification. Its the qualitative stuff that gets us going. And even when there are precise qualitative measures, not all art is defined by its precision. Some art, in fact, is best known for its groundbreaking exploration. Some art aims at specific qualities and others see how far you can go in particular directions. Some art is simply serendipitous and improvisational. The rules for judging them may not have been written yet. How do you measure that? By what right do you measure that? Sometimes the exercise of art consists in finding or establishing that new source of measure. The art itself is the measure, not something that can or should be measured by other things…….
Our problem is that we expect the arts to be organized and behave like other more quantifiable practices. We are simply admitting a preference for quantification. But until we can make a case for qualitative measures, and even resisting the need to measure, the arts will be compromised. Quantification is a poor measure of the arts any way you slice it. Its like judging a painting by how wide the canvass is. Its like judging a piece of music by how loud it gets played. Its like judging a sculpture by how much it weighs. You can measure a sunset, but lumens will not add up to the experience of beauty anymore than that one extra hair will stop me from being bald.
I must be missing something. The arts in education are not particularly difficult to test. Music students can be tested on their ability to play scales, master a certain set of pieces with graduated difficulty, identify pieces in the standard repertoire, identify biographical aspects of famous composers lives, etc. Similar tests can be offered in other performing and visual arts. Engagement with the arts can be measured in tickets sold, the amount of arts funding available, recordings sold, etc.
Data and tests can always be abused, but a society that mistrusts art and the artists, and a society that under supports them, will naturally not want to face the implications objective data about the arts provides. We refuse to see what we don’t want to see. And besides, in the hyper-utilitarian world of America’s unmitigated capitalism, if it doesn’t make a buck, it can’t be worth anything. The dollar is the only true measure of anything. Just ask Ronald Reagan and Milton Friedman.
William: You’re right – scales, brush skill, monologues – all can be measured. But we’re talking about standardized tests that are taken by millions of students and whose answers have to be empirical and easily graded. That’s why most standardized tests consist of multiple choice answers so they can be machine-graded. Even having students writing essays for these tests is problematic.
But these standardized tests don’t really measure independent creativity or many other useful life skills. Nor do they measure creativity. It would be possible to have students perform or draw or dance, but this kind of testing doesn’t “scale” very well. Fortunately, some colleges are starting to abandon standardized test scores as an entrance requirement. But there are still those who want to allocate funding to schools based on how well they teach to the tests. And when the arts aren’t in included in standardized tests, then teachers and schools pass over the arts to concentrate on the subjects on which they (the teachers) will be graded.
Thank you. I now better understand your point. Testing knowledgeability and technique in the arts is one thing, testing creativity is another. Why assume that testing in arts subjects would need to measure creativity – especially given that there are no objective measures to begin with? I assume the goal would be to measure the basic knowledge necessary for the students to have the tools to eventually become creative. The means is tested, not the ends.
Even in the so-called core subjects, creativity is not measured. Students might be measured on their knowledge of the basics of algebra, geometry, and calculus, but not expected to provide solutions for the Hodge conjecture or the Riemann hypothesis.
Many aspects of arts knowledge could be measured by machine graded multiple choice tests. The British have used standard tests nationally for their music students for decades – it’s called the Associated Board of the Royal Schools of Music. They define ear training and theory skills, establish repertoires to be mastered for various grade levels, etc. One can check out their tests and approaches here:
http://us.abrsm.org/en/our-exams/
In the States we’ve also have a system, though we don’t call it standardized testing. We measure statewide comparative accomplishment through All-State Band programs, and through district band competitions, and district solo and ensemble festivals. In states like Texas and Iowa, schools go nuts to win the competitions and to put as many students as possible in the All-State bands and orchestras. There ain’t much creativity involved, but these programs provide good measures of accomplishment and really motivate students.
I dislike the idea of standardized tests and recognize the vast problems they have created, but when we see the stats for illiteracy in the USA, we try just about anything in desperation to solve the inadequacy of our schools. In Cleveland, for example, Seeds of Literacy found that 67% of the people were functionally illiterate – defined as not reading at a fourth grade level. Perhaps the problem is that our standardized tests are not well designed, and their application too simplistic. And that they excluded the arts from core subjects. Happily, better minds than mine are working on these problems.
William: Agreed. When I was a music student in Canada we had national tests by the Royal Conservatory of Toronto. Actually, those annual tests were very helpful – something to work for, aspire to. But it’s interesting to me that at a time when standardized tests are coming under fire for their lack of ability to measure mastery or predict success that we’re obsessing about getting arts included in the standardized test regime. Totally understand reasons, but a flawed system into which we’re trying to be admitted.
The pressure to make the arts amenable to standardized testing puts us in a position the arts were not necessarily designed to handle. There’s all sorts of stuff we CAN measure, but are those the right things? Knowledgeability and technique are surely important at some stage and in some respects, perhaps more so in specific disciplines, but are they universally informative? Are we testing merely for the sake of testing? We need to test, and so whatever can be quantified gets lumped into the process? Are we putting the test before the thing tested? Are we prioritizing things that are testable simply because they CAN be tested?
Tests are one way of asking questions, and there are good questions and bad questions. There are relevant questions and questions that have very little to do with our interests. Often, even a great answer to the wrong question is less worth pursuing than even a tentative answer to the right question. Bad questions can lead us into sterile territory. Bad questions can be misleading. Bad questions are sometimes even worse than not having asked them in the first place. We pursue answers to bad questions at the expense of answers to good questions. Does cramming for the test help us or lead us astray?
So a question is, and maybe this is a good question, are the respects in which the arts can be tested for important for the arts or is the fact of being testable more important to us? Are we trying to fit the arts into a world that does not share our values, or are we trying to make our values more understood and acceptable to a wider audience? Is it incumbent on the arts to change to fit some other part of society, or is it necessary to change how the arts are perceived in themselves?
We need to ask the right questions, better questions at least, and I get the sense that any questions covering the quantitative testability of the arts is seriously biased against, perhaps, some of what the arts ought to promote as worthy. That seems like a question worth asking, at least.
I guess standardized testing for the arts is a strategy to get the arts back into the core curriculum that uses those tests. The American concept of locally funded and administered school systems has failed many children due largely to the economic inequalities between neighborhoods. Standardized testing tries to level out this inequality without addressing the cause which is mainly poverty, and worse, a poverty that is often a legacy of racism.
Two points: the premise, and the promise. The “premise:” As a professional evaluator of programs that feature learning in or through the arts, I have been puzzled why over the past 40 years there has been almost no questioning of the premise regarding the place of the arts in schools and the presence or absence of standardized testing in the arts. I am particularly skeptical about the premise; schools (leaders, parents, teachers) that value the arts as important aspects of the curriculum usually do so because they KNOW that the arts are an important component in developing young people’s skills AND sensibilities. They KNOW that the arts when well taught and where students find their voices for considering political, social, psychological and aesthetic issues, create the foundation for a highly developed society. Advocates of arts education have repeatedly tied the fortune of the arts with the opportunities that funding (if you can test it, we will fund it) and politicing. There is virtually no push back from those who in their hearts know that the so called arts assessments do not cause school principals to go beyond the minimum requirements. In private conversations, I have found acknowledgement of this sad state of affairs. Going where the money is in the minds of most advocates seems to be the basic strategy for keeping the arts alive in schools and local communities. Re “promise:” A policy that truly stimulates a deeply rooted respect for the arts as an essential part of every community probably requires a DNA transplant into the “ambition gene” of our political and social leaders. Is there a solution to this conundrum? I tend to doubt it, and like many of my fine colleagues, I evaluate and dig up evidence of how the arts positively influence the individual and society on a micro level and hope that my reports and published books help wider swaths of audience understand. Cultivating the arts through education continues to be successful where the the creative arts are taught well, not just where test results “count.”
Maybe this is a short term/long term issue. We seem to find plenty of money for new buildings and roads but fail to invest in the upkeep of our existing infrastructure. We find it easier to raise money to build concert halls than the funds to make the programming vibrant. There seem to be endless studies on the benefits of the arts when well taught. But those benefits are, if you will, infrastructural – they contribute to a good foundation but take years to manifest themselves, and even then, often in diffuse ways.. But our mindset in education seems to be – as in so many things – the short term impact rather than taking the longer view.
The way we test reading, for example, doesn’t measure critical thinking skills. It’s easier to test pieces – vocabulary and “comprehension” – so we do. But that doesn’t mean we’ve taught someone how to think critically. I get the need for minimum standards and developing basic tools. But as you said so well: “Cultivating the arts through education continues to be successful where the the creative arts are taught well, not just where test results ‘count’.”