Quality Metrics

There's a very good piece by the tremendous Charlotte Higgins responding the the Arts Council's recent suggestion that they will start to use 'quality metrics' in their evaluation of the work they fund. That is, they will use quantitative methods to judge the artistic success of all the poetry, music, theatre, and more than is a recipient of arts funding.

I have a small dog in this fight, in that I have been working with the British Theatre Consortium and UKTheatre/SOLT in generating a lot of quantitative data about British theatre (reports here and here), in part for the Arts Council. It's very revealing about the work being done with arts funding. But nowhere does it say anything about artistic quality. I don't think it means that a show is good just because it had a huge box office or lots of people went to see it. I doesn't mean a show is bad or good if it was written by a writer from a particular democratic. 

This is because facts do not entail values. In other words, I don't think you can logically deduce a statement of value from a statement (or a series of statements) of fact. For instance, 'Harold Shipman killed more than 200 people' does not, of itself, entail 'Harold Shipman was a bad man'. You need some other value premise to come in between ('killing people is wrong' for example). And that's a value statement. 

A judgment of aesthetic quality is another kind of value statement. No matter how many statements of fact you generate about a piece of theatre, it will not tell you whether it's good or not. Even if you believe that there are 'rules' about what makes a good play (which I sort of don't), those rules will only ever be necessary; they won't be sufficient. In other words, let's say, for the sake of argument, that a good play must be a complete representation of the arc of a single action (as Aristotle suggests), with a beginning, middle and end, the three parts connected by strict necessity; if that's true that means a good play cannot be written which does not conform to that rule; but it does not mean that if the play conforms to that rule, it will definitely be good. It's crucial that it has that aspect, but it is not enough. 

And we know this because, obviously, if everyone knew how to guarantee a good piece of theatre, they'd do it all the time, but we've all been to see terrible theatre so it can't just be a matter of rules. And - even more obvious - no one agrees what those rules are.

So no matter how many facts you amass about a show, you won't know how good it is. And that means that quality metrics are bound to fail as an evaluative method in itself.

Charlotte Higgins's piece is a lovely overview of these argument with some good points of its own. I was briefly phone interviewed about this and I have a comment towards the end, though, in truth, my view is better summed up in the words of Tim Etchells right at the top. Quality metrics is 'horseshit'.

You can read it here: https://www.theguardian.com/culture/2016/oct/04/quality-metrics-arts-council-england-funding