How do you measure the impact of a work of art?

How do you measure the impact of a work of art?

12.11.2014 - 11:34

- Notes from Nordic Culture Forum 2014 in Stockholm

The Nordic Council of Ministers commissioned Nordic Culture Point to organise Nordic Culture Forum in Stockholm during the 2014 Session of the Nordic Council. The aim was to discuss ways of measuring the impact of cultural policies and why it is important to do so. 

Most people have felt the effect of a work of art, either intellectually or emotionally. Few can be in any doubt that art and culture have an impact upon us. The question is how to demonstrate and measure this impact on the individual and society. 

But why bother? Most people agree that art and culture have intrinsic value. Can’t we just enjoy them? The reality is that – just as in any other sector – a return on investment has to be shown. Otherwise, the cultural sector would find itself in a weak position compared to other sectors when its budgets are negotiated. 

In the Council of Ministers’ Strategy for Cultural Co-operation 2013–2020, the culture ministers express a desire for more research and knowledge on which to base and implement Nordic cultural policies. 

Nordic Culture Point has been working on this theme for some time. It organised Culture Forum, held during the Session of the Nordic Council in Stockholm, to discuss the issue with a range of experts from various countries who spoke about their research and experience.


People develop through culture
Illugi Gunnarsson, Icelandic Minister for Education, Science and Culture, and chair of the Nordic Council of Ministers for Culture in 2014, opened the event by saying that culture is an aid to personal development. He also stressed that culture plays a highly important role in Nordic co-operation – indeed, culture has helped make it what it is today. “But measuring the impact of culture is a challenge, and that is why we have arranged this forum today,” he said. 

“Culture can lose out quite easily when we try to define its impact. Other sectors are often able to make a more compelling case for funding. My main hope is that this forum will help improve our knowledge and put us in a stronger negotiating position, as we will be better able to explain the effects of culture.”

per stor kult

Why a culture forum?
Per Voetmann, director of Nordic Culture Point, elaborated on the background to the forum. All over the Nordic Region, the sector faces a range of challenges that recur again and again. The Swedish Agency for Cultural Policy Analysis has listed some of those that broadly apply in all of the Nordic countries:

• Culture-policy objectives look more like vision statements. If the targets are too specific, then there is a risk of them being meaningless in a wider perspective. Targets that focus on what is measurable don’t always reflect what it is most important to achieve. 
• What gets measured gets done – but not everything that is relevant is measurable. Anything that can’t be readily expressed in a meaningful quantitative or qualitative way risks being overlooked. 
• It’s difficult to establish causal relationships or define the impact of a particular policy. Cause and effect in culture policy are not easily identifiable. 
The agency also noted the relative lack of data in the sector. Since all of the Nordic countries face similar challenges, it makes sense to look for joint solutions. This was the ambition behind the new strategy adopted by the ministers in 2012. 

“We want to leave here with greater knowledge and feeling inspired. That way, our future work will have a solid and well-founded basis. We’d also very much like to take with us tangible proposals that we can follow up, both individually and at Nordic level.”

Big Data as cultural indicator
“Big Data” has been a hot topic in information technology in recent years. Since more or less all of our movements and actions leave electronic footprints, it is now possible to use the vast amounts of data amassed to track and predict patterns. What does a woman who doesn’t yet know she is pregnant buy in a supermarket that other women don’t? Big Data makes it possible to identify very specific patterns of great commercial interest. But is it possible to use Big Data to measure the impact of art and culture? 

The example of the pregnant woman was cited by Anthony Lilley, CEO of Magic Lantern and Professor of Creative Industries at the University of Ulster, in his presentation “Counting what counts: Towards data-led decision-making”.

According to Lilley, the first challenge is that, when it comes to the culture sector, Big Data is something that Google has but the cultural institutions don’t. A good example is Google’s “Flu Trends” – a concept based on predicting an influenza epidemic by monitoring how many people are looking up flu symptoms. 

“Social media may be the way forward for cultural institutions. People increasingly use social media to express opinions, and it’s conceivable that they could be used to quantify the impact of cultural activities. But the challenge is to mine data that can be used as the basis for political decisions.”

Research into cultural policy is a space for reflection 
The Nordic Council of Ministers commissioned Erik Henningsen and Heidi Stavrum of the Telemark Research Institute to produce the report Kulturpolitikkforskning i Norden – En kunnskapsoversikt (Nordic culture policy research – a knowledge overview). The report is based on surveys of cultural organisations across the Region, and therefore has a Nordic rather than national perspective. 

Henningsen believes that culture policy plays a part in democracy, but that it needs to be more transparent to people outside the sector. He said that more people need to be drawn into the debate about culture, and that this debate needs to be widened and promoted by initiatives designed to meet culture-policy targets.

According to Henningsen, Nordic culture policies haven’t achieved their declared goals, but things could be different if there was a wider debate on policy. “Culture-policy research is a space for reflection rather than an instrument. Research can help us devise policies that learn from their own mistakes,” he said.

He also pointed out that policy research tends to be commissioned, but stressed that it still has to observe certain professional standards. It does so in part by being cumulative and not spreading itself too thinly. This may be possible at Nordic rather than just national level, as echoed in one of the report’s main conclusions – that Nordic partnerships can provide a boost to national culture policies. 
In conclusion, Henningsen proposed a research programme and challenged Nordic cultural politicians to take the idea seriously, signing off with the words: 

“If you really mean it when you say you want a knowledge-based policy, then this is the way to do it!”

Co-ordinating statistics
The Nordic Council of Ministers asked Christian Nissen of NORDICOM to look at options for co-ordinating Nordic culture statistics. 

He opened his presentation by pointing out the vast distance between impact reports and the data on which they’re based. We have to travel all the way from data (statistics) to indicators of knowledge (identifying indicators) to action (the indicators’ causal contexts) before any conclusions can be drawn about impact. This places heavy demands on statistics – the supply chain is very long and the three groups of people who work with and use the statistics have different needs. Cultural statisticians often find that their work has low priority and is poorly funded; cultural researchers work in isolated environments and only receive ad-hoc funding; and civil servants generally only require cultural statistics in specific, often acute situations. 

Nissen argued that the specification and production of statistics is predicated on continuous, politically based demand. He also questioned whether co-ordination is even realistic given that it has never happened before. Nonetheless, the growing interest in quantification of impact has put the issue back on the agenda, making co-ordination a relevant issue once again. 

Nissen proposed that a Nordic research unit linked to an existing institution should be used to co-ordinate culture statistics. 

“We need to get the ball back into play – or punt it into very long grass!”

Study: Significant impact of art and culture in England
Richard Russell is Policy and Research Director at the Arts Council England – the national development agency for art, museums and libraries. The Arts Council’s mission statement is “Great art and culture for everyone”, and earlier this year it published the report The value of arts and culture to people and society. 

The report studies the impact of art on the economy, health, welfare, society and education at national level in England. In general, it paints a picture of art and culture playing a highly significant role in the sectors studied. For example, 32% of all tourists in 2011 engaged with art and culture, generating revenue of £7.6 billion. The study also revealed that people who actively participate in the arts and cultural life enjoy better health, perform better in education and participate more actively in democracy and in their local communities. 

In his presentation, Russell said the report shows very clearly how investment in culture contributes to the economy, society, education and public health, but that we still don’t know how involvement with art and culture affects us as individuals. 

He also noted the need to define the return on investment that we expect in art and culture, concluding that both the sector and the researchers have significant gaps in their knowledge in this area.

What use is Open Data?
Sanna Marttila of Open Knowledge Finland spoke about the “Open Data” concept – i.e. that data should be available to all, without exception. Open Data from various sources could also be used to analyse the culture sector. According to Marttila, if we make data available to the public, there are some who fear it could be misused. However, the research conducted in this area has found no evidence to suggest that Open Data might present serious risks. 

Marttila used, which reveals how public spending is divided up in the UK, as an example of open data. If the same degree of openness were applied to the Nordic countries, it would help provide a clearer picture of the cultural sector’s contribution. Marttila pointed out, for example, that specific artworks could be more closely linked to foundations in order to make their impact more visible. 

“Open Data is nothing in itself – we make it meaningful.”

Communicating the tangible results of abstract objectives
Eva Englund, former programme advisor at Nordic Culture Point, spoke briefly about our own work. Nordic Culture Point’s job is to render visible the results and outcomes of projects it funds on behalf of the Nordic Council of Ministers. We use a range of materials and methods to learn more about the projects’ impact, e.g. contacting recipients and experts, reading reports and issuing questionnaires.

Eva explained that a 2013 questionnaire completed by funding recipients in a variety of spheres showed the impact of the projects based on a number of parameters:

• Nordic projects can attract greater media attention than national projects, particularly outside the Nordic Region
• New partners trigger new idioms, content and forms of communication
• New partners from other countries encourage new local partners to emerge
• Face-to-face meetings are key: contacts and networks lead to productive meetings, knowledge and inspiration.

Nordic Culture Point also found that a high proportion of funding recipients agreed that their projects reached a new audience, helped develop their genres and generated results that could be used even after the project came to an end.

“We need to listen and look at everything through the lens of results.”


Beyond reception
Martha Fleming, director of the Collections-Based Research Programme at the University of Reading, argued in her presentation that we need a more complex method of evaluation to demonstrate that cultural experiences make a difference. We need to look backwards, forwards and within ourselves, go beyond studying public opinion or statistics, and give culture producers the same status as researchers. 

When quantifying the impact of culture, we mustn’t overlook the artistic process. It’s important not to just look at the audience, as there is no reception without production. “You can have dancers without an audience but you can’t have an audience without the dancers,” she explained.

One of her main points was that there is often too great a focus on the audience and too little on the practitioners. In looking at the artistic process, we must also remember that the artists are a kind of audience, too. It would be a big mistake not to look at what artists actually do when producing their art. According to Fleming, the work of an artist is just like the work of the researcher. 

“Evidence isn’t what we lack, it’s analysis. But for that we would need a holistic, long-term view of the whole of the artistic landscape,” she said. 

“We need to look in every direction to see how much we already know.”


Panel debate: What do politicians and funding bodies expect?
The panel for the closing debate consisted of Anna Enemark (senior advisor, Nordic Council of Ministers), Eva Englund (former programme advisor at Nordic Culture Point), Martha Fleming (director of the Collections-Based Research Programme, University of Reading) and Peter Johnsson (the Nordic Council’s Culture, Education and Training Committee). Mikael Schultz (Swedish Ministry of Culture and a member of the Nordic Committee of Senior Officials for Culture) acted as chair.

The debate opened with the statement that culture is part of welfare – it’s culture that changes instruments of policy, not vice versa. It’s up to politicians to facilitate culture, not direct it. The cultural sector has trouble knowing what politicians want, since they don’t express it clearly. Equally, politicians need to know what they are talking about in order to make policy decisions. They need input, in the form of evidence, emotions and responses. We should therefore make better use of artists’ and practitioners’ skills and make sure that their feedback is put to good use. 

One contributor pointed out that politicians hope to get re-elected and that this can be used to influence them. There is also a misconception in political circles that cultural activity will continue no matter what the politicians do – and this may well be the case, but not in a positive manner if it doesn’t get the necessary political understanding and support. 

Later in the debate, the need was identified for specific mechanisms to show politicians how funding is spent and illustrate the results achieved by arts and culture. Big Data is an example of one such specific tool. It was suggested that the results of the Nordic-Baltic Mobility Programme for Culture should be presented to politicians. 

The Nordic Council of Ministers rounded off the debate by looking forward to the focus on knowledge and by saying that it will, along with Christian Nissen and Erik Henningsen, follow up on the idea of a Nordic cultural research environment. Regarding this, it was remarked that the idea of an institution working on these issues was only possible in the Nordic countries because of Nordic co-operation.

Closing remarks by Jesper Hermansen 
Summarising the day’s input, Jesper Hermansen, head of the Danish Ministry of Culture, said we now know where we stand and what challenges we face. He stressed that there is a real desire on the part of the Council of Ministers to see a more knowledge-based culture policy, and that the Danish Presidency will initiate debate on working towards this. 

He concluded by saying that Christian Nissen’s report will form the basis for a recommendation to the Council of Ministers in early 2015. After that, possibly following the mid-point evaluation of the Council of Ministers’ strategy for cultural co-operation, consideration can be given to incorporating data and knowledge into a revision of the strategy.

Here you can download some of the presentations from the Culture Forum: