Skip to content

Why I hate Lab in a Lorry & the Einstein Ballet

July 18, 2012

Okay, hate is a strong word. But they do annoy me. More in a moment.

Earlier today @BenGoldacre tweeted: “i have a real problem with most state funded science engagement stuff in general. huge expense, small audiences, wrong approach. they all try to do commercial stuff, but without commercial success. can’t help noticing that outside state funded sci eng world there is (a) people like me and @robinince who make it (with hard ideas) work commercially and (b) blogs that get big audiences without state funding. i’ll stop now. but srsly, give me one thousandth of the cash that’s spent on science engagement and i would make shit happen.” [N.B. I have merged the tweets.]

I retweeted Ben and added that: “Institute of Physics: 2 mentions for least cost-effective science engagement ideas labinalorry.org.uk & the Einstein Ballet 2005. I’d be v interested to read blogs championing either of these bad ideas. In terms of wonderful science engagement, look at www.periodicvideos.com & Chemistry Top Trumps (chemists seem to do it better).”

So, please add comments below. In particular, how would you defend Lab in a Lorry and the Einstein Ballet?

Before you comment, here are a few thoughts:

1. Why did I pick on Lab in a Lorry and the Einstein ballet?
They involve massive costs. If you want to defend either of these please include a note of the total running cost. For example, for Lab in a Lorry, this might include IOP staff time back at base, lorry running costs, lorry lab staff time, PR cost, contributions from sponsors, money from regional development agencies and money from schools. There is also the initial cost of the lorry and kitting it out. Remember, even if most of the money came from sponsors, this is money that did not go to other science engagement projects. Also, in your defence of Lab in a Lorry, you will need to estimate the number of schools visited, the number of pupils engaged and for how long, and any wider impact. Also, I am not sure how a lab in a lorry is different from a lab in a school, except it is more cramped.
By the way, I think doing real experiments in schools is brilliant, but I think there are smarter, cheaper ways to deliver this than sticking a lab in the back of an HGV.

2. One way to think about this is to treat the “teacher” as the SI unit of science engagement. The science engagement activity (£ for £) should achieve more than a single teacher, who works with 100 students across a day for roughly 150 days a year.

3. Also, you could compare Lab in a Lorry with other methods of science engagement. I think some of the 6th form lecture days are cost-effective. 1,000 students hear 5 speakers in a large venue. However, I can see that this reaches only a specific audience (i.e., already semi-committed to science and willing to pay £10), but on the other hand I think that the events deliver something worthwhile. In any case, all engagement activities are specific in one way or another, so the key question is whether that specific audience gets something out of it at a reasonable cost.

4. For me, cost-effectiveness is the key thing, and innovation counts for very little. I would rather have a cheap, tried and tested idea that really delivers, rather than a contemporary dance production that costs £100k and which reaches very few and has very little impact.

5. Why did I pick periodicvideos.com as something to admire? Currently at 184 videos, which have been watched on YouTube over 20 million times. I don’t know the cost, but I am confident that it is a fraction of Lab in a Lorry. Even if the cost was on par with Lab in a Lorry, Periodic Videos wins on reach and longevity.

6. Why did I pick Top Trump chemistry cards? I was in a rush. However, at first sight it looked clever, geeky and cheap. And the product was being sold. If people are willing to pay for your product then it probably means that it is good (homeopathy excluded). And if it does not cost much in the first place, then that is good too. By contrast, I don’t think schools pay or are allowed to pay for Lab in the Lorry. If they did, it would be interesting to see how much they would pay.

Please add your thoughts below. In particular, let’s have an analysis of Lab in a Lorry. If it turns out to be brilliant and cost-effective, then I will be very happy.

On the other hand, if it turns out to be expensive and not cost-effective, then the funders need to think seriously about how they spend their money. I am sorry to have picked on Lab in a Lorry, because it is just one of dozens of such wasteful projects. On the other hand, lots of other wasteful projects is not an excuse for the Institute of Physics to be wasteful. Physics education is in crisis and throwing away money on big lorries is not helping.

Ps. Just about to dash to get a train to Coventry for Skeptics in the Pub. SitP is arguably one of the most cost-effective science engagement projects in the UK. Well done to Sid Rodrigues and everyone else who helped start and grow this network. The Institute of Physics could spend a small amount of money encouraging half a dozen physicists to speak at Skeptics in the Pub or it could offer some seed money to establish SitP in towns that don’t have them. On the other hand, SitP seems to do very well without any interference, so I am not sure if this is a good or bad idea.

Advertisements

From → Uncategorized

14 Comments
  1. AndrewHolding permalink

    Hi Simon,I can’t comment on some of the things you mention. However, I think the key thing is really social entrepreneurism, I hate the phrase but things should really be able to keep themselves afloat or be supported by smaller grants.Bright Club (Steve Cross’ comedy event) is brilliant at reaching new audiences and totally funded by the ticket prices. I wish though that there was more support for these sorts of things, I run SITP, Bright Club and trying to get SciBar off the ground in Cambridge, yet receive no real backing.I recently have been trying to get out of organising SITP in Cambridge, and response is that no-one is really willing to take up the role. It’s a sad truth I’m now in a situation I want to spend more time with my family and concentrate on other projects yet I can’t find any support to keep it going. If I was given the amount of money that spent on some of these projects I could take 1 day a week off to organise all these things for the rest of my life!I also can’t guarantee at the end of my current academic position I’ll even be based in Cambridge.

  2. Anonymous permalink

    It also depends where you are. What is not cost-effective in the South-East may be the only solution in the Highlands. See http://www.scifun.ed.ac.uk. It is still expensive, but getting folk north of Edinburgh is not always easy and it does engage the kids.

  3. davidmpyle permalink

    I don’t know about Lab in a Lorry, but student geologists in Cambridge have run a ‘Time Truck’ visiting local schools during National Science Week for at least the last decade. I don’t know about the success in terms of the audience, but it is a great way for undergraduate scientists to get involved in SciComm, and for some it has been the start of a route into public engagement or teaching. As for the costs – well, raised from sponsors (corporate, as well as charities) each year; again, great experience for the student organisers.

  4. Sarah Martin permalink

    I’m worried that the only two indicators you use as "impact" are "brilliant and cost-effective" versus "expensive and not cost-effective" – which is actually saying the same thing twice. LIAL and pubnights provide two very different services, and to very different age groups too. Comparing them on cost and attendance alone is simplistic and emphasizes quantity alone as more beneficial than quality. That’s a direction nobody in outreach and education should be forced to aspire to.The author states no competitive interest in either scheme. 🙂

  5. David McGloin permalink

    I’m not sure that comparing 11 million pages views on periodicvideos.com necessarily means it’s any better than a much smaller audience for Lab in a Lorry, for example. I’ve watched some of those videos and they are interesting, but I can’t remember which ones I’ve seen (so clearly didn’t learn anything!).I have not volunteered on Lab in a Lorry but I have some access to rough data. From the start of this year (for the Scottish Lab in a Lorry) there were, I believe, 7000 or so pupil visitors with a few hundred hours of volunteer time and a projector co-ordinator who gets paid ~??24k. I assume there is also a lorry driver to be paid and the initial truck outlay and petrol. So ?? for ?? more expensive than a single teacher, perhaps. However some of my students have been volunteers, as have many STEM Ambassadors, and the role they play in interacting with pupils and giving them info about careers and personal experiences is not something that lots of school kids get – so it’s valuable from this point of view too. Schools also get networking opportunities (see this blog by a volunteer from BAESystems, for example: http://networking.stemnet.org.uk/blog/lab-lorry-lakes-school-windermere-cumbria ). The scheme also provides, what I expect, may be one of the few interactions with physicists that many pupils receive, especially in remoter parts of Scotland, far removed from local Universities and physics related businesses.I do agree that maybe there should be more assessment of this type of scheme to assess it’s value. Lab in a Lorry has some evaluation from 2005 (http://www.labinalorry.org.uk/about_lab_in_a_lorry/evaluation_summary.cfm), but a more up to date one would be welcome.

  6. suziesheehy permalink

    Hi Simon, I wanted to chat with you about this stuff after the IoP Communicators group meeting but didn’t get a chance. What I think programs like Lab in a Lorry provide that many other fail on is a hands-on experience, the chance to actually interact directly with real scientists and the chance to ask questions (I seem to remember LIAL is mostly staffed by volunteer STEM ambassadors, but could be wrong). I’m not defending it because without knowing the cost and seeing the evaluation I don’t want to pass judgement.In my experience of sci comm with schools (over 10 years now) there is a big difference between this kind of hands-on experience and just seeing a talk or watching a video. I have no evidence to back this up but I really hope some of our sci comm colleagues might be able to find something which discriminates between an actual tactile experience and a ‘virtual’ one where the person just watches. I also agree with the person above whose argument about ‘hard to reach places’ is valid. Having spent years doing sci comm in Australia I understand that geographical isolation can be the biggest boundary to communicating science. That said, when I first moved to the UK I was really shocked at how much sci comm there was which cost a lot of money but didn’t do much – I remember commenting to someone that there was "quantity but not quality" – too many similar programs not working together etc. Anyway, that’s my two cents.

  7. amyplatypus permalink

    Measuring the impact of such initiatives is something that science engagement and education have struggled with, though many are seriously trying both through evaluation and published research (and maybe even randomly controlled trials).We could do better at sharing this so that we’re not just commenting on projects based on our own assumptions (that isn’t very scientific is it?) whether these are about what the project was aiming to do, how much it cost and the degree of success in its own or someone else’s terms. We do also need to invite criticism to keep us on our toes, and share failure when it happens. I should declare that I work for Wellcome Trust where we have the luxury of being able to fund projects on all sorts of scales for all sorts of people and with all sorts of particular aims. It is hard to compare them against each other, even when you have evaluation data to help, which we often do. But to me anyway, its clear that this should be about more than a cost-per-head calculation. We also look at the type and depth of engagement, whether we reached people we wouldn’t otherwise, (who might run a mile from a science-themed event, in a pub or elsewhere), and whether the scientists involved and the people running the project learnt something from the process too.

  8. Brady permalink

    Cheers for the encouragement Simon… we’re actually on 23.6 million official views (not counting multiple views when classrooms watch and downloads… that is just initial plays from YouTube)! The physics version (sixtysymbols) is past 11 million.And David McGloin… I went to a Lab in A Lorry and liked it, but I can’t remember what they did either. Not being able to recall exactly what you saw and when doesn’t mean you don’t learn!!! That’s not fair… I can’t remember what I did at school on which days and with which teachers, but I’m pretty sure I learned stuff! :)PS: When i went to the Lab in a Lorry, I reckon about 20 students used it all day… It may have been ultra-effective, but I reckon it cost a bomb too… I make no claims to be a world expert, but like Ben I reckon I could do better with a lorry’s worth of money!?

  9. Anonymous permalink

    Some important questions raised here, but let???s just start with the ???hate??? thing. Sad to see such an emotional and non-evidence-based starting point. ???Hate??? is an emotive word and a cheap and nasty shot, not made better by the semi-apology in the initial text. I hope it will be edited out of the title.But the substance is significant. We all want ???value for money??? but a big problem is how to judge ???value??? (actually even measuring money is not straightforward in many of these case; depends what one counts and how). Ultimately judging value depends on measures of impact, but impact on what, over what period? And indeed for what purpose or purposes? Is it to inculcate scientific knowledge, which seems to be the implied objective? And what does ???a single teacher, who works with 100 students across a day for roughly 150 days a year??? actually achieve by comparison? How on earth to measure all that impact in all its complexity?For example, what is the ???value??? of a direct conversation with a scientist compared to watching a video? I can imagine situations in which the former has an impact on the person several orders of magnitude greater than the latter (it might completely change their career aspirations), but equally the reverse in the right context. How to judge? How to put a (possibly monetary, for comparison) ???value??? on it?There are of course multiple, often overlapping, and sometimes conflicting purposes for public engagement (take a look at the original Science for All Report and the ???engagement triangle??? for a refresher if you want). I have seen one or two attempts to compare impact and value for money for engagement programmes, but they are not convincing at all to those who know the programmes.And then the point that ???innovation counts for very little???. From a scientist! Of course we should identify, nurture and support tried and tested approaches for different purposes, and we should evaluate to improve (as we do with all British Science Association programmes). But we have to innovate. Science and technology change. Social media and communication change. Society changes and our understanding of learning and engagement changes. Let???s have some balance.Finally, and I hold no brief for the IOP, what about the ballet? It???s a ballet, for goodness sake, not a piece of science education. I actually remember it and enjoyed it, and I???m not a ballet fan. In value for money terms an apt comparison would be with a production of Swan Lake at Covent Garden, and it would probably come out of that comparison quite well.

  10. Andy Lloyd permalink

    Hi Simon, while the desire to capture the impact of public engagement is laudable (and shared by pretty much everyone in the sector) I think there is a danger here of trying to compare apples and orang-utans. As others have hinted at, while the impact of Lab in a Lorry may seem poor value from the perspective of well-educated, scientifically interested adults in SE England I would suggest that the perspective of schoolchildren in the rural Highlands might be different. And unless you are proposing to replace LIAL with something else for the same audience that can achieve the same impact (however you choose to assess that) then I think this comparison makes for a very weak argument. I’m not even sure I would characterise Skeptics in the Pub as a science engagement activity – it’s more accurately a club for people with an existing shared interest who would arguably be engaged with the subject whether or not the specific events take place. Is that impact?A more worthwhile discussion might be why so much science communication, whether funded publicly, privately or not at all, depends on the goodwill and enthusiasm of people who are prepared and able to give up their spare time. What skills do we lose by excluding people with family and work committments because they can’t give up time? What segments of the population not represented? Are we devaluing the amount of effort required to create high quality engagement programmes? And do the grassroots programmes like Skeptics in the Pub and Bright Club, while providing a service for their audiences, run the risk of perpetuating this?

  11. JensenWarwick permalink

    Judgement about whether a particular science communication activity or event is effective should be made based on quality evaluation evidence, not attendance/viewing figures or speculation (as both can be highly misleading). Any form of communication can has the potential for both negative and positive impact (so high viewing figures of an ineffective or problematic video can simply mean higher levels of negative impact). For recent comments on the need for better quality evaluation in public engagement and science communication (first published in Nature, the second in the British Science Association publication People and Science), see:http://warwick.academia.edu/EricJensen/Papers/402817/Evaluate_Impact_of_Communicationhttp://www.britishscienceassociation.org/NR/rdonlyres/46A434CE-6039-4331-AD74-FB4AD5409EE0/0/SoundingoffandWatercooler.pdf

  12. Richard Ellam permalink

    Several Points:1. Simon cites two projects as being ones he particularly ‘hates’. The ballet came and went in 2005 without much notice. Lab in a Lorry started in 2005 and is still going. This suggests that ‘LiL’ is considered a success by its funders, as they are still happy to go on paying for it. The public stayed away from the Einstein Ballet in their millions which is why it isn’t opening tomorrow on Broadway. We are really talking chalk and cheese here.2. In any activity where there are lots of players there will be a spread of results. Simon is a reasonably successful author – most authors are not even reasonably successful, so its not surprising that there will always be some science communication or science engagement activities that flop, sometimes expensively. Its reasonable to ask why the flops happen, to avoid repeating the mistakes, but its not reasonable to pick on a couple of things that you happen not to like and use them as evidence that the whole enterprise they represent is foolish or unproductive. 3. Simon suggests that a reasonable criterion for judging the cost effectiveness of an enterprise like Lab in a Lorry is that it should deliver the goods no more expensively per head than a teacher could in a school. I’ve never worked on Lab in a Lorry and I don’t know the details, but I think there are plausible reasons to suggest that LiL might actually meet this criterion, once you understand what the true costs on both sides actually are.Lets start with the experiments: they are not things that you do as part of ordinary science teaching, and most of them use specialist apparatus that isn’t in school prep rooms, and can’t be used to deliver the core curriculum. If schools were to try and replicate the LiL experiments, they’d either have to invest in expensive kit which would come out of hibernation once or twice a year, or there’d have to be some kind of lending library of equipment that they could borrow or hire to do the job. Neither option is cost free.Then there’s the business of meeting the volunteers who run the lab sessions. These people are not schoolteachers, they are ‘proper’ scientists and engineers and they meet the kids in small groups and have a chance to talk to them not just about the experiments but about life, the universe, and football. So you’re not just doing some experiments you are meeting s different people, people who perhaps challenge your ideas about what scientists and engineers are like. Again, you could stage this kind of work in school, by bringing in the volunteers and the equipment and setting up a temporary lab in (say) the school hall, but you’re starting to disrupt the school day, and there are hidden costs that go with this.OK, so how about forgetting the traveling aspect, and making the activities avaliable in a science centre, or a lab attached to a university or whatever?First, you’ve got to provide the space, and that costs – quite possibly the cost per square meter of lab space in a building is more than the equivalent cost in a lorry trailer. Second the kids have to travel, with teachers to the activity. This costs big time – a coach plus driver these days will be at least ??500 a day, and then you need to pay for cover for all the lessons the staff on the trip can’t teach back at base, and there’s the inevitable disruption to the rest of the school caused by the absence of teaching staff. Remember Lil is for 11-14 year olds, not primary school kids. Primary school trips are easier to run because taking a class out of school is much less disruptive.Plus, if you want to have a fixed lab then it can only serve a limited area, and if your ambition is to go nationwide then you need a network of labs which ratchets up the costs… On this perspective LiL starts to look positively cheap. OK it can only reach a tiny number of students because there’s only one of it, and it can’t take a whole class at once, but these are not of themselves arguments for declaring it to be a futile activity. Indeed Simon says he favours the idea of funders supporting the wider roll-out of proven activities rather than ‘novelty’. As LiL has been going since 2005 this could be an argument for increasing its funding to allow multiple Labs in Lorries to be run, provided of course that proper and rigourous evaluation proves that it delivers…

  13. bosprowal permalink

    Apropos of cost-effectiveness, I taught what might have been my best lesson all year a couple of days ago. It was about what happens when you boil water. Beakers, measuring cylinders, Bunsen burners and thermometers. It was their last lesson of the year, which usually has the soundtrack of "Can’t we do something fun/watch a film?", but there were no whinges whatsoever. I was amazed!(year 8 science lesson, large comprehensive in Cornwall)Good spot on the top trumps, and thanks for the links to those videos.You know http://www.ptable.com has links to those Nott Uni videos, right? By far the best online periodic table IMHO.

  14. drpeterrodgers permalink

    Lab in a Lorry and Constant Speed (the ???Einstein ballet???) both featured in Einstein Year, a year of activities in 2005 that was intended ???To enthuse young people, and those who influence them, about physics whilst building a sustainable increase in public awareness of physics and its role in society??? [http://www.einsteinyear.org/index.html]Einstein Year was later evaluated by researchers from the Institute of Education, and they produced a 96-page report [www.iop.org/activity/outreach/activity/file_39565.pdf]. This report, as I recall, is more qualitative than quantitative and it does not look at the cost effectiveness of Einstein Year or any elements of it. (One could read several things into this ??? that the IOP did not want to know about the cost effectiveness (for whatever reason), or that it is very difficult to measure the cost effectiveness of outreach activities)I don???t know much about Lab in a Lorry but I recall thinking at the time that Constant Speed was a brave and ambitious thing for the IOP (actually IOP Publishing) to support. It was also, I recall, intended to reach audiences who were not interested in physics, notably young girls. It was also more than some performances at Sadler???s Wells ??? it toured the country, including workshops and talks at schools. PR was also a large part of the motivation for the project and, if I recall correctly, it got quite a lot of media coverage.Disclosure: I am a member of the IOP and worked for Physics World, the IOP magazine, between 1990 and 2005. However, the comments above are my own.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: