Skip to content

Why I am mildly perturbed by Lab in a Lorry

July 19, 2012

Good to see lots of comments after my blog “ Why I hate Lab in a Lorry & the Einstein Ballet“.

Perhaps I should start by making two further points.

1. My main criticism is with funders, not with those who actually conduct the projects. Lab in a Lorry might be conducted with enthusiasm and skill by highly motivated and knowledgeable people (including volunteers), but if the running costs of the project are astronomical then it remains a poor project. The problem resides not with those running the project on the ground, but those who commissioned it and who continue to fund it from on high. Similarly, I am not blaming sponsors, who are trying to support science with hard cash or resources. I am blaming those who spend that money and use those resources. As I have said before, I might be wrong, in which case I am keen to see the analysis, formal or informal.

2. There are good science engagement projects. Richard Wiseman’s videos cost virtually nothing, yet they are witty and thought-provoking. Take a look at Top Ten Quirky Science Tricks for Parties, which has over 5 million hits. Or listen to the Pod Delusion, which reaches 10,000 people every week. The audience includes many non-scientists, because of the podcast’s mix of content. It’s run on a shoestring budget and would probably benefit from some support. I helped start the Undergraduate Ambassadors Scheme, a credited undergraduate degree module that currently operates in over 100 departments around the UK and which sends over 1,000 undergraduates into schools each year. At the moment, as far as I know, it runs on about £5,000 per year. It does require lecturers to devote time to it, but they are involved because the module helps them achieve an important goal, namely graduates with transferable skills. I am also closely involved in the Enigma Project, which takes my Enigma to schools in order to explore maths and its links with history, technology and, of course, codes. It ran over 100 events last year, half of which were full day workshops, and it reached 12,000 students … and it operates at a profit, including the loan fee for the Enigma. I am also impressed with the various activities at the maths and computing departments at Queen Mary, University of London. Anything involving Matt Parker seems to be priceless, yet relatively cheap. Maths Inspirations is also worth a mention. It lacks anything innovative, flashy or stylish, but it delivers inspiring maths lectures to students in large numbers. It mimics the sixth form study day format, which is known to be profitable and appreciated by teachers, but it makes the format more affordable and transportable, so that the lectures range from Kent to Durham. It baffles me why learned institutions have not jumped on this idea and created Physics Inspirations or Chemistry Inspirations or Science Inspirations.

I am still hoping for a response from the people behind Lab in a Lorry or the Institute of Physics. I would be delighted to receive a robust defence of the project, which showed that is actually excellent value for money. I am often grumpy and critical, but I prefer being happy and impressed.

In the meantime, here is my best effort at addressing as many as possible of the existing comments on the blog in 30 minutes.
(Apologies in advance for typos, errors, omissions, etc. I am trying to answer every point quickly and briefly, rather than provide a long, structured response to one particular point.)

Rolandjackson (Chief Executive of the British Science Association)

You criticise my use of the word ‘hate’.

Happy to change it. I have put in a holding phrase. Let me know the most appropriate word or phrase and I will update it.

The word was partly put there to generate interest, and it was partly put there because it is not far from how I feel.

Physics education in the UK is in a terrible state (is the word ‘terrible’ ok? Probably not. Sorry) and it makes me sad and angry when I think money is being wasted when it could be spent on trying to fix the problem.

You raise the problem of metrics

I agree that it is hard to measure the impact of a project and even to assess its real cost. However, if the goal is clear (e.g., improve take up of A level by A and A* GCSE students or increase positive attitudes towards science among parents), then it should be possible to assess projects.

In particular, if £1 million (fair guess?) has been spent on Lab in a Lorry, then the organisers ought to be able to give some idea of the impact, particularly if it is being run year after year after year.

I have sat in meetings where we have discussed the assessment of Lab in a Lorry and other engagement projects, and my recollection is that the level of scrutiny is poor. There seems to be an understanding that nobody is allowed to say anything too critical, because after all people are doing their best.

If absolute measurement is hard, then organisations should at least be able to rank their projects in relative terms, so that top projects can be expanded and bottom projects abandoned or revamped.

One approach would be to have an independent panel assess science engagement projects. Experienced and successful science communicators have a good nose for what works and what does not. This is far from ideal, but I think it would filter out the worst projects sooner rather than later.

By the way, I don’t think I am arguing just that some projects are a bit below par, but rather that some projects are far below par, inasmuch as they are hugely expensive and not having nearly the proportionate impact.

You support innovation

So do I. My point is that I often see innovation for innovation’s sake. Let’s make it interactive. Let’s give the audience buttons to press. Let’s do it through mime. Let’s pre
sent it in Esperanto. Let’s turn it into a ballet.

You enjoyed the Einstein Ballet and say it should be judged in comparison with Swan Lake, not as a piece of science education

It was commissioned by the Institute of Physics. I don’t know how much was required in terms of IoP money or sponsorship, but I am arguing that the money could have been better spent. Perhaps I am wrong. Ultimately it is up to the members of the IoP to judge if their money is being well spent. Lots of people are bored by physics and lots of people are bored by ballet, so IMHO this project merely allowed people to be bored more efficiently.

(Apologies to ballet lovers. Ballet has no meaning for me, but I realise many others (more cultured than me) do appreciate it. Ballet has its place and I am not saying that we should cut funding to ballet or the arts. However, IMHO physicists should not be spending its money on ballet projects. BTW I also don’t recommend that the Ballet Rambert commits funding to the Isis muon and neutron source at the Rutherford Appleton Laboratory.)

Nigelbrown & David McGloin & Andy Lloyd

You seem to support Lab in a Lorry in the Highlands

Maybe Lab in a Lorry is cost-effective in that area of the UK. I certainly would not want the money to be spent elsewhere. I just think the money could be spent more effectively and efficiently, even in the Highlands.

David M Pyle

You support Time Truck in Cambridge

I am sure that it achieves great things, but could those great things be achieved in cheaper ways? Perhaps the project could then be run twice a year. How about trying to work out the full cost of the project?

Sarah Martin

You say “expensive and not cost-effective” are the same thing.

No. Sorry if it was not clear. I don’t mind if projects are expensive as long as they have a huge positive impact, which then makes the projects cost-effective.

You say “Comparing them on cost and attendance alone is simplistic”

I agree. WRT Lab in a Lorry I mentioned “any wider impact”. Impact is not just numbers of people, but the difference made per person. Wider impact could include, for example, positive local press that might influence parents to be more supportive of science. I agree with the approach suggested by @amyplatypus at Wellcome: “We also look at the type and depth of engagement, whether we reached people we wouldn’t otherwise…”

David McGloin & Suzie Sheehy

You point out the good things that result from Lab in a Lorry (eg hands on experience for pupils & science communication experience for undergraduates)

As with Time Truck, I am sure that it achieves great things, but could those great things be achieved in cheaper ways? Lab in a suitcase?

Andy Lloyd

You suggest Skeptics in the Pub is not real science engagement

Having spoken at maybe 20 pubs, I think one third of the audience is made up of non-scientists, eg. partners, friends, curious folk. But it would be interesting to do a survey. Perhaps all the pubs could pick a month (October?) and ask how many people in the audience have a degree in science or engineering or maths?

In any case, it does not really matter. Skeptics in the Pub does not take money from anyone, so it can do whatever it wants.

My concern is with projects funded by research councils or other public bodies. There is a limited (but not insignificant) pot of money for UK science engagement, and I am asking if it is being well spent.

Jensen Warwick

You say assessment is hard

Fair point and I talk a bit about this in response to Roland Jackson. However, the problem starts even before assessment. I sometimes wonder how some of these projects get funded in the first place.

Andrew Holding

You ask for help running Cambridge Skeptics in the Pub.

Someone help, please.

Brady

Correction – Periodic Table videos have 23.6 million viewings, not 11 million.

Sorry, I was thinking of the copycat project for physics, namely Sixty Symbols, also out of Nottingham University. Copycat is meant as compliment, because good ideas are hard to find, so once you have a good idea then make the most of it and replicate it.

Advertisements

From → Uncategorized

9 Comments
  1. David McGloin permalink

    The point about Lab in a Lorry versus Lab in a suitcase is a fair one. I suppose Physics in the Field (run by IoP) is sort of the latter, but aimed more at science festivals as opposed to schools: http://www.physics.org/article-activity.asp?id=61and IoP does have schools lectures, not quite like Maths Inspires, but maybe could be adapted: http://www.iop.org/education/teacher/extra_resources/lecture_tour/page_41642.htmlI too would be interested in a decent analysis of Lab in a Lorry from the IoP.

  2. majikthijs permalink

    Periodic Videos and Sixty Symbols were all done by Brady Haran (along with Numberphile and Deep Sky videos). If you count up the views of these four feeds it comes to 41.6million.

  3. majikthijs permalink

    Analysis of Lab in a Lorry is already on their website http://t.co/v45k0mEX although uncertain as to how robust and thorough it is. In any case it goes agains the principles of Occam’s razor – "It is vain to do with more what can be done with fewer". Roll on the labs in a suitcase…Better still, the IOP could urge its members to get out into their local schools – perhaps by giving them a discount on membership per outreach hour.

  4. Helen permalink

    I can see the point about the cost effectiveness of Lab in a lorry, but having recently left 6th form, I’d wonder about engagement in rural areas. Our nearest Physics offering university was 1.5 hrs away, lectures were offered there or at another venue about an hour and a quarter away at 4pm, with no public transport links. It was infeasible for us to get to them (the school only offered school trips if the teacher booked them a year in advance), so initiatives such as travelling lectures were the only way we got any engagement at all. This obviously isn’t as cost effective as larger lectures, and periodic videos is a good idea but nothing compares to actually meeting scientists. Lab in a suitcase may be cheaper, but with the vast distances involved, I can’t see even that being ‘cost-effective’ for my area.

  5. Maria permalink

    I agree with Helen, my Parents live in the Highlands, and their community is 45 minutes away from the nearest town. The Highlands and Islands have a small population spread over a large area. Their cinema & library services are all in mobile vans because that;s the best way to reach the people with minimal kit . It would seem that Lab in a Lorry is following that model? There is also a huge difference between the audiences of all these engagement activities. The websites and videos and pub events you linked to are all aimed at older people who whilst are not specialists, are in some way already engaged with the topic and have chosen to access that event. Services for schools, enable them to provide a richer curriculum and reach students who are less engaged with science in the first place.

  6. Beth Taylor permalink

    SimonWhile it???s never pleasant seeing a project you???re involved in criticised so vehemently, I do welcome the debate your comments have raised about how we, as a community, measure the effectiveness of what we???re doing. Resources are limited and we should be ensuring that we are achieving our stated aims through our projects.As has been said already, there is a plurality of aims/agendas/approaches in science engagement and Lab in a Lorry is aiming to do something quite different to some of the other projects that have been mentioned. For those who don???t know, Lab in a Lorry takes hands-on physics experiments to schools and gives 11-14 year olds an experience of doing science that is not possible in a school lab as well as the opportunity to meet scientists and engineers. The aim is to inspire them and show them what is possible if you carry on studying science at higher levels. Lab in a Lorry is free to schools. The teachers involved have to do very little by way of organisation. We align the sessions on board the lorry with the school???s normal timetable so students don???t miss other lessons. The activity takes place on school grounds so there???s none of the hassle/cost involved in taking kids off site or needing cover teachers. All of this, along with active targeting, means that we are successful at getting in to schools that are not already taking advantage of other existing STEM enrichment offers.The project has been evaluated independently twice: in 2005 and 2009. And both reports showed that we were meeting our aim as well as identifying areas where we could improve. In addition, each school we visit is asked to complete a feedback form on their experiences with the project. This allows us to act quickly on any issues, but the vast majority of feedback we get from both teachers and students is positive and provides on-going evidence that we are meeting our aim to inspire students. Anyone who is interested in reading the full evaluation reports is welcome to ask for a copy (the summary of the 2005 report is available here: http://www.labinalorry.org.uk/about_lab_in_a_lorry/evaluation.cfm).Another point that has also been made previously is that the wider impact of any project should be taken in to account. Our main focus is on the students in a school, but we also work with volunteers, improving their communication skills and confidence; we raise the profile of physics within a school and, importantly, with their senior management; we generate local press coverage for the school; we share ideas, resources and information about what other STEM activity is available with teachers and the whole school gets involved in a visit, not just those who get to come on board.I???m not going to argue that Lab in a Lorry is cheap, it patently isn???t. And while numbers aren???t everything, they can help to put the project in context: for the period April 2011-March 2012, Lab in a Lorry Scotland hosted 10,533 students from 44 schools. This was possible with the help of 255 individuals who gave a total of 447 days of volunteering. The cost of this was ??84,000 or ??7.97 per head.Taking suitcases to schools may be cheaper, but what it would save in cash terms, it would lose in whole school engagement. Anyone who is interested in finding out more about the project is welcome to come and see Lab in a Lorry in action. Just email labinalorry@iop.org to arrange a time.Beth TaylorDirector of Communications Institute of Physics

  7. A few years ago I was one of several physics PhD students who helped bring Lab in a Lorry to a cluster of schools in north-east England. Though this area is not nearly as remote as the Scottish highlands, students in these schools nevertheless had a distinct lack of hands-on science options. The labs at the school I served as an ambassador were in a terrible state (a budgeting error by the now-retired head meant they couldn’t buy so much as a paperclip) and every off-site trip I tried to organize — public lectures at the nearest university, visits to a research-incubator campus, trips to a science day at a more distant university, you name it — fell through because of fears they would take time away from regular teaching, combined with a lack of transport, money, teacher supervisors and, above all, organizational wherewithal from the school.Lab in a Lorry, by contrast, was simplicity itself as far as the school was concerned. It required no teacher time, no changes to the timetable, no money and no logistical support of any kind — not even speakers or an auditorium for a lecturer. Very few other hands-on, in-person projects I’ve seen can claim that, and while I think that virtual projects like SixtySymbols are fantastic, I’d agree with previous posters that they cannot be the only thing on offer — we need science encounters IRL, too.As for the impact on the students — well, I can’t speak to long-term outcomes, but at the time they seemed to think the Lorry was pretty amazing. They liked wearing lab coats, they liked talking to real scientists, they liked playing with green goo and breaking wine glasses with resonant sound, and unfortunately they weren’t doing *any* of this in their ordinary physics classes. Also, a few dozen of them came to visit the lab with their parents on Saturday, which suggests they were engaged enough to devote some free time to it, rather than just using it as a handy way of escaping lessons for an hour.Along those same lines, though: you really shouldn’t underestimate the "cool factor" for kids of having a special, out of the ordinary place to visit during the school day. Something big and exciting really does help make a splash at the school, and it makes a big impression on a lot of kids. Before the Lorry arrived, the only comparable big exciting thing that had ever visited my school was something called The Jesus Bus, a similar-sized vehicle that’s been doing the rounds on behalf of RE teachers. Gratifyingly, one student told me that the Lorry "is so much better than The Jesus Bus," and I somehow doubt that they would have said that about a Lab in a Suitcase!Might there be a better, cheaper way of doing science outreach? Of course there might. But for schools like the one I worked at, it does not currently seem to exist, and for them, Lab in the Lorry seems to be filling a real need. I also detect a hint of "the stuff I’m doing is so much better, why don’t people give their money to meeee?" from both Simon Singh and Ben Goldacre, and this really doesn’t do either of them any favours.

  8. Robert Minchin permalink

    Simon, In your original blog post, you said, "Remember, even if most of the money came from sponsors, this is money that did not go to other science engagement projects." I don’t believe this is necessarily true. Selling a project to sponsors is not like bidding for a certain pot of money set aside by a research council (for instance). If a sponsor had not given money to Lab in a Lorry, it is not possible to say that this money would have been spent on other science engagement projects – it might have been spent on something completely different.

  9. Anonymous permalink

    Thanks to everyone who posted comments on this and the previous blog. Particular thanks to Beth Taylor from the Institute of Physics for providing some information about Lab in a Lorry. Beth and I have discussed Lab in a Lorry in the past when I sat on the Institute of Physics External Engagement Committee, where I also raised concerns about this project.In many cases, the comments are accurate but miss the point. For example, yes, Lab in a Lorry does visit lots of schools in areas that usually miss out on science engagement, but could that also be achieved more cheaply (with the same impact), so that even more schools could be reached?In some cases, the comments and tweets are based on a misunderstanding. Ben and I are keen to encourage a new generation and the existing generation of science communicators, whether part-time, professional, academic, etc. However, we are concerned about the way that the biggest funders allocate their biggest pots of money. And, in the worst cases, it is not just about good science being communicated badly, it is about bad science being communicated badly … although promoting bad science badly is probably better than promoting it well. In the last 12 months, there have been three cases (at least ) of the Science Museum promoting bad science, and then digging its heels in when this has been challenged.I am currently thinking about the best way to move the discussion forward. I will return to this debate later in the summer/autumn, but I am not sure how or where.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: