I’ve repeatedly heard that every academic, at some stage in their
career, has that one project that just won’t behave itself and become a nice
little publishable package. Experimental results lead to more questions than
answers. Reviewers say the ides are interesting but are unconvinced by the
conclusions or have issues in the reliability of the method. You're rejected but
encouraged to re-submit. The whole thing drags on for years, quietly ticking
away in the background, while research avenues with more promise for short-term
gain are chased instead. But the nemesis project never dies, it stays there at
the back of your head. Too much time and effort has already been invested, you’re
in too deep to give up on it now.
My own mini-version of this has just ended. My nemesis project
has just been published in Astrobiology (link, and link to non-paywalled
pre-print) 3 years after my supervisor scribbled down a ‘cool idea which won’t
take much time to test’ (I may be paraphrasing there, it was a long time ago).
This is not my usual post-publication summary of my work, I will hopefully
write that soon, instead this is a story of how much behind the scenes failure
can have gone into one, small, relatively insignificant, successfully published
paper.
The idea for the project came out of the PlanetaryProtection of the Outer Solar System project, which I have written about before
(link). In one of the early meetings back in 2016 my supervisor, clearly paying
full attention to whatever was being discussed at the time, scribbled a vague
experimental idea onto a scrap of paper. This idea was to see if we could use a
well-established environmental sampling technique (solid phase micro extraction,
SPME) to test spacecraft hardware surfaces for organic contamination. Now this was interesting as organic contamination
is a big issue in planetary protection, we don’t want to send dirty spacecraft
with highly sensitive instruments to the (currently) pristine icy moons of the
outer solar system. We’d end up only detecting muck from Earth and so either getting
all excited over nothing, misinterpreting it for evidence of alien life, or, a
real interesting extra-terrestrial signal would be missed, lost in background
noise from the contaminants. Current methods employed for detecting contaminants
on surfaces tend to be time consuming, complicated and may involve multiple
solvents being used in the process – themselves potential contaminants.
The premise was therefore simple: Get some stainless steel
to use as a budget stand in for a spacecraft surface, contaminate it, see if
SPME (coupled with GC-MS) is sensitive enough to detect contaminants at the levels
of cleanliness required for life-detection missions.
The first version of this study just involved leaving some
stainless steel L-shaped brackets (bought from a hardware store) out in the lab
to collect fallout contamination from the air and also handling them with and
without gloves to see if they picked up anything detectable from hand transfer.
To be scientifically valid a study like this must be reproducible,
so many repetitions of everything being tested are needed, simple, but
monotonous, time consuming work – perfect for an undergraduate summer
internship! Georgios, a 2nd year undergraduate and now co-author on
the final paper, gave up 6 weeks of his summer in 2017 to do this, creating
loads of data for me to work up afterwards. Now initially we thought this first
version of the study was pretty good, however the reviewers had other thoughts.
Reject but encourage re-submission.
The issues basically boiled down to our method being a bit
woolly and bullsh#t (again, paraphrasing). How could we know what was on the
surface to detect and therefore how sensitive the method was if we hadn’t
specifically contaminated it ourselves at a known concentration? We’d basically
skipped the proof of concept stage and gone straight to real-word
testing (well as real as you can get without a real spacecraft)...So yeah, fair enough.
Not having scope to dedicate 6 weeks of lab time to
completely redo the experimental side of this study myself, the project had to
get shelved until the following summer (2018) when I could get a second student,
Yuting, who was keen to get some experience in the mind-numbing, soul
destroying boredom of repetitive lab work.
In the meantime, I took the reviewer comments, which despite
being rather critical were all very valid and helpful, and developed a whole
new method for testing the sensitivity of this technique. This was to be much
more scientific, creating a whole range of solutions of astrobiologically-relevant
contaminants to contaminate a surface with much better-defined properties (although
it was still basically just a steel nut).
Once again, the student project seemed a success. Yuting
produced a shed load of nice replicate data over the summer, which I turned
into a completely new manuscript. None of the data set from Georgios’s original
experimental run even made it into the new work, and after a few weeks of
tidying up and writing we re-submitted.
Again, however, the reviewers didn’t quite agree, while they
did think the method was now (mostly) sound, they didn’t agree the results were
as promising as we did and wanted more and better data. Almost annoyingly this
wasn’t a rejection this time, there was now a time pressure involved with a
re-submission deadline. I could have ignored it and waited another year, but
there was an end in sight, a way to kill this thing. Jon just needed to work
some magic to tweak the mass spec settings to decrease the noise and make the
data more convincing. Unfortunately, this meant I now had to repeat all the experimental
work with the new settings myself, replicating a whole summer student project
in about 3 weeks.
This was not fun, but it worked.
Now at the end of it all, it is clear that without the
multiple knock backs and the intermediary time periods to just think about how
to improve the methodology, this study would’ve been pretty rubbish. This is definitely
a case where the review process has greatly improved an original idea and has shown me that rejections don’t always have to be a bad thing, they can be an opportunity.
However, this only took 3 years to take down, I’m not sure how I’d feel if this
had grown into some 5 or 10 year, or even career-spanning, monster.
Maybe that’ll be the next quick project…
