Earlier this week, as promised, I had a chat with Andrew MacDonald about the recent back and forth and back again about the use of writing tools in ecology. In typical Canadian fashion, we both really enjoyed our chat, and learned something from each other in the process
Absolutely fantastic chat with @polesasunder on Markdown, open science, & writing tools. Will digest some thoughts & post an update tomorrow
— Lab and Field (@thelabandfield) August 27, 2013
may you often have conversations that improve & change your thinking on a topic, like the one I just had with @thelabandfield — Andrew MacDonald (@polesasunder) August 27, 2013
Let me say that Markdown (and integrating it with R using the R package knitr) is REALLY NEAT. Andrew showed me an example of the departmental coffee co-op he runs where after entering each month’s consumption, the document automatically produces the most recent stats, graphs, and customer balances. Whereas I would have entered the data, generated new figures, and pasted them into the document manually before, the combination of Markdown and R/knitr is fantastic. In a more ecological application, consider an annual report where similar elements are required each year (e.g., temperature and precipitation plots, or a running graph of a bird population’s reproductive success). Just enter the data, “knit” the file, and you’re off to the races. Bloody brilliant if you ask me. Consider, too, a manuscript with a complex analysis that keeps getting tweaked. Instead of separating the stats/modeling, plotting, and writing, one could do them all in one self-updating document (well, except the writing. But if anyone figures that out, let me know). In my original response, I highlighted three issues that would be required for me to break out of the Word paradigm that dominates ornithology like wetness dominates the ocean:
- Integration with a reference database
- Track changes made by coauthors
- Available offline
There’s a command in the program git that will compare two markdown files, and highlight changed words in different colours. I haven’t played around with it, but Andrew sent an example:
here is an example of git diff –color-words @thelabandfield pic.twitter.com/TpmOWY0smi
— Andrew MacDonald (@polesasunder) August 31, 2013
Markdown doesn’t handle citations natively, but integrating with the program pandoc can take advantage of the LaTeX reference format, and programs like JabRef can be used to facilitate inline citations. It’s not as slick as the Endnote/Word interface in my view (especially when dealing with two papers that could be cited as “Smith et al. 2013”), but it’s workable.
But what we agreed was the biggest hurdle is the cultural shift required, particularly among field ecologists. i still know folks who manage citations in their manuscripts by manually typing them in. Among the 15-20 manuscripts I’m asked to review by journals and colleagues each year, I’d say that half have errors in the literature cited that reveal such a manual strategy. So getting people comfortable with any sort of command line operation or mark-up coding will require a significant shift in a field not known for its shifting.
Another challenge is in the way that tools like R, Markdown, and the open science movement are proselytized*. As I pointed out earlier, when confronted by open science advocates, many of us feel like the kid who got caught by the dentist not flossing between his incisors, and is subsequently chided. The focus can sometimes be on why Word, Endnote, and SPSS are bad, not why Markdown, and R are better options than Word, and SPSS. I’ve also found that concepts that some people take for granted get turned into jargon that makes communicating this to non-users challenging. As an example, I finally had to ask Andrew what “git” actually was – a program, a built-in command, or a term of derision often preceded by “smarmy”. For the record, git is a program that, depending on one’s operating sytem, has several graphical user interfaces (GUIs) that avoid the need for command line typing.
*Obviously this isn’t a blanket statement, but it’s been mentioned to me by at least 2 other independent sources, so it must be a thing.
As Andrew and I wrapped up our 2-hour-long Skype/Google video chat, he asked me whether I would use Markdown et al. If I were writing a single-authored manuscript or report, I might. But right now, it will take some playing around to see what if I can make it work. I’ll let you all know.
Postscript: I wrote the original text of the post in Markdown, and thanks to Andrew for looking it over this weekend.
The Lab and Field 