Everything Hertz

We answer our first audio question, on whether academia is too broken to fix, and a second question on whether we’ve ever worried about the possible repercussions of our public critiques and commentary on academia. Show details: Our first audio question is from Erin Williams (@DrErinWill), who asks whether academia is too broken to fix The letter to the editor that got rejected, despite the publication of the response to the letter Harassment in academia Have we ever been worried that someone might say, "I'd never hire those dudes" because of what we say? Other stuff that has happened to us as a result of the podcast Fahrenheit vs. Celsius Supply and demand for academic jobs The criticism that comes with putting yourself out there Links - @ReproRocks (https://twitter.com/ReproRocks): for those working in reproduction to share their work through twitter - The Steven Pinker book - The Better Angels of Our Nature: Why Violence Has Declined (https://www.amazon.com/Better-Angels-Our-Nature-Violence/dp/0143122010

We chat with Sean Rife, who the co-founder of scite.ai (https://scite.ai), a start-up that combines natural language processing with a network of experts to evaluate the veracity of scientific work. Here's what we cover and links for a few things we mention * What is scite.ai? * The Winnower (https://thewinnower.com) * Why is there no good (and free) plagiarism detector? * Grobid (https://grobid.readthedocs.io/en/latest/Introduction/) - A machine learning library for extracting, parsing and re-structuring PDFs * Meta-analysis can prop up flawed bodies of literature * The "Too meta (https://xkcd.com/1447/)" XKCD cartoon * What’s the end game for scite? * The 80,000 hours game (https://80000hours.org/articles/can-you-guess/) * Spooner (http://spooner.lyceum.ws), a utility that allows authors of scientific publications to make their work available to the general public (probably) without violating publishing agreements Other links - Sean on twitter (www.twitter.com/seanrife) - Dan on twitter (www.twitter.com/dsq

We chat with Henry Drysdale (University of Oxford), co-founder of the COMPare trials project (http://compare-trials.org), which compared clinical trial registrations with reported outcomes in five top medical journals and qualitatively analysed the responses to critical correspondence. Discussion points and links galore: The history behind the COMPare project The two papers that were published: a prospective cohort study (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3173-2) correcting and monitoring 58 misreported trials and a qualitative analysis (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3172-3) of researchers’ responses to critical correspondence Ben Goldacre's books (https://www.amazon.co.uk/l/B002C1VRBQ?_encoding=UTF8&redirectedFromKindleDbs=true&rfkd=1&shoppingPortalEnabled=true) What is outcome switching? What were some of the responses to query letters from the authors and journals? Misreporting trials (usually) doesn't lead to patient harm, but it ha

In this episde, we chat with Lisa DeBruine (University of Glasgow) about her experience with large-scale collaborative science and how her psychology department made the switch from SPSS to R. Discussion points and links galore: Deborah Apthorp's tweet on having to teach SPSS (https://twitter.com/deborahapthorp/status/1092599860212068352), "because that's what students know" People who are involved with teaching R for psychology at the University of Glasgow: @Eavanmac @dalejbarr @McAleerP @clelandwoods @PatersonHelena @emilynordmann Why the #psyTeachR started teaching R for reproducible science Data wrangling vs. statistical analysis The psyTeachR website (https://psyteachr.github.io) Danielle Navarro (https://djnavarro.net), and her R text book (https://learningstatisticswithr.com) that you should read Lisa's "faux" package (https://github.com/debruine/faux) for data simulation Sometimes you can't share data, simulations are a good way around this problem "synthpop" is the name of the package (https://cran.r

Dan and James discuss how to deal with the problem of scientists who start talking about topics outside their area of expertise. They also discuss what they would do differently if they would do their PhDs again Here's what they cover... The podcast will now be permanently archived on Open Science Framework (https://osf.io/zj7y3/) James did a talk at the Sound Education conference on podcasting for early career researchers. Here's the video (https://www.youtube.com/watch?v=26t6660_f-A) if you want to see him squirm uncomfortably in his chair for 20 minutes and/or hear his thoughts our approach to podcasting The temptation for academics to believe their own press and to have their thoughts reinforced by the praise they get Keeping a handle on what you know and don't know Nassim Nicholas Taleb (https://twitter.com/nntaleb) has FANS The "Pete Evans" effect, James' solution, that we should eat Pete Evans (https://medium.com/@jamesheathers/i-think-i-have-a-solution-i-m-going-to-eat-pete-evans-7e2da6f3967f)

Peer review is typically conducted behind closed doors. There's been a recent push to make open peer review standard, but what's often left out of these conversations are the potential downsides. To illustrate this, Dan and James discuss a recent instance of open peer review that led to considerable online debate. Here's what they cover... How should we navigate the open review of preprints? Gate keepers gonna gate keep, but is this better out in the open? Weaponising openness Some people don't realise that some data can’t be shared Should the reviewers of rejected papers follow them to the next journal? When bad papers that you reject pop up in another journal, unchanged Does the venue and timing of the open peer review matter? Signing your reviews Using publons to track your reviews Links - Brad Love’s blog post: http://bradlove.org/blog/open-review - Niko’s blog post: https://nikokriegeskorte.org/2019/01/09/whats-the-best-measure-of-representational-dissimilarity/ - Publons: https://publons.com - Dan on t

We’re joined by Daniele Marinazzo (University of Ghent) to chat about the recently launched overlay journal Neurons, Behavior, Data analysis and Theory (NBDT), for which he on the Editorial Board. An overlay journal is organised a set of manuscripts that is published and hosted by a seperate entity (in this case, the Arxiv server), a feature that dramatically reduces publication costs. We discuss the unique overlay model, how this can drive article fees to essentially zero, and what it takes to build a good community journal. Here’s what we cover: Why launch a new neuroscience journal and how is it different from currently established journals? The unique way that editor’s decide which papers to send out for review How does the journal operate when it’s open access and submissions only cost $10? How do you build a journal that your target community will recognise as a ‘good’ journal? The process of submitting a manuscript to NBDT Should journals allow or encourage authors to suggest potential reviewers for t

In this episode, Dan and James chat with microbiologist Elisabeth Bik about about the detection of problematic images in scientific papers, the state of microbiome research, and making the jump from academia to industry. More info on what they cover: How Elisabeth get into error detection of scientific images The process of detecting errors in images How groups of authors tend to publish multiple papers with problematic images The association between journal prestige and problematic images Providing monetary incentives for publications Making the jump from academia to industry The current state of microbiome research Links - Patreon: www.patreon.com/hertzpodcast - Elisabeth on Twitter:  www.twitter.com/microbiomdigest - Elisabeth online: https://microbiomedigest.com - The problematic image paper: https://mbio.asm.org/content/7/3/e00809-16.short - Pubpeer: https://pubpeer.com - Dan on twitter: www.twitter.com/dsquintana - James on twitter: www.twitter.com/jamesheathers - Everything Hertz on twitter: www.twi

Dan and James discuss what's missing from biobehavioral science course syllabi. Here's the episode lowdown: - A thank you to our new Patron supporters - The (supposed)CRISPR baby - SPSS vs. R: What should be used for instruction? - What would Dan and James include in a new syllabus? - Should students be taught scientific communication? - If we’re going to add new stuff to a syllabus, what gets removed? - Are courses too big these days? - Should students be taught how to set up a side hustle to apply their research skills outside of academia? Links - Patreon: https://www.patreon.com/hertzpodcast - CRISPR baby story: https://www.vox.com/science-and-health/2018/11/30/18119589/crispr-technology-he-jiankui - Dan on twitter: https://www.twitter.com/dsquintana - James on twitter: https://www.twitter.com/jamesheathers - Everything Hertz on twitter: https://www.twitter.com/hertzpodcast - Everything Hertz on Facebook: https://www.facebook.com/everythinghertzpodcast/ Music credits: Lee Rosevere freemusicarchive.org/musi

Dan and James discuss a new journal of "controversial ideas" that will allow authors to publish articles anonymously. They also launch their Patreon page, in which listeners can support the show and get bonus features. Here's the episode lowdown - James describes his first experience eating a “ding dong” - Why James and Dan do the show - What is Patreon? - The Journal of Controversial Ideas - The link between a vaccine batch and narcolepsy in Norway - Can you “claim” our anonymous article a few years into the future? - What’s the difference between anonymous blogging and anonymous journal articles? - The new Neurons, Behavior, Data analysis and Theory journal Links - Patreon: https://www.patreon.com/hertzpodcast - Narcolepsy paper: https://www.sciencedirect.com/science/article/pii/S1389945713001548 - Journal of Controversial ideas story: https://www.bbc.com/news/education-46146766 - PsyBrief twitter account: twitter.com/psybrief - NBDT journal: nbdt.scholasticahq.com - Dan on twitter: https://www.twitter.c

In this episode, we chat about whether it’s necessary to move for an academic job to demonstrate “independence”. Here's a rundown of the other stuff we cover: - James' appearance at the “Sound education” conference - Dan’s first day of school as a latino in a white neighbourhood - Our thoughts on the restrictive social media policy at the SfN (Society for Neuroscience)conference - Why and how Dan and James moved overseas from Australia - Do you really need to move overseas to demonstrate independence? - The two-body problem - Can you demonstrate independence with sole-author papers? Links SFN social media policy https://twitter.com/fedeadolfi/status/1058760331747581953 Dan on twitter https://www.twitter.com/dsquintana James on twitter https://www.twitter.com/jamesheathers Everything Hertz on twitter https://www.twitter.com/hertzpodcast Everything Hertz on Facebook https://www.facebook.com/everythinghertzpodcast/ Music credits: Lee Rosevere freemusicarchive.org/music/Lee_Rosevere/

Dan and James discuss the recent "grievance studies" hoax, whereby three people spent a year writing twenty-one fake manuscripts for submission to various cultural studies journals. They also discuss a new proposal to shift publication culture in which researchers pledge to publish exclusively in community-run journals but only when a pre-specified threshold of support for this commitment by the research community has been met. Here's an overview of the episode: - It’s fat bear week! - The new proposal to fix the stranglehold of commercial publishers in academia - Flipping journals to open access - The ‘grievance studies’ hoax - When James first came across the “dog rape” paper - What if you were to design the dog study properly? - Should we systematically try and hoax journals? - Astronomy already injects fake data, can we learn from this? - Should these new hoaxes all be associated with Sokal? Links - Brian Resnick’s fat bear week story: https://www.vox.com/science-and-health/2018/10/9/17955432/fat-bear-w

We’re joined by Brian Nosek (Centre for Open Science and University of Virginia) to chat about building technology to make open science easier to implement, and shifting the norms of science to make it more open. We also discuss his recent social sciences replication project in which researchers accurately predicted which studies would replicate. Here’s what we cover: - What is the Centre for Open Science? - How did Brian go from Psychology professor to the director of tech organisation? - How can researchers use the Open Science Framework (OSF)? - How does OSF remove friction for conducting open science? - Registered reports (now available at 131 journals!) - What factors converged to cause the emerging acceptance of open science? - The social sciences replication project - Can researchers anticipate which findings can replicate? - What happened when Brian and his team tried to submit their replication attempts of Science papers to Science? - The experience of reviewing registered reports Links: Centre for

This episode includes part two of a chat with Nathan Hall (McGill University), who is the person behind the ’Shit academics say’ account (@AcademicsSay), which pokes fun of all the weird stuff that academics say. Before getting to the discussion, James and Dan answer two listener questions on grants and data cleaning. Here’s what is covered in the episode: People talk about papers all the time, but the grant process is not discussed openly—why? Speaking to your funding body’s relevant program officer Assembling a team that complements your weaknesses Data carpentry and the tidyverse Outlier analysis Nathan Hall on big publishing Upending the publication system by getting journals to bid for papers Using peer review quality to judge the quality of journals Debunking learning styes Academics chasing after celebrity and hype The cost of chasing academic prestige Using twitter hashtags like #PhDChat and #ECRchat to learn more about the experiences of other people Links Data carpentry https://datacarpentry.org

We’re joined by Nathan Hall (McGill University) to chat about the role of humour in academia. Nathan is the person behind the ’Shit academics say’ Twitter account (@AcademicsSay), which pokes fun of all the weird stuff that academics say. Here’s what we cover: How Nathan got started with the account The story behind Nathan's 'Research Wahlberg' Twitter account (@ResearchMark) The risk of social media usage being perceived as “unprofessional” The amount of free labor that academics are pressured to do How alcohol is becoming an unspoken coping strategy in academia Academic guilt and glamorising overwork Why Nathan changed his mind about making Imposter Syndrome jokes Leaving tweets in your draft folder Links Nein Quarterly https://twitter.com/NeinQuarterly Shit my Dad says https://twitter.com/shitmydadsays Cern and comic sans https://www.theverge.com/2012/7/4/3136652/cern-scientists-comic-sans-higgs-boson Ate the onion https://www.reddit.com/r/AteTheOnion/ Shit Academics Say on twitter https://www.twitter.c

Dan and James answer listener questions on tips for starting your PhD and the role of statistics in exploratory research. Other stuff they cover: James new paper on people that voluntarily give themselves goosebumps Dan’s new podcast: Physiology and Behavior A preview of next weeks guest, Nathan Hall When things are taken out of context on Twitter What do you do when people are angry with you on the internet? Tips for people starting a PhD Can inferential statistics play a role in exploratory research? Why don’t journals publish peer review reports? Why is PsycNet so bad? Links James’ paper https://peerj.com/articles/5292/ Physiology and Behavior podcast from Dan https://shows.pippa.io/dsquintana The tweet we discuss https://twitter.com/andpru/status/1024005699737509888?s=21 Dan on twitter https://www.twitter.com/dsquintana James on twitter https://www.twitter.com/jamesheathers Everything Hertz on twitter https://www.twitter.com/hertzpodcast Everything Hertz on Facebook https://www.facebook.com/everythingh

Dan and James chat with Jon Brock (Cognitive scientist at Frankl) about the use of blockchain technology for open science. Here's what they cover: What is the blockchain? Why Jon made the jump from academia to Frankl A cryptocurrency for open science What do institutional review boards think about using blockchain for data collection and storage? Autism heterogeneity How will this approach scale to biological signals and genetics data? What’s something that Jon’s changed him mind about in regards to academia? Links Frankl https://frankl.io Five reasons Frankl has a token https://medium.com/franklopenscience/why-does-frankl-need-a-frankl-token-4129d718ab74 Bjoern Brembs blog post http://bjoern.brembs.net/2018/05/after-24-years-when-will-academic-culture-finally-shift/ An explainer on cryptographic hashes https://en.wikipedia.org/wiki/Cryptographichashfunction#Illustration Frankl in a nutshell https://medium.com/franklopenscience/frankl-in-a-nutshell-9b488c554dea Frankl for autism https://medium.com/franklope

Dan and James talk about the recent SIPS conference answer a listener question on "salami slicing" the outcomes from one study into multiple papers. Here's what they cover: What is the SIPS conference? [0:24] A SIPS proposal for Google scholar to highlight commentaries and replication attempts on specific articles [15:42] James and Dan’s favourite Hertz episodes [20:43] We answer a listener question on Salami slicing [28:45] Can you publish too much? [48:10] Links - SIPS conference: https://www.improvingpsych.org/SIPS2018/ - Reproducibilitea podcast: https://soundcloud.com/reproducibilitea - Salami slicing tweet: https://twitter.com/academicswrite/status/1008719899940786176 - Cumulative impact factors: http://khakhalin.blogspot.com/2012/11/cumulative-impact-factor-benchmarking.html - A working document from SIPS on making replications discoverable (including Google scholar) https://osf.io/57zxa/ Find us on Twitter: www.twitter.com/hertzpodcast www.twitter.com/dsquintana www.twitter.com/jamesheathers Music cr

Dan and James chat about science journalism with Brian Resnick (@b_resnick), who is a science reporter at Vox.com. Here’s what they cover: Should scientists be worried that their work will be misrepresented when talking to the media? [0:58] How Brian approaches science journalism [8:53] It’s ok to challenge the assumptions of science journalists [16:57] How do you write a great headline? [19:13] How does Brian appraise the quality of research? [29:50] Should psychiatrists (or journalists) diagnose the US President? [32:50] Stories in science that no one knows the answer to [36:58] How to promote your research without going via your institution’s media department [40:24] The best way to pitch your research to a science journalist [44:25] How pre-preprints are great for research addressing current events [48:45] How scientists can improve their science communication writing [53:15] Dick jokes in science writing — yes or no? [54:30] What has Brian changed his mind about? [56:37] Brian’s book recommendation [58

Dan and James chat about the adoption of open science practices with Dorothy Bishop, Professor of Developmental Neuropsychology at the University of Oxford. Here are some highlights from the show: Why Dorothy starting adopting open science practices Forking paths of analysis Dorothy’s experience with her first registered report Issues around data deposition When someone finds an error in your data What happens when a senior researcher is using questionable research practices? What to do when you’re caught doing something wrong Detecting errors in other papers The potential for open data to be weaponised How error detection is interpersonally difficult Carving out time for non-work pursuits The benefits of writing fiction when you're a scientist Using video for science communication James Heathers: Cat dealer Promoting your research vs. promoting yourself Dorothy’s book recommendation Links Dan Gilbert’s paper: http://science.sciencemag.org/content/351/6277/1037.2 Merchants of doubt [book]: https://www.amazo