Everything Hertz

Dan and James cover a new paper which discusses whether research misconduct should be criminalised. If so, where do we draw the line and who should investigate these cases? Here's an episode overview and links to stuff we mentioned: We’re a pop science podcast, apparently Elizabeth Bik’s wikipedia page (https://en.wikipedia.org/wiki/Elisabeth_Bik) Elizabeth’s Patreon page (https://www.patreon.com/elisabethbik) The original consortium letter (https://presspage-production-content.s3.amazonaws.com/uploads/1508/coalitionletteropposinglowerembargoes-864869.pdf?54750) The apology letter (https://www.psychologicalscience.org/policy/to-aps-members-from-the-board-of-directors.html) from the APS The “love of science (https://twitter.com/NobelPrize/status/1219530825320083456?s=20)” tweet How James got into science Tal’s “science is not a jobs program (https://twitter.com/talyarkoni/status/960296870080925696?s=20)” tweet The 'Should research misconduct be criminalised? (https://journals.sagepub.com/doi/10.1177/174701

To celebrate our 100th episode, which we video-streamed live, Dan and James were joined by three special guests: Daniel Lakens, Amy Orben, and Chris Chambers. Here's what they covered in this episode: James and Dan share their favourite episodes The power of the Twitter direct message Daniel Lakens joins us to discuss his recent work on helping people make better statistical decisions Can you create cross-discipline effect size guidelines? What would Jacob Cohen say if we could bring him back to life? Academic backup career plans Our new partnership with Prolific James' piece on not treating your research participants like cattle (https://medium.com/@jamesheathers/stop-treating-your-experimental-participants-like-cattle-b5fab7fbfca7) Amy Orben joins us to discuss multiverse analysis and the Reproducibilitea community The Latin Modern Roman font (https://www.fontsquirrel.com/fonts/latin-modern-roman) We speak with Chris Chambers and get an update of what's happening with Registered reports Other links - Dan

Dan and James answer a listener question on science advocacy. Is this an activity that all scientists should do, and if so, how much advocacy work should we be doing? Here's other stuff they cover and links to stuff they mention: James’ thoughts on thanksgiving James’s hot mic tweet (https://twitter.com/jamesheathers/status/1199676892460261376?s=20) The Tom Bartlett story in the Chronicle (https://www.chronicle.com/interactives/20190924-Criminology) about the criminologist accused of cooking the books The SCORE DAPRA project (https://www.darpa.mil/program/systematizing-confidence-in-open-research-and-evidence) A listener question from Crystal Steltenpohl (https://twitter.com/CNSteltenpohl): What is a scientist’s role in advocacy? Julieanne Smolinski piece (https://medium.com/@boobsradley/between-my-hectic-job-and-nourishing-social-life-it-s-not-always-easy-to-find-the-time-to-make-aa9c48a5458b) on Hollywood diets Sunbathe your arsehole (https://jezebel.com/sunbathe-your-asshole-for-wellness-1840031856),

We chat with Sophia Crüwell (Meta-Research Innovation Center Berlin) about pre-registration and her recent work introducing pre-registration templates for cognitive modelling research. Here's what we cover and some links: * Sophia’s PhD research * Sophia’s recent preprint: Preregistration in Complex Contexts: A Preregistration Template for the Application of Cognitive Models (https://psyarxiv.com/2hykx) * The first version of the pre-print (https://osf.io/download/5dbb4af2af84c3000eea7685/?version=1&displayName=Preregistration%20is%20redundant,%20at%20best-2019-10-31T20%3A58%3A26.782Z.pdf) titled, preregistration is redundant, at best * The updated version (https://psyarxiv.com/x36pz/) of the preprint titled, "Is preregistration worthwhile?" * The Bayesian Spectacles blogpost (https://www.bayesianspectacles.org/a-breakdown-of-preregistration-is-redundant-at-best/) on the first version of the pre-print * Data simulation (https://debruine.github.io/lmem_sim/index.html) from former guest Lisa DeBruine * The late

Dan and James discuss the concept of "slow science", which has been proposed in order to improve the quality of scientific research and create a more sustainable work environment. Here's what they cover in this episode Thank you patrons day! Social media algorithms reward outrage, not quality of substance A paper on slow science (https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(19)30242-6) from Uta Frith, which includes a proposal of publication limits Is information overload really a problem? The META platform (https://www.meta.org/) for a weekly research digest Would reducing the volume of publications really improve quality? The working paper (https://osf.io/z47w3/) that simulated the quality vs. quantity question The slow professor book https://utorontopress.com/ca/the-slow-professor-3 Michael Frank’s paper on N-best evaluation (https://www.sciencedirect.com/science/article/abs/pii/S1364661319302347) Some institutions are now screening papers before submission to check for e

Dan and James discuss the results of this year's Nature survey of PhD students. Despite a majority of students reporting general satisfaction with their decision to undertake a PhD, many described a sense of uncertainty, harassment in the lab, and gruelling work hours. Things they discuss... James met an Australian member of parliament (https://twitter.com/deborahapthorp/status/1192343606985555968?s=20) and won a commendation (https://twitter.com/senseaboutsci/status/1195284321617293313?s=20) from Sense under Science The Doing Good symposium (https://www.cbs.mpg.de/doing-good) The Nature PhD survey (https://www.nature.com/articles/d41586-019-03459-7) Bloat in academia What people like the most about being a PhD student Are we just not hearing that much from people who are having a good time? Financial pressure in PhDs Harassment and discrimination in PhD programs Alternative academic careers James' cat whisker collection South Korean kids turn one when they’re born (https://www.theguardian.com/world/201

Dan and James discuss why academia tolerates bad presentations and the strange distrust of polished presentations. Here's what else they discuss... James had a Filipino feast https://twitter.com/jamesheathers/status/1188582859528949766?s=20 We’re approaching 100 episodes! ReproducibiliTea (https://osf.io/3qrj6/wiki/home/) is spreading worldwide! Why do some people not trust polished presentations? The Mike Morrison episode (https://everythinghertz.com/87) on the Better Poster The “I want a refund for a bad presentation” blog post (https://www.jratcliffe.net/post/i-want-a-refund-for-your-conference-presentation) What does James consider a ‘good’ presentation? Conference apps Why don’t we teach PhD students to do things that they’ll need further in their careers, like making presentations and writing emails? Vague emails James wants to help out an email spammer Email vs. Twitter DM Anonymous people on the internet James discovers coin collecting Blocking people on Twitter Dan’s got a show recommendation: Mon

Dan and James chat with Fiona Fidler (University of Melbourne), who is leading the repliCATS project (https://replicats.research.unimelb.edu.au/), which aims to develop accurate techniques to elicit estimates of the replicability of research. This is also the first time they interview a guest live! Here's what they discuss... * The story behind repliCATS * Australia's best export, Tim Tams (https://en.wikipedia.org/wiki/Tim_Tam) * The SCORE project (https://www.wired.com/story/darpa-wants-to-solve-sciences-replication-crisis-with-robots/) organised by DARPA * Can anyone use the repliCATS methodology? * Dan, Fiona, and James talk about did their honours theses (this is roughly the Australian equivalent of a Masters) * What would a successful repliCATS project look like? * What sort of heuristics do people use to assess replicability? * The AIMOS conference (https://www.aimos2019conference.com/) * The role of replicability in public policy * This is Bob Katter (https://www.youtube.com/watch?v=1i739SyCu9I) * S

Dan and James answer a listener question on how to navigate open science practices, such as preprints and open code repositories, in light of double-blind reviews. Stuff they cover: How common is double-blind review? How many journals don’t accept preprints? Bias in the review process How practical is blinded review? Do the benefits of preprints outweighs not having blinded review? James' approach to getting comments on his preprints Convincing your supervisor to adopt open science practices The preprint that James won’t submit for publication, for some reason We get reviews... Our first live guest! Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more

Dan and James discuss the role of Google Scholar in citation patterns and whether we should limit academics to only publishing two papers a year. Links and details: James has a new Hertz-quarters The Metascience conference (https://www.metascience2019.org) How is google scholar influencing citation patterns A slide from @Jevinwest's presentation (https://twitter.com/gmusser/status/1170466414345375746?s=20) on Google Scholars Is this a symptom of lazy citaton practices rather than the algorithm? What are the alternatives to google scholar? Should google open up the algorithm? GS will find your preprint and link it to the paywall link Why is Google Scholar free? What would make GS better? Using the Zotero plugin for to collected citation info in bulk from search results in GS Top recommended articles in GS are phenomenal GS is not great for meta-analysis GS reduces friction Should we limit academics to only publishing two papers a year (https://twitter.com/joseph_fridman/status/1170447576505520128?s=20), as s

We chat with Kristin Sainani (Stanford University) about a popular statistical method in sports medicine research (magnitude based inference), which has been banned by some journals, but continues to thrive in some pockets of scholarship. We also discuss the role of statistical inference in the current replication crisis. Links and info * What is magnitude based inference and how did Kristin get involved in this? * The response to Kristin’s critiques * This is really an issue of small sample sizes * Kristin’s Coursera course (https://coursera.org/learn/sciwrite) on scientific writing * The readability of scientific articles is decreasing (https://elifesciences.org/articles/27725) * The role of statistical inference in the replication crisis * Kristin has changed her mind about… Twitter * Dan made international news by posting a picture of a bird on Twitter (https://twitter.com/dsquintana/status/1163083819605475328) * Kristin recommends this paper: P values are just the tip of the iceberg (https://www.nature

Dan and James discuss two listener questions on performing secondary data analysis and the potential for prestige to creep into open science reforms. More info and links: Why generate your own dataset when you can get a high impact paper using public data? Thanks to Stu Murray (https://twitter.com/DrStuartBMurray) for the question Will people steal your ideas? The journal Scientific Data (https://www.nature.com/sdata/) Are we now incentivising data mining rather than data collecting? Synthetic data Dan’s recent synthetic data preprint primer (https://psyarxiv.com/dmfb3/) Ego and prestige got us into the mess we’re trying to fix with open science, but how can we stop this from happening again? Thanks to Robin Kok (https://twitter.com/robinnkok) for the question, listen to our episode with him on e-health (https://everythinghertz.com/34)! Did all the people who co-authored the paper to change statistical significance the default p-value threshold to .005 (https://www.nature.com/articles/s41562-017-0189-z) actu

We chat with Tom about whether psychology has a conflict-of-interest problem and how to best define such conflicts. Links and other stuff we cover... Tom's article (https://www.nature.com/articles/d41586-019-02041-5) on conflicts of interest in psychology How can we define a conflict an interest without falling down a rabbit hole? Communication statistics to the layperson How science journalism focuses on single studies rather than the larger story Tom’s new book: The AI does not hate you (https://www.amazon.com/Does-Not-Hate-You-Superintelligence-ebook/dp/B07K258VCV) Win Tom’s book! Tweet your favourite Hertz episode and we’ll pick one at random, who'll get sent Tom's book How do journalists go about hearing from new voices for story comments? What has Tom changes his mind about? Tom’s book recommendation: Galileo's Middle Finger (https://www.amazon.com/Galileos-Middle-Finger-Heretics-Activists/dp/0143108115) Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesh

Dan and James apply the pomodoro principle by tackling four topics within a strict ten-minute time limit each: James' new error detection tool, academic dress codes, the "back in my day..." defence for QRPs, and p-slacking. Here are links and details... * James won an award * James’ new error detection tool, DEBIT (https://osf.io/pm825/) * Academic dress codes * P-slacking * The p-slacking paper (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3408200) * Marcus Crede’s paper: A Negative Effect of a Contractive Pose Is Not Evidence for the Positive Effect of an Expansive Pose (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3198470) * A preview of our next episode on conflicts of interest in psychology Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/

We chat with Mike Morrison, a former User Experience (UX) designer who quit his tech career to research how we can bring UX design principles to science. We discuss Mike's recently introduced 'better poster' format and why scientists should think carefully about UX. Here's what we cover: What’s the story behind the “better poster?” The Better Poster video (https://www.youtube.com/watch?v=1RwJbhkCA58) The Better Poster template (https://osf.io/ef53g/) The importance of minimising cognitive load Science isn’t badly designed, it’s not even designed at all What is good User Experience (UX)? The most important feature of SciHub Version 2 of the ‘better poster’ Weird poster designs that James has seen over the years The Fish Market study (https://peerj.com/articles/7138/) Common misunderstandings of the better poster Empirically investigating the performance of the Better Poster The meta-meta poster How better posters get better questions Mike’s next target: Better Presentations Andrew York’s Github paper (https:

Dan and James answer a listener question on whether they should stick it out for a few months in a toxic lab to get one more paper or if they should leave. Other stuff they cover: * We don’t like cricket, oh no, we love it (https://www.youtube.com/watch?v=cewfIUQ6q8o) * James is bad at tribalism * We answer a listener question about a bad lab environment * The “Dutch Bounce” * The Golden Lab Child * Demonstrating independence by writing a sole author paper * What should you do if there’s a late authorship switch on your paper? * Having an upfront conversation about authorship * James on NPR (https://www.npr.org/2019/02/17/695536889/volunteers-fight-bad-science?t=1560712522537) Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon

We chat with Kevin Mitchell (Trinity College Dublin) about what the field of psychology can learn from genetics research, how our research theories tend to be constrained by our research tools, and his new book, "Innate (https://press.princeton.edu/titles/13255.html)". Other stuff we cover: * Kevin's book, "Innate: How the Wiring of Our Brains Shapes Who We Are (https://press.princeton.edu/titles/13255.html)" * The story with link between Serotonin transporter gene 5-HTTLPR variation and it’s link to depression * Gene wide association studies (GWAS) vs. candidate gene studies * Correcting for multiple tests in GWAS * What can psychology learn from genetics when it comes to improving methods? * When did people lose faith in candidate gene studies? * Common vs. rare gene mutations * How our theories are shaped by our technology and resources * The third overlooked area of genetic variation (other than nature and nurture): How the developmental program plays out * Why intelligence is just an index of how well

We chat with Amy Orben, who applies "multiverse" methodology to combat and expose analytical flexibility in her research area of the impact of digital technologies on psychological wellbeing. We also discuss ReproducibiliTea, an early career researcher-led journal club initiative she co-founded, which helps young researchers create local open science groups. Here are some more details and links: * The tweet (https://twitter.com/_vdeni_/status/1126485212337143808) pointing our Dan's gramatical error in his usual introduction. THANKS DENIS * Is Twitter melting our brains? * The history of "new technology" panic * What's the next panic? * Moral entrepreneurs: profiting from moral panic * Specification curve analysis (https://socialsciences.nature.com/users/200472-amy-orben/posts/42763-beyond-cherry-picking): a way to run all theoretically defensible analysis options on a given dataset * Amy's Nature Human Behavior paper (https://www.nature.com/articles/s41562-018-0506-1) * Amy's PNAS paper (https://www.pnas.

By popular demand, Dan and James are kicking it old school and just shooting the breeze. They cover whether scientists should be on Twitter, if Fortnite is ruining our youth, book recommendations, and null oxytocin studies. Stuff they cover and links to obsure references * Should scientists be on twitter? * James runs a Twitter experiment * Scite has now gone live, listen to our episode (https://everythinghertz.com/80) on this platform * Our dreams of a live Hertz episode * Is Fortnite (https://en.wikipedia.org/wiki/Fortnite) killing our youth and the parallels with the “heavy metal” scare * Amy Orben’s screen time study (https://www.nature.com/articles/s41562-018-0506-1) * Multiverse analysis * Book recommendations: Kevin Mitchell’s "Innate" (https://www.amazon.com/Innate-How-Wiring-Brains-Shapes/dp/0691173885), Gareth Leng's "Heart of the brain" (https://mitpress.mit.edu/books/heart-brain) * Daryl dug a hole reference (https://www.youtube.com/watch?v=mDl1i3mpKwM), from the Aussie classic, "The Castle"

We answer a listener question on the possible negative consequences of the open science movement—are things moving too quickly? Links and things we discuss in the episode: * We have a new logo, if you haven't already noticed... * Contact us via our website form (https://everythinghertz.com/contact)! * Considering the potential downsides of open science * Here come dat boi meme explination (https://knowyourmeme.com/memes/dat-boi) * The dangers of open access by fiat * The role of commercial entities in open science * The “University of Oslo fancy Norway people-pay-taxes oil money bloody library (https://www.ub.uio.no/english/)” * Dropping the success rate of grants to increase the quality of evaluation * Reframing open science reform efforts to a mission of equity and fairness * We don’t know the process behind university sexual harassment/misconduct investigations * Does transparency even matter if people won’t follow up on problems? * James' prediction: If someone starts a journal that ONLY does Register