Center For Open Science Puns

Last Thursday, our article "Estimating the Reproducibility of Psychological Science" was published in Science. Coordinated by the Center for Open Science, we conducted 100 replications of published results in psychology with 270 authors and additional volunteers. We observed a substantial decline effect between the original result and the replications. This community-driven project was conducted transparently, and all data, materials, analysis code, and reports are available openly on the Open Science Framework.

Ask us anything about our process and findings from the Reproducibility Project: Psychology, or the initiatives to improve transparency and reproducibility in science more generally.

We will be back at 12pm EDT (9 am PT, 4 pm UTC), AUA!

Responding are:

• Brian Nosek, Center for Open Science & University of Virginia
• Johanna Cohoon, Center for Open Science
• Mallory Kidwell, Center for Open Science

[EDITED BELOW] Some links for context:

1. PDF of the paper: http://www.sciencemag.org/content/349/6251/aac4716.full.pdf

2. OSF project page with data, materials, code, reports, and supplementary information: https://osf.io/ezcuj/wiki/home/

3. Open Science Framework: http://osf.io/

4. Center for Open Science: http://cos.io/

5. TOP Guidelines: http://cos.io/top/

6. Registered Reports: https://osf.io/8mpji/wiki/home/

12:04. Hi everyone! Mallory, Brian, and Johanna here to answer your questions!

12:45. Our in house statistical consultant, Courtney Soderberg, has joined us in responding to your methodological and statistical questions.

3:50. Thanks everyone for all your questions! We're closing up shop for the holiday weekend but will check back in over the next few days to give a few more responses. Thanks to all the RPP authors who participated in the discussion!

I heard about it on this great podcast from 80,000 Hours.. From what they say, direct initiatives like these tend to have a good track record of passing, and indeed Fargo's won by a landslide. Approval Voting is the clear favorite among experts in voting methods.

North and South Dakota, Oklahoma, Missouri, Arkansas, and Ohio seem like possibly the best first moves in terms of absolute number of signatures needed and percent of the state population, and this brave soul is going for a constitutional amendment in Florida. I reached out to a friend in Oklahoma and he's interested in starting an Approval Voting petition there. Who wants to take on these other states?

EDIT: I should've included Massachusetts, which requires 75,542 signatures (or 1.1% of the population) to get a direct initiative on the ballot.

Scientists value transparency and reproducibility, but are rewarded for highlighting the novelty of unexpected findings. This is one reason why published research findings are hard to reproduce. See, for example, the recent work done by us and the scientists involved in the Open Science Collaboration on Estimating the Reproducibility of Psychological Science (https://osf.io/ezcuj/wiki/home/).

When scientists preregister their research, they are making key decisions without being biased by the data they collect, which makes standard statistical tests more effective. Though preregistration is required by law for clinical research involving human medical studies, it is not widely practiced by most scientists. We at the Center for Open Science have $1,000,000 to hand out as prizes for researchers who publish the results of their preregistered research. See https://cos.io/prereg

We'll be back at 12 pm ET (9 am PT, 5 pm UTC) to answer your questions, Ask us anything!

Answering questions today:

Courtney Soderberg is our Statistical and Methodological Consultant who advises researchers on best practices in experimental design and statistical analysis to make their work more reproducible.

Jolene Esposito works with researchers in Africa to to improve the rigor of their work using the tools we've made, such as the Open Science Framework (osf.io)

April Clyburne-Sherin is our Reproducible Research Evangelist who conducts workshops to train researchers on reproducible research methods and open science tools.

David Mellor works on encouraging researchers to preregister their work on the Open Science Framework.

Hello Reddit! http://imgur.com/DpMrjKV

(edits for formatting, picture, our names)

Edit 2 PM EST: Thanks for all of your questions everyone! We've enjoyed talking to you. We will come back later today to see if any more questions are up. Follow us on Twitter! @OSFramework

Hello! Current employee of the Carnegie science center here, throwaway to avoid being doxxed. So even with the ban on events over 25 people, the science center is open.

Be aware we have closed many normal attractions (including the ropes course and rock wall)

Everything is constantly cleaned by our staff

We are lowering our capacity to 10% for this week,

All employees are wearing masks and all customers over the age of 2 must wear one as well.

We are working hard to keep things clean but there are going to be holes. There are a lot of spots that are touched often that can’t be cleaned after every use.

Employees are just as afraid of the virus as you, we lose our unemployment now that we are open so we have to be here.

If you don’t feel comfortable, don’t come If you are high risk and cannot wear a mask, don’t come If you disagree with us being open for this time, please don’t come.

Heya!

So I'm an intern over at the Center for Open Science in Charlottesville, VA. We'd really like someone who knows 'R' and python; who is personable; who has research experience; and who knows and cares about reproducibility and open science. Is this order too tall?

If you've got these particular skills and feel like you want to apply with us, follow this link to have a look at the full description and to have a look around our website : http://centerforopenscience.org/jobs/

From Job posting: "SKILLS: Substantial training and experience in scientific research, quantitative methods, and reproducible research practices; Extremely high social and communication skills; Exceptional organization and attention to detail; Service-oriented mindset; Diverse experience with technologies for data collection, analysis, and visualization; Ability to use web communication and documentation software effectively; Team-oriented; Very strong work ethic; Self-starter and industrious; Adaptivity to rapidly changing demands in a high performance workplace; Very strong writing skills. Skills in programming (at least R, ideally Python or Julia), web development, and data analysis are essential."

(We're a great office, and there are a bunch of redditors here. Lots of quality people too, but I'm biased :D )

Anyway... any help with how you might suggest we attract the right statistical folks would be appreciated.

Cheers!

TL;DR We are looking for qualified statisticians who love stats and open science, any suggestions on where to look?

EDIT: We've had a lot of good applications, just not as many as we would have hoped. AND I have nothing to do with the hiring process so if you're going to apply, please follow the link provided.

EDIT EDIT: This got way more attention that I expected, I've explored some of the routes that were suggested, and I've taken some feedback from this back to the company. Thanks everyone!

EDIT EDIT EDIT: Also, we are looking for a consultant to work for us. Because we do consulting. I'm not sure if that was clear. http://centerforopenscience.org/stats_consulting/

I am one of the co-founders of the COS (http://centerforopenscience.org). As I said in the title, we're a newly-founded, newly-funded non-profit with the goal of decreasing the gap between scientific values and scientific practices. We're in the process of hiring developers, but we'll be a predominately Python, exclusively open source shop.

The center has three main activities:

1. create an infrastructure for facilitating open science (the Open Science Framework; OSF--a dissertation project that will be getting overhauled)
2. foster the open source/open science community (coordinate events, give out grants, etc.)
3. conduct meta-science (science of science) research.

You can find out more about us at our website or through the press-release announcing the starter grant we received (also some press coverage: science, nat. geo.).

Why am I here? We've already benefited a lot from reddit: I've asked reddit questions, we discovered a great design/UI consulting group that we are eager to work with, and I've met a few student programmers who have taken on projects with us. Now, I'm looking to see if others are interested in getting involved. We're working on a variety of packages that we'll use for the OSF, but want to be highly modular to maximize use for others in the community. Our stack currently consists of Flask, Mako, pymongo, beaker, git, jquery, ember.js, d3.js, mongodb, nginx, uwsgi, celery, and rabbitmq. We also want to support API-style collaborations with projects that will facilitate openness (github, for instance; lots of API development needed) and have a redditor-led project creating a citation parser and input API to create a freely accessible dataset of the academic citation network (parser, database, visualization help/advice needed).

We'll be hosting a sprint this year at PyCon, and welcome any help/advice we can get. Even if you aren't going to PyCon, I'd like to make your acquaintance (maybe we can do a group Google Hangout) to talk about common interests. You'll also be seeing ads for jobs pretty soon, so turn off ad-block. ;)

So, if you're interested, leave a comment, send me a message, or co