Unjournal wants YOU: to do an evaluation. Piloting "Independent evaluations" – (updated, with kickstarter bounty)

By david_reinstein @ 2024-08-16T20:26 (+17)

This is a linkpost to https://docs.google.com/document/d/1Cj6LuHpKYN6SR77t5YZYte1lqu_wGSu3zMB2dcEtmJw/edit?pli=1#heading=h.yr7junmt8irj

Update: we have added a kickstarter bounty incentive: After the first eight quality submissions (or by January 1, 2025 - whichever comes later) we will award a prize of $500 to the most valuable evaluation.

The Unjournal is now inviting, facilitating, and rewarding people to do "independent evaluations". This will happen in parallel with our main mode (commissioning expert evaluations, guided by an ‘evaluation manager’ and compensating people for their work.  

We are also now sharing a database of ~‘all the research we are considering, evaluating, or evaluated’ here.

The Unjournal focuses on commissioning expert evaluations, guided by an ‘evaluation manager’ and compensating people for their work.  

However, we see a range of benefits from also encouraging independent evaluation work, thus we're trying out this new initiative.  

We're looking for your interest, questions, and feedback on the proposed (linked) initiative.  Comments here or in the Gdoc are very welcome! We aim to follow  up with a more detailed post, as we also flesh out some of the details and guidelines.

 

Excerpts from the proposal doc

The Unjournal is seeking academics, researchers, and students to submit structured evaluations of the most impactful research emerging in the social sciences. Strong evaluations will be posted or linked on our PubPub community, offering readers a perspective on the implications, strengths, and limitations of the research.  

[Also] keen to work with students and professors to integrate ‘independent evaluation assignments’ (aka ‘learn to do peer reviews’) into research training.

 

We aim to provide feedback to help you become a better researcher and reviewer. We’ll also give prizes for the strongest evaluations. Lastly, writing evaluations will help you build a portfolio with The Unjournal, making it more likely we will commission you for paid evaluation work in the future. 

 

... See “What specific areas do we cover?” .... We’re especially eager to receive independent evaluations of:

  1. Research we publicly prioritize
  2. Research we previously evaluated (see pubpub.unjournal.org)...
  3. Work that other people and organizations suggest as having high potential for impact/value of information  ...