Talk:WikiJournal of Science/WP Experiment

Ideas behind this experiment

See this Talk page and this essay. Sylvain Ribault (discuss • contribs) 19:04, 5 June 2019 (UTC)

Suggested Wikipedia article

w:Quantum electrodynamics: a GA that was not submitted to WJS in spite of an invitation to do so, with no dominant author, with technical content, on a rather fundamental subject. Sylvain Ribault (discuss • contribs) 21:36, 14 June 2019 (UTC)

That looks like a sensible first test. As you say, it's a core topic (level-4 vital) and will be interesting to see what external reviewers reckon of it. It was made a GA 2010 and there's also been plenty of edits since then. There's also quite a few Level 3 vitals that could be good candidates for followup tests. T.Shafee(Evo﹠Evo)^talk 01:05, 18 June 2019 (UTC)

Looking at vital articles is an interesting suggestions. Level 3 vitals are often not very technical, level 4 and level 5 vitals might be easier to deal with. Sylvain Ribault (discuss • contribs) 21:13, 18 June 2019 (UTC)

Procedure

We describe the experiment's workflow, with the case of w:Quantum electrodynamics as the first example.

1. Announce the intention to do peer review on the article's Talk page, with a link to the experiment's page, and a permanent link to a reference version of the article. (The current version at that time.)
2. Send invitations to reviewers. Draft email template:

Dear X,

With about 400 pageviews per day, the Wikipedia article on w:Quantum electrodynamics is an important resource on the subject, particularly for non-specialists. However, this article was written by pseudonymous authors, and has not been reviewed by specialists.

On behalf of the WikiJournal of Science, I am inviting you to review this article: give your opinion on its scientific merit, and suggest improvements.

The article on Quantum electrodynamics was not submitted by its authors, but selected by WikiJournal editors. The reviews will be publicly available, indexed and citable. (Reviewers will not be anonymous.) The reviewers and WikiJournal editors may improve the article themselves, or wait for Wikipedians to act on their recommendations. For more details on the procedure, see this page.

Do not hesitate to contact me if you have any questions on this invitation. 

Best regards,

Y (Z & Editorial board, WikiJournal of Science)

------------------------------------

Permanent link to the version under review: https://en.wikipedia.org/w/index.php?title=Quantum_electrodynamics&oldid=896095671

The WikiJournal of Science has been organizing peer review of a number of scientific Wikipedia articles, which has led to important improvements and/or the discovery of important flaws, including in articles that were rated Good Articles by Wikipedia. (For example, w:Surface tension.) The journal has so far worked on Wikipedia articles that were submitted by their main authors. With Quantum electrodynamics, we are now testing a new review procedure.

Provisional outcome and discussion

Three reviewers accepted in principle to review w:Quantum electrodynamics, but none of them did it after a few months and a few reminders. Assuming this outcome is significant (i.e. not a random fluctuation), the reasons might be:

• As is generally the case with the WikiJournal of Science, the editor in charge is not a specialist of the particular topic, and is therefore taken less seriously by the reviewers.
• We are asking people to review Wikipedia, when they do not write in Wikipedia. Disguising Wikipedia articles as journal articles might really be needed for stimulating reviewers.
• The absence of an identified author may make reviewers take the process less seriously. In the presence of an author who is a known specialist of the subject, the fact that the editor is not a specialist may matter less. Moreover, the presence of an author makes it more likely that the reviewers' recommendations will be useful.

Possible conclusion: Reviewers prefer working for an unknown journal, than for the well-known Wikipedia. We should focus on inciting academics to write in Wikipedia and submit to WikiJournals.

---

I would disagree that reviewers prefer to work with unknown journals. I have reviewed many articles as have many of my colleagues. I have reviewed for PLoS One, Nature and several other high ranking journals that are hardly unknown. The fame of a journal in some ways may go against whether I review for them.

Wikipedia may be partly the issue. In academia Wikipedia generally does not have a good reputation. Personally I think this is largely misunderstanding. But there are some issues as well. As such I think its more likely that because its Wikipedia is the problem and hence not taken seriously.

The absence of identified authors is an issue. In general as a reviewer the author contributions is an important issue, if an article is multiply authored this detracts from the confidence in what is written. This would extend to making it difficult for people to review these articles.

Another factor may be the method of review, it may be seen as unusal by many academics.

Cheers Scott Thomson (Faendalimas) ^talk 02:34, 21 December 2019 (UTC)

Disappointing, but useful to know. I was actually expecting that the problem would be more in the implementation of reviewer recommendations without nominated people who agree to take responsibility. Disappointing that reviewers were also difficult to secure. As for the well known encyclopedia vs unknown journal aspect, I've found that the combination of being a journal plus being relevant to Wikipedia seems to be attractive to potential reviewers from a "bringing Wikipedia up to academic standards" direction. T.Shafee(Evo﹠Evo)^talk 07:02, 5 January 2020 (UTC)

The idea that the authors matter even if they do nothing may not be that surprising from a psychological point of view. Sylvain Ribault (discuss • contribs) 16:52, 5 January 2020 (UTC)

By chance, a related conversation came up at Facebook's Wikipedia Weekly, so I've summarised a few of the links shared here: w:Wikipedia_talk:Expert_help. Seems like there are a few features in common for similar experiments. T.Shafee(Evo﹠Evo)^talk 09:53, 9 January 2020 (UTC)