Open academic practice and Excellence in Research for Australia

Abstract

Type classification: this is an article resource.

Educational level: this is a research resource.

Completion status: this resource is ~25% complete.

Advocates of open academic practices argue greater public and private benefits than closed practices. Recent developments in Australian and international lobbying and resulting government policy, show an increasing support for such practices. Australian universities and research centres are yet to respond however, with key drivers of their governance limiting development. Perspectives on intellectual property remain protective, and support for publishing in open journals is limited.

This paper examines the extent to which the Australian Research Council's Excellence in Research Australia (ERA) journal lists, recognise and value open academic journals. A team of researchers in the fields of health, education and governance at the University of Canberra have reviewed the 2010 and 2012 guideline documents of ERA, looking for statements or processes that either encourage or discouraged open academic practices. They also examined ERA-ranked and unranked journal lists, rating journals that appear in that list to a criteria of open access, open licensing and open formats.

The 2010 and 2012 ERA Guideline documents make X mention of open academic practices, and the journal ranking lists are made up of X% open journals in fields of health, education and governance. This study concludes that the ERA initiative offers X recognition or valuing for avenues of open academic practices, and recommends that the ERA project should X it's criteria, and X to reflect Federal Government recommendations, and similar international trends.

Keywords

open academia, open journals, open publishing, open access, open licensing, open formats, Excellence in Research for Australia (ERA)

Introduction

This picture is necessary because Leigh has never been known to edit a page which does not have a picture . It also helps to convey that this article is in part about the politics of academic journals.

Open academic practices arguably offer greater benefits to public and private investors in research and education than do closed practices, however the majority of scientific and peer-reviewed research and other academic work, is published with exclusions on access, with all copyrights reserved, and using formats that are difficult to distribute and reuse. Restricting access and reuse of academic work offers little in the way of wider dissemination of research findings, such as distribution through niche and target publishing channels, or translation and transformation into other languages and media. In other words, restrictions on access and reuse offer limited public benefit, and little to no new business opportunity.

Recent developments in Australian and other national and international policy toward intellectual property indicate growing acceptance, if not encouragement of free and open access and reuse of research. In Australia, the Australian Ministry for Finance and Deregulation endorsed recommendations made to it by the Gov 2.0 Taskforce (2010), that Public Service Information be free and open for reuse and adaptation, by adopting the Creative Commons Attribution copyright license as a default copyright statement.. Public Service Information arguably includes that of agencies tasked with funding research.

In New Zealand, their Government Open Access and Licensing Framework (NZGOAL) also instructs state service agencies to make their works available online for re-use under the Creative Commons Attribution license.

In Europe, The Commission of the European Communities recommends that member states promote the broad dissemination of knowledge created with public funds, by taking steps to encourage open access to research results.

In the United States, The USA Federal Research Public Access Act] requires 11 U.S. government agencies with annual extramural research expenditures over $100 million to make manuscripts of journal articles publicly available via the Internet.

On the other hand, major publishers of academic literature present their case for restricted access and reuse as being necessary and beneficial to the dissemination of information. they cite their management of a trusted brand, reliable information management, review processes, and clear and established lines of contact with major academic institutions and libraries, as the value in their business model, and argue that restricting access reuse to a user pay model is critical to ensuring a financial return.

Weighing up the arguments for and against open academic practices, it is our position that academics, particularly those whose work is majority public-funded, should be encouraged and supported towards open academic practices. At the very least this would mean publishing research findings in journals which provide free and open access to all their articles, with copyright licensing that permits free reuse, and that publishes articles using open file formats that facilitate ease of reuse. Further along a continuum of openness, researchers and their publishers might strive toward transparency in their processes, such as review of articles being made accessible, data and research processes being made accessible, even the data that illustrates the finer detail of research project itself, so that emerging researchers might learn more from past work, and to widen the opportunities for participation in research and learning. An over used but appropriate comparison would be with the principles and practices of open source research and development, which includes computing software for servers, desktops and portables; content such as encyclopedias, books, dictionaries, news; architecture, design, engineering, farming, cooking, and many other fields of research and development that either have their roots in Commons based development and distribution, or are moving toward it.

This research project has evaluated the Australian Research Council's primary mechanism for recognising and compensating public Australian research - the Excellence in Research Australia (ERA), for recognition and support for open academic practices. To make such an evaluation, we first arrived at three criteria for open academic publishing, and surveyed the ERA's journal lists for journals that meat that criteria:

1. open access - where access to published work is available online.
2. free copyright - where people are free to make copies of and derivatives from the original (such as translations and new formats), and to redistribute those copies and derivatives either commercially or non commercially.
3. and reusable formats - where the original is made available in a format that facilitates easy access, copying and reuse, such as an HTML document.

We see these three criteria as minimum standards for open academic publishing, and acknowledge there is plenty of scope for furthering a culture of openness beyond these 3 criteria, such as with open governance - where the data, processes and reviews of research projects are likewise made accessible and reusable.

This paper describes our methods in detail, summarising our findings - with full access to the data and discussion record that make up the process of the project. Finally, we make recommendations to independent and institution based researchers who would like to progress toward open academic practices, while at the same time benefiting from the ERA recognition and compensation, as well as the distribution channels presently exploited by exclusive journals.

The position of open academic practices

Shortcut to this page:
http://tinyurl.com/openkca
Other versions and resources:
Video on Wikimedia Commons
Video on Archive.org
Video on Blip.tv
Video in two parts on Youtube
Image stack on Picassa web galleries
Slides: Leigh Blackall, James Neill

The continuum of open academic practices

1. Open access to published findings
2. Free copyrights over published findings
3. Reusable formats for published findings
4. Open access and reusable formats to reviews of research findings
5. Open access, free copyrights, and reusable formats for the data gathered in a research work
6. Transparency in the processes, discussion topics and decision points of research work
7. A general culture of disclosure, transparency, openness, inclusion, participation and action

• Our focus is on a minimum standard. Our standard ends at 3/7, where published findings should be open access, free in copyright, reusable in format. We have however, in the conduct of this research project, demonstrated the exercise of the full spectrum of open academic practices, where the reviews to our methods and findings are accessible, that data that makes up our research is accessible, freely licensed and available in reusable formats, the processes of the project have been transparently documented and made accessible, and our conduct has been in the spirit of disclosure, transparency, openness, inclusion, participation and action through the use of a popular wiki, and the development of a scalable and reusable method.

What makes academic practice open and why does it matter?

• Arguments for/against open academic practices
• Arguments for/against openly published research findings
• Criteria and examples of open journals
• Each discipline to present one good example of an open journal in their field
• Examples of new, innovative or emerging alternative research publishing practices

The problem with closed academic journals

• Publications are authored, edited and often reviewed by publicly funded academics, and the journals do not compensate for this time
• Review and publishing turn around time for most journals is slow
• Journals take the copyrights from the authors, often obtaining exclusive rights
• Journals charge fees on a pay-per-view basis
• Journals charge public libraries access fees
• Journals collect royalties for copies made of a publication that they own copyright for, but the collection agencies do not check that the royalties go back to the author in full

The purpose of the Australian Research Council

• ARC mission and why the ERA
• ERA Guidelines
• ERA journal listing process
• ERA Criteria
• ARC and ERA public consultation process

Research questions

1. To what extent does the Australian Research Council's Excellence in Research Australia (ERA) initiative recognise international policy trends in open publishing as they relate to research?
2. To what extent does ERA include open journals in its journal lists?

Method

Seven early to mid-career researchers in the fields of education, health and government, sought review what open academic practices are, determine to what extent such practices are trending internationally, establish a minimum standard criteria for evaluating academic practices for openness, and to evaluate the Australian Research Council (ARC) for awareness and representation of open research practices.

The discipline areas represented in the team of researchers were established based on staff availability and interest at the University of Canberra, largely due to the unfunded and voluntary nature of the project. Two of the lead researchers conducted a literature review, looking for existing research, coverage and commentary on open academic practices broadly. The team used that review to establish a common understanding of open academic practices, and to developed a simple 3 point evaluation criteria that represented a minimum standard of open practice. They then used the ARC's Excellence in Research Australia (ERA) initiative to evaluate ARC's readiness to recognise and support open research practice. The team reviewed the ERA guideline documents for any mention of open academic practice, then surveyed a 50 item random sample, for each of the discipline areas represented in the research, from the journals listed by ERA, evaluating each listed journal on a 3 point criteria of:

1. open access,
2. free copyright and,
3. reusable format.

The result of each journal's evaluation for each of these criteria was recorded as a 1 for "yes", a 0.5 for "yes with conditions", or a 0 for "no".

From the findings of the survey of the ERA journal list samples and the review of the ERA guideline documents, this project believes it can make some quantifiable judgement on the extent to which the ARC recognises, supports and encourages open academic practices.

Researchers

Researchers were recruited from a cross-section of the University of Canberra's (UC) disciplines to check and rate their discipline's Excellence in Research Australia (ERA) ranked journal list on openness criteria. Where disciplines had large fields, the top-ranked journals were selected.

1. Kasia Ball, Nursing, University of Canberra (UC)
2. Leigh Blackall, Education, UC
3. James Neill, Psychology, UC
4. Diane Phillips, Government, UC
5. Jamie Ranse, Disaster management, UC
6. Ben Rattray, Sport and exercise, UC
7. Sarah Stewart, Midwifery, Griffith University

Review of literature

List references here initially. Turn into an annotated bibliography at least. Incorporate it into discussion at best.

• http://www.ands.org.au/publishing/licensing.html

"A fundamental aspiration of the Australian Research Data Commons is that more data is re-usable. One of the essential ingredients of re-usable data is clarity of re-use permissions, terms, and conditions. Prospective re-users need to know exactly what they can do with the data. Those conditions and permissions should be explicit. "

• Butler, L 2003, Modifying publication practices in response to funding formulas, Research Evaluation, vol. 12, no. 1, pp. 39-46.

"The reaction of Australian academics to the use of one performance measure, raw publication counts, can be starkly illustrated using data from the Science Citation Index. The mid-1990s saw the first distribution of research funds to Australian universities based on a formula encapsulating a number of performance measures (graduate student numbers or completion rates, research income, and publications). Many universities reinforce the sector-wide signals by allocating the money they receive under these programs back to the departments, or even individuals, who ‘earn’ them. The reaction of Australians to these signals is entirely predictable — their publication output has increased dramatically in the last decade. But as quality is paid scant regard in the measures, there is little incentive to strive for the top journals, and this paper shows that the biggest increase has been in those journals at the lower end of the impact scale."

• ERA Key Documents 2010
• 2012 Draft Guidelines
• Ranked Journal List Development 2010
• ERA journal lists
• ERA listed journals (Lamp. 2010; Deakin uni)
• ROMEO/SHERPA
• UCANSearch
• E-Journals
• Directory of Open Access Journals

Formulation of minimum standard evaluation criteria

How did we arrive at the three point evaluation standard...

• Open access
• Free copyright
• Reusable format

Open access

• The online access to a digital format of a publication is not limited or restricted in any way

Conditions

1. Time based: After a certain time period, the journal allows open access
2. Issue Based: After number of issues, the journal allows open access
3. Topic based: The journal has open access for a particular theme / topic issue
4. Selected Manuscripts: Open access to a select amount / type of manuscript after free registration
5. Note: this is not the final set of access conditions - see spreadsheet

Free copyright

• The copyright used to manage the copy and reuse of the document does not restrict someone's options beyond the moral obligation of recognition to original authors and publishers.
• To this end we look for copyright licenses equivalent to:

• Public Domain
• Creative Commons Attribution
• Creative Commons Attribution Share Alike
• General Public License

Conditions

1. Note: this is not the final set of access conditions - see spreadsheet

Reusable format

• In a format that is usable on all devices, such as HTML. (PDF is not accessible to many portable computing devices, and can (depending on the publisher) be rendered unusable for people relying on assisted technology.
• In a format that can be easily edited, copied, and otherwise appropriated. For example, the ease in which a person can copy and paste text from a document (where PDF too often presents inconvenience), or so that a more technically skilled person can program software to automatically search and lift content from a document based on a standard formatting structure in the document, such as in HTML and XML. Such formats enable re purposing of the content such as for special printing processes, republishing online, and syndication.

Conditions

1. Note: this is not the final set of access conditions - see spreadsheet

Review of the 2010 and 2012 ERA guideline documents

Leigh read through the 2010 and draft 2012 guideline documents, highlighting words, sentences, sections and chapters that mention or explain open academic practices, or where there are links and references out about it. Leigh wrote a brief report on both documents. James checked the data and report for accuracy. The research team read, suggested edits and agreed to a final version. The final version was submitted to ERA for feedback, along with the journal list survey, with acceptable changes made based on that feedback.

1. Found the link to ERA 2012 on the ERA site: http://www.arc.gov.au/era/default.htm > http://www.arc.gov.au/era/era_2012/era_2012.htm
2. Followed through the the ERA 2012 Key Documents link > http://www.arc.gov.au/era/era_2012/key_documents_2012.htm
3. The only format provided for the 2012 Guidelines is a 1.5 meg PDF > http://www.arc.gov.au/pdf/era12/ERA2012_SubmissionGuidelines.pdf
4. Passed that PDF to a friend who is totally blind, for his assessment of the document's accessibility for people like him...
5. Word searched the document for:

• open academia = 0
• open academic = 0
• open access = 5
• creative commons = 0
• copyleft = 0
• accessible = 1
• ...

Open access

The ERA 2012 Guidelines mention open access five times in the document. Three appear within section six of the document - Other Matters, and two appear within the same one item of the glossary.

With regard to the first three, they appear in sub section 6.6 - Managing Copyright in Research Outputs Nominated for ERA Peer Review and Related Material. In subsection 6.6.1 - Where Copyright Is Owned by Institutions, the first is used within quotation marks:

"Where an institution owns the copyright in relevant material, the institution must provide the ARC and ERA reviewers with access to such relevant material, as described in 6.6.

For journal articles, book chapters and conference publications in cases where the research output is available in electronic form, this should take the form of storage in an institutionally supported repository on an 'open access' basis"

The next two uses of the phrase are found in 6.6.2 - Where Copyright Is Owned by Eligible Researcher, both are again within quotation marks:

"Where one or more eligible researchers own the copyright in research outputs nominated for ERA peer review, then the researcher(s) should be encouraged to give permission for that material to be used for the purposes of ERA so as to allow access to it as described in 6.6. The permission should allow relevant research outputs, where available in electronic form, to be stored in a repository on an 'open access' basis.

Where such eligible researchers do not expressly consent to the provision of access to relevant material for ERA purposes in the manner described in 6.6, copyright will be managed in accordance with 6.6.3.

Where depositing research outputs in an 'open access' repository was a condition of any funding which enabled the research to be undertaken, full public access to the research output(s) should exist, irrespective of the ERA submission process, as a result of the eligible researcher complying with that funding condition."

The final two uses of the phrase open access are both within the same item of the glossary for the document: Open access repositories, defined as: "Open access repositories provide free public access to research outputs via the internet."

The use of the phrase open access is both important and significant. The Australian Research Council is instructing institutions that participate in Excellence in Research Australia, to make research publications publicly available in open access repositories. In section 6.6.3. it is explained that the ARC can require this, even of material that is owned by third parties, because it is a Commonwealth Agency exercising it's provision under section 183(1) of The Copyright Act.

"Acting under section 183(1), the ARC (representing the Commonwealth) on 1 June 2009 authorised each institution participating in ERA to do acts in the copyright of relevant material owned by third parties for the purposes of ERA. This authorisation means that an institution participating in ERA may make all uses of relevant material owned by third parties that are necessary or convenient to enable it to participate in ERA."

In short, the participating institutions are to make a copy of all ERA related works - including those where copyright is owned by a third party, publicly available on an open access repository.

However, it appears that within section 6.6. is an opportunity to conservatively interpret these ERA Guidelines. At the outset it stipulates a term 'relevant material' which "...means research outputs identified by institutions for ERA peer review...". The "ERA Review" is outlined in section 3.5.2. and described as a process where where a discipline goes under review, and is required to provide 30% of its materials submitted to the ERA process. That there, that 30% is what is meant by 'relevant material'. At best, this can be surmised to mean that less than 30% of research submitted for ERA assessment, should be placed on a publicly accessible open access repository, as previously described. At worst, it only means 30% of research if, and only if under an ERA Review.

A comparison between the 2010 and 2012 documents was made difficult by the ARC, when the 2010 Guidelines were taken off the ERA website (sometime before 26 September 2011). A copy was obtained for this review however, and the sections referring to 'open access, 'relevant material' and 'ERA Review' remain exactly the same. It seems to us that the ARC is overly conservative in it's use of the provision open to it under section 183(1) of The Copyright Act, and could go further and require all research that is submitted to it - a publicly funded body for recognising and compensating research, be made publicly available on an open access repository. Doing so would then afford the institutions that participate in the ERA process, the protections they need under that provision also

Managing and offering such a repository could however, be an expensive and cumbersome operation for most institutions, compounding the costs and problems already encumbered in the existing research publication and subscription system of traditionally closed access journals. Institutions would be better to encourage their researchers to engage in copyright negotiations with their publishers, and to take steps toward ensuring that their work is publicly accessible and reusable at the first point of publication - the journal. Failing that, or as well as, individual researchers should take independent charge of the publicly accessible online archiving of their work, by publishing it on an number of free and fee based hosting services, as a second level of backup, or a primary source for access. Failing that, or as a third layer of backup and accessibility, the institution should provide their researchers with a service of publicly accessible online archive, either in its own capacity, or through partnership with third parties such as Archive.org, Wikimedia Foundation, or the Australian National Library.

Survey of ERA ranked journals against open publishing criteria

The researchers used the official downloadable ERA 2010 finalised journal lists and the copy of the ERA lists made available through Deakin University.

Journals were checked for three criteria which identify openness:

1. Free and open access to publications online
2. Open copyrights such as the Creative Commons Attribution and/or Share Alike license or equivalent
3. Open standard file formats such as in Hyper Text Markup Language (HTML), Rich Text Format (RTF), Open Document Text (ODT) or equivalent. Portable Document Format (PDF) while recently becoming an open standard, was not included owing to the difficulties in usability still commonly found in the format's use.

Data was obtained for each journal through a combination of the following strategies:

1. Web search for the journal's official homepage
2. Search the SHERPA/ROMEO database for information about Author republication rights
3. Check subscription-based electronic journal access via insitutional subscription, e.g., E-Journals @ UC, UCANSearch (Federated institutional subscription journal database search)

The researchers recorded the data for the surveyed journals via a Google Docs spreadsheet using separate discipline sheets. In some instances, journals conditionally satisfied a criteria, such as access being made available after a time embargo, or if an author paid a fee. While not strictly satisfying the criteria of free and open access, such a case was rated 'yes with conditions' and the condition was explained.

Surveyed information was correct to the best of the researchers' knowledge at the time of surveying (mid-2011). Journal data was based on the journal publisher's default policies, however changes to policy occur and exceptions are sometimes negotiated by individual authors.

Notes on the survey of midwifery related journals

There is no field of research that is specific to midwifery in the ERA ranked journals. Midwifery research tends to be lumped in with Nursing, or scattered across a number of other disciples such as Medical and Health Sciences. To find journals that are related to midwifery, Sarah Stewart used the following key words:

• midw, birth, neonatal (nursing, not medicine), obstetric (nursing, not medicine), paediatric ( and pediatric) (nursing, not medicine), pregnancy, antenatal, postnatal, antepartum, postpartum, intrapartum, childbirth, breastfeeding, lactation, newborn, mother, baby, women, family, perinatal.

Notes on the survey of psychology related journals

"Psychology and Cognitive Sciences" is an ANZ-recognised field of research (ANZFoR; 17). The 2010 finalised ERA list for ANZFoR 17 (Psychology) provided 1608 journal/ANZFoR pairs (see Table *), based on 1324 unique journals.

Table *. Journal/ANZFoR pairs for ANZFoR 17 (Psychology) by FoR codes (Australian Research Council, 2010)

ANZFoR Code	Field of Research	No of Journal/ANZFoR pairs
17	Psychology and Cognitive Sciences	22
1701	Psychology	754
1702	Cognitive Sciences	829
1799	Other Psychology and Cognitive Sciences	3
Total		1608

Given the large number of psychology-related journals, the current survey focused only on the 593 Psychology journals with an ANZFor1 code of 1701 (i.e., not journals with 1701 as a second or third ANZFor code; see Table *).

Table *. Frequency and percentage of rankings for journals with ANZFoR1 code of 1701 (ARC, 2010)

Rank	No of Journals	% of journals	Sample size	Sample %
A*	27	4.6	27	100.0
A	85	14.3	25	?
B	177	29.8	25	?
C	293	49.4	25	?
Not ranked	11	1.9	0	0
Total	593	100	102	?

Notes on the survey of education related journals

1. Used John Lamp's data base to collect ranked journals from the 2010 ANZFoR code 13 (Education)
2. Took the first 67 (A-B) journals to assess
3. Assessed all 3 criteria for the first 45 using the SHERPA/RoMEO assessment tool, and Google Search where a RoMEO did not list a journal
4. For the remainder, I stopped using the SHERPA/RoMEO tool, finding Google Search and self assessment quicker and more reliable
5. Also for the remainder, I adopted Jamie and Kasia's method of first assessing Access and if it was accessible, then assessing license and format. If it wasn't accessible, I assessed no further.
6. I returned to John Lamp's database, but it had been updated since the ERA announcement to no longer rank journals, making it difficult to draw out Journal titles I had not already assessed. Lamp's new list was now listed in alphabetical order. I reordered my list accordingly, but owing to my list being based on original rankings, I had less journals listed.
7. Another problem detected by Kasia midway through, was that assessing journals when logged in to UC's computer network, may yield false positive access readings, as the UC network may at times be registered on the journal database and therefore granted access as a fee paying institution. While I have double checked, there is room for error here, so my findings a maximum readings.
8. Am not proceeding at this point, until I can work out the most efficient way to grow my list. Need to perhaps copy past Lamps full list, and somehow delete repeated items.
9. Findings to date:

1. of the 67 Education journals assessed to date, 16 grant open access.
2. of those 16, none offer free and reusable copyrights.
3. of those 16, 2 offer their publications in open, reusable and accessible formats (where PDF is generally not considered an open, reusable or accessible format)

Notes on the survey of government related journals

Included in this section of the survey is Governance and Public Administration.

Results

Review of the 2010 ERA guideline documents

• ERA Key Documents 2010

Survey of ERA ranked journals against open publishing criteria

• Overall results
• General comments on the method
• General comments on the criteria
• General comments about ERA

Discipline results

Psychology

One hundred and twenty two A* and A ERA ranked journals (Commonwealth of Australia, 2010) were reviewed, as well as * non-ranked open journals.

Validation

• Networked discussion on the formulation of evaluation criteria (see discussion page)
• Pre publication response from ARC
• Project delay pending clarification on the dropping of ranking in the ERA list

Discussion

Recommendations

See also

• Collaboration, open research and the commons
• Editable survey of journals datasheet
• Surveying ERA influence on open academic practice: Open academia survey
• ERA Journal List - includes downloadable xls and zip files
• Open academia survey

See also

• Open academic practice survey

External links

• ERA Journal List - includes downloadable xls and zip files
• [1]

References

<references> Pappalardo, Kylie M., Fitzgerald, Brian F., Fitzgerald, Anne M., Kiel-Chisholm, Scott D., Georgiades, Jenny, & Austin, Anthony C. (2008). Open access journals. In Understanding Open Access in the Academic Environment: A Guide for Authors (Ch 5).

Hartsock, N,(1986), Money, Sex, and Power: Toward a Feminist Historical Materialism, Northeastern University Press <references>