CoARA-ERIP Questionnaire for Cooperation with CoARA Members

Coalition for Advancing Research Assessment (CoARA) Ethics and Research Integrity Policy in Responsible Research Assessment for Data and Artificial Intelligence (ERIP) Working Group (website: https://coara.eu/coalition/working-groups/wg-erip/)

Question Title

1. What are the biggest challenges you currently face in assessing digital contributions to science and AI? (Required.)

Lack of standardized assessment criteria: difficulty in finding or applying consistent criteria for evaluating digital research outputs.

Data quality and integrity issues: challenges in ensuring the accuracy, reliability, and integrity of data used in assessments.

Difficulties in identifying AI research outputs in the context of research assessment.

Difficulties in assessing the value and impact of research outputs created using AI.

Integration with existing research management systems with research assessment systems and practices.

Lack of sufficient technical knowledge or tools to effectively assess complex digital contributions.

Limited in-house access to training and resources needed for the proper assessment of digital contributions.

Difficulties in assessing the compliance of digital contributions with relevant regulations, such as the GDPR or the EU AI Act.

Difficulties in facilitating cross-disciplinary, cross-institutional, or cross-border collaboration in research assessments.

Challenges in ensuring that research assessments are transparent and that results can be reproduced.

Limited time or resources available to conduct thorough assessments of digital research outputs.

Other (please specify)

Question Title

2. Which of the following needs do you see as needed to advance research assessment in the digital space?
(Select all that apply) (Required.)

Automated assessment tools automatically evaluate digital research outputs, such as software, data sets, and algorithms for their value, impact, and eventual compliance with ethics and quality standards.

Ethics and research integrity guidance best practices for ensuring ethical considerations are integrated into the development and use of AI in research and research assessment.

Integration of digital assessment tools with existing research management systems to seamlessly integrate with existing research management and evaluation systems within your organization.

Visualization tools for research assessment practices in the digital space

Interactive visualization tools that help illustrate the integrity, value, impact, and ethics considerations of digital research contributions and assessment.

Checklists and templates designed to help researchers and institutions comply with regulatory standards, such as GDPR and the EU AI Act.

Customizable evaluation metrics/features that allow users to tailor assessment criteria and metrics to fit the specific needs of their projects or institutional standards.

Collaborative integrated platforms within the toolkit that enable cross-disciplinary and cross-border collaboration on research assessments.

The portability of research assessment tools for integration on disparate systems and platforms and within various organizations or national infrastructures.

Training and onboarding resources providing access to training materials, tutorials, and guides that facilitate the effective use of the toolkit and the implementation of best practices for research assessment.

Reporting and feedback mechanisms/tools that allow users to generate detailed reports on research assessments and provide feedback on the toolkit’s functionality for regular improvement.

Other (please specify)

Question Title

3. What platform(s) would you like to see used for building a digital research assessment infrastructure?
(Please select all that apply.) (Required.)

European Open Science Cloud (EOSC)

Global Open Science Cloud (GOSC)

African Open Science Platform (AOSP)

Open Science Framework (OSF)

Interplanetary File System (IPFS)

Google Cloud Platform (GCP)

Other (please specify)

Question Title

4. What software, systems, or tools are you aware of that would support or could be integrated into the development of a digital platform for research assessment?
(Please select all that apply.) (Required.)

ORCID

OpenAIRE

Datacite

Dataverse

Figshare

Open Review

Altmetric

SciVal

Dimensions

CrossRef

Other (we are particularly interested in other software or ongoing projects that could be integrated)

Question Title

5. In which areas of research assessment, data integrity, or AI ethics do you see synergies or areas for cooperation with CoARA-ERIP? (Required.)

Question Title

6. Is your institution a member of the Coalition for Advancing Research Assessment (CoARA)? (Required.)

Yes

Not yet, considering becoming a member

Question Title

7. In what country is your organization/institution located? (Required.)

Question Title

8. Name of your organization/institution (Required.)

Question Title

9. Contact Person (Required.)

Question Title

10. Contact E-mail Address (Required.)

Question Title

11. The responses provided here to questions 1 to 7 will be shared with the members of CoARA-ERIP and more widely. The personal information provided here in questions 8 to 10 will be shared only with the CoARA-ERIP Co-chairs and Facilitators. (Required.)

I agree to having the personal data I provided here in questions 8 to 10 processed by the CoARA-ERIP Co-chairs and Facilitator. Questions 1 to 7 may be processed and shared by the CoARA-ERIP Co-chairs and Facilitator.

CoARA-ERIP Questionnaire for Cooperation with CoARA Members

Question Title

* 1. What are the biggest challenges you currently face in assessing digital contributions to science and AI? (Required.)

Question Title

* 2. Which of the following needs do you see as needed to advance research assessment in the digital space?(Select all that apply) (Required.)

Question Title

* 3. What platform(s) would you like to see used for building a digital research assessment infrastructure?(Please select all that apply.) (Required.)

Question Title

* 4. What software, systems, or tools are you aware of that would support or could be integrated into the development of a digital platform for research assessment?(Please select all that apply.) (Required.)

Question Title

* 5. In which areas of research assessment, data integrity, or AI ethics do you see synergies or areas for cooperation with CoARA-ERIP? (Required.)

Question Title

* 6. Is your institution a member of the Coalition for Advancing Research Assessment (CoARA)? (Required.)

Question Title

* 7. In what country is your organization/institution located? (Required.)

Question Title

* 8. Name of your organization/institution (Required.)

Question Title

* 9. Contact Person (Required.)

Question Title

* 10. Contact E-mail Address (Required.)

Question Title

* 11. The responses provided here to questions 1 to 7 will be shared with the members of CoARA-ERIP and more widely. The personal information provided here in questions 8 to 10 will be shared only with the CoARA-ERIP Co-chairs and Facilitators. (Required.)

1. What are the biggest challenges you currently face in assessing digital contributions to science and AI? (Required.)

2. Which of the following needs do you see as needed to advance research assessment in the digital space?
(Select all that apply) (Required.)

3. What platform(s) would you like to see used for building a digital research assessment infrastructure?
(Please select all that apply.) (Required.)

4. What software, systems, or tools are you aware of that would support or could be integrated into the development of a digital platform for research assessment?
(Please select all that apply.) (Required.)

5. In which areas of research assessment, data integrity, or AI ethics do you see synergies or areas for cooperation with CoARA-ERIP? (Required.)

6. Is your institution a member of the Coalition for Advancing Research Assessment (CoARA)? (Required.)

7. In what country is your organization/institution located? (Required.)

8. Name of your organization/institution (Required.)

9. Contact Person (Required.)

10. Contact E-mail Address (Required.)

11. The responses provided here to questions 1 to 7 will be shared with the members of CoARA-ERIP and more widely. The personal information provided here in questions 8 to 10 will be shared only with the CoARA-ERIP Co-chairs and Facilitators. (Required.)