Source-linked AI summary
COCI: Conference Organisers and Content Identifier
Angelo Salatino, Francesco Osborne, Alexis Vizcaino, Aliaksandr Birukou, Enrico Motta
TL;DR
Calls for Papers are important but difficult to analyse because they are heterogeneous, informal, and poorly represented in scholarly knowledge graphs. COCI combines LLM-based extraction with semantic mapping to structure CfP metadata and connect it to established knowledge bases. Evaluation across 40 CfPs showed that the framework processed all cases, while also exposing dependence on external Linked Data quality.
Problem
Calls for Papers remain difficult to analyse at scale because they lack bibliographic control, structure, and stable distribution channels.
Method
COCI uses LLM extraction, entity disambiguation, and semantic mapping to convert raw CfPs into structured metadata linked with scholarly knowledge bases.
Results
COCI successfully processed all 40 evaluated CfPs and extracted their metadata.
Takeaways & Limitations
The resulting structured data supports integration of CfPs into Scientific Knowledge Graphs and analysis of conference organisers and events.
Takeaways & Limitations
Enrichment reliability can depend on errors in external Linked Data, including incorrect author aliases in OpenAlex.
Abstract
from arXiv · showhide
Despite the critical role of grey literature in scholarly communication, artefacts such as Calls for Papers (CfPs) remain largely isolated from modern Scholarly Knowledge Graphs. The unstructured and highly heterogeneous nature of these documents has traditionally hindered their large-scale processing. In this demo paper, we present the Conference Organisers and Content Identifier (COCI), an AI-based framework designed to extract fine-grained, structured metadata from raw CfP texts. COCI employs a multi-stage pipeline that combines Large Language Models (LLMs) with semantic mapping techniques to integrate extracted entities with established knowledge bases, including OpenAlex, DBLP, TIB ConfIDent, and the AIDA Dashboard. By disambiguating authors and semantically aligning topics and conference series, COCI bridges the gap between informal scholarly dissemination and structured Semantic Web resources, laying the foundation for systematic analysis of non-publisher-based academic events.
1. Introduction
COCI addresses the difficulty of analysing heterogeneous Calls for Papers by extracting structured metadata and aligning it with established scholarly knowledge bases.
- Grey literature captures emerging research directions before they reach formal publication channels.
- Calls for Papers lack bibliographic control, bypass standard publishing pipelines, and circulate through ephemeral informal channels.
- COCI uses Large Language Models to automate granular metadata extraction from unstructured CfP texts.
- The framework identifies conference series, locations, organisers, roles, affiliations, and topics of interest.
- COCI semantically aligns extracted entities with OpenAlex, DBLP, TIB ConfIDent, and the AIDA Dashboard.
2. The COCI Application
The COCI application transforms raw CfPs into rich conference, organiser, and topic representations, then connects them to external scholarly knowledge graphs for broader analysis.
- COCI transforms raw CfP text into structured conference metadata, organiser records, persistent identifiers, and mapped topics.
- The interface displays conference metadata, extracted organisers with affiliations and identifiers, and semantically mapped topics.
- Structuring committee lists supports assessment of organisational diversity, seniority, and conference scope.
- External databases can be used to investigate retractions and flagged questionable practices among organisers.
- COCI could support formal, quantifiable credit for researchers’ community contributions, analogous to Publons for peer review.
3. Architecture
COCI combines LLM extraction with entity matching, semantic validation, and controlled-vocabulary mapping to convert CfPs into interoperable scholarly data.
- Architecture: The workflow begins with prompt engineering and LLM extraction, followed by entity matching and semantic validation modules.
- Architecture: GPT-4o parses CfPs to identify event metadata, topics, and organising committees using a prompt refined on more than 40 CfP files.
- Author and Entity Disambiguation: Author disambiguation matches organisers to OpenAlex using institutions, names, publication counts, and Levenshtein similarity, enriching records with ORCID and ROR identifiers.
- Conference Series Matching: Conference series matching uses 384-dimensional embeddings to align extracted names with DBLP, the AIDA Dashboard, and TIB ConfIDent.
- Topic Mapping: Topic mapping compares embeddings with OpenAlex concept embeddings and applies a default similarity threshold of 0.6.
- User Interface: The Streamlit interface presents enriched records and supports export of the dataset as a Microsoft Excel file.
4. Evaluation
COCI was evaluated on 40 CfPs across several disciplines and processed all of them, while revealing that external Linked Data quality can constrain enrichment reliability.
- 40 CfPs from Computer Science, Engineering, Scientometrics, and Materials Science were manually evaluated.
- COCI successfully processed all evaluated CfPs and extracted their metadata.
- An organiser was incorrectly mapped to an OpenAlex profile because the external database listed the name as another researcher’s alias.
- COCI includes heuristics that reject affiliation data when unique organisers vastly outnumber unique organisations.
5. Conclusions
COCI transforms heterogeneous Calls for Papers into structured metadata linked to established knowledge bases, supporting richer Scientific Knowledge Graphs and new analyses of scholarly events. Future work will extend this foundation through continuous harvesting and longitudinal analysis.
- 5. Conclusions: COCI automates entity extraction and semantic mapping from unstructured CfPs into standardised metadata suitable for Scientific Knowledge Graph integration.It links extracted information to DBLP, TIB ConfIDent, the AIDA Dashboard, and OpenAlex.
- 5. Conclusions: The enriched data could help identify researchers’ recurring community duties and assess conference scope, organising-team seniority, and misconduct histories.
- 5. Conclusions: COCI’s interconnected event and organiser records could support formal, quantifiable credit for researchers’ community contributions.
- 5. Conclusions: Future work will develop continuous crawling from registries and professional mailing lists to harvest new Calls for Papers.
- 5. Conclusions: The planned longitudinal analyses will track emerging paradigms and identify disciplinary intersections or collaboration networks at their inception.
Declaration on Generative AI
The authors used Gemini for grammar and spelling checks, then reviewed and edited the content and retained full responsibility for the publication.
- Declaration on Generative AI: The authors used Gemini for grammar and spelling checks and reviewed and edited the content afterward.
- Declaration on Generative AI: The authors state that they retain full responsibility for the publication’s content.