LongEval 2026 Test Dataset

This repository contains the testing datasets for the 2026 LongEval  (https://clef-longeval.github.io/), organized at CLEF. It includes the dataset for subtasks: (1) Task 1 - LongEval-Sci: Ad-Hoc Scientific Retrieval, (2) Task 3 - LongEval-USim: User Simulation, and (3) Task 4. LongEval-RAG: Retrieval Augmented Generation (RAG)

The testind data for all tasks were collected from June to August 2025 and from September to November 2025.

• Task 1 - LongEval-Sci: Ad-Hoc Scientific Retrieval

The collection consists of queries and documents extracted from the CORE scholarly literature search engine (https://core.ac.uk/). The queries in this collection were issued by CORE users, and a specific pipeline was designed to capture user queries, their corresponding search results, and user interactions. Using this pipeline, the dataset for this task was extracted, consisting of two main components: 

1. Search Information: unique, anonymous identifiers for individual user sessions, search queries, and the returned results. 
2. Click Information: unique, anonymous identifiers for individual user sessions, the links clicked within the results list, and the positions of the clicked links. 

The documents in the collection were selected based on the search results of user queries. In addition to these selected documents, the collection also contains randomly chosen documents from the CORE index. 

In total, the collection comprises 381 testing queries. The document set consists of downloaded, cleaned, filtered scholarly articles, and divided into two snapshots: (1) from June to August 2025: 1322720 documents , and (2) from September to November 2025: 1661900 documents.  The document sets are incremental, where each future snapshots contains the previous snapshots documents and extra documents.

• Task 3 - LongEval-USim: User Simulation

The dataset consists of : (1) 118 sessions collected from June to August 2025, and (2) 181 sessions collected from September to November 2025. The sessions contain the following rich set of features:

    - The sequence of queries submitted.

    - Timestamps for query submissions.

    - The top-10 SERP (Search Engine Results Page) retrieved for each query.

    - Which documents were clicked, along with their corresponding click type.

• Task 4. LongEval-RAG: Retrieval Augmented Generation (RAG)

The dataset contains 47 queries, each paired with a corresponding set of document IDs. Each query includes the following elements:

    - A question related to research papers available on the CORE website.

    - A set of 10 document IDs from which participants must justify their answer, selecting at least two ids.