# Artifact Description ### Data-Driven Performance in Anomaly Detection and Clustering: Evaluation Experiments This artifact provides code and results corresponding to experimental setting used in the paper: **When Does the Algorithm Matter? Data-Driven Performance in Anomaly Detection and Clustering**. Code is designed to evaluate anomaly detection and clustering algorithms at scale under controlled experimental conditions. Aug 2026 ## 0. Executed Environment and Results We used a single machine with an Intel Xeon E5-2620 v4 CPU (8 physical cores, 16 threads), and 160 GB RAM. Experiments were executed single-threaded (n_jobs=1). In the code, thread-level parallelism is disabled via environment variables to ensure fair and reproducible runtime measurements. Therefore, the complete experiments (*run.sh*) may take some hours/days to complete. Note that time out is fixed to 5 minutes (*run_analysis_real.py*, *run_analysis_synt.py*). By default, scripts create a results/ folder where outcomes are saved in .CSV files. ### 0.1. Reported Results Complete experimental results are provided in the *results_Aug2026.zip* file. ## 1. Requirements - Python 3.9.6 - Linux or macOS (recommended) Main packages are listed in the `pip_list_main.txt`, whereas all required Python packages are listed in `docker/requirements.txt`. ## 2. Setup *You can alternatively use the Docker image (recommended). See section 7.* Create and activate a virtual environment: ```bash python3.9 -m venv venv source venv/bin/activate ``` Install dependencies: ```bash pip install -r docker/requirements.txt ``` Note: *If installing dependencies causes errors due to cross-compatibility issues, try removing the specific versions (e.g., `hdbscan` instead of `hdbscan==0.8.42`), or use the Dockerized version directly.* ## 3. Reproducing Experiments ### 3.0. Run all experiments To run the full experimental suite: ```bash bash run.sh ``` This executes: - all combinations of OFAT analysis (size, dims, outs, clus) in easy and hard scenarios. - the factorial analysis in easy and hard scenarios. - anomaly detection analysis on the ADBench collection. - clustering analysis on the ClusBench collection. ### 3.1. OFAT analysis (synthetic data) Experiments are executed via: ```bash python run_analysis_synt.py --analysis_type --scenario ``` #### Parameters - `analysis_type`: `size`, `dims`, `outs`, `clus` - `scenario`: `easy`, `hard` ### 3.2. Factorial analysis (synthetic data) Experiments are executed via: ```bash python run_analysis_synt.py --analysis_type interaction --scenario ``` #### Parameters - `scenario`: `easy`, `hard` ### 3.3. Experiments with real dataset collections Experiments are executed via: ```bash python run_analysis_real.py --mode cl --datasets_dir datasets/clusbench --max_samples 0 python run_analysis_real.py --mode ad --datasets_dir datasets/adbench --max_samples 0 ``` `--max_samples 0` ensures that datasets are fully analyzed (no subsample). ## 4. Output Results are stored in the `results/` directory as CSV files. Each file contains: - evaluation metrics - execution time - memory usage - execution status (success, timeout, memory, error) ## 5. Reproducibility - Python version is fixed (3.9.6) - Dependencies are fully specified in `docker/requirements.txt` - Random seeds are internally controlled ## 6. Structure . ├── datasets │   ├── adbench --> ADBench dataset collection │   │   ├── 10_cover.npz │   │   ├── ... │   ├── clusbench --> ClusBench dataset collection │   │   ├── penguins_original_n1.npz │   │   ├── ... │   ├── clusbench_metadata.csv --> metadata (from the ClusBench source repo) │   ├── adbench.tex --> metadata (extracted by us) │   └── clusbench.tex --> metadata (extracted by us) ├── docker │   ├── docker-compose.yml --> docker-compose configuration │   ├── Dockerfile --> Docker container configuration │   ├── Makefile --> For automating the creation, execution, and cleanup of the Docker environment │   └── requirements.txt --> Python packages required ├── docker_image │   └── docker-reproducible-env.tar --> built docker image with all Python requirements already installed ├── FIRST_OF_ALL.txt ├── results_Aug2026.zip --> already generated results and presented in the paper (August 2026) ├── gen_data.py --> script to generate synthetic datasets ├── pip_list_main.txt --> list with main Python libraries used ├── README.md --> repository documentation ├── run_analysis_real.py --> script for running experiments on real datasets ├── run_analysis_synt.py --> script for running experiments on synthetic datasets (OFAT and factorial) ├── run.sh --> script for running all experiments conducted for the research └── utils.py --> script with data and algorithm configurations ## 7. Docker To facilitate reproducibility and reuse, we provide a pre-built Docker image that can either be loaded directly or rebuilt from the provided Docker configuration files. Using the pre-built Docker image is the recommended option for reproducing the experiments, as it already contains the required Python environment and dependencies. In this case, there is no need to create or configure a Python virtual environment on the host system. You need to have **Docker** and **Docker Compose** (or the Docker Compose plugin) installed on your system. > **Note:** Depending on your system configuration, Docker commands may require `sudo`. Alternatively, you can configure your user account to run Docker without `sudo`. ### Using the pre-built Docker image The pre-built Docker image is the file: `docker-reproducible-env.tar`. Load it into your local Docker installation with: ```bash docker load -i docker_image/docker-reproducible-env.tar ``` Instead of loading the image, you can build it by using the Docker configuration files provided in the `docker` folder: - `docker-compose.yml` - `Dockerfile` - `Makefile` - `requirements.txt` (Python package dependencies) The `Makefile` simplifies the management of the Docker container through the following commands (run them from a terminal opened in the /docker folder): - `build`: builds the Docker image using the available cache. - `run`: run all the experiments. - `shell`: opens a shell inside the running container. - `clean`: stops and removes the Docker containers. - `nuke`: **Warning:** removes all Docker images from the system. The Docker container mounts the local artifact directory as a volume. Therefore, files generated by the container are available directly on the host system.