Artifact Description

This artifact provides codes and results corresponding to the experimental setting used in the paper: Limits of Clustering Models Based on Distance Similarity: A Sensitivity Study of SDOclust by Sabina Khazari, Tanja Zseby, Félix Iglesias Vázquez.

Codes are designed to conduct analysis to evaluate SDOclust main hyperparameters sensitivity.

Aug 2026

1. Reported Results

Complete experimental results are provided in the results.zip file. Extract it as the [results] folder, which contains:

  • [results/run20260430_213142] --> outs analysis.
  • [results/run20260430_214742] --> clus analysis.
  • [results/run20260430_220412] --> inspread analysis.
  • [results/run20260430_222012] --> outspread analysis.
  • [results/run20260430_223512] --> dims analysis.
  • [results/run20260430_231155] --> factorial analysis.
  • [results/run20260501_184618] --> size analysis.

2. Requirements

  • Python 3.9.6
  • Linux or macOS (recommended)

All required Python packages are listed in requirements_complete.txt.

3. Setup

You can alternatively user the Docker image (recommended). See section 8.

Create and activate a virtual environment:

python3.9 -m venv venv
source venv/bin/activate

Install dependencies:

pip install -r requirements_complete.txt

4. Reproducing Experiments

Each experiment is configured via a JSON file and executed as:

python sensitivity_analysis.py --config configs/<config_name>.json

To run the full experimental suite:

bash run.sh

This executes all configurations in sequence:

python sensitivity_analysis.py --config configs/outs.json
python sensitivity_analysis.py --config configs/clus.json
python sensitivity_analysis.py --config configs/inspread.json
python sensitivity_analysis.py --config configs/outspread.json
python sensitivity_analysis.py --config configs/dims.json
python sensitivity_analysis.py --config configs/factorial.json
python sensitivity_analysis.py --config configs/size.json

Configuration Files

Each JSON config specifies the analysis to run. Available analysis_type values:

  • size — varies number of samples
  • dims — varies number of features
  • clus — varies number of clusters
  • outs — varies outlier fraction
  • inspread — varies inlier spread
  • outspread — varies outlier spread
  • factorial — joint sweep over clusters and inlier spread
  • all — runs all of the above in sequence

Example config:

{
  "analysis_type": "size",
  "i": 10,
  "o": "results",
  "seeds": "0,1,2"
}

5. Output

Results are stored in a timestamped subfolder inside the results/ directory (e.g. results/run20260501_143022/).

Each run produces:

  • ALL_combinations_LONG.csv — one row per (seed, dataset config, algorithm) with ARI, execution time, effective parameters used, and error info
  • args.json — copy of the config used for that run

6. Reproducibility

  • Python version is fixed (3.9.6)
  • Dependencies are fully specified in requirements_complete.txt
  • Random seeds are controlled via the seeds field in each config file
  • A pre-built docker image (and the docker configuration files to build the image) to replicate the experimental environment.

6. Structure

  • sensitivity_analysis.py — main experiment runner
  • configs/ — JSON configuration files for each analysis type
  • run.sh — batch execution script
  • results/ — output folder with CSV results and run metadata

7. Notes

  • The SDO algorithm grid is configured via the optional sdo_grid field in the JSON config.
  • Results are written incrementally to CSV so experiments can be interrupted and resumed.
  • ARI is computed on inliers only (outliers are excluded from clustering evaluation).

8. Docker

To facilitate reproducibility and reuse, we provide a pre-built Docker image that can either be loaded directly or rebuilt from the provided Docker configuration files.

Using the pre-built Docker image is the recommended option for reproducing the experiments, as it already contains the required Python environment and dependencies. In this case, there is no need to create or configure a Python virtual environment on the host system.

You need to have Docker and Docker Compose (or the Docker Compose plugin) installed on your system. The docker/requirements.txt file is exactly the same as requirements_complete.txt file.

Note: Depending on your system configuration, Docker commands may require sudo. Alternatively, you can configure your user account to run Docker without sudo.

Using the pre-built Docker image

The pre-built Docker image is the file: docker-reproducible-env.tar. Load it into your local Docker installation with:

docker load -i docker-reproducible-env.tar

Instead of loading the image, you can build it by using the Docker configuration files provided in the docker folder:

  • docker-compose.yml
  • Dockerfile
  • Makefile
  • requirements.txt (Python package dependencies)

The Makefile simplifies the management of the Docker container through the following commands:

  • build: builds the Docker image using the available cache.
  • run: run all the experiments.
  • shell: opens a shell inside the running container.
  • clean: stops and removes the Docker containers.
  • nuke: Warning: removes all Docker images from the system.

The Docker container mounts the local artifact directory as a volume. Therefore, files generated by the container are available directly on the host system.