# Comparison Tests for Clustering and Anomaly Detection

This repository contains a benchmarking framework to compare parallel and distributed clustering and anomaly detection algorithms under controlled synthetic scenarios. The main script is `comparison_tests.py`, which generates datasets, runs multiple algorithms, measures performance and resource usage, and stores results.

> **How to run this** (locally, with Docker, or with Apptainer/SLURM): see the root `README.md`. This document only covers what the script measures and how results are structured.

Aug 2026

---

## 1) Overview

The script evaluates algorithms under different experimental conditions:

* Dataset size (`N`)
* Dimensionality (`d`)
* Number of clusters (`k`)
* Outlier fraction
* Dataset structure (cardinality, density, shapes, groups)
* Random seeds

Algorithms currently evaluated include:

### Clustering

* SDOclust (NumPy backend)
* SDOclust (Dask backend)
* HDBSCAN 
* MiniBatchKMeans 
* DaskKMeans

### Anomaly Detection

* SDO (NumPy backend)
* SDO (Dask backend)
* IsolationForest 
* KMeans-based distance scoring

### Metrics

| Task              | Metrics                                          |
| ----------------- | ------------------------------------------------ |
| Clustering        | ARI, AMI                                         |
| Anomaly Detection | AUROC, Average Precision (AP)                    |
| Resources         | Time, memory, CPU usage, threads, Dask resources |

---

## 2) Dataset Generation

Datasets are generated using `datagen.generate_dataset` with parameters:

```python
generate_dataset(
    N=N,              # number of samples
    d=d,              # dimensionality
    k=k,              # number of clusters
    outlier_frac=out, # fraction of outliers
    seed=rseed,
    option=option     # dataset type
)
```

### Dataset types (`option_list`)

| Option        | Description                  |
| ------------- | ---------------------------- |
| `cardinality` | Varying cluster sizes        |
| `density`     | Varying cluster densities    |
| `shapes`      | Non-spherical cluster shapes |
| `groups`      | Cluster group structure      |

Datasets are generated per experiment and stored temporarily as Parquet files under `results/` (deleted after each run).

---

## 3) Output Files

Results are stored in the `results/` folder as CSV files:

| File              | Description                     |
| ----------------- | -------------------------------- |
| `results_N.csv`   | Scaling with dataset size       |
| `results_d.csv`   | Scaling with dimensionality     |
| `results_k.csv`   | Scaling with number of clusters |
| `results_out.csv` | Sensitivity to outliers         |

Each row corresponds to one algorithm run.

### Example output columns

| Column          | Meaning                   |
| --------------- | -------------------------- |
| task            | `clustering` or `anomaly` |
| method          | Algorithm name            |
| ARI / AMI       | Clustering quality        |
| AUROC / AP      | Anomaly detection quality |
| time_sec        | Runtime                   |
| peak_mem_mb     | Peak memory usage         |
| cpu_avg_percent | Average CPU usage         |
| cpu_max_percent | Max CPU usage             |
| N, d, k         | Dataset parameters        |
| outlier_frac    | Outlier fraction          |
| dataset_type    | Dataset structure         |
| rseed           | Random seed               |

---

## 4) Design Choices

### Block-based processing

Large datasets are processed in chunks to control memory usage:

```python
chunksize = 100000
```

### Dask vs NumPy backends

* NumPy backend: single-node, high performance
* Dask backend: distributed / large-scale experiments

### Resource monitoring

The decorator `monitor_resources` ensures fair comparison between algorithms.

---

## 5) Reproducibility

* Random seeds are controlled via `rseed`
* Thread counts are fixed via environment variables:

```python
OMP_NUM_THREADS=1
MKL_NUM_THREADS=1
OPENBLAS_NUM_THREADS=1
```

* Datasets are regenerated per experiment.