TIF_E41253310/Model/README.md

# 🤖 Machine Learning Model

## Deskripsi

Model Machine Learning untuk clustering alumni berdasarkan kompetensi menggunakan Text Mining (TF-IDF) dan K-Means Clustering.

## Struktur Folder

```
Model/
├── __init__.py
├── main.py                    # Pipeline utama
├── preprocessing/
│   └── text_preprocessing.py  # Text preprocessing
├── feature_engineering/
│   └── tfidf.py               # TF-IDF extraction
├── clustering/
│   └── kmeans.py              # K-Means clustering
├── utils/
├── data/
│   ├── raw/                   # Data mentah
│   ├── processed/              # Data sudah diproses
│   └── result/                # Hasil clustering
└── README.md
```

## Instalasi Dependencies

```bash
pip install pandas scikit-learn nltk sastrawi
```

## Penggunaan Dasar

### 1. Import Pipeline

```python
import pandas as pd
from Model.main import AlumniClusteringPipeline

# Siapkan data
data = pd.DataFrame({
    'nama': ['Ali', 'Budi', 'Citra'],
    'perusahaan': ['Google', 'Tokopedia', 'Gojek'],
    'jabatan': ['Engineer', 'Analyst', 'Manager']
})

# Jalankan pipeline
pipeline = AlumniClusteringPipeline(n_clusters=3)
result = pipeline.fit_transform(data)

print(result)
```

### 2. Auto-Detect K Optimal

```python
pipeline = AlumniClusteringPipeline(n_clusters=3, auto_k=True)
result = pipeline.fit_transform(data)
```

### 3. Simpan Hasil

```python
# Simpan ke CSV
result.to_csv('hasil_clustering.csv', index=False)

# Lihat distribusi cluster
print(result['cluster_label'].value_counts())

# Lihat ringkasan
summary = pipeline.get_cluster_summary(result)
print(summary)
```

## Input Model

Model menerima DataFrame dengan kolom:

| Kolom | Deskripsi |
|-------|-----------|
| `nama` | Nama lengkap alumni |
| `perusahaan` | Nama perusahaan/instansi |
| `jabatan` | Jabatan/posisi |

## Output Model

DataFrame dengan kolom tambahan:

| Kolom | Deskripsi |
|-------|-----------|
| `cluster` | Nomor cluster (0, 1, 2, ...) |
| `cluster_label` | Label cluster ("Cluster_0", "Cluster_1", ...) |

## CLI Usage

```bash
cd Model
python main.py [input.csv] [n_clusters] [output.csv]
```

Contoh:
```bash
python main.py ../data/raw/alumni.csv 3 ../data/result/hasil.csv
```

## Integrasi dengan Backend

Model sudah diintegrasikan dengan Backend FastAPI melalui `ml_service.py`.

Endpoint untuk clustering tersedia di `/api/v1/clustering`.

Lihat `Backend/app/services/ml_service.py` untuk detail.

----------------------------------------

## Update Progress (2026-08-28)

### Pipeline ML Selesai (6 Tahap)

| Tahap | Script | Fungsi |
|-------|--------|--------|
| 1 | `column_mapping.json` | Mapping kolom 2019-2025 |
| 2 | `merge_data.py` | Merge data multi-file CSV |
| 3 | `text_preprocessing.py` | Text preprocessing + Sastrawi |
| 4 | `clustering.py` | TF-IDF + K-Means K=4 |
| 5 | `interpretasi.py` | Interpretasi C0-C3 |
| 6 | `dashboard.py` | Generate visualisasi |

### Cara Menjalankan Pipeline

```bash
cd Model/preprocessing
python merge_data.py           # Tahap 1-2
python text_preprocessing.py  # Tahap 3
python clustering.py          # Tahap 4
python interpretasi.py        # Tahap 5
python dashboard.py           # Tahap 6
```

### Data Sources

| Tahun | File CSV | Records |
|-------|----------|---------|
| 2019 | Data_2019_Tracer_study.csv | 30 |
| 2020 | Data_2020_Tracer_study.csv | 31 |
| 2021 | Data_2021_Tracer_study.csv | 237 |
| 2022 | Data_2022_Tracer_study.csv | 211 |
| 2023 | Data_2023_Tracer_study.csv | 268 |
| 2024 | Data_2024_Tracer_study.csv | 361 |
| 2025 | Data_2025_Tracer_study_*.csv (5 file) | 458 |
| **Total** | | **1596** |

### Hasil

- **Total Alumni:** 1596
- **Data untuk Clustering:** 1114
- **K-Means K:** 4 (sesuai PRD)
- **Silhouette Score:** 0.1034
- **TF-IDF Features:** 500

### Distribusi Cluster

| Cluster | Jumlah | % | Top Terms |
|---------|--------|---|-----------|
| 0 | 587 | 52.7% | perintah, bumn, bnpp, jember, staff |
| 1 | 146 | 13.1% | freelance, kerja lepas |
| 2 | 183 | 16.4% | wiraswasta, usaha sendiri |
| 3 | 198 | 17.8% | swasta, usaha swasta, indonesia |

### Distribusi Kategori C0-C3

| Kategori | Deskripsi | Jumlah | % |
|----------|-----------|--------|---|
| C0 | Sesuai Profil Lulusan | 223 | 13.97% |
| C1 | Bidang Diketahui (BUMN, pemerintah, dll) | 385 | 24.12% |
| C2 | Bidang Tidak Jelas | 506 | 31.70% |
| C3 | Belum/Tidak Bekerja | 482 | 30.20% |

### Catatan Hasil

- **C0 (14%)**: Alumni bekerja di bidang sesuai prodi (IT, jaringan, dll)
- **C1 (24%)**: Bekerja di bidang non-teknis (BUMN, pemerintah, dll)
- **C2 (32%)**: Bidang pekerjaan tidak teridentifikasi jelas
- **C3 (30%)**: Alumni belum/tidak bekerja saat tracer study

### Distribusi per Prodi

| Prodi | Kode | C0 | C1 | C2 | C3 |
|-------|------|----|----|----|----|
| TIF | 55301 | 13 | 135 | 365 | 169 |
| MIF | 57401 | 10 | 66 | 226 | 136 |
| TKK | 56401 | 13 | 48 | 217 | 122 |

### Output Files

```
Model/data/
├── standardized/
│   └── unified_dataset.csv       # Data setelah merge
├── processed/
│   ├── dataset_untuk_clustering.csv
│   ├── clustering_results.csv
│   ├── model_info.json
│   └── dashboard/
│       └── dashboard.html        # Dashboard interaktif
└── result/
    └── interpretasi_results.csv  # Hasil C0-C3
```