210 lines
5.3 KiB
Markdown
210 lines
5.3 KiB
Markdown
# 🤖 Machine Learning Model
|
|
|
|
## Deskripsi
|
|
|
|
Model Machine Learning untuk clustering alumni berdasarkan kompetensi menggunakan Text Mining (TF-IDF) dan K-Means Clustering.
|
|
|
|
## Struktur Folder
|
|
|
|
```
|
|
Model/
|
|
├── __init__.py
|
|
├── main.py # Pipeline utama
|
|
├── preprocessing/
|
|
│ └── text_preprocessing.py # Text preprocessing
|
|
├── feature_engineering/
|
|
│ └── tfidf.py # TF-IDF extraction
|
|
├── clustering/
|
|
│ └── kmeans.py # K-Means clustering
|
|
├── utils/
|
|
├── data/
|
|
│ ├── raw/ # Data mentah
|
|
│ ├── processed/ # Data sudah diproses
|
|
│ └── result/ # Hasil clustering
|
|
└── README.md
|
|
```
|
|
|
|
## Instalasi Dependencies
|
|
|
|
```bash
|
|
pip install pandas scikit-learn nltk sastrawi
|
|
```
|
|
|
|
## Penggunaan Dasar
|
|
|
|
### 1. Import Pipeline
|
|
|
|
```python
|
|
import pandas as pd
|
|
from Model.main import AlumniClusteringPipeline
|
|
|
|
# Siapkan data
|
|
data = pd.DataFrame({
|
|
'nama': ['Ali', 'Budi', 'Citra'],
|
|
'perusahaan': ['Google', 'Tokopedia', 'Gojek'],
|
|
'jabatan': ['Engineer', 'Analyst', 'Manager']
|
|
})
|
|
|
|
# Jalankan pipeline
|
|
pipeline = AlumniClusteringPipeline(n_clusters=3)
|
|
result = pipeline.fit_transform(data)
|
|
|
|
print(result)
|
|
```
|
|
|
|
### 2. Auto-Detect K Optimal
|
|
|
|
```python
|
|
pipeline = AlumniClusteringPipeline(n_clusters=3, auto_k=True)
|
|
result = pipeline.fit_transform(data)
|
|
```
|
|
|
|
### 3. Simpan Hasil
|
|
|
|
```python
|
|
# Simpan ke CSV
|
|
result.to_csv('hasil_clustering.csv', index=False)
|
|
|
|
# Lihat distribusi cluster
|
|
print(result['cluster_label'].value_counts())
|
|
|
|
# Lihat ringkasan
|
|
summary = pipeline.get_cluster_summary(result)
|
|
print(summary)
|
|
```
|
|
|
|
## Input Model
|
|
|
|
Model menerima DataFrame dengan kolom:
|
|
|
|
| Kolom | Deskripsi |
|
|
|-------|-----------|
|
|
| `nama` | Nama lengkap alumni |
|
|
| `perusahaan` | Nama perusahaan/instansi |
|
|
| `jabatan` | Jabatan/posisi |
|
|
|
|
## Output Model
|
|
|
|
DataFrame dengan kolom tambahan:
|
|
|
|
| Kolom | Deskripsi |
|
|
|-------|-----------|
|
|
| `cluster` | Nomor cluster (0, 1, 2, ...) |
|
|
| `cluster_label` | Label cluster ("Cluster_0", "Cluster_1", ...) |
|
|
|
|
## CLI Usage
|
|
|
|
```bash
|
|
cd Model
|
|
python main.py [input.csv] [n_clusters] [output.csv]
|
|
```
|
|
|
|
Contoh:
|
|
```bash
|
|
python main.py ../data/raw/alumni.csv 3 ../data/result/hasil.csv
|
|
```
|
|
|
|
## Integrasi dengan Backend
|
|
|
|
Model sudah diintegrasikan dengan Backend FastAPI melalui `ml_service.py`.
|
|
|
|
Endpoint untuk clustering tersedia di `/api/v1/clustering`.
|
|
|
|
Lihat `Backend/app/services/ml_service.py` untuk detail.
|
|
|
|
----------------------------------------
|
|
|
|
## Update Progress (2026-08-28)
|
|
|
|
### Pipeline ML Selesai (6 Tahap)
|
|
|
|
| Tahap | Script | Fungsi |
|
|
|-------|--------|--------|
|
|
| 1 | `column_mapping.json` | Mapping kolom 2019-2025 |
|
|
| 2 | `merge_data.py` | Merge data multi-file CSV |
|
|
| 3 | `text_preprocessing.py` | Text preprocessing + Sastrawi |
|
|
| 4 | `clustering.py` | TF-IDF + K-Means K=4 |
|
|
| 5 | `interpretasi.py` | Interpretasi C0-C3 |
|
|
| 6 | `dashboard.py` | Generate visualisasi |
|
|
|
|
### Cara Menjalankan Pipeline
|
|
|
|
```bash
|
|
cd Model/preprocessing
|
|
python merge_data.py # Tahap 1-2
|
|
python text_preprocessing.py # Tahap 3
|
|
python clustering.py # Tahap 4
|
|
python interpretasi.py # Tahap 5
|
|
python dashboard.py # Tahap 6
|
|
```
|
|
|
|
### Data Sources
|
|
|
|
| Tahun | File CSV | Records |
|
|
|-------|----------|---------|
|
|
| 2019 | Data_2019_Tracer_study.csv | 30 |
|
|
| 2020 | Data_2020_Tracer_study.csv | 31 |
|
|
| 2021 | Data_2021_Tracer_study.csv | 237 |
|
|
| 2022 | Data_2022_Tracer_study.csv | 211 |
|
|
| 2023 | Data_2023_Tracer_study.csv | 268 |
|
|
| 2024 | Data_2024_Tracer_study.csv | 361 |
|
|
| 2025 | Data_2025_Tracer_study_*.csv (5 file) | 458 |
|
|
| **Total** | | **1596** |
|
|
|
|
### Hasil
|
|
|
|
- **Total Alumni:** 1596
|
|
- **Data untuk Clustering:** 1114
|
|
- **K-Means K:** 4 (sesuai PRD)
|
|
- **Silhouette Score:** 0.1034
|
|
- **TF-IDF Features:** 500
|
|
|
|
### Distribusi Cluster
|
|
|
|
| Cluster | Jumlah | % | Top Terms |
|
|
|---------|--------|---|-----------|
|
|
| 0 | 587 | 52.7% | perintah, bumn, bnpp, jember, staff |
|
|
| 1 | 146 | 13.1% | freelance, kerja lepas |
|
|
| 2 | 183 | 16.4% | wiraswasta, usaha sendiri |
|
|
| 3 | 198 | 17.8% | swasta, usaha swasta, indonesia |
|
|
|
|
### Distribusi Kategori C0-C3
|
|
|
|
| Kategori | Deskripsi | Jumlah | % |
|
|
|----------|-----------|--------|---|
|
|
| C0 | Sesuai Profil Lulusan | 223 | 13.97% |
|
|
| C1 | Bidang Diketahui (BUMN, pemerintah, dll) | 385 | 24.12% |
|
|
| C2 | Bidang Tidak Jelas | 506 | 31.70% |
|
|
| C3 | Belum/Tidak Bekerja | 482 | 30.20% |
|
|
|
|
### Catatan Hasil
|
|
|
|
- **C0 (14%)**: Alumni bekerja di bidang sesuai prodi (IT, jaringan, dll)
|
|
- **C1 (24%)**: Bekerja di bidang non-teknis (BUMN, pemerintah, dll)
|
|
- **C2 (32%)**: Bidang pekerjaan tidak teridentifikasi jelas
|
|
- **C3 (30%)**: Alumni belum/tidak bekerja saat tracer study
|
|
|
|
### Distribusi per Prodi
|
|
|
|
| Prodi | Kode | C0 | C1 | C2 | C3 |
|
|
|-------|------|----|----|----|----|
|
|
| TIF | 55301 | 13 | 135 | 365 | 169 |
|
|
| MIF | 57401 | 10 | 66 | 226 | 136 |
|
|
| TKK | 56401 | 13 | 48 | 217 | 122 |
|
|
|
|
### Output Files
|
|
|
|
```
|
|
Model/data/
|
|
├── standardized/
|
|
│ └── unified_dataset.csv # Data setelah merge
|
|
├── processed/
|
|
│ ├── dataset_untuk_clustering.csv
|
|
│ ├── clustering_results.csv
|
|
│ ├── model_info.json
|
|
│ └── dashboard/
|
|
│ └── dashboard.html # Dashboard interaktif
|
|
└── result/
|
|
└── interpretasi_results.csv # Hasil C0-C3
|
|
```
|