dark-ui/automl-fraud-detection
🛡️ AutoML-X — Intelligent Fraud Detection System
📌 Overview
AutoML-X is a production-ready, end-to-end machine learning system for credit card fraud detection. It automatically trains, evaluates, and selects the best model from 5 competing algorithms using cross-validation, applies feature engineering on raw transaction data, optimizes the decision threshold based on real business costs, and serves predictions through a FastAPI REST API with full SHAP explainability.
Built as a Minor Project at Madhav Institute of Technology and Science.
🎯 Key Results
🌐 Live Demo
📊 Evaluation Reports
ROC Curve (AUC = 0.97482)
Confusion Matrix (Threshold = 0.20603)
🔍 SHAP Explainability
Feature Importance — Mean |SHAP| (Top 20 Features)
Feature Impact Direction — Beeswarm Plot
Top fraud indicators:V14,V4,V12,V10,V3— consistent with published research on this dataset.
🏗️ System Architecture
creditcard.csv
│
▼
DataLoader ──► imbalance detection (0.17% fraud rate)
│
▼
DataCleaner ──► remove duplicates, impute missing, log outliers
│
▼
Feature Engineering (+5 derived features)
├── Hour (fraud varies by time of day)
├── Night_txn (10pm–6am binary flag)
├── Amount_log (log transform — compresses skew)
├── Amount_zscore (how unusual is this amount)
└── High_amount (Amount > $1,000 binary flag)
│
▼
SMOTE Oversampling ──► balanced training set (50/50)
│
▼
AutoModelSelector — 5 models compete via cross-validation
├── LogisticRegression (ROC-AUC: 0.99812)
├── RandomForest (ROC-AUC: 0.99999) ✅ selected
├── LightGBM_Balanced (ROC-AUC: 0.99998)
├── LightGBM_HighRecall (ROC-AUC: 0.99997)
└── XGBoost_HighRecall (ROC-AUC: 0.99993)
│
▼
BusinessCostOptimizer ──► threshold = 0.20603
│ cost = $113,800
▼
best_model.pkl ──► FastAPI ──► /predict
│ └──► /predict/batch
│ └──► SHAP explanation per transaction
▼
DriftDetector ──► PSI + KS test on new data
│
▼
ModelMonitor ──► SQLite logging + alert rules
│
▼
AlertManager ──► Slack webhook + log file
│
▼
MLflow ──► experiment tracking + model registry🔧 New: Monitoring & Alerting
Alert rules:
- 🔴 CRITICAL — Fraud rate exceeds 30% in last 100 predictions
- 🟡 WARNING — Average model confidence drops below 60%
- 🟡 WARNING — Data drift detected (PSI ≥ 0.1 on 10%+ of features)
🧪 Experimentation Journey
💰 Business Cost Optimization
Total Cost = (Missed Frauds × $10,000) + (False Alarms × $200)
Final result at threshold 0.20603:
10 frauds missed × $10,000 = $100,000
69 false alarms × $200 = $13,800
─────────────────────────────────────────
Total minimum cost = $113,800🚀 Quick Start
# 1. Clone
git clone https://github.com/Sujal-baghela/Automl-fraud-detection.git
cd Automl-fraud-detection
# 2. Install
pip install -r requirements.txt
# 3. Add dataset
# Download creditcard.csv from Kaggle → place at Data/creditcard.csv
# 4. Train
python -m Scripts.train
# 5. Run API
uvicorn app.api:app --reload --port 8000
# 6. Run Dashboard
streamlit run app/dashboard.py
# 7. Run MLflow
mlflow ui --port 5000
# 8. Docker (all services)
docker-compose up --build🌐 API Endpoints
📂 Project Structure
automl-x/
├── .github/workflows/ci.yml
├── app/
│ ├── api.py # FastAPI + monitoring integration
│ ├── dashboard.py # Streamlit dashboard
│ └── universal.py
├── src/
│ ├── monitor.py # ModelMonitor — SQLite logging
│ ├── alerting.py # AlertManager — Slack + log file
│ ├── drift_detector.py
│ ├── fraud_system.py
│ ├── inference_engine.py
│ ├── model_selector.py
│ └── ...
├── tests/
│ ├── test_monitor.py # 96 tests
│ ├── test_pipeline.py
│ └── ...
├── models/
│ ├── best_model.pkl
│ └── drift_reference.json
├── Dockerfile
├── supervisord.conf
├── docker-compose.yml
└── requirements.txt📦 Tech Stack
👥 Authors
Sujal Baghela — @Sujal-baghela
Sameer Bhilware — @SameerBhilware-ui
Institution: Madhav Institute of Technology and Science
📄 License
MIT License