# Evaluation Results

This document summarizes the performance, accuracy, latency, and resource utilization of the TinyML-enabled induction motor fault detection system. All experimental validation was conducted on the ESP32-S3 microcontroller using real-world data collected from a physical three-phase induction motor testbed.

---

## 📊 1. Dataset Summary
The system was trained and evaluated on a **real-world, multi-sensor dataset** containing electrical and mechanical signals.

### **Dataset Distribution**
| Class | Samples |
|-------|---------|
| Healthy | 129,812 |
| Single Phase Fault | 93,075 |
| Bearing Fault | 68,376 |
| Overcurrent Fault | 65,197 |

### **Sensor Modalities**
- Phase Voltage (A, B, C)
- Phase Current (A, B, C)
- Temperature
- Vibration (801S analog sensor)

Sampling Frequency: **200 Hz**  
Window Size: **2 seconds (400 samples)**

---

## 🤖 2. Model Performance (TinyML)
The final deployed model is the **quantized int8 TensorFlow Lite Micro (TFLM)** version.

### **Classification Metrics**
| Metric | Value |
|--------|-------|
| **Accuracy** | **94.3%** |
| Precision (avg) | 0.96 |
| Recall (avg) | 0.96 |
| F1-score (avg) | 0.96 |
| Inference Latency | **287 ms** |
| Model Size | **164 KB** |

---

## 📉 3. Confusion Matrix Summary

![Confusion Matrix](./images/confusion%20matrix.png)

Misclassifications were mainly between:
- Healthy ↔ Early Bearing Fault
- Healthy ↔ Early Phase Fault

This occurs due to subtle vibration changes in early fault stages. Overall separation between the four classes remains highly reliable.

---

## ⚡ 4. Benchmark Comparison
This comparison shows how the TinyML model performs relative to high-resource algorithms.

| Model | Accuracy | Latency | Memory | Suitable for MCU? |
|--------|----------|---------|---------|--------------------|
| KNN (k=5) | 97.22% | High | High | ❌ No |
| Deep Learning CNN | 97.32% | 200–500 ms | 50MB+ | ❌ No |
| SVM | 68% | 50 ms | Moderate | ⚠️ Hard |
| **TinyML (Proposed)** | **94.3%** | **287 ms** | **164 KB** | ✅ Yes |

The TinyML model sacrifices <3% accuracy but reduces memory by **~300×**.

---

## 🔧 5. Model Optimization (Edge Impulse Benchmarks)
### Float32 (Unoptimized)
- **Total latency:** 307 ms
- **RAM:** 86.3 KB
- **Model size:** 551 KB

### Quantized int8 (Optimized)
- **Total latency:** 287 ms
- **RAM:** 86.3 KB
- **Model size:** 164 KB

The quantized model was chosen for deployment due to:
- Lower Flash footprint
- Faster classifier stage
- Nearly identical accuracy

---

## 📈 6. Prognostic Ability (EMA Trend Prediction)
The system uses **Exponential Moving Average (EMA)** to detect rising probability trends.

### Early Warning Example
During a bearing fault degradation test:
- Smoothed probability for *Bearing Fault* increased steadily
- **15 inference cycles** (≈30 seconds) before crossing the 70% detection threshold
- System raised **early warning alert** successfully

This verifies the model’s predictive capability.

---

## 🧪 7. System-Level Validation
### Summary
- Fully operational for **7 days continuous testing**
- TFT display and Web Dashboard updated in real time
- Automatic fail-safe shutdown triggered correctly during overcurrent
- Remote command (START/STOP) tested via IoT dashboard

### Reliability Indicators
| Parameter | Result |
|-----------|--------|
| Power Consumption | <600 mW |
| Total Hardware Cost | ~$200 |
| Real-Time Operation | ✔ Stable |
| UI Rendering | ✔ Smooth |
| TinyML Inference | ✔ Consistent |

---

## 🏁 8. Conclusion
The evaluation confirms that the system achieves:
- **High diagnostic accuracy (94.3%)**
- **Low inference latency (287 ms)** suitable for real-time edge deployment
- **Robust prognosis capability** using EMA-based probability trend analysis
- **Industrial-grade reliability** on low-cost hardware

The TinyML-based approach delivers near–deep-learning performance while being deployable on microcontrollers, making it ideal for scalable predictive maintenance solutions.

---

# ✅ End of Evaluation Results