Our AI Models
Learn about the two machine learning models powering Sign-Speak AI.
Hand Landmark Recognition Model (Random Forest)
Goal: Recognize static or simple gestures using 3D hand landmark coordinates.
How It Works:
-
•
Feature Extraction: Uses
MediaPipe Hands to detect 21 landmarks per hand (x, y, z → 63 features per hand).
-
•
Data Representation: Landmarks are stored in a CSV file; each row represents one gesture sample with its label (e.g., "A", "B", "1").
-
•
-
•
Recognition: Captures real-time landmarks, scales features, and predicts the gesture using the trained model with
temporal smoothing for stability.
Key Strengths:
-
•
Lightweight & Fast: Runs on CPU.
-
•
High Accuracy: 90–97% for static gestures
-
•
Efficient: Low storage & quick retraining
Video-Based CNN + LSTM Model (Deep Learning)
Goal: Recognize dynamic signs that involve motion and temporal changes.
How It Works:
-
•
Input Data: Uses short 3–5 second video clips (≈30 frames, resized to 224×224).
-
•
Feature Extraction: EfficientNet-B0 extracts spatial features (hand shape, orientation, facial cues).
-
•
Temporal Modeling: Features are passed through a
Bidirectional LSTM to learn motion and time-based patterns.
-
•
Classification: LSTM outputs go through dense layers to predict the gesture class.
-
•
-
•
Recognition: Processes live video input →
CNN →
LSTM → gesture prediction with smoothing for stability.
