A modern cybersecurity dashboard that combines Machine Learning and Flow-Based Network Analysis to detect suspicious network activities from CSV datasets and PCAP files.
The Intelligent Network Intrusion Detection & Risk Analysis System is a cybersecurity analysis platform developed during the Microsoft AI Innovators Internship Program.
The system combines machine learning and flow-based traffic analysis to identify suspicious network activities and assess potential security risks.
It provides two complementary analysis modules:
- 🤖 Machine Learning-Based CSV Analysis using a trained Random Forest model on the CICIDS2017 dataset.
- 🌐 Flow-Based PCAP Analysis using NFStream to extract network flows and perform explainable rule-based security assessment.
The dashboard provides attack predictions, model probability scores, interactive visualizations and downloadable reports through an intuitive Streamlit interface.
Note: The PCAP module performs rule-based traffic assessment and does not use machine learning classification.
- Random Forest attack classification
- CICIDS2017 dataset support
- 77 network traffic features
- 15 attack classes
- Model Probability Score
- Second Most Probable Class
- Risk Level Assessment
- Interactive charts
- CSV report export
- Missing / NaN / Infinite value validation
- NFStream flow extraction
- Protocol statistics
- Traffic visualization
- Top Talkers analysis
- Port analysis
- Rule-based security assessment
- Search & filtering
- Sortable flow table
- CSV report export
| Category | Technologies |
|---|---|
| Programming Language | Python |
| User Interface | Streamlit |
| Machine Learning | Scikit-learn |
| Classification Model | Random Forest |
| Network Flow Analysis | NFStream |
| Data Processing | Pandas |
| Visualization | Matplotlib |
CSV FILE
│
▼
Feature Validation
│
▼
Random Forest Model
│
▼
Attack Classification & Risk Analysis
│
▼
Streamlit Dashboard
PCAP / PCAPNG FILE
│
▼
NFStream
│
▼
Flow Extraction
│
▼
Rule-Based Security Assessment
│
▼
Streamlit Dashboard
The Random Forest model was evaluated on a stratified test set containing 462,762 network flows.
| Metric | Score |
|---|---|
| Accuracy | 99.85% |
| Macro Precision | 92.50% |
| Macro Recall | 83.83% |
| Macro F1-Score | 86.39% |
| Weighted F1-Score | 99.85% |
| Property | Value |
|---|---|
| Algorithm | Random Forest |
| Dataset | CICIDS2017 |
| Features | 77 |
| Attack Classes | 15 |
| Test Samples | 462,762 |
| Evaluation Strategy | Stratified Train/Test Split |
The CICIDS2017 dataset is highly imbalanced. Benign traffic dominates the dataset, while several attack classes contain only a limited number of samples.
Therefore, Macro F1 and class-wise metrics are reported together with Accuracy.
The evaluation was performed on 462,762 test samples using:
- Accuracy
- Macro Precision
- Macro Recall
- Macro F1-Score
- Weighted F1-Score
- Class-wise Precision
- Class-wise Recall
- Class-wise F1-Score
The noticeable difference between Macro F1 (86.39%) and Weighted F1 (99.85%) indicates that the model performs exceptionally well on majority classes while minority attack categories remain significantly more challenging.
| Attack Type | Precision | Recall | F1-Score |
|---|---|---|---|
| Benign | 99.91% | 99.96% | 99.93% |
| Bot | 83.61% | 69.10% | 75.67% |
| DDoS | 99.98% | 99.94% | 99.96% |
| DoS Hulk | 99.82% | 99.54% | 99.68% |
| DoS Slowhttptest | 94.94% | 98.76% | 96.81% |
| DoS Slowloris | 99.63% | 99.07% | 99.35% |
| FTP-Patator | 99.92% | 99.33% | 99.62% |
| PortScan | 92.39% | 90.03% | 91.19% |
| SSH-Patator | 100.00% | 97.83% | 98.90% |
| Web Attack Brute Force | 74.67% | 77.21% | 75.92% |
| Web Attack XSS | 43.01% | 30.77% | 35.87% |
Note: Rare attack classes such as Heartbleed, SQL Injection, and Infiltration contain very few samples. Their reported metrics should therefore be interpreted with caution.
The normalized confusion matrix provides a class-wise visualization of the model's predictive performance. It highlights that common attack categories are classified with high accuracy, whereas minority classes exhibit lower recall because of the severe class imbalance.
The model was trained using the CICIDS2017 dataset, which is highly imbalanced. While common attack categories contain hundreds of thousands of samples, several attack classes contain only a handful of instances.
Consequently:
- Accuracy alone is not sufficient to evaluate the model.
- Macro F1-Score is reported alongside Accuracy and Weighted F1-Score.
- Minority attack classes may exhibit lower Recall despite the high overall Accuracy.
The PCAP module performs rule-based traffic assessment rather than machine learning classification.
Instead of predicting attack categories, it detects suspicious behaviors such as:
- Port Scanning
- High TCP Activity
- Abnormal Flow Statistics
using heuristic rules extracted from NFStream flow statistics.
The reported evaluation metrics were obtained using the CICIDS2017 benchmark dataset. Although the model achieved high performance under this experimental setup, results may differ in real-world network environments due to variations in traffic patterns, attack diversity, and deployment conditions.
The model was evaluated using a stratified train-test split to preserve the original class distribution.
Although this strategy prevents label imbalance, the CICIDS2017 dataset contains network flows collected during the same capture days. Therefore, highly similar network flows may appear in both training and testing subsets.
As a result, the reported evaluation metrics should be interpreted as benchmark results for the selected experimental setup rather than absolute real-world deployment performance.
Future work may include:
- Day-based splitting
- Time-based validation
- Cross-day evaluation
- External dataset validation
intelligent-network-intrusion-detection-risk-analysis-system
│
├── app.py
├── csv_analyzer.py
├── pcap_analyzer.py
├── ui.py
├── helpers.py
├── constants.py
├── network_attack_detector.pkl
├── label_encoder.pkl
├── requirements.txt
├── LICENSE
├── notebooks/
│ └── Network_Attack_Detection_Training.ipynb
└── screenshots/
dashboard.png
csv-analysis.png
csv-visualization.png
pcap-analysis.png
class-distribution.png
confusion-matrix-normalized.png
git clone https://github.com/efloq/Intelligent-Network-Intrusion-Detection-Risk-Analysis-System.git
cd intelligent-network-intrusion-detection-risk-analysis-system
python -m venv venv
# macOS / Linux
source venv/bin/activate
# Windows
venv\Scripts\activate
pip install -r requirements.txt
streamlit run app.py| Module | Supported Format |
|---|---|
| Machine Learning Analysis | .csv |
| Flow-Based Analysis | .pcap, .pcapng |
Recommended maximum file sizes
- CSV: 100 MB
- PCAP / PCAPNG: 500 MB
To reduce excessive memory usage and improve application stability, upload limits are applied before analysis.
| File Type | Maximum Size |
|---|---|
| CSV | 100 MB |
| PCAP / PCAPNG | 500 MB |
Files exceeding these limits are rejected before processing.
- Real-time packet capture
- Threat intelligence integration (AbuseIPDB / VirusTotal)
- Automatic model calibration
- Unit and integration tests
- Live network monitoring
- Docker deployment
- REST API support
- User authentication
- Explainable AI (XAI) support
Elifnur Sertkaya
Computer Engineering Student
Microsoft AI Innovators Internship Project
This project uses the CICIDS2017 dataset developed by the Canadian Institute for Cybersecurity (CIC).
Reference
Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A.
Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization.
International Conference on Information Systems Security and Privacy (ICISSP), 2018.
This project is licensed under the MIT License.
| Property | Value |
|---|---|
| Language | Python |
| Dataset | CICIDS2017 |
| ML Model | Random Forest |
| Features | 77 |
| Attack Classes | 15 |
| Evaluation Samples | 462,762 |
| PCAP Engine | NFStream |
| UI Framework | Streamlit |
| Flow Analysis | NFStream |