rkmagama/tinyllama-lora-security-log-analysis

OpenSOC-AI-B200

Fine-tuned TinyLlama-1.1B for intelligent Security Operations Center (SOC) log analysis using Parameter-Efficient Fine-Tuning (LoRA) on the NVIDIA DGX B200 platform.

Overview

This model is designed to automate cybersecurity log analysis by identifying threats from raw security logs. It was fine-tuned using LoRA adapters on a custom SOC dataset consisting of firewall, authentication, intrusion detection, and web server log samples.

The model performs:

• —Threat Classification
• —Severity Classification
• —MITRE ATT&CK Technique Identification
• —Security Log Understanding

The implementation is based on the OpenSOC-AI framework and optimized for NVIDIA DGX B200 hardware.

Base Model

TinyLlama/TinyLlama-1.1B-Chat-v1.0

Framework:

• —PEFT (LoRA)
• —Transformers
• —TRL
• —BitsAndBytes (4-bit Quantization)

Evaluation Results

Evaluation Dataset:

• —50 unseen SOC log samples

Training Dataset:

• —450 security log samples

Hardware

Model fine-tuned using:

• —NVIDIA DGX B200
• —CUDA 12.8
• —4-bit Quantization (NF4)
• —LoRA Rank = 16
• —LoRA Alpha = 32

Training Configuration

• —Base Model: TinyLlama-1.1B-Chat-v1.0
• —Trainable Parameters: 12.6 Million
• —Total Parameters: 1.11 Billion
• —Fine-tuning Method: LoRA
• —Quantization: QLoRA (4-bit NF4)
• —Learning Rate: 2e-4
• —Batch Size: 4
• —Gradient Accumulation: 4
• —Effective Batch Size: 16

Intended Uses

This model can be used for:

• —Security Operations Centers (SOC)
• —Threat Detection
• —Cybersecurity Research
• —Log Analysis
• —Incident Response
• —Threat Intelligence

license: mit ---