erikbranmarino/DeepSeek-PRCT

🔍 DeepSeek-PRCT

A LoRA fine-tuned DeepSeek-R1-14B model for detecting Population Replacement Conspiracy Theory (PRCT) content, optimized for formal news discourse with good cross-domain performance.

 

Overview

DeepSeek-PRCT is a LoRA adapter fine-tuned on Portuguese Telegram messages for detecting Population Replacement Conspiracy Theories. Despite training on informal social media, the model achieves exceptional performance on formal Italian news (0.892 F1-macro), demonstrating remarkable cross-domain transfer from informal to formal discourse.

Key Metrics

⚠️ Important: This model exhibits unusual behavior - it performs better on cross-domain (formal news) than on its training domain (informal Telegram), suggesting architectural sensitivity to reasoning-intensive formal text.

Model Description

DeepSeek-PRCT is a LoRA fine-tuned version of DeepSeek-R1-Distill-Qwen-14B, a reasoning-optimized language model. Trained on Portuguese Telegram messages, it demonstrates exceptional cross-domain transfer to Italian news headlines, achieving the highest F1-macro (0.892) among all evaluated configurations.

What are PRCTs?

Population Replacement Conspiracy Theories are false narratives claiming deliberate orchestration of demographic substitution through immigration. Main variants include:

• —The Great Replacement Theory
• —White Genocide
• —Kalergi Plan
• —Eurabia

These narratives are linked to extremist violence (Christchurch 2019, Utøya 2011) and pose serious threats to democratic discourse.

Model Configuration

Architecture

• —Base Model: DeepSeek-R1-Distill-Qwen-14B (14B parameters)
• —Adapter Type: LoRA (Low-Rank Adaptation)
• —LoRA Rank: 16
• —LoRA Alpha: 32
• —Target Modules: qproj, vproj, kproj, oproj
• —Special Feature: Reasoning chains with <think> tags

Label Mapping

• —0: Non-PRCT content
• —1: PRCT content (supports/mentions replacement narratives)

Input Requirements

• —Maximum sequence length: 4096 tokens
• —Input type: Text (Portuguese, Italian, Spanish)
• —Preprocessing: DeepSeek tokenization

Intended Uses & Limitations

✅ Intended Uses

• —Formal news analysis: Optimized for detecting PRCT in journalistic content
• —Cross-domain deployment: Training on informal data, deployment on formal text
• —Research: Understanding domain transfer in conspiracy detection
• —High-accuracy applications: Where F1-macro >0.85 is required

⚠️ Limitations

• —Inverse domain performance: Worse on training domain (Telegram) than test domain (News)
• —Catastrophic forgetting: Fine-tuning degrades informal discourse understanding
• —Inference cost: 16.3s per sample (slowest among evaluated models)
• —Memory requirements: 14B parameters require substantial GPU memory
• —Reasoning overhead: <think> tags increase token usage

Critical Note: This model's unusual performance profile (cross-domain > in-domain) makes it ideal for formal text analysis but not recommended for social media monitoring.

Training Data

• —Primary training: Portuguese Telegram messages (n=919)
• —Domain: Informal social media discourse, conspiracy-oriented channels
• —PRCT prevalence: 15.7%
• —Annotation: Expert annotators (Krippendorff's α=0.58)
• —Time period: 2020-2024

Training Procedure

Hyperparameters

• —Learning rate: 2e-5
• —Batch size: 2 (with gradient accumulation)
• —Training steps: 600
• —Optimizer: AdamW 8-bit
• —LoRA dropout: 0.05
• —Weight decay: 0.01

Hardware

• —GPU: NVIDIA A100 40GB
• —Training time: ~4 hours
• —Framework: PyTorch + PEFT

Results

Cross-Domain Performance (News ITA) ⭐

Best-in-class performance: Highest F1-macro among all evaluated models on formal news text.

In-Domain Performance (Telegram PT)

Performance degradation: 23.7pp drop from cross-domain to in-domain, indicating architectural sensitivity to text formality.

Usage

Installation

pip install transformers peft torch

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
import re

# Load base model and tokenizer
base_model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-14B"
model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, "erikbranmarino/DeepSeek-PRCT")

# Prepare prompt
text = "Your Italian or Portuguese text here"
prompt = f"""Classify if the following text contains Population Replacement Conspiracy Theory (PRCT) content.

Text: {text}

Classification (YES/NO):"""

# Generate prediction
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.0,
    do_sample=False
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Parse response (remove <think> tags if present)
response = re.sub(r'<think>.*?</think>', '', response, flags=re.DOTALL)
print(response)

Processing with Reasoning Chains

DeepSeek-R1 outputs reasoning chains in <think> tags. You can preserve or remove them:

def extract_classification(response):
    """Extract classification from DeepSeek response"""
    # Remove thinking process
    cleaned = re.sub(r'<think>.*?</think>', '', response, flags=re.DOTALL)
    
    # Extract YES/NO
    if "YES" in cleaned.upper():
        return "PRCT"
    elif "NO" in cleaned.upper():
        return "Non-PRCT"
    else:
        return "Uncertain"

# With reasoning visible
full_response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("Reasoning:", full_response)
print("Classification:", extract_classification(full_response))

Architectural Insights

Why Cross-Domain > In-Domain?

DeepSeek-R1's reasoning architecture benefits from formal text structure:

1. 1.Explicit logic: News headlines state claims directly
2. 2.Reasoning alignment: Formal text matches model's reasoning chains
3. 3.Catastrophic forgetting: Fine-tuning on informal text disrupts implicit pattern recognition

Recommendation: Use this model for formal news analysis, not social media monitoring.

Bias and Ethical Considerations

Known Biases

• —Formality bias: Optimized for formal journalistic discourse
• —Domain paradox: Underperforms on training domain
• —Language bias: Best on Italian, weaker on Portuguese
• —Reasoning bias: May over-analyze simple informal text

Ethical Use

• —⚠️ Not for automated censorship: Requires human review
• —✅ News monitoring: Ideal for tracking PRCTs in media
• —✅ Research purposes: Understanding cross-domain transfer
• —❌ Social media: Better alternatives exist (Mistral-PRCT)

We advocate for freedom of speech and constitutional rights.

Comparison with Mistral-PRCT

Choose DeepSeek-PRCT if: High accuracy on news, can afford slow inference Choose Mistral-PRCT if: Social media focus, need faster processing

Citation (to appear)

@inproceedings{marino2025prct,
  title={Population Replacement Conspiracy Theories Detection on Telegram and News Headlines: 
         benchmarking LLMs and BERT models in Portuguese and Italian},
  author={Marino, Erik Bran and Vieira, Renata},
  booktitle={Proceedings of PROPOR 2026},
  year={2026}
}

Model Card Authors

Erik Bran Marino (Universidade de Évora, HYBRIDS Project)

Contact

• —Email: erik.marino@uevora.pt
• —Project: MSCA HYBRIDS (Grant Agreement No. 101073351)
• —Institution: Universidade de Évora, Portugal

License

MIT License - Free for research and educational purposes.

Developed as part of the HYBRIDS Marie Skłodowska-Curie Actions project