firfircelik/etl-error-explainer-v2

ETL Error Explainer v2

Model Description

ETL Error Explainer v2 is a domain-specific instruction-tuned language model designed to analyze ETL and data pipeline failures.

Given an ETL error message and execution context (environment, severity, cloud provider), the model produces a structured JSON response containing:

• —category
• —root_cause
• —immediate_fix
• —longtermfix

The model is intended to assist Data Engineers, Analytics Engineers, Platform Engineers, and DevOps teams during troubleshooting and incident response.

Model Details

Developed by

Fırat Çelik

Model type

Instruction-tuned causal language model

Base model

Qwen/Qwen2.5-1.5B-Instruct

Fine-tuning method

QLoRA (4-bit NF4 Quantization + LoRA)

Language

English

License

Apache-2.0

Model Sources

Hugging Face Repository

https://huggingface.co/firfircelik/etl-error-explainer-v2

Dataset

Synthetic ETL troubleshooting instruction dataset created by the author.

Intended Uses

Direct Use

The model is designed for structured ETL troubleshooting.

Example applications include:

• —Spark error explanation
• —SQL exception analysis
• —Databricks job failures
• —Azure Data Factory pipeline failures
• —Kafka ingestion errors
• —Snowflake query failures
• —Schema evolution issues
• —Authentication and permission errors
• —Data validation failures

Input:

Context:
environment=prod
severity=High
cloud=Azure

Analyze the following Databricks failure:

org.apache.spark.sql.AnalysisException:
cannot resolve 'customer_email'

Output:

{
  "category": "...",
  "root_cause": "...",
  "immediate_fix": "...",
  "long_term_fix": "..."
}

Downstream Uses

The model can be integrated into:

• —AI Data Engineering copilots
• —Internal troubleshooting assistants
• —Incident response systems
• —RAG applications
• —Chatbots
• —IDE assistants
• —Knowledge base generation
• —Support automation

Out-of-Scope Uses

The model is not intended for:

• —General-purpose chat
• —Code generation
• —SQL execution
• —Production system diagnosis
• —Security analysis
• —Legal advice
• —Medical advice

Outputs should always be reviewed before applying fixes to production environments.

Bias, Risks and Limitations

The model has been trained specifically for ETL troubleshooting.

Limitations include:

• —It does not inspect live systems.
• —It may suggest common fixes that are not applicable in every environment.
• —It cannot validate infrastructure configurations.
• —It may hallucinate missing context if insufficient information is provided.
• —It should complement—not replace—human expertise.

Recommendations

Best performance is achieved when prompts include:

• —Full error messages
• —Stack traces
• —Cloud provider
• —Environment (dev/test/prod)
• —Severity level
• —Relevant SQL or Spark snippets

Getting Started

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "firfircelik/etl-error-explainer-v2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = """
Context:
environment=prod
severity=High
cloud=Azure

Analyze the following Databricks failure:

org.apache.spark.sql.AnalysisException:
cannot resolve 'customer_email'
"""

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Base Model

Qwen/Qwen2.5-1.5B-Instruct

Fine-Tuning Method

QLoRA

• —4-bit NF4 quantization
• —LoRA adapters
• —PEFT

LoRA Configuration

Optimizer

pagedadamw8bit

Training Hyperparameters

Training Data

The model was trained on a synthetic instruction dataset covering realistic ETL failures.

Covered domains include:

• —Apache Spark
• —Databricks
• —SQL
• —Azure Data Factory
• —Snowflake
• —Kafka
• —Delta Lake
• —Data validation
• —Schema evolution
• —Authentication
• —File ingestion
• —Cloud storage

Each sample contains:

• —execution context
• —instruction
• —structured target JSON

Evaluation

A held-out validation split (10%) was used during training.

Additionally, the model was manually tested using unseen ETL failure scenarios.

Evaluation focused on:

• —JSON validity
• —Root cause correctness
• —Practical remediation quality
• —Instruction following

Environmental Impact

Hardware

NVIDIA Tesla T4

Platform

Kaggle Notebooks

Training Method

QLoRA (4-bit)

Using QLoRA significantly reduces GPU memory usage and energy consumption compared to full fine-tuning.

Technical Specifications

Architecture

• —Transformer Decoder
• —Causal Language Modeling

Objective

Instruction-following generation of structured ETL troubleshooting responses.

Output format:

{
  "category": "...",
  "root_cause": "...",
  "immediate_fix": "...",
  "long_term_fix": "..."
}

Supported Domains

Citation

@misc{celik2026etlexplainer,
  author = {Fırat Çelik},
  title = {ETL Error Explainer v2},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/firfircelik/etl-error-explainer-v2}}
}

Contact

GitHub

https://github.com/firfircelik

Hugging Face

https://huggingface.co/firfircelik