Modelpublic

AvinashRicky/AViGPT

sourceHugging Facemitupdated 21d agoView on Hugging Face

1likes483downloads

Model Card

AViGPT: 183M Parameter Core with Native NVMe Hardware Bus

[image]

AViGPT is a 183-million parameter autoregressive language model designed and pretrained from scratch by Avinash Ricky Yadlapalli.

Rather than increasing parameter count to memorize factual records inside dense neural weights, AViGPT separates syntactic reasoning from factual storage. It couples a compact 183M reasoning core with a dedicated local NVMe SSD hardware memory bus. The model emits explicit control tokens to pause inference, execute sub-millisecond SQLite FTS5 full-text lookups on local storage, inject verified records into context, and complete generations with verified factual precision.

—Author: Avinash Ricky Yadlapalli (@AvinashRicky)
—DOI: 10.5281/zenodo.22856047
—Code Repository: GitHub - Avinashricky211/AviGPT
—License: MIT Open Source License

Technical Specifications

Parameter	Specification
Model Size	183,926,400 parameters (183M)
Architecture	Autoregressive Decoder-only Transformer (`GPT2LMHeadModel` compatible)
Layer Count	16 Transformer Layers
Hidden Dimension ($d_{\text{model}}$)	896
Attention Heads	14 heads (head dimension = 64)
Context Window	1,024 tokens
Vocabulary	32,009 custom BPE tokens (including 9 hardware control tokens)
Pretraining Volume	~5.0 Billion tokens (English Wikipedia + FineWeb-Edu subset)
Alignment Dataset	25,850 multi-step hardware trajectories
Storage Engine	Local SQLite 3 FTS5 (WAL mode, normal synchronous disk writes)
SSD Latency	1.18 milliseconds (NVMe average read latency)
RAM Footprint	~0.4 GB (runs comfortably on CPU or edge devices)

Hardware Memory Bus Protocol

AViGPT manages external execution through nine dedicated vocabulary tokens:

User Instruction
       │
       ▼
[ AViGPT Neural Core (183M) ]
       │
       ├── Emits <|intent_start|> ... <|intent_end|> (Goal framing)
       ├── Emits <|mem_query|> ... <|mem_query_end|>
       │         │
       │         ▼
       │     [ NVMe SSD / SQLite FTS5 Engine ] ── Latency: 1.18 ms
       │         │
       ├── Emits <|mem_payload|> ... <|mem_payload_end|> (Injects factual record)
       ├── Emits <|calc|> ... <|calc_end|> (Optional arithmetic sandbox)
       │
       ▼
   <|synthesize|> (Produces final verified answer)

Special Token	Role	Runtime Action
`<	intent_start	> `/` <	intent_end	>`	Intent formulation	Deconstructs request into concise query parameters.
`<	mem_query	> `/` <	memqueryend	>`	Disk retrieval trigger	Generation pauses; search string dispatches to SQLite FTS5.
`<	mem_payload	> `/` <	mempayloadend	>`	Ground truth injection	Retrieved database record injected directly into KV cache.
`<	calc	> `/` <	calc_end	>`	Arithmetic dispatch	Sandboxed AST evaluator computes exact numeric result.
`<	synthesize	>`	Final synthesis	Model resumes decoding to deliver grounded final response.

Empirical Benchmarks & Performance Charts

1. Training Convergence Curve

Cross-entropy loss declined from 3.3698 to 0.6935 over 1,800 steps on an NVIDIA T4 GPU:

[image]

2. External Retrieval Latency (NVMe Bus vs. Network RAG)

Direct NVMe storage retrieval operates in 1.18 milliseconds, compared to 500 to 1,500 milliseconds for network-based RAG architectures:

[image]

3. Hardware Footprint Comparison

AViGPT operates with a 0.4 GB memory footprint, running entirely on consumer CPUs without requiring dedicated GPU accelerators:

[image]

Hardware & Efficiency Comparison

System	Parameters	Minimum Hardware	Retrieval Latency	Knowledge Updates
AViGPT	183M	0.4 GB RAM (Any CPU)	1.18 ms (NVMe SSD)	Immediate (0-cost disk write)
SmolLM-135M	135M	0.3 GB RAM	None (Parametric only)	Retraining required
LLaMA-3-8B	8.0B	16 GB VRAM	N/A	Retraining required
Standard RAG	8B+	16 GB + Vector DB	450 ms – 1,200 ms	Index re-embedding

Quickstart

1. Standard Hugging Face Generation

You can load and query the model directly via the transformers library:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "AvinashRicky/AViGPT"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)

prompt = (
    "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n"
    "### Instruction:\nWho is your owner and creator?\n\n### Response:\n"
)

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=120,
        temperature=0.2,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=False))

2. Autonomous Hardware Bus Loop (Full System)

To run AViGPT with active sub-millisecond SSD queries and arithmetic execution:

bash

git clone https://github.com/Avinashricky211/AviGPT.git
cd AviGPT
pip install -r requirements.txt
streamlit run app.py

Or programmatically in Python:

python

import torch
from transformers import GPT2LMHeadModel, GPT2TokenizerFast
from autonomous_bus import AutonomousHardwareBus

model_id = "AvinashRicky/AViGPT"
tokenizer = GPT2TokenizerFast.from_pretrained(model_id)
model = GPT2LMHeadModel.from_pretrained(model_id).to("cuda" if torch.cuda.is_available() else "cpu")

# Initialize hardware bus controller with local SQLite FTS5 engine
bus = AutonomousHardwareBus(model=model, tokenizer=tokenizer)

# Execute query with hardware-accelerated memory retrieval
response, metrics = bus.generate_autonomous_response("When did Apollo 11 land on the Moon?")

print("Response:\n", response)
print(f"SSD Retrieval Latency: {metrics['ssd_latency_ms']:.2f} ms")
print(f"Total Response Latency: {metrics['total_latency_s']:.2f} s")

Training Details

—Phase 1: Pretraining from Scratch
—Hardware: NVIDIA H100 SXM5 80GB GPU.
—Optimizer: AdamW ($\beta1=0.9, \beta2=0.95$, weight decay $0.1$, learning rate $6 \times 10^{-4}$ with cosine decay).
—Data: 5.0B tokens combining English Wikipedia and the educational FineWeb-Edu subset.
—Starting Loss: 8.42 $\rightarrow$ Final Pretraining Loss: 2.84.

—Phase 2: Hardware Bus Alignment
—Dataset: 25,850 multi-step hardware trajectories with token-level supervisor loss.
—Initial Alignment Loss: 3.3698 $\rightarrow$ Final Convergence Loss: 0.6935 (Step 1,800).
—Checkpoint Validation: Trajectory validation passed across identity, retrieval, and math routing.

Citation

bibtex

@article{Avinash2026avigpt,
  title={AViGPT: Decoupling Neural Reasoning from Parametric Memory via a Sub-Millisecond Native NVMe Hardware Bus},
  author={Avinash Ricky Yadlapalli},
  year={2026},
  journal={Zenodo},
  doi={10.5281/zenodo.22856047},
  howpublished={\url{https://doi.org/10.5281/zenodo.22856047}},
  url={https://github.com/Avinashricky211/AviGPT}
}