Modelpublic

guicybercode/br-sovereign-llm-smoke

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes72downloads

Model Card

BR Sovereign LLM Local Smoke Model

Status: engineering artifact only

This is a tiny, randomly initialized Llama checkpoint trained for six CPU optimizer steps on eight original synthetic Brazilian Portuguese documents. It exists only to validate the local from-scratch training, in-process interruption/resume simulation, integrity, export, and loading paths. It does not establish recovery across a process, node, or scheduler restart.

Do not use this model for language generation, evaluation, downstream tasks, or any scientific conclusion. Its loss values are path-validation evidence, not a model-quality result. It was not trained on a supercomputer.

Measured configuration

Field	Value
parameters	149,696
decoder layers	2
hidden size	64
attention heads	4
key/value heads	2
context length	64
observed vocabulary	447
numeric type	float32
initialization seed	1701
optimizer steps	6
interruption after step	3
execution device	CPU

The default implementation used PyTorch 2.13.0, Transformers 5.15.1, and Tokenizers 0.22.2 under Python 3.13.12.

Recovery evidence

Measurement	Value
initial loss	`6.131303787231445`
uninterrupted final loss	`6.114025115966797`
resumed final loss	`6.114025115966797`
exact recovery	`true`
gradients finite and nonzero	`true`
model weights changed	`true`
Python and PyTorch RNG restored	`true`
uninterrupted state SHA-256	`c68fc0da7e6126f320bda2479d2d7de83bc3e7d09c250abe3c2ccbc1ff6b4055`
resumed state SHA-256	`c68fc0da7e6126f320bda2479d2d7de83bc3e7d09c250abe3c2ccbc1ff6b4055`
checkpoint manifest file SHA-256	`c3b64c12e3d65286586e66c6a2541b18a07729c9c8fc78038d80b560de294a88`
smoke report file SHA-256	`989859964a6020b69783b71a66d761f85a8b8de5a31047b08fd3387c2a8a58f4`
tokenizer SHA-256	`fa8ce3f9d3e665e0caaffa380e2ee0b6d0e4f71e2084f5dda478c82e3e9f8e90`

The full loss trajectories and final tensor hashes matched between the uninterrupted and interrupted-then-resumed executions. The published folder contains safetensors weights, model configuration, tokenizer, and the smoke training report. Optimizer state and the internal .pt recovery state are not published.

Loading

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "guicybercode/br-sovereign-llm-smoke"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

Loading success does not make the generated text meaningful.

Data and license

The eight synthetic documents were drafted with AI assistance for BR Sovereign LLM, reviewed and adopted by the project owner, and released under Apache-2.0 to the extent of his rights. The smoke weights are also released under Apache-2.0. This choice does not preselect a license for future scientific weights trained on another corpus.

Contamination, memorization, and safety review

—Training input was limited to the public eight-document fixture. Its source JSONL SHA-256 is 6896439e772f212c2848846f1f908410a606d81cd8845583024c16496a07979f; the derived Parquet SHA-256 is e2b5d4c01739e92240cfe99e6dccebce3903729d083e61e86afa37f6dd86c8a2.
—No benchmark examples or third-party corpus documents are present, so there is no benchmark-contamination result to report for this engineering run.
—No formal extraction or memorization evaluation was run. With eight short training documents, reproduction of fixture phrases must be assumed possible.
—No safety evaluation was run. The model is randomly initialized and trained for only six steps; its output is not suitable for use.
—Hosted inference is disabled in the card. Loading remains documented only to verify artifact compatibility.

The scoped approval and weight-license decision are in PUBLICATION_REVIEW.md; the machine-readable removal history is takedown-ledger.json.

Author

Guilherme Monteiro, FIAP ORCID: <https://orcid.org/0009-0008-5294-224X>