TREEIndustries/Llama-3-ARDY-Text-Encoder-ONNX

Llama 3 ARDY Text Encoder ONNX

Built with Meta Llama 3.

This is the reusable INT4 ONNX text encoder for NVIDIA ARDY runtime motion prompting. It converts a raw prompt token sequence into the 4096-dimensional text feature consumed by every Core and G1 ARDY denoiser in the TREE Industries ONNX releases:

• —`ARDY-Core-RP-20FPS-Horizon40-ONNX`
• —`ARDY-Core-RP-20FPS-Horizon8-ONNX`
• —`ARDY-G1-RP-25FPS-Horizon52-ONNX`
• —`ARDY-G1-RP-25FPS-Horizon8-ONNX`

The graph combines:

1. 1.meta-llama/Meta-Llama-3-8B-Instruct
2. 2.the McGill NLP LLM2Vec MNTP adapter
3. 3.the McGill NLP LLM2Vec supervised adapter
4. 4.bidirectional Llama attention and ARDY-compatible mean pooling

The two adapters were merged before ONNX export. Constant-weight matrix multiplications were quantized to symmetric INT4 MatMulNBits operators with block size 32. The graph retains FP16 activations and returns an FP32 pooled embedding.

Files

• —onnx/text_encoder_int4.onnx is the ONNX graph.
• —onnx/text_encoder_int4.onnx.data contains its external INT4 weights.
• —tokenizer/tokenizer.json and tokenizer/tokenizer_config.json are the standard Llama 3 tokenizer assets.
• —tokenizer/tokenizer.ardy is a compact case-sensitive tokenizer asset for native runtimes.
• —validation/text_encoder_reference.json contains five FP16 reference embeddings and their exact tokenized inputs.
• —SHA256SUMS.txt provides integrity hashes.

ONNX interface

Inputs are fixed to a 64-token sequence:

Output:

Prompts are formatted using the Llama 3 user header and end-of-turn token before tokenization. Padding is on the left; truncation is on the right.

Validation

The graph passed full ONNX checking and was executed with ONNX Runtime 1.20.1 using DmlExecutionProvider on an NVIDIA GeForce RTX 4080 Laptop GPU.

Across five stored FP16 reference prompts:

• —minimum cosine similarity: 0.9772086
• —mean cosine similarity: 0.9805653
• —maximum cosine similarity: 0.9823330
• —every output had shape [1, 4096]
• —every output value was finite

Detailed per-prompt results are in validation/VALIDATION.md.

Source provenance

Licenses and required notices

The Llama Materials and this derivative model are distributed under the Meta Llama 3 Community License. Use must also comply with the Meta Llama 3 Acceptable Use Policy.

Required Meta attribution:

Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

The LLM2Vec adapters and project code are MIT licensed. See LICENSES/MIT-LLM2Vec.txt and NOTICE.

LLM2Vec citation

@inproceedings{llm2vec,
  title={LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders},
  author={BehnamGhader, Parishad and Adlakha, Vaibhav and Mosbach, Marius and Bahdanau, Dzmitry and Chapados, Nicolas and Reddy, Siva},
  booktitle={First Conference on Language Modeling},
  year={2024},
  url={https://openreview.net/forum?id=IW1PR7vEBf}
}