RichardErkhov/LeroyDyer_-_SpydazWebAI_VisionEncoderDecoderModel_Mini548m-4bits

1Quantization made by Richard Erkhov.2 3[Github](https://github.com/RichardErkhov)4 5[Discord](https://discord.gg/pvy7H8DZMG)6 7[Request more models](https://github.com/RichardErkhov/quant_request)8 9 10SpydazWebAI_VisionEncoderDecoderModel_Mini548m - bnb 4bits11- Model creator: https://huggingface.co/LeroyDyer/12- Original model: https://huggingface.co/LeroyDyer/SpydazWebAI_VisionEncoderDecoderModel_Mini548m/13 14 15 16 17Original model description:18---19license: mit20language:21- en22library_name: transformers23tags:24- vision25- VISION-ENCODER-DECODER-MODEL26pipeline_tag: image-to-text27---28 29# ADD HEAD30```31 32Mistral33VISION-ENCODER-DECODER-MODEL34 35print('Add Vision...')36# ADD HEAD37# Combine pre-trained encoder and pre-trained decoder to form a Seq2Seq model38 39 40 41Vmodel = VisionEncoderDecoderModel.from_encoder_decoder_pretrained(42    "google/vit-base-patch16-224-in21k", "LeroyDyer/Mixtral_AI_Tiny"43)44_Encoder_ImageProcessor = Vmodel.encoder45_Decoder_ImageTokenizer = Vmodel.decoder46_VisionEncoderDecoderModel = Vmodel47# Add Pad tokems48LM_MODEL.VisionEncoderDecoder = _VisionEncoderDecoderModel49# Add Sub Components50LM_MODEL.Encoder_ImageProcessor = _Encoder_ImageProcessor51LM_MODEL.Decoder_ImageTokenizer = _Decoder_ImageTokenizer52LM_MODEL53 54 55 56```57 58