Team Ai
Modelpublic

YKYSpatz/ragproject_ver2

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes99downloads
Model Card

SentenceTransformer based on YKYSpatz/ragproject

This is a sentence-transformers model finetuned from YKYSpatz/ragproject. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • —Model Type: Sentence Transformer
  • —Base model: YKYSpatz/ragproject <!-- at revision 8cb94db3c73c2bb485ad485aaece8f362b60bae7 -->
  • —Maximum Sequence Length: 512 tokens
  • —Output Dimensionality: 384 dimensions
  • —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash
pip install -U sentence-transformers

Then you can load this model and run inference.

python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Is this a clean medical document?',
    'specific examples of protein tethering, see Figure 3–54 and Figure 16–18; for scaffold RNA molecules, see Figure 7–49B.) a few of the modifying groups with known regulatory roles. As in phosphate and ubiquitin additions described previously, these groups are added and then removed from proteins according to the needs of the cell.  A large number of proteins are now known to be modified on more than one amino acid side chain, with different regulatory events producing a different pattern of such',
    'latent period between injury and onset of neurologic sequelae, diagnostic imaging is performed based on identified risk factors (Fig. 7-55).92 After identification of an injury, antithrombotics are administered if the patient does not have contraindications (intra-cranial hemorrhage, falling hemoglobin level with solid organ injury or complex pelvic fractures). Heparin, started without a loading dose at 15 units/kg per hour, is titrated to achieve a PTT between 40 and 50 seconds or antiplatelet',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset
  • —Size: 2,150 training samples
  • —Columns: <code>sentence0</code>, <code>sentence1</code>, and <code>label</code>
  • —Approximate statistics based on the first 1000 samples: | | sentence0 | sentence1 | label | |:--------|:-------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:-----------------------------------------------| | type | string | string | int | | details | <ul><li>min: 9 tokens</li><li>mean: 9.0 tokens</li><li>max: 9 tokens</li></ul> | <ul><li>min: 66 tokens</li><li>mean: 122.43 tokens</li><li>max: 253 tokens</li></ul> | <ul><li>0: ~7.10%</li><li>1: ~92.90%</li></ul> |
  • —Samples: | sentence0 | sentence1 | label | |:-----------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------| | <code>Is this a clean medical document?</code> | <code>Postsynaptic potentials and action potential generation. A shows the voltage recorded upon entry of a micro-electrode into a postsynaptic cell and subsequent recording of a resting membrane potential of −60 mV. Stimulation of an excitatory pathway (E1, left) generates transient depolarization called an excitatory postsynaptic potential (EPSP). Simultaneous activation of multiple excitatory synapses (E1 + E2, middle) increases the size of the depolarization, so that the threshold for action</code> | <code>1</code> | | <code>Is this a clean medical document?</code> | <code>the dysarthrias being considered in this section. Also, in many cases of partially recovered Broca aphasia and in the “mini-Broca” syndrome, the patient is left with a dysarthria that may be difficult to distinguish from a pure articulatory defect. Careful testing of other language functions, especially writing reveals the aphasic aspect of the defect. A severe dysarthria that is difficult to classify, but resembles that of cerebellar disease, may occur with a left hemiplegia, usually the</code> | <code>1</code> | | <code>Is this a clean medical document?</code> | <code>al: faternal obesity, length of gestation, risk of postdates pregnancy and spontaneous onset of labour at term. B JOG 115:720,t2008 Dixon JB: Obesity in 2015: advances in managing obesity. Nat Rev Endocrinol 12:65,t2016 Dixon JB, Dixon ME, O'Brien PE: Birth outcomes in obese women after laparoscopic adjustable gastric banding. Obstet Gynecol lO6:965, 2005 Ducarme G, Parisio L, Santulli P, et al: Neonatal outcomes in pregnancies after bariatric surgery: a retrospective multi-centric cohort</code> | <code>1</code> |
  • —Loss: <code>SoftmaxLoss</code>

Training Hyperparameters

Non-Default Hyperparameters
  • —per_device_train_batch_size: 32
  • —per_device_eval_batch_size: 32
  • —num_train_epochs: 2
  • —multi_dataset_batch_sampler: round_robin
All Hyperparameters

<details><summary>Click to expand</summary>

  • —overwrite_output_dir: False
  • —do_predict: False
  • —eval_strategy: no
  • —prediction_loss_only: True
  • —per_device_train_batch_size: 32
  • —per_device_eval_batch_size: 32
  • —per_gpu_train_batch_size: None
  • —per_gpu_eval_batch_size: None
  • —gradient_accumulation_steps: 1
  • —eval_accumulation_steps: None
  • —torch_empty_cache_steps: None
  • —learning_rate: 5e-05
  • —weight_decay: 0.0
  • —adam_beta1: 0.9
  • —adam_beta2: 0.999
  • —adam_epsilon: 1e-08
  • —max_grad_norm: 1
  • —num_train_epochs: 2
  • —max_steps: -1
  • —lr_scheduler_type: linear
  • —lr_scheduler_kwargs: {}
  • —warmup_ratio: 0.0
  • —warmup_steps: 0
  • —log_level: passive
  • —log_level_replica: warning
  • —log_on_each_node: True
  • —logging_nan_inf_filter: True
  • —save_safetensors: True
  • —save_on_each_node: False
  • —save_only_model: False
  • —restore_callback_states_from_checkpoint: False
  • —no_cuda: False
  • —use_cpu: False
  • —use_mps_device: False
  • —seed: 42
  • —data_seed: None
  • —jit_mode_eval: False
  • —use_ipex: False
  • —bf16: False
  • —fp16: False
  • —fp16_opt_level: O1
  • —half_precision_backend: auto
  • —bf16_full_eval: False
  • —fp16_full_eval: False
  • —tf32: None
  • —local_rank: 0
  • —ddp_backend: None
  • —tpu_num_cores: None
  • —tpu_metrics_debug: False
  • —debug: []
  • —dataloader_drop_last: False
  • —dataloader_num_workers: 0
  • —dataloader_prefetch_factor: None
  • —past_index: -1
  • —disable_tqdm: False
  • —remove_unused_columns: True
  • —label_names: None
  • —load_best_model_at_end: False
  • —ignore_data_skip: False
  • —fsdp: []
  • —fsdp_min_num_params: 0
  • —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
  • —fsdp_transformer_layer_cls_to_wrap: None
  • —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
  • —deepspeed: None
  • —label_smoothing_factor: 0.0
  • —optim: adamw_torch
  • —optim_args: None
  • —adafactor: False
  • —group_by_length: False
  • —length_column_name: length
  • —ddp_find_unused_parameters: None
  • —ddp_bucket_cap_mb: None
  • —ddp_broadcast_buffers: False
  • —dataloader_pin_memory: True
  • —dataloader_persistent_workers: False
  • —skip_memory_metrics: True
  • —use_legacy_prediction_loop: False
  • —push_to_hub: False
  • —resume_from_checkpoint: None
  • —hub_model_id: None
  • —hub_strategy: every_save
  • —hub_private_repo: None
  • —hub_always_push: False
  • —gradient_checkpointing: False
  • —gradient_checkpointing_kwargs: None
  • —include_inputs_for_metrics: False
  • —include_for_metrics: []
  • —eval_do_concat_batches: True
  • —fp16_backend: auto
  • —push_to_hub_model_id: None
  • —push_to_hub_organization: None
  • —mp_parameters:
  • —auto_find_batch_size: False
  • —full_determinism: False
  • —torchdynamo: None
  • —ray_scope: last
  • —ddp_timeout: 1800
  • —torch_compile: False
  • —torch_compile_backend: None
  • —torch_compile_mode: None
  • —include_tokens_per_second: False
  • —include_num_input_tokens_seen: False
  • —neftune_noise_alpha: None
  • —optim_target_modules: None
  • —batch_eval_metrics: False
  • —eval_on_start: False
  • —use_liger_kernel: False
  • —eval_use_gather_object: False
  • —average_tokens_across_devices: False
  • —prompts: None
  • —batch_sampler: batch_sampler
  • —multi_dataset_batch_sampler: round_robin

</details>

Framework Versions

  • —Python: 3.11.12
  • —Sentence Transformers: 4.1.0
  • —Transformers: 4.52.2
  • —PyTorch: 2.6.0+cu124
  • —Accelerate: 1.7.0
  • —Datasets: 2.14.4
  • —Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers and SoftmaxLoss
bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->