SentenceTransformer

This is a sentence-transformers model trained on the mnri_dataset and contrastive_dataset datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Maximum Sequence Length: 2500 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Datasets:
  • mnri_dataset
  • contrastive_dataset

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2500, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 57\nSex: Male\nCancer type: Gastric adenocarcinoma (HER2\u2011amplified, intestinal type)\nHistology: Moderately differentiated intestinal\u2011type adenocarcinoma\nCurrent extent: Metastatic disease with hepatic metastases (dominant lesion ~4\u202fcm in segment\u202fVII, multiple smaller lesions) and diffuse peritoneal carcinomatosis; bulky omental implants and moderate ascites; persistent circumferential thickening of the gastric cardia (~2.5\u202fcm). No longer receiving disease\u2011directed therapy; transitioned to best\u2011supportive hospice care (ECOG\u202f\u2248\u202f4).\nBiomarkers: HER2 IHC\u202f3+, HER2 (ERBB2) amplification confirmed by FISH (ratio\u202f5.2); Microsatellite stable (MSS); KRAS G12V (pathogenic); TP53 R273C (pathogenic); CDH1 G274E (likely pathogenic); PIK3CA E542K (activating); MET amplification (Copy number\u202f\u223c6); FGFR2 amplification (Copy number\u202f\u223c5); CDKN2A homozygous deletion; Tumor mutational burden\u202f\u2248\u202f8\u202fMut/Mb; Ki\u201167 \u223c45%; CK7\u207a/CK20\u207b; retained MLH1,PMS2,MSH2,MSH6.\nTreatment history:\n# 1/2018\u2011mid\u20112018: Front\u2011line trastuzumab (loading 8\u202fmg/kg then 6\u202fmg/kg q21\u202fd)\u202f+\u202fcisplatin 80\u202fmg/m\u00b2 iv day\u202f1 q21\u202fd\u202f+\u202fcontinuous infusional 5\u2011fu 1000\u202fmg/m\u00b2 days\u202f1\u20114 q21\u202fd (four cycles). Best response: Partial response (shrinkage of gastric wall thickening and hepatic lesions).\n# Late\u202f2018 (cycles\u202f5\u20116): Continuation of the same triplet regimen to complete six cycles; maintained partial response.\n# 12/2018: Restaging CT confirmed ongoing partial response.\n# Early\u202f2019: Planned radical gastrectomy aborted intra\u2011operatively due to diffuse peritoneal disease; feeding jejunostomy placed.\n# 5/2019: Elective laparoscopic right hemicolectomy for synchronous ascending colon adenocarcinoma (pT1a\u202fN0\u202fM0, R0). No adjuvant therapy required.\n# 6/2019 onward: Second\u2011line ramucirumab 10\u202fmg/kg i.v q2\u2009wks\u202f+\u202fweekly paclitaxel 80\u202fmg/m\u00b2 (initiated after progression on first\u2011line). Delivered two cycles; development of grade\u202f2 peripheral sensory neuropathy led to dose reduction of paclitaxel to 70\u202fmg/m\u00b2 and eventual cessation after disease progression.\n# 10/2019: Further disease progression evidenced by growing hepatic metastases and new peritoneal implants; cardiology consulted after NSTEMI; recommendation to cease cytotoxic and anti\u2011angiogenic agents.\n# 1/2020: Evaluated for experimental HER2/PD\u2011L1 bispecific antibody trial; found ineligible due to therapeutic anticoagulation (apixaban) begun shortly after NSTEMI.\n# 4/2020: Transitioned to hospice/best\u2011supportive care; all systemic anticancer agents (incl. trastuzumab) discontinued. Focus shifted to symptom control (opioids, anti\u2011emetics, nutritional support via jejunostomy) and advance\u2011directive implementation.\nCancer type: Ascending colon adenocarcinoma\nHistology: Well\u2011to\u2011moderately differentiated tubular adenocarcinoma, grade\u202f1\nCurrent extent: Resected, pathologic stage\u202fIA (pT1a\u202fN0\u202fM0), R0 margins; no evidence of disease on surveillance; no further therapy required.\nBiomarkers: KRAS wild\u2011type; BRAF V600E negative; MSI stable; CK20\u207a, CDX2\u207a.\nTreatment history:\n# 5/2019: Laparoscopic right hemicolectomy with regional node sampling (0/15 positive); postoperative course uncomplicated.",
]
documents = [
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Any solid malignant tumor. Histology allowed: NA. Cancer burden allowed: Advanced unresectable or metastatic disease where standard therapies have failed, are intolerable, or ineffective. Prior treatment required: Progression after standard anticancer therapy. Prior treatment excluded: Any prior gene or cell therapy product or any prior therapy directly targeting KRAS G12V mutation (e.g., KRAS\u202fG12V‐specific small‑molecule inhibitor or cellular therapy). Biomarkers required: KRAS G12V mutation and HLA‑A*11:01 positivity (both assessed during screening). Biomarkers excluded: NA.',
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: NA. Sex allowed: Male and Female. Cancer type allowed: Non‑small cell lung cancer. Histology allowed: Adenocarcinoma, large\u202fcell, neuroendocrine or any non‑squamous histology (predominantly squamous histology excluded). Cancer burden allowed: Unresectable, locally advanced, or metastatic disease. Prior treatment required: Disease refractory to or progressive after all standard‐of‑care therapies demonstrating clinical benefit, unless no applicable standard therapy exists or the patient chooses to decline. Prior treatment excluded: Receipt of concurrent systemic anticancer therapy other than protocol‑permitted localized palliative radiation or hormone ablative therapy. Biomarkers required: NA. Biomarkers excluded: NA.',
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Any solid malignant tumor. Histology allowed: NA. Cancer burden allowed: Locally advanced or metastatic disease. Prior treatment required: NA. Prior treatment excluded: NA. Biomarkers required: NECTIN4 gene amplification positive. Biomarkers excluded: NA.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4265, 0.1800, 0.3230]])

Training Details

Training Datasets

mnri_dataset

• Dataset: mnri_dataset
• Size: 913,338 training samples
• Columns: patient_summary_trunc and this_space_trunc
• Approximate statistics based on the first 1000 samples:

  	patient_summary_trunc	this_space_trunc
  type	string	string
  details	min: 264 tokens mean: 805.57 tokens max: 1600 tokens	min: 112 tokens mean: 170.21 tokens max: 388 tokens

• Samples:

  patient_summary_trunc	this_space_trunc
  Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 15 Sex: Female Cancer type: B‑cell acute lymphoblastic leukemia (precursor B‑cell ALL) Histology: Precursor B‑lymphoblasts (flow: CD19⁺, CD22⁺, CD10⁺, CD34⁺, TdT⁺; CD20−, MPO−, surface IgM−) Current extent: Systemic disease with diffuse bone‑marrow involvement; now in partial remission after induction (marrow blasts ≈15 %, MRD low‑level positive ∼0.02 %) Biomarkers: TCF3‑PBX1 fusion (t(1;19)), hyperdiploid karyotype (extra chromosomes 4, 10, 17), PAX5 truncating mutation (p.R38*), CDKN2A/CDKN2B homozygous deletion, NRAS activating mutation (p.Q61K), low‑frequency CREBBP p.R1746H (subclonal), negative for BCR‑ABL1 and ETV6‑RUNX1; immunophenotype as above Treatment history: # 08/2017 – 09/2017 (Day 1‑28 of induction): Multidrug induction per COG AALL1131 – vincristine ...	Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 0 through 45 years. Sex allowed: both sexes. Cancer type allowed: acute lymphoblastic leukaemia. Histology allowed: B‑cell precursor acute lymphoblastic leukaemia. Cancer burden allowed: newly diagnosed, never relapsed, de novo disease. Prior treatment required: NA. Prior treatment excluded: systemic corticosteroids ≥10 mg/m²/day prednisone equivalents for longer than one week before diagnosis; any chemotherapeutic agent administered within four weeks before diagnosis. Biomarkers required: surface immunoglobulin negative phenotype; IG::MYC rearrangement accepted only when BCL2 and BCL6 rearrangements are absent. Biomarkers excluded: KMT2A‑rearranged B‑cell precursor ALL in patients younger than 1 year; Philadelphia chromosome‑positive (t[9;22]/BCR‑ABL) ...
  Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 53 Sex: Male Cancer type: Lung cancer Histology: Poorly differentiated squamous cell carcinoma (left lower lobe) Current extent: Metastatic disease – primary left lower‑lobar lesion ~3.5 cm, new left adrenal metastasis (1.8 cm), right hilar lymph node enlargement; overall progressive disease per RECIST 1.1 Biomarkers: PD‑L1 tumor proportion score 10 %; HER2 (ERBB2) amplification (≈9 copies); TP53 missense mutation p.R273C (likely pathogenic); CDKN2A homozygous deletion; FGFR1 amplification (≈7 copies); PIK3CA H1047R activating mutation; NOTCH1 L1575P variant of uncertain significance; KRAS wild‑type; EGFR wild‑type Treatment history: # 1/Jan 2017: Diagnostic bronchoscopic forceps biopsies of left lower lobar endobronchial lesion → confirmation of poorly differentiate...	Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Advanced solid tumor. Histology allowed: Any. Cancer burden allowed: Locally advanced or metastatic disease refractory to standard therapy or lacking effective treatment. Prior treatment required: Progressive disease after standard systemic therapy. Prior treatment excluded: Prior receipt of targeted ROR1 inhibitor therapy. Biomarkers required: ROR1 positive (assessment planned during screening). Biomarkers excluded: NA.
  Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 9 Sex: Male Cancer type: B‑myeloid Mixed Phenotype Acute Leukemia (MPAL) Histology: Biphasic B‑cell/myeloid acute leukemia Current extent: Post‑second allogeneic HSCT, in remission; flow MRD < 0.05 % (0.02 %), low‑level FLT3‑ITD persistence (allelic ratio ∼0.38) Biomarkers: FLT3‑ITD (persisting, AR ∼ 0.38); WT1 truncating mutation p.Arg430*; NRAS p.Gly12Asp; DNMT3A p.Arg882His; CDKN2A/B homozygous deletion (9p21); IDH2 p.Arg140Gln (VAF ≈ 21 %→7 % across samples); NUP98‑NSD1 fusion (detected at diagnosis, subsequently undetectable); Additional routine panels negative for KMT2A rearrangement, BCR‑ABL1, other actionable hits. Treatment history: # 1/2017‑5/2017: FLAG‑IDA induction (Fludarabine, Cytarabine, G‑CSF, Idarubicin) + intrathecal Methotrexate; best response comp...	Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 1 Year to 39 Years. Sex allowed: Male and Female. Cancer type allowed: Mixed phenotype acute leukemia, B‑myeloid. Histology allowed: B‑myeloid mixed phenotype acute leukemia. Cancer burden allowed: Relapsed or refractory disease. Prior treatment required: Full recovery from prior hematopoietic stem cell transplantation or anthracycline exposure. Prior treatment excluded: Current administration of anticancer agents (except intrathecal agents or hydroxyurea). Biomarkers required: NA. Biomarkers excluded: KMT2A rearrangement, Philadelphia chromosome/BCR‑ABL1 fusion.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json Copy

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

contrastive_dataset

• Dataset: contrastive_dataset
• Size: 1,526,022 training samples
• Columns: patient_summary_trunc, this_space_trunc, and label
• Approximate statistics based on the first 1000 samples:

  	patient_summary_trunc	this_space_trunc	label
  type	string	string	float
  details	min: 264 tokens mean: 840.28 tokens max: 1600 tokens	min: 121 tokens mean: 185.75 tokens max: 388 tokens	min: -1.0 mean: -0.4 max: 1.0

• Samples:

  patient_summary_trunc	this_space_trunc	label
  Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 15 Sex: Female Cancer type: B‑cell acute lymphoblastic leukemia (precursor B‑cell ALL) Histology: Precursor B‑lymphoblasts (flow: CD19⁺, CD22⁺, CD10⁺, CD34⁺, TdT⁺; CD20−, MPO−, surface IgM−) Current extent: Systemic disease with diffuse bone‑marrow involvement; now in partial remission after induction (marrow blasts ≈15 %, MRD low‑level positive ∼0.02 %) Biomarkers: TCF3‑PBX1 fusion (t(1;19)), hyperdiploid karyotype (extra chromosomes 4, 10, 17), PAX5 truncating mutation (p.R38*), CDKN2A/CDKN2B homozygous deletion, NRAS activating mutation (p.Q61K), low‑frequency CREBBP p.R1746H (subclonal), negative for BCR‑ABL1 and ETV6‑RUNX1; immunophenotype as above Treatment history: # 08/2017 – 09/2017 (Day 1‑28 of induction): Multidrug induction per COG AALL1131 – vincristine ...	Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 0 through 45 years. Sex allowed: both sexes. Cancer type allowed: acute lymphoblastic leukaemia. Histology allowed: B‑cell precursor acute lymphoblastic leukaemia. Cancer burden allowed: newly diagnosed, never relapsed, de novo disease. Prior treatment required: NA. Prior treatment excluded: systemic corticosteroids ≥10 mg/m²/day prednisone equivalents for longer than one week before diagnosis; any chemotherapeutic agent administered within four weeks before diagnosis. Biomarkers required: surface immunoglobulin negative phenotype; IG::MYC rearrangement accepted only when BCL2 and BCL6 rearrangements are absent. Biomarkers excluded: KMT2A‑rearranged B‑cell precursor ALL in patients younger than 1 year; Philadelphia chromosome‑positive (t[9;22]/BCR‑ABL) ...	0.6000000000000001
  Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 53 Sex: Male Cancer type: Lung cancer Histology: Poorly differentiated squamous cell carcinoma (left lower lobe) Current extent: Metastatic disease – primary left lower‑lobar lesion ~3.5 cm, new left adrenal metastasis (1.8 cm), right hilar lymph node enlargement; overall progressive disease per RECIST 1.1 Biomarkers: PD‑L1 tumor proportion score 10 %; HER2 (ERBB2) amplification (≈9 copies); TP53 missense mutation p.R273C (likely pathogenic); CDKN2A homozygous deletion; FGFR1 amplification (≈7 copies); PIK3CA H1047R activating mutation; NOTCH1 L1575P variant of uncertain significance; KRAS wild‑type; EGFR wild‑type Treatment history: # 1/Jan 2017: Diagnostic bronchoscopic forceps biopsies of left lower lobar endobronchial lesion → confirmation of poorly differentiate...	Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Advanced solid tumor. Histology allowed: Any. Cancer burden allowed: Locally advanced or metastatic disease refractory to standard therapy or lacking effective treatment. Prior treatment required: Progressive disease after standard systemic therapy. Prior treatment excluded: Prior receipt of targeted ROR1 inhibitor therapy. Biomarkers required: ROR1 positive (assessment planned during screening). Biomarkers excluded: NA.	0.19999999999999996
  Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 9 Sex: Male Cancer type: B‑myeloid Mixed Phenotype Acute Leukemia (MPAL) Histology: Biphasic B‑cell/myeloid acute leukemia Current extent: Post‑second allogeneic HSCT, in remission; flow MRD < 0.05 % (0.02 %), low‑level FLT3‑ITD persistence (allelic ratio ∼0.38) Biomarkers: FLT3‑ITD (persisting, AR ∼ 0.38); WT1 truncating mutation p.Arg430*; NRAS p.Gly12Asp; DNMT3A p.Arg882His; CDKN2A/B homozygous deletion (9p21); IDH2 p.Arg140Gln (VAF ≈ 21 %→7 % across samples); NUP98‑NSD1 fusion (detected at diagnosis, subsequently undetectable); Additional routine panels negative for KMT2A rearrangement, BCR‑ABL1, other actionable hits. Treatment history: # 1/2017‑5/2017: FLAG‑IDA induction (Fludarabine, Cytarabine, G‑CSF, Idarubicin) + intrathecal Methotrexate; best response comp...	Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 1 Year to 39 Years. Sex allowed: Male and Female. Cancer type allowed: Mixed phenotype acute leukemia, B‑myeloid. Histology allowed: B‑myeloid mixed phenotype acute leukemia. Cancer burden allowed: Relapsed or refractory disease. Prior treatment required: Full recovery from prior hematopoietic stem cell transplantation or anthracycline exposure. Prior treatment excluded: Current administration of anticancer agents (except intrathecal agents or hydroxyurea). Biomarkers required: NA. Biomarkers excluded: KMT2A rearrangement, Philadelphia chromosome/BCR‑ABL1 fusion.	0.19999999999999996

• Loss: CoSENTLoss with these parameters:
  json Copy

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 10
• learning_rate: 2e-05
• warmup_ratio: 0.01
• bf16: True

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.13.12
• Sentence Transformers: 5.2.3
• Transformers: 4.57.6
• PyTorch: 2.9.1+cu128
• Accelerate: 1.13.0
• Datasets: 4.6.1
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

CoSENTLoss

@article{10531646,
    author={Huang, Xiang and Peng, Hao and Zou, Dongcheng and Liu, Zhiwei and Li, Jianxin and Liu, Kay and Wu, Jia and Su, Jianlin and Yu, Philip S.},
    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
    title={CoSENT: Consistent Sentence Embedding via Similarity Ranking},
    year={2024},
    doi={10.1109/TASLP.2024.3402087}
}