Lauther/d4-embeddings-v1.0

SentenceTransformer based on intfloat/multilingual-e5-large-instruct

This is a sentence-transformers model finetuned from intfloat/multilingual-e5-large-instruct on the d4-embeddings-multiple_negative dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/multilingual-e5-large-instruct
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • d4-embeddings-multiple_negative

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Lauther/d4-embeddings-v1.0")
# Run inference
sentences = [
    'ECOMP-GUARAREMA - BY-PASS',
    'What is a flow computer?\nA flow computer is a device used in measurement engineering. It collects analog and digital data from flow meters and other sensors.\n\nKey features of a flow computer:\n- It has a unique name, firmware version, and manufacturer information.\n- It is designed to record and process data such as temperature, pressure, and fluid volume (for gases or oils).',
    'What is a Fluid?\nA Fluid is the substance measured within a measurement system. It can be a gas or liquid, such as hydrocarbons, water, or other industrial fluids. Proper classification of fluids is essential for ensuring measurement accuracy, regulatory compliance, and operational efficiency. By identifying fluids correctly, the system applies the appropriate measurement techniques, processing methods, and reporting standards.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

d4-embeddings-multiple_negative

• Dataset: d4-embeddings-multiple_negative at 299ee20
• Size: 3,632 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 8.91 tokens max: 19 tokens	min: 80 tokens mean: 208.8 tokens max: 406 tokens

• Samples:

  anchor	positive
  differential pressure	What is an Uncertainty Curve Point? An Uncertainty Curve Point represents a data point used to construct the uncertainty curve of a measurement system. These curves help analyze how measurement uncertainty behaves under different flow rate conditions, ensuring accuracy and reliability in uncertainty assessments. Key Aspects of an Uncertainty Curve Point: - Uncertainty File ID: Links the point to the specific uncertainty dataset, ensuring traceability. Equipment Tag ID: Identifies the equipment associated with the uncertainty measurement, crucial for system validation. - Uncertainty Points: Represent a list uncertainty values recorded at specific conditions, forming part of the overall uncertainty curve. Do not confuse this uncertainty points with the calculated uncertainty. - Flow Rate Points: Corresponding flow rate values at which the uncertainty was measured, essential for evaluating performance under varying operational conditions. These points are fundamental for generating unce...
  FQI-4300.44-101	What is a Measurement Type? Measurement types define the classification of measurements used within a system based on their purpose and regulatory requirements. These types include fiscal, appropriation, operational, and custody measurements. - Fiscal measurements are used for tax and regulatory reporting, ensuring accurate financial transactions based on measured quantities. - Appropriation measurements track resource allocation and ownership distribution among stakeholders. - Operational measurements support real-time monitoring and process optimization within industrial operations. - Custody measurements are essential for legal and contractual transactions, ensuring precise handover of fluids between parties. These classifications play a crucial role in compliance, financial accuracy, and operational efficiency across industries such as oil and gas, water management, and energy distribution.
  FQI-4300-44116	What is a Meter Stream? A Meter Stream represents a measurement system configured within a flow computer. It serves as the interface between the physical measurement system and the computational processes that record and analyze flow data. Key Aspects of a Meter Stream: - Status: Indicates whether the meter stream is active or inactive. - Measurement System Association: Links the meter stream to a specific measurement system, ensuring that the data collected corresponds to a defined physical setup. - Flow Computer Association: Identifies the flow computer responsible for managing and recording the measurement system's data. Why is a Meter Stream Important? A meter stream is a critical component in flow measurement, as it ensures that the measurement system is correctly integrated into the flow computer for accurate monitoring and reporting. Since each flow computer can handle multiple meter streams, proper configuration is essential for maintaining data integrity and traceability.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

d4-embeddings-multiple_negative

• Dataset: d4-embeddings-multiple_negative at 299ee20
• Size: 778 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 778 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 8.92 tokens max: 19 tokens	min: 27 tokens mean: 214.51 tokens max: 406 tokens

• Samples:

  anchor	positive
  FQI-4150.6122A1	What is an Uncertainty Curve Point? An Uncertainty Curve Point represents a data point used to construct the uncertainty curve of a measurement system. These curves help analyze how measurement uncertainty behaves under different flow rate conditions, ensuring accuracy and reliability in uncertainty assessments. Key Aspects of an Uncertainty Curve Point: - Uncertainty File ID: Links the point to the specific uncertainty dataset, ensuring traceability. Equipment Tag ID: Identifies the equipment associated with the uncertainty measurement, crucial for system validation. - Uncertainty Points: Represent a list uncertainty values recorded at specific conditions, forming part of the overall uncertainty curve. Do not confuse this uncertainty points with the calculated uncertainty. - Flow Rate Points: Corresponding flow rate values at which the uncertainty was measured, essential for evaluating performance under varying operational conditions. These points are fundamental for generating unce...
  FQI-4150.63122C	What is uncertainty? Uncertainty is a measure of confidence in the precision and reliability of results obtained from equipment or measurement systems. It quantifies the potential error or margin of error in measurements. Types of uncertainty: There are two main types of uncertainty: 1. Uncertainty of magnitudes (variables): - Refers to the uncertainty of specific variables, such as temperature or pressure. - It is calculated after calibrating a device or obtained from the equipment manufacturer's manual. - This uncertainty serves as a starting point for further calculations related to the equipment. 2. Uncertainty of the measurement system: - Refers to the uncertainty calculated for the overall flow measurement. - It depends on the uncertainties of the individual variables (magnitudes) and represents the combined margin of error for the entire system. Key points: - The uncertainties of magnitudes (variables) are the foundation for calculating the uncertainty of ...
  Pressão Absoluta	What is uncertainty? Uncertainty is a measure of confidence in the precision and reliability of results obtained from equipment or measurement systems. It quantifies the potential error or margin of error in measurements. Types of uncertainty: There are two main types of uncertainty: 1. Uncertainty of magnitudes (variables): - Refers to the uncertainty of specific variables, such as temperature or pressure. - It is calculated after calibrating a device or obtained from the equipment manufacturer's manual. - This uncertainty serves as a starting point for further calculations related to the equipment. 2. Uncertainty of the measurement system: - Refers to the uncertainty calculated for the overall flow measurement. - It depends on the uncertainties of the individual variables (magnitudes) and represents the combined margin of error for the entire system. Key points: - The uncertainties of magnitudes (variables) are the foundation for calculating the uncertainty...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 100
• per_device_eval_batch_size: 100
• learning_rate: 2e-05
• weight_decay: 0.01
• max_grad_norm: 0.5
• num_train_epochs: 10
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• fp16: True
• dataloader_num_workers: 4

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 100
• per_device_eval_batch_size: 100
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 0.5
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 4
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.1351	5	4.6063	-
0.2703	10	4.5611	-
0.4054	15	4.5136	-
0.5405	20	4.4686	-
0.6757	25	4.3974	-
0.8108	30	4.3636	-
0.9459	35	4.3199	-
1.0811	40	4.1099	-
1.2162	45	4.32	-
1.3514	50	4.2257	-
1.4865	55	4.2873	-
1.6216	60	4.2686	-
1.7568	65	4.2479	-
1.8919	70	4.2432	-
2.0270	75	4.0026	-
2.1622	80	4.187	-
2.2973	85	4.2103	-
2.4324	90	4.2152	-
2.5676	95	4.2713	-
2.7027	100	4.2239	-
2.8378	105	4.1924	-
2.9730	110	4.1704	-
3.1081	115	4.0059	-
3.2432	120	4.1686	-
3.3784	125	4.1632	-
3.5135	130	4.2151	-
3.6486	135	4.2386	-
3.7838	140	4.234	-
3.9189	145	4.1404	-
4.0541	150	3.9627	4.2148
4.1892	155	4.1705	-
4.3243	160	4.1642	-
4.4595	165	4.2132	-
4.5946	170	4.2082	-
4.7297	175	4.2228	-
4.8649	180	4.1663	-
5.0	185	3.9486	-
5.1351	190	4.1747	-
5.2703	195	4.1028	-
5.4054	200	4.1586	-
5.5405	205	4.1668	-
5.6757	210	4.2009	-
5.8108	215	4.1822	-
5.9459	220	4.1669	-
6.0811	225	3.9627	-
6.2162	230	4.1673	-
6.3514	235	4.1455	-
6.4865	240	4.0968	-
6.6216	245	4.1569	-
6.7568	250	4.1978	-
6.8919	255	4.1343	-
7.0270	260	3.9438	-
7.1622	265	4.1094	-
7.2973	270	4.1836	-
7.4324	275	4.1104	-
7.5676	280	4.138	-
7.7027	285	4.1784	-
7.8378	290	4.1437	-
7.9730	295	4.141	-
8.1081	300	3.8248	4.3043
8.2432	305	4.1369	-
8.3784	310	4.128	-
8.5135	315	4.1231	-
8.6486	320	4.1153	-
8.7838	325	4.1667	-
8.9189	330	4.1659	-
9.0541	335	3.8298	-
9.1892	340	4.182	-
9.3243	345	4.1639	-
9.4595	350	4.1651	-
9.5946	355	4.0624	-
9.7297	360	4.1012	-
9.8649	365	4.0938	-
10.0	370	3.857	-

Framework Versions

• Python: 3.11.0
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}