A possible explanation is you have a new CUDA version which isn't
yet compatible with FA2? Please file a ticket to Unsloth or FA2.
We shall now use Xformers instead, which does not have any performance hits!
We found this negligible impact by benchmarking on 1x A100.
ImportError                               Traceback (most recent call last)
File /usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py:1603, in _LazyModule._get_module(self, module_name)
   1602 try:
-> 1603     return importlib.import_module("." + module_name, self.__name__)
   1604 except Exception as e:

File /usr/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File /usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:119
    118 if is_accelerate_available():
--> 119     from accelerate.hooks import AlignDevicesHook, add_hook_to_module
    121 NEED_SETUP_CACHE_CLASSES_MAPPING = {
    122     "static": StaticCache,
    123     "sliding_window": SlidingWindowCache,
    124     "hybrid": HybridCache,
    125     "mamba": MambaCache,
    126 }

File /usr/local/lib/python3.10/dist-packages/accelerate/__init__.py:16
     14 __version__ = "0.33.0"
---> 16 from .accelerator import Accelerator
     17 from .big_modeling import (
     18     cpu_offload,
     19     cpu_offload_with_hook,
   (...)
     24     load_checkpoint_and_dispatch,
     25 )

File /usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:36
     34 from huggingface_hub import split_torch_state_dict_into_shards
---> 36 from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
     37 from .data_loader import DataLoaderDispatcher, prepare_data_loader, skip_first_batches

File /usr/local/lib/python3.10/dist-packages/accelerate/checkpointing.py:24
     22 from torch.cuda.amp import GradScaler
---> 24 from .utils import (
     25     MODEL_NAME,
     26     OPTIMIZER_NAME,
     27     RNG_STATE_NAME,
     28     SAFE_MODEL_NAME,
     29     SAFE_WEIGHTS_NAME,
     30     SAMPLER_NAME,
     31     SCALER_NAME,
     32     SCHEDULER_NAME,
     33     WEIGHTS_NAME,
     34     get_pretty_name,
     35     is_torch_xla_available,
     36     is_xpu_available,
     37     save,
     38 )
     41 if is_torch_xla_available():

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/__init__.py:190
    181     from .deepspeed import (
    182         DeepSpeedEngineWrapper,
    183         DeepSpeedOptimizerWrapper,
   (...)
    187         HfDeepSpeedConfig,
    188     )
--> 190 from .bnb import has_4bit_bnb_layers, load_and_quantize_model
    191 from .fsdp_utils import load_fsdp_model, load_fsdp_optimizer, merge_fsdp_weights, save_fsdp_model, save_fsdp_optimizer

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/bnb.py:29
     24 from accelerate.utils.imports import (
     25     is_4bit_bnb_available,
     26     is_8bit_bnb_available,
     27 )
---> 29 from ..big_modeling import dispatch_model, init_empty_weights
     30 from .dataclasses import BnbQuantizationConfig

File /usr/local/lib/python3.10/dist-packages/accelerate/big_modeling.py:24
     22 import torch.nn as nn
---> 24 from .hooks import (
     25     AlignDevicesHook,
     26     CpuOffload,
     27     UserCpuOffloadHook,
     28     add_hook_to_module,
     29     attach_align_device_hook,
     30     attach_align_device_hook_on_blocks,
     31 )
     32 from .utils import (
     33     OffloadedWeightsLoader,
     34     check_cuda_p2p_ib_support,
   (...)
     48     retie_parameters,
     49 )

File /usr/local/lib/python3.10/dist-packages/accelerate/hooks.py:30
     29 from .utils.modeling import get_non_persistent_buffers
---> 30 from .utils.other import recursive_getattr
     33 _accelerate_added_attributes = ["to", "cuda", "npu", "xpu", "mlu", "musa"]

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/other.py:36
     35 from .modeling import id_tensor_storage
---> 36 from .transformer_engine import convert_model
     37 from .versions import is_torch_version

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/transformer_engine.py:21
     20 if is_fp8_available():
---> 21     import transformer_engine.pytorch as te
     24 def convert_model(model, to_transformer_engine=True, _convert_linear=True, _convert_ln=True):

File /usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/__init__.py:34
     31     spec.loader.exec_module(solib)
---> 34 _load_library()
     35 from transformer_engine.pytorch.module import LayerNormLinear

File /usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/__init__.py:29, in _load_library()
     28 spec = importlib.util.spec_from_file_location(module_name, so_path)
---> 29 solib = importlib.util.module_from_spec(spec)
     30 sys.modules[module_name] = solib

ImportError: /usr/local/lib/python3.10/dist-packages/transformer_engine/transformer_engine_torch.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZNK3c105Error4whatEv

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[4], line 1
----> 1 from unsloth import FastLanguageModel
      2 import torch
      3 max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!

File /usr/local/lib/python3.10/dist-packages/unsloth/__init__.py:154
    144         warnings.warn(
    145             "Unsloth: CUDA is not linked properly[.\n](http://<IP_ADDRESS>:8889/lab/workspaces/auto-4/tree/RTC%3Atom_FT/n)"\
    146             "Try running `python -m bitsandbytes` then `python -m xformers.info`\n"\
   (...)
    150             "Unsloth will still run for now, but maybe it might crash - let's hope it works!"
    151         )
    152 pass
--> 154 from .models import *
    155 from .save import *
    156 from .chat_templates import *

File /usr/local/lib/python3.10/dist-packages/unsloth/models/__init__.py:15
      1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from .loader  import FastLanguageModel
     16 from .llama   import FastLlamaModel
     17 from .mistral import FastMistralModel

File /usr/local/lib/python3.10/dist-packages/unsloth/models/loader.py:15
      1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from ._utils import is_bfloat16_supported, HAS_FLASH_ATTENTION, HAS_FLASH_ATTENTION_SOFTCAPPING
     16 from .llama import FastLlamaModel, logger
     17 from .mistral import FastMistralModel

File /usr/local/lib/python3.10/dist-packages/unsloth/models/_utils.py:213
    210     HAS_FLASH_ATTENTION = False
    211 pass
--> 213 from transformers.models.llama.modeling_llama import logger
    215 # =============================================
    216 # Get Xformers
    217 from xformers import __version__ as xformers_version

File /usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py:41
     33 from ...modeling_outputs import (
     34     BaseModelOutputWithPast,
     35     CausalLMOutputWithPast,
   (...)
     38     TokenClassifierOutput,
     39 )
     40 from ...modeling_rope_utils import ROPE_INIT_FUNCTIONS
---> 41 from ...modeling_utils import PreTrainedModel
     42 from ...pytorch_utils import ALL_LAYERNORM_LAYERS
     43 from ...utils import (
     44     add_start_docstrings,
     45     add_start_docstrings_to_model_forward,
   (...)
     48     replace_return_docstrings,
     49 )

File /usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:46
     44 from .configuration_utils import PretrainedConfig
     45 from .dynamic_module_utils import custom_object_save
---> 46 from .generation import GenerationConfig, GenerationMixin
     47 from .integrations import PeftAdapterMixin, deepspeed_config, is_deepspeed_zero3_enabled
     48 from .pytorch_utils import (  # noqa: F401
     49     Conv1D,
     50     apply_chunking_to_forward,
   (...)
     56     prune_linear_layer,
     57 )

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File /usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py:1593, in _LazyModule.__getattr__(self, name)
   1591     value = self._get_module(name)
   1592 elif name in self._class_to_module.keys():
-> 1593     module = self._get_module(self._class_to_module[name])
   1594     value = getattr(module, name)
   1595 else:

File /usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py:1605, in _LazyModule._get_module(self, module_name)
   1603     return importlib.import_module("." + module_name, self.__name__)
   1604 except Exception as e:
-> 1605     raise RuntimeError(
   1606         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1607         f" traceback):\n{e}"
   1608     ) from e

RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
/usr/local/lib/python3.10/dist-packages/transformer_engine/transformer_engine_torch.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZNK3c105Error4whatEv

When running
from unsloth import FastLanguageModel
import torch
max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

# 4bit pre quantized models we support for 4x faster downloading + no OOMs.
fourbit_models = [
    "unsloth/Meta-Llama-3.1-8B-bnb-4bit",      # Llama-3.1 15 trillion tokens model 2x faster!
    "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
    "unsloth/Meta-Llama-3.1-70B-bnb-4bit",
    "unsloth/Meta-Llama-3.1-405B-bnb-4bit",    # We also uploaded 4bit for 405b!
    "unsloth/Mistral-Nemo-Base-2407-bnb-4bit", # New Mistral 12b 2x faster!
    "unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit",
    "unsloth/mistral-7b-v0.3-bnb-4bit",        # Mistral v3 2x faster!
    "unsloth/mistral-7b-instruct-v0.3-bnb-4bit",
    "unsloth/Phi-3.5-mini-instruct",           # Phi-3.5 2x faster!
    "unsloth/Phi-3-medium-4k-instruct",
    "unsloth/gemma-2-9b-bnb-4bit",
    "unsloth/gemma-2-27b-bnb-4bit",            # Gemma 2x faster!
] # More models at https://huggingface.co/unsloth

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Mistral-Nemo-Base-2407",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
)

I am in a pytorch:24.07 docker environment
this is the cuda version (nvcc -V)
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Jun__6_02:18:23_PDT_2024
Cuda compilation tools, release 12.5, V12.5.82
Build cuda_12.5.r12.5/compiler.34385749_0

This is the nvidia-smi
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 PCIe               Off |   00000000:01:00.0 Off |                    0 |
| N/A   54C    P0             62W /  350W |       4MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Yeah IMO it’s tricky to install all correctly, there’s some dependency hell going on. I took some good hours to figure out some installation order that works.
You can try this:
python3 -m pip install torch==2.2.1+cu121 torchvision --index-url https://download.pytorch.org/whl/cu121
python3 -m pip install "unsloth @ git+https://github.com/unslothai/unsloth.git@d0ca3497eb5911483339be025e9924cf73280178"
python3 -m pip install --no-deps "xformers<0.0.26" --force-reinstall
python3 -m pip install flash_attn==2.6.3

They have to be in that specific order and not together in the same pip command, or the dependencies get resolved in a way they’re not compatible.
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
==((====))==  Unsloth 2024.8: Fast Llama patching. Transformers = 4.43.4.
   \\   /|    GPU: NVIDIA A100 80GB PCIe. Max memory: 79.151 GB. Platform = Linux.
O^O/ \_/ \    Pytorch: 2.2.1+cu121. CUDA = 8.0. CUDA Toolkit = 12.1.
\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.25.post1. FA2 = True]
 "-____-"     Free Apache license: http://github.com/unslothai/unsloth

Hey,
So I tried to run all of this, but the code still ask me to download some dependencies bitsandbytes transformers trl peft
And I still get an error
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[10], line 1
----> 1 from unsloth import FastLanguageModel
      2 import torch
      3 max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!

File /usr/local/lib/python3.10/dist-packages/unsloth/__init__.py:154
    144         warnings.warn(
    145             "Unsloth: CUDA is not linked properly[.\n](http://<IP_ADDRESS>:8889/lab/tree/RTC%3Atom_FT/n)"\
    146             "Try running `python -m bitsandbytes` then `python -m xformers.info`\n"\
   (...)
    150             "Unsloth will still run for now, but maybe it might crash - let's hope it works!"
    151         )
    152 pass
--> 154 from .models import *
    155 from .save import *
    156 from .chat_templates import *

File /usr/local/lib/python3.10/dist-packages/unsloth/models/__init__.py:15
      1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from .loader  import FastLanguageModel
     16 from .llama   import FastLlamaModel
     17 from .mistral import FastMistralModel

File /usr/local/lib/python3.10/dist-packages/unsloth/models/loader.py:15
      1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from ._utils import is_bfloat16_supported, HAS_FLASH_ATTENTION, HAS_FLASH_ATTENTION_SOFTCAPPING
     16 from .llama import FastLlamaModel, logger
     17 from .mistral import FastMistralModel

File /usr/local/lib/python3.10/dist-packages/unsloth/models/_utils.py:559
    553 pass
    556 # =============================================
    557 # Weirdly LoraLayer.update_layer downcasts PEFT layers to float16??
    558 # For mixed precision, we need it to be in float32 not float16.
--> 559 from peft import __version__ as peft_version
    560 if Version(peft_version) < Version("0.12.0"):
    561     from peft.tuners.lora.layer import LoraLayer

File /usr/local/lib/python3.10/dist-packages/peft/__init__.py:22
      1 # flake8: noqa
      2 # There's no way to ignore "F401 '...' imported but unused" warnings in this
      3 # module, but to preserve other warnings. So, don't check this module at all.
   (...)
     17 # See the License for the specific language governing permissions and
     18 # limitations under the License.
     20 __version__ = "0.12.0"
---> 22 from .auto import (
     23     AutoPeftModel,
     24     AutoPeftModelForCausalLM,
     25     AutoPeftModelForSequenceClassification,
     26     AutoPeftModelForSeq2SeqLM,
     27     AutoPeftModelForTokenClassification,
     28     AutoPeftModelForQuestionAnswering,
     29     AutoPeftModelForFeatureExtraction,
     30 )
     31 from .mapping import (
     32     MODEL_TYPE_TO_PEFT_MODEL_MAPPING,
     33     PEFT_TYPE_TO_CONFIG_MAPPING,
   (...)
     36     inject_adapter_in_model,
     37 )
     38 from .mixed_model import PeftMixedModel

File /usr/local/lib/python3.10/dist-packages/peft/auto.py:31
     19 from typing import Optional
     21 from transformers import (
     22     AutoModel,
     23     AutoModelForCausalLM,
   (...)
     28     AutoTokenizer,
     29 )
---> 31 from .config import PeftConfig
     32 from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING
     33 from .peft_model import (
     34     PeftModel,
     35     PeftModelForCausalLM,
   (...)
     40     PeftModelForTokenClassification,
     41 )

File /usr/local/lib/python3.10/dist-packages/peft/config.py:24
     21 from huggingface_hub import hf_hub_download
     22 from transformers.utils import PushToHubMixin
---> 24 from .utils import CONFIG_NAME, PeftType, TaskType
     27 @dataclass
     28 class PeftConfigMixin(PushToHubMixin):
     29     r"""
     30     This is the base configuration class for PEFT adapter models. It contains all the methods that are common to all
     31     PEFT adapter models. This class inherits from [`~transformers.utils.PushToHubMixin`] which contains the methods to
   (...)
     36         peft_type (Union[[`~peft.utils.config.PeftType`], `str`]): The type of Peft method to use.
     37     """

File /usr/local/lib/python3.10/dist-packages/peft/utils/__init__.py:23
     21 from .loftq_utils import replace_lora_weights_loftq
     22 from .peft_types import PeftType, TaskType
---> 23 from .other import (
     24     TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING,
     25     TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING,
     26     TRANSFORMERS_MODELS_TO_ADALORA_TARGET_MODULES_MAPPING,
     27     TRANSFORMERS_MODELS_TO_IA3_TARGET_MODULES_MAPPING,
     28     TRANSFORMERS_MODELS_TO_IA3_FEEDFORWARD_MODULES_MAPPING,
     29     TRANSFORMERS_MODELS_TO_LNTUNING_TARGET_MODULES_MAPPING,
     30     TRANSFORMERS_MODELS_TO_VERA_TARGET_MODULES_MAPPING,
     31     TRANSFORMERS_MODELS_TO_FOURIERFT_TARGET_MODULES_MAPPING,
     32     CONFIG_NAME,
     33     WEIGHTS_NAME,
     34     SAFETENSORS_WEIGHTS_NAME,
     35     INCLUDE_LINEAR_LAYERS_SHORTHAND,
     36     _set_trainable,
     37     bloom_model_postprocess_past_key_value,
     38     prepare_model_for_kbit_training,
     39     shift_tokens_right,
     40     transpose,
     41     _get_batch_size,
     42     _get_submodules,
     43     _set_adapter,
     44     _freeze_adapter,
     45     ModulesToSaveWrapper,
     46     _prepare_prompt_learning_config,
     47     _is_valid_match,
     48     infer_device,
     49     get_auto_gptq_quant_linear,
     50     get_quantization_config,
     51     id_tensor_storage,
     52     cast_mixed_precision_params,
     53 )
     54 from .save_and_load import get_peft_model_state_dict, set_peft_model_state_dict, load_peft_weights

File /usr/local/lib/python3.10/dist-packages/peft/utils/other.py:21
     18 from contextlib import nullcontext
     19 from typing import Optional, Tuple
---> 21 import accelerate
     22 import torch
     23 from accelerate.hooks import add_hook_to_module, remove_hook_from_module

File /usr/local/lib/python3.10/dist-packages/accelerate/__init__.py:16
      1 # Copyright 2020 The HuggingFace Team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
     14 __version__ = "0.33.0"
---> 16 from .accelerator import Accelerator
     17 from .big_modeling import (
     18     cpu_offload,
     19     cpu_offload_with_hook,
   (...)
     24     load_checkpoint_and_dispatch,
     25 )
     26 from .data_loader import skip_first_batches

File /usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:36
     33 import torch.utils.hooks as hooks
     34 from huggingface_hub import split_torch_state_dict_into_shards
---> 36 from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
     37 from .data_loader import DataLoaderDispatcher, prepare_data_loader, skip_first_batches
     38 from .hooks import AlignDevicesHook

File /usr/local/lib/python3.10/dist-packages/accelerate/checkpointing.py:24
     21 from safetensors.torch import load_model
     22 from torch.cuda.amp import GradScaler
---> 24 from .utils import (
     25     MODEL_NAME,
     26     OPTIMIZER_NAME,
     27     RNG_STATE_NAME,
     28     SAFE_MODEL_NAME,
     29     SAFE_WEIGHTS_NAME,
     30     SAMPLER_NAME,
     31     SCALER_NAME,
     32     SCHEDULER_NAME,
     33     WEIGHTS_NAME,
     34     get_pretty_name,
     35     is_torch_xla_available,
     36     is_xpu_available,
     37     save,
     38 )
     41 if is_torch_xla_available():
     42     import torch_xla.core.xla_model as xm

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/__init__.py:190
    180 if is_deepspeed_available():
    181     from .deepspeed import (
    182         DeepSpeedEngineWrapper,
    183         DeepSpeedOptimizerWrapper,
   (...)
    187         HfDeepSpeedConfig,
    188     )
--> 190 from .bnb import has_4bit_bnb_layers, load_and_quantize_model
    191 from .fsdp_utils import load_fsdp_model, load_fsdp_optimizer, merge_fsdp_weights, save_fsdp_model, save_fsdp_optimizer
    192 from .launch import (
    193     PrepareForLaunch,
    194     _filter_args,
   (...)
    199     prepare_tpu,
    200 )

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/bnb.py:29
     22 import torch.nn as nn
     24 from accelerate.utils.imports import (
     25     is_4bit_bnb_available,
     26     is_8bit_bnb_available,
     27 )
---> 29 from ..big_modeling import dispatch_model, init_empty_weights
     30 from .dataclasses import BnbQuantizationConfig
     31 from .modeling import (
     32     find_tied_parameters,
     33     get_balanced_memory,
   (...)
     37     set_module_tensor_to_device,
     38 )

File /usr/local/lib/python3.10/dist-packages/accelerate/big_modeling.py:24
     21 import torch
     22 import torch.nn as nn
---> 24 from .hooks import (
     25     AlignDevicesHook,
     26     CpuOffload,
     27     UserCpuOffloadHook,
     28     add_hook_to_module,
     29     attach_align_device_hook,
     30     attach_align_device_hook_on_blocks,
     31 )
     32 from .utils import (
     33     OffloadedWeightsLoader,
     34     check_cuda_p2p_ib_support,
   (...)
     48     retie_parameters,
     49 )
     50 from .utils.other import recursive_getattr

File /usr/local/lib/python3.10/dist-packages/accelerate/hooks.py:30
     22 from .utils import (
     23     PrefixedDataset,
     24     find_device,
   (...)
     27     set_module_tensor_to_device,
     28 )
     29 from .utils.modeling import get_non_persistent_buffers
---> 30 from .utils.other import recursive_getattr
     33 _accelerate_added_attributes = ["to", "cuda", "npu", "xpu", "mlu", "musa"]
     36 class ModelHook:

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/other.py:36
     34 from .imports import is_deepspeed_available, is_torch_distributed_available, is_torch_xla_available
     35 from .modeling import id_tensor_storage
---> 36 from .transformer_engine import convert_model
     37 from .versions import is_torch_version
     40 logger = get_logger(__name__)

File /usr/local/lib/python3.10/dist-packages/accelerate/utils/transformer_engine.py:21
     17 from .imports import is_fp8_available
     20 if is_fp8_available():
---> 21     import transformer_engine.pytorch as te
     24 def convert_model(model, to_transformer_engine=True, _convert_linear=True, _convert_ln=True):
     25     """
     26     Recursively converts the linear and layernorm layers of a model to their `transformers_engine` counterpart.
     27     """

File /usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/__init__.py:34
     30     sys.modules[module_name] = solib
     31     spec.loader.exec_module(solib)
---> 34 _load_library()
     35 from transformer_engine.pytorch.module import LayerNormLinear
     36 from transformer_engine.pytorch.module import Linear

File /usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/__init__.py:29, in _load_library()
     27 module_name = "transformer_engine_torch"
     28 spec = importlib.util.spec_from_file_location(module_name, so_path)
---> 29 solib = importlib.util.module_from_spec(spec)
     30 sys.modules[module_name] = solib
     31 spec.loader.exec_module(solib)

ImportError: /usr/local/lib/python3.10/dist-packages/transformer_engine/transformer_engine_torch.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZNK3c105Error4whatEv

Apologies @C0casio45, I should've shared a Dockerfile instead.
I created this Dockerfile where it's working: https://github.com/zytedata/unsloth_docker/blob/main/Dockerfile

It's working like a charm
Thank you
Have a good day

@C0casio45 It's not necessary to install flash-attn! Xformers is on par or sometimes faster than FA2! I would simply uninstall it via pip uninstall flash-attn


Simulation mode is missing feature extraction argument

"asreview simulate --help" shows me there is no option to specify the  feature_extraction
sage: simulate [-h] [-m MODEL] [-q QUERY_STRATEGY] [-b BALANCE_STRATEGY]
                [--n_instances N_INSTANCES] [--n_queries N_QUERIES]
                [-n N_PAPERS] [--embedding EMBEDDING_FP]
                [--config_file CONFIG_FILE]
                [--prior_included [PRIOR_INCLUDED [PRIOR_INCLUDED ...]]]
                [--prior_excluded [PRIOR_EXCLUDED [PRIOR_EXCLUDED ...]]]
                [--extra_dataset EXTRA_DATASET] [--log_file LOG_FILE]
                [--save_model SAVE_MODEL_FP] [--verbose VERBOSE]
                [--n_prior_included N_PRIOR_INCLUDED]
                [--n_prior_excluded N_PRIOR_EXCLUDED]
                X

Thanks for reporting. We are having a look asap.


No-connection view while upgrading agent

on windows

Have an old agent installed.

Click Upgrade
Screenshare - 2023-09-21 3_23_06 PM.zip

Now we will  see the no-connection view until the new containers are up and running.

@shaykeren
please decide how urgent it is

@shaykeren @asafchen-dig @doppleware
I tried it , it really looks a bit strange. we can think to show something else while upgrade is in progress.


npm test does not work out of the box

[READ] Step 1: Are you in the right place?
[REQUIRED] Step 2: Describe your configuration

Extension name: firestore-send-email
Extension version: head

[REQUIRED] Step 3: Describe the problem
Hi there, I'm the release owner on the extension team this week and while verifying that all extension tests are passing, I ran into an issue.
npm test fails with failure in e2e.test.ts in firestore-send-email. Upon closer investigation, this test requires an emulator to be running by running __test__/emulators/start.sh first.
Can we update the npm test command to require no other magic commands to be ran first? One possible solution is run the test through firebase emulators:exec jest.

Hi @elvisun.
I agree this is an issue when running locally.
We currently have a Github action command cd ./firestore-send-email/functions/__tests__/emulator && sh./start.sh that runs in our ci environment.
I'll add this to the project board


feature: add serverice task parse in statelang

What this PR does:
This PR involves the serverice task parse in statelang module. At the same time, the parse of other models related to servicetask is also implemented.
Which issue(s) this PR fixes:

Fixes #https://github.com/apache/incubator-seata-go/issues/636
Special notes for your reviewer:
There are still some parse functions that have not been implemented in statelang refer to seata-java.
Does this PR introduce a user-facing change?:

no change

LGTM


Revert "nodePackages.uglify-js: repackage with buildNpmPackage"

Reverts NixOS/nixpkgs#323702
Broke eval of various packages using uglify-js.
❯ nix-build -A discourse
error:
       … while calling the 'derivationStrict' builtin
         at /builtin/derivation.nix:1:208:
       … while evaluating derivation 'discourse-3.2.2'
         whose name attribute is located at /home/hexa/git/nixos/master/pkgs/stdenv/generic/make-derivation.nix:331:7

       … while evaluating attribute 'installPhase' of derivation 'discourse-3.2.2'
         at /home/hexa/git/nixos/master/pkgs/servers/web-apps/discourse/default.nix:384:5:
          383|
          384|     installPhase = ''
             |     ^
          385|       runHook preInstall

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Alias uglify-js is still in node-packages.nix

First of all: How did ofborg not catch this? Second: a fix is here #324399


I don't know if this can be classified as an issue, but would there ever be a xamarin control gallery on the store

That's a good question for the Xamarin team - I'd recommend filing a new issue over there: https://github.com/xamarin


📊 artificial_intelligence: Epoch dataset

@lucasrodes thanks a lot for taking a look at this. I am not sure why the changes aren't showing up but I think I've now implemented all of your suggestions :)


Google Recaptcha stops bouncer

If I set up a form to be validated, it works beautifully. But if I try to incorporate Google Recaptcha into the submit process, the field validation stops working.
I suspect that  this is a Google issue, but getting them to change anything seems a forlorn hope. Is there a fix or a workaround, or do I have to choose between Bouncer and Recaptcha?

Do you have a reduced test case I can look at?

I'm sorry, I can't make any sense of your test case page.
Here is the stripped-down html

Skip
You need to fill in all the boxes. If you leave any blank, it won't work.

Your Details
Your Name *

What number is two more than ten? *
 sorry, we need this to reduce spam
Email *

Phone *

Clicking Submit runs recaptcha OK, but not bouncer.
Clicking Test runs bouncer OK, but (obviously) not Recaptcha

After several tries, I still can't get the code to show properly. The google submit button is:
<input type="submit" class="button g-recaptcha" data-callback="onSubmit" data-sitekey="6LfMmcQZAAAAAB1daDepQvZev9vUGF_KUcsMxKu1" title="Send the message" tabindex="200" id="send" value="   Send   " />

You want to wrap your code in three backticks on each end, like in markdown.


Migrate  faiss search migrate to jina 0.4.1

This PR was aligned with issue #116 , migrate faiss search example for Jina 0.4.1 release, includes:

Code migration based on migration guide.
Minor changes on Readme.

To run this example, you should build faiss image locally using jina 0.4.1 with Dockerfile:
FROM conda/miniconda3-centos7:latest

WORKDIR /

RUN conda update -n base conda && \
    conda install -y -c pytorch faiss-cpu

COPY faiss.yml .

RUN conda install pip
RUN pip install jina==0.4.1

ENTRYPOINT ["jina", "pod", "--uses", "faiss.yml"]

And run command:
docker build -f Dockerfile -t jinaai/hub.executors.indexers.vector.faiss:latest .

@nan-wang should we merge this example?

@JoanFM one moment, let me test again

I proposed change in #138.
When I created this example, I wanted to have a dummy encoder and crafter to showcase a much more common pattern than  simply index. It may be able to work directly against the Index.

With the new recursive document, this example should be refactored to use only 0-depth-level

Made some small changes in #139, I included the dockerfile folder into the example, otherwise COPY faiss.yml . will show an error.

@YueLiu-jina @JoanFM I saw your PRs, thanks for that.How we are going to proceed?

@YueLiu-jina @JoanFM I saw your PRs, thanks for that.How we are going to proceed?

You can merge both into your branch!

See comment https://github.com/jina-ai/examples/pull/138#issuecomment-669111243

LGTM!

@YueLiu-jina can you resolve the conflict?

@bwanglzu Merged my branch, I'll give it a test again

Still works?

@JoanFM not really, it says no driver for search request, the problem seems from the yaml/indexer.yml file, I'm taking a look in it
🐳 Traceback (most recent call last):
🐳 File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 267, in msg_callback
🐳 self.zmqlet.send_message(self._callback(msg))
🐳 File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 256, in _callback
🐳 self.pre_hook(msg).handle(msg).post_hook(msg)
🐳 File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 154, in handle
🐳 self.executor(self.request_type)
🐳 File "/usr/local/lib/python3.7/site-packages/jina/executors/__init__.py", line 553, in __call__
🐳 raise NoDriverForRequest(req_type)
🐳 jina.excepts.NoDriverForRequest: SearchRequest

it works well, I think this PR is ready to merge

@YueLiu-jina, it prints results? can u post a screenshot of what u get in query output? just double checking

Is this the effect we want?

Good to merge then!


Overwrite BigQuery tables

Hi Team,
I am not able to overwrite tables while insert/update rows. Is there any option where I can mention to overwrite while inserting into tables? Here I am using this method:
response = await table.InsertRowsAsync(bqRows);
public Task<BigQueryInsertResults> InsertRowsAsync(IEnumerable<BigQueryInsertRow> rows, InsertOptions options = null, CancellationToken cancellationToken = default)

I can see InsertOptions provides me following configuration, but could not find for overwrite table:
public bool? AllowUnknownFields { get; set; }
public bool? SkipInvalidRows { get; set; }
public bool SuppressInsertErrors { get; set; }
public string TemplateSuffix { get; set; }
public bool AllowEmptyInsertIds { get; set; }

Could you please look into this and provide any update on this thread.
Thanks!

As far as I'm aware, you can only update data using DML - see the documentation for updating with DML for details. This isn't a limitation of the .NET client library - it's inherent in the nature of BigQuery. (Bear in mind that when defining BigQuery tables there's no primary key, so it would be difficult for an operation such as InsertRows to perform an update anyway.)

That's assuming you want to selectively update rows, of course. If you want to simply truncate the existing data in the table and then insert the new data, that's certainly feasible with a load job (using a WriteDisposition of Truncate) - I don't know the simplest way of doing it with InsertRows though.

When we open GBQ and from UI choosing

More Settings > Query Settings option on query editor top bar

I am talking about above functionality, can we achieved through this library?
Thanks!

Okay, so you're not wanting to call the InsertRows API after all - you're performing a query and wanting to truncate any existing table. Yes, that's entirely doable. If you don't need the results within your code, you can do this with BigQueryClient.CreateQueryJob, specifying QueryOptions like this:
var queryOptions = new QueryOptions
{
    DestinationTable = client.GetTableReference(datasetId, tableId),
    WriteDisposition = WriteDisposition.WriteTruncate
};
var job = client.CreateQueryJob(sql, parameters, queryOptions)
    .PollUntilCompleted()
    .ThrowOnAnyError();

Here we can only pass one query inside CreateQueryJob function, Is there any way to pass DataTable or records in forms to overwrite the table

I would suggest inserting into a temporary table, then to use CreateCopyJob to truncate the existing table.
I don't know whether that's atomic, mind you. That sort of detailed API-specific question is best put straight to the BigQuery team, and isn't really about the .NET client library - I suggest you follow one of the BigQuery support options for that.

@jskeet I am facing the same issue at https://github.com/googleapis/google-cloud-dotnet/issues/8146 (thanks for the tip of @amanda-tarafa)
Since the data is eventual consistent, so the new table might be not available at the time performing truncate which means the issue remains the same.
Just thinking loudly what is recommended way to recreate table every time as it is very common request.

@iqb-dawn: It's not clear that this is exactly the same issue (or that it's a "very common request"). I'm watching the other issue - there's no need to draw attention in other places.

Closing this issue now as I don't believe there's any change needed (or feasible) in the client libraries.


NO-JIRA: [release-4.16] test/e2e: remove api budget checks

presub e2e's a failing on this again and with no way to quickly diagnose the situation, we should not block PRs on this.
This isn't a great check as the number of mutating calls is somewhat outside our control with delays in cloud resource create/delete contirbuting to long test durations and more mutating calls.
We need a fundamentally better solution here based on KAS audit log analysis.

/lgtm

/test e2e-aws

Hit this here as well https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_hypershift/4385/pull-ci-openshift-hypershift-release-4.15-e2e-aws/1813949514063548416
/cherry-pick release-4.15


Feature map of network output and target seem not match in tracking

Take Alexnet for example, As far as I know, the network take 303✖303 search image as input and do a few downsampling and finally output 17✖17 feature map of classification score and center score (fcos_cls_score_final in code). So the 17✖17 matches the 303✖303. Even considering some frontier info loss caused by no padding, the 17✖17 will still match maybe some area like 200✖200 in my opinion.
However when it comes to target generation in 'make_densebox_target.py', we just crop a (17✖8)✖(17✖8)=(136✖136) area from the center of 303✖303 and use it to calculate target. This area seems too small, and doesn't match the network output.
In other words, 17*17 seems to match a much larger area of input search image in network, and match a very small area in target calculating. Which in my opinion is not reasonable.
Maybe I misunderstand some code, welcome to point out.

Convolutional layer without padding can only guarantee the alignment of center point. Therefore, the position of pixels on the feature map will be mapped to the original search image by (w//2, h//2) + stride*(dist_from_center). It should be noted that this is the mapping between centers, not the corresponding search image area. In other worlds, the 136 is farmost position in the  center points.

I see. Thanks for replying.


Using Variables to Carry Over ID's of Responses in the Postman Collection

Discussed in https://github.com/eclipse-edc/MinimumViableDataspace/discussions/350

Originally posted by nikschul October  1, 2024
Currently - when executing the REST requests inside the Postman collection in the order described in step 7 of the README.md - some IDs of previous responses have to be copied over manually to the next request. Even though this could have a certain learning effect for new users as they have to understand which ID has to be used when, this could also be simplified by automating it with Postman variables.
Here is an example:

Instead of manually copying the "contractAgreementId" over to the next request, it could be automatically carried over to the next request.
This could be accomplished with post-request scripts: https://blog.postman.com/extracting-data-from-responses-and-chaining-requests/

I don't think I currently have the permission to assign myself, but I am currently working on the issue.

@paullatzelsperger regarding the Postman Flows: I think there is currently not way to export the flows in Postman (https://community.postman.com/t/export-postman-flow/60756/3), so I guess we would have to create a Postman Workspace for the repository in order to share flows or do you see another option?
regarding software tests: would you suggest to test the pre- and postscript with Postman and also to implement some integration tests for the postman collection?

I think there is currently not way to export the flows in Postman

yeah, seems that way. we'll have to hold off on that, but maybe we could document that possibility in the README somewhere, like "hey, you can either follow the sequence in this document, or you define a flow to chain together the requests" and an explanation on how to do it.

we would have to create a Postman Workspace for the repository

this gets complicated quickly and will likely involve cost etc. not likely to happen, TBH.

would you suggest to test the pre- and postscript with Postman and also to implement some integration tests for the postman collection?

Thats a good idea, maybe we just add a CI workflow that runs all requests in the "ControlPlane Management" folder using the newman tool. Basically the same thing that the TransferEndToEndTest does from JUnit.


 But failed to run the demo.
Please see the log below:
2018-10-29 17:20:23.728 demo[27003:4540970] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'
*** First throw call stack:
(
0   CoreFoundation                      0x00007fff314742fb __exceptionPreprocess + 171
1   libobjc.A.dylib                     0x00007fff57dffc76 objc_exception_throw + 48
2   AppKit                              0x00007fff2f1a02cf -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 4167
3   libSDL2-2.0.0.dylib                 0x00000001054201fb Cocoa_PumpEvents + 211
4   libSDL2-2.0.0.dylib                 0x00000001053a4847 SDL_PumpEvents_REAL + 23
5   libSDL2-2.0.0.dylib                 0x00000001053a48c2 SDL_WaitEventTimeout_REAL + 76
6   demo                                0x00000001050bd35d monitor_sdl_refr_thread + 141
7   libSDL2-2.0.0.dylib                 0x00000001053ce6d3 SDL_RunThread + 60
8   libSDL2-2.0.0.dylib                 0x000000010542c668 RunThread + 9
9   libsystem_pthread.dylib             0x00007fff58c786c1 _pthread_body + 340
10  libsystem_pthread.dylib             0x00007fff58c7856d _pthread_body + 0
11  libsystem_pthread.dylib             0x00007fff58c77c5d thread_start + 13
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6

Seems both of 'main' and 'monitor_sdl_refr_core' handled the he SDL events.

It should be solved here:
https://github.com/littlevgl/lv_drivers/blob/master/display/monitor.c#L299
Have you updated lv_drivers?

Seems both of 'main' and 'monitor_sdl_refr_core' handled the he SDL events.

It should be solved here:
https://github.com/littlevgl/lv_drivers/blob/master/display/monitor.c#L299
Have you updated lv_drivers?

It works, Thanks.


Unreal 5.4 Support

With the preview build out, I've tried using the plugin with 5.4. 5.4 is a lot more strict in its compilation, which appears to be causing issues when building the plugin.
Sorry for lack of logs/data, I can't provide them at the moment.

Same, 5.4 doesn't build with this SDK right now.

Could you please let me know the current status of this issue?

I created a pull request to fix the build issues. https://github.com/aws/amazon-gamelift-plugin-unreal/pull/61

guys is anyone working on an update to this? kinda need 5.4

Damn it, I already upgraded up project to 5.5.

We need support by 5.5 quickly..


changes some css

Changes Made

Updated PNG Images:

Replaced old PNG images with new ones to enhance the visual appeal and consistency of the user interface.

Hover and Transition Effects:

Added hover effects to images for better interactivity.
Implemented smooth transition effects to make the UI more dynamic and user-friendly.
Corrected the alignment issues in the navbar for a more attractive and responsive layout.
Ensured the navbar looks consistent across different screen sizes.

closes . I've made the necessary changes in #88

Which issue did you work upon? Share your issue link here!
I guess you didnt create any issue for this !?

Sorry, @vansh-codes, actually I'm new to open source and I didn't have any idea about the correct way to contribute. I created an issue, and obviously, this PR is related to that issue.
- closes #86


Tmate sesssion step is just hanging

tmate session has been running over 5 minutes. Stuck on the bottom step. Never populating the session URL
openssh-client is already the newest version (1:8.2p1-4ubuntu0.4).
xz-utils is already the newest version (5.2.4-1ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 71 not upgraded.

It finally went thru at 11 minutes

I'd like to open this issue again since it's still an issue @mxschmitt
Getting stuck here for a very long time:

@monkybrain, yes, tmate is currently looking for new sponsors: https://github.com/sponsors/nviennot
In the meantime it's only running with a limited capacity.


[BUG] SKShader.CreatePicture not working properly with SKPaint.Reset and SKPath.Reset

Description

SKShader.CreatePicture not working with SKPaint.Reset and SKPath.Reset
Code

https://gist.github.com/wieslawsoltes/08a9b739fb3df25e191f9e99b943a7c5
For reference code without Reset() calls, using new instances.
https://gist.github.com/wieslawsoltes/c615e2dccf7fc9ce86f7ef5c8153cafb
Expected Behavior

Actual Behavior

Basic Information

Version with issue:  2.80.2
Last known good version:
IDE:  Visual Studio / Visual Studio Code
Platform Target Frameworks:

Android:
iOS:
Linux:
macOS:
Tizen:
tvOS:
UWP:
watchOS:
Windows Classic:

Target Devices:

Detailed IDE/OS information (click to expand)

PASTE ANY DETAILED VERSION INFO HERE

Screenshots

Reproduction Link

@wieslawsoltes does anything different happen when you don't reset the path? Does having just the paint reset also stop working?
I originally added the reset for the paint after a request, and I never got any negative feedback from it.
A SKPaint reset is literally creating a new pain instance in the exact same place: https://github.com/google/skia/blob/master/src/core/SkPaint.cpp#L96
And, when it is passed around, I believe it is copied. However, a picture may be using a reference.
Could you confirm that it is both the paint AND the path having issues, or maybe it is one and the other is just causing it?

Both have issue when used separately and together. I was under impression that Reset() does not allocate new objects?

It does not in C# and the C++ side is interesting. It technically creates a new object over the old one. So not really a new object, just "re-initializes" it.

This pointer constructor magic.

@wieslawsoltes  I have been looking at this and I think there might be a few things going on here.
First, I need to check some more, but I don't think you can reset the paths. This might be held by reference when drawing, so that is why the top part of the picture - the bit made up of the small path blocks - is reset.
The other part is where it appears a font size is being set, then reset and then drawn.
Over here, the paint is set:
https://gist.github.com/wieslawsoltes/08a9b739fb3df25e191f9e99b943a7c5#file-pservers-pattern-01-b-svg-cs-L90
Then reset over here:
https://gist.github.com/wieslawsoltes/08a9b739fb3df25e191f9e99b943a7c5#file-pservers-pattern-01-b-svg-cs-L134
Then drawn here:
https://gist.github.com/wieslawsoltes/08a9b739fb3df25e191f9e99b943a7c5#file-pservers-pattern-01-b-svg-cs-L159
Even though you are referencing the old paint object variable you have actually reset it twice since.
This might be a result of a container setting some properties and then the elements changing them? You may need to investigate some sort of stack-based paint collection. I think it might be possible to have multiple paint objects at the same time due to the hierarchical nature of the xml.


change destination ?

Hello !
I'm trying to modify the img so that instead of writing the cleaned documents to another usb drive, it writes them to a network samba share folder.
I'll do some tests modifying some scripts in /opt/groomer , but if anyone already tried / achieved that and has advices, or if it raises some security issues, thank you for telling me !
best regards

Hello!
The main security issue I can come up with is if there is something malicious on the source key that triggers network activities. If that's not a concern, I do not think it is a problem.
If you want to use the core of the project on something else than a raspberry pi, this is the standalone library btw: https://github.com/CIRCL/PyCIRCLean


filter on NULL and some other value using GUI generates wrong sql code

System information:

Operating system (distribution) and version: Windows
DBeaver version: 21.1.3

Describe the problem you're observing:
When filtering on a table using the GUI you can select from all values entered for that field. If NULL and some other values are selected like in

it generates a WHERE clause as

So far no problem.
After selecting some more fields the new selection is AND-ed to the rest resulting in e.g.:

This gives wrong results because the selection with the OR should be between parentheses.
The total WHERE clause shoud be
habitat IN (4,11) AND ( quality IS NULL OR quality IN (1.0,2.0) )

@JeroenNienhuis Thank you for the report

VErified


Vaadin 7 + Spring security configuration

Hello,
first of all - thank you so much for the awesome vaadin4spring add-on! I used it to configure spring with my existing vaadin application.
But right now I run into the problem while trying to configure Spring security with vaadin.
Also, I'm unable to correctly launch spring-boot-sample-security and continuously getting white screen in the my browser without any errors.
Could someone please take a look into my issue described here  http://stackoverflow.com/questions/25952514/vaadin-7-spring-security-configuration
What I'm doing wrong ?

@GJRTimmer  Thanks It solved it

Spring boot based security sample is now available. Located within the samples directory.
Note: This is a multi UI security sample with a fully working springSecurityFilterChain.
Next update of the security-sample will be the implementation of 'remember-me' through the use of the official RememberMeAuthenticationFilter within the springSecurityFilterChain.
When this is completed the creation of a security-sample 'single-UI' will be started.
Anyone interested in helping out?

Basic implementation of Spring-Security is done. See samples/security-sample


docs: add contributors

@all-contributors please add @saadpasta for contributions

@all-contributors add @saadpasta  for mr

@all-contributors add @saadpasta  for code


Remove "qx contrib" as an alias of "qx package"

We left qx contrib as a backward-compatibility alias when we changed to qx package, however, at this point, the alias is no longer necessary, as nobody still uses qx contrib.
I think it is confusing to have two aliases of qx package, i.e. qx pkg  and qx contrib, so qx contrib should be removed.

Done in https://github.com/qooxdoo/qooxdoo-compiler/pull/709


Amount is not localized on IOU Details page, LHN and Report

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Request Money - Click on Green Badge (Top Right)
Observe Amount in IOU Details Page, LHN and Report page

Expected Result:
Amount should be localized (ex: for Spanish 100,90 and in English 100.90)
Actual Result:
Amount is not localized in report page and IOU details page
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] MacOS / Chrome / Safari

Version Number: 1.2.58-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:

https://user-images.githubusercontent.com/43996225/214114792-bc6983fb-09da-40e8-8672-ae74dafdfa9a.mov
https://user-images.githubusercontent.com/43996225/214114896-4b9c5ff6-3dff-4f35-8e85-c9e1a4ee9606.mp4
Expensify/Expensify Issue URL:
Issue reported by: @varshamb
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1674233388653609
View all open jobs on GitHub

Bug0 Triage Checklist
Note: see this SO for more information.

[ ] The "bug" is actually a bug
[ ] The bug is not a duplicate report of an existing GH.

If it is, close the GH and add any novel details to the original GH instead

[ ] The bug is reproducible, following the reproduction steps.

[ ] If the GH doesn’t have steps to reliably reproduce the bug and you figure it out, then please add them.
[ ] If you’re unable to reproduce the bug, add the Needs reproduction label.
[ ] Comment on the issue outlining the steps you took to try to reproduce the bug, your results and tag the issue reporter and the Applause QA member who created the issue. Ask them to clarify reproduction steps and/or to check the reproduction steps again. Close issue.

[ ] The GH template is filled out as fully as possible

The GH body and title are clear (ie. could an external contributor understand it and work on it?)

[ ] The GH was created by an Expensify employee or a QA tester
[ ] If the bug is an OldDot issue, you should decide whether it is widespread enough to justify fixing or it is better to wait for reunification. If it’s better to wait, close the GH & provide this justification
[ ] If there's a link to Slack, check the discussion to see if we decided not to fix it
[ ] Decide if the GH should be resolved by an External contributor or Internal engineer, add the appropriate label

Testing this and going to raise a question about the expected behavior. I see that we're localizing in most places but in the original chat comment for the request and the request history we aren't.

We're going to close this as a not a bug. The behavior being experienced here is expected at this time and it relies on internationalization to be completed on the backend.


License should be specified.

I can't seem to find the license for this code anywhere meaning that technically anyone using it is violating your copyright and shouldn't be using it.
Would you mind putting a license (I'd use LGPL personnally)?

Huh, you're right. I usually add one before I publish it. Guess I missed this one.
FYI, I believe that bintray does distribute copies under the specified license (apache-2.0 if I remember correctly) so you wouldn't be violating anything by using that, just the actual contents of the repo.
I'll add a license and update this sometime tomorrow. It's late here, and despite most programmer stereotypes I do actually need to sleep (lol)

Haha. It's fine. No worries.
I noticed because the license didn't show up in Alarmio.


Add Synapse Statistics grafana dashboard

Overview
This PR adds a Grafana dashboard for metrics collected by the Synapse Stats Exporter.

Rationale
Offer metrics visualization. So far, only two: number of rooms and number of users.
Juju Events Changes

Module Changes

Library Changes

Checklist

[x] The charm style guide was applied
[x] The contributing guide was applied
[x] The changes are compliant with ISD054 - Manging Charm Complexity
[x] The documentation is generated using src-docs
[ ] The documentation for charmhub is updated.
[x] The PR is tagged with appropriate label (urgent, trivial, complex)

No need of updating charmhub documentation.

Sorry folks, I added the screenshot only to the story, now is on the description as well. :)


About the implementation of LWD

Thanks for your great work: Camouflaged Object Detection with Feature Decomposition and Edge Reconstruction.
I have some questions about the implementation of LWD. I read the code:

It seems from the code of the LWD that the LWD is a  unlearnable Haar wavelet. I'm afraid that I cannot find the code of learnable HF and LF filters. I read the reference AWD. AWD seems to update coefficients via the loss function:

I think I must miss something about your code. Would you like to help me figure out the question? Thank you very much.

I have the same question

I have the same question. LWD seems to be unlearnable


Reversing the position of haystack and needle

Issue 1
This fixes the issue where the browser may send a generic language header such as en or cy as opposed to the whole en-GB or cy-GB.
In that case we end up looking for cy-GB inside cy which stripos doesn't like. Flipping params fixes this issue.
Issue 2
Due to legal reasons, no one language can be considered default. Fluent will currently return the first locale in the list regardless of the default flag being set or not. Introducing the disable_implicit_default flag to not have it do this allows some flexibility around this. By default, Fluent will behave as it currently does unless the flag is explicitly set to true.
Issue 3
Once the correct domain has been determined, there is no need to prefix the locale to URLs as Fluent will serve the page in the correct locale thereafter. Introducing the prefix_locale_in_link flag will allow us to disable this globally. Also it's really useful for outputting the canonical URL without the prefix. Again, by default Fluent will continue to behave as it currently does unless the flag is explicitly set to false.

@robbieaverill @tractorcow For your consideration.

Am in China will let robbie handle this 😁

Are you able to get the tests passing to start with?

@robbieaverill There's an issue for this which I'm currently fixing. Please hold off from reviewing for a bit.

Okay, that has been issue amended, and a new feature has been added.
Issue with FluentExtension::getSourceLocale
Doesn't seem like SourceLocale was being populated correctly when fetching a record from the database which existed in one locale but not in the current locale. We're adding a fallback to use Locale instead, and then finally falling back to using the default locale.
Feature to show full locale title in the badge
FluentBadgeExtension::getBadge will only output the suffix of a domain which is fine in most cases, however, there are locales where the suffix is the same. Staying without our Welsh example, both cy_GB and en_GB will show "GB" as the badge. Pretty bad UX you can imagine.
The best way around this is to provide support to show the full title instead of just the suffix. show_full_locale_badge has been introduce to do just this:

As with the other flags, Fluent will by default show only the suffix unless the flag is explicitly set to true.

@robbieaverill so do these changes keep the current functionality but provides devs the ability to set it to show a full language eg. "en_GB"? If so, I've got no problems with this.


Paper: Update box type

See https://arxiv.org/pdf/2104.14722.pdf pp. 25-30.
Should only have a box type (not boxas/boxto). The box takes an option [duration] argument w/ square brackets. Name is not included.
The box is used to scope timing for a particular piece of a circuit.

Addressed in https://github.com/Qiskit/openqasm/pull/204.

Closed via #204.


RTL layouts

in RTL layouts it shows messed up!
thanks for good library
please fix it!

I had the same issue, luckily android studio lets u edit the resources files of a library,
find the layout file of this library and either edit it so that it's compatible in RTL orientation or create a separate file for RTL orientation!


chore: bump metallb to v0.13.9

Related release notes: https://metallb.universe.tf/release-notes/#version-0-13-9
Also release v0.30.1

Codecov Report
Patch coverage has no change and project coverage change: -0.16 :warning:

Comparison is base (85bf1f7) 54.14% compared to head (cfdcecf) 53.99%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #602      +/-   ##
==========================================
- Coverage   54.14%   53.99%   -0.16%
==========================================
  Files          47       47
  Lines        3869     3869
==========================================
- Hits         2095     2089       -6
- Misses       1508     1519      +11
+ Partials      266      261       -5

Flag
Coverage Δ

integration-test
58.09% <ø> (-0.06%)
:arrow_down:

unit-test
5.17% <ø> (-0.48%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/clusters/addons/metallb/metallb.go
66.80% <ø> (ø)

... and 3 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


chore: optimize the apisix cluster processing

The processing for the apisix cluster was optimized, so it can be easier to implement the multiple clusters support in the future.

Codecov Report

Merging #414 (ed24de7) into master (8817dce) will increase coverage by 0.10%.
The diff coverage is 39.13%.

@@            Coverage Diff             @@
##           master     #414      +/-   ##
==========================================
+ Coverage   39.20%   39.30%   +0.10%
==========================================
  Files          41       40       -1
  Lines        3301     3328      +27
==========================================
+ Hits         1294     1308      +14
- Misses       1846     1859      +13
  Partials      161      161

Impacted Files
Coverage Δ

pkg/ingress/apisix_upstream.go
0.00% <0.00%> (ø)

pkg/ingress/controller.go
0.00% <0.00%> (ø)

pkg/ingress/manifest.go
66.66% <0.00%> (-1.13%)
:arrow_down:

pkg/apisix/apisix.go
64.70% <40.00%> (-19.51%)
:arrow_down:

cmd/ingress/ingress.go
76.82% <100.00%> (+0.87%)
:arrow_up:

pkg/config/config.go
84.00% <100.00%> (+2.18%)
:arrow_up:

test/e2e/e2e.go

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8817dce...ed24de7. Read the comment docs.

I have two questions.
How to configure multi-clusters ?
How to match cluster name and admin url and key.

I have two questions.
How to configure multi-clusters ?
How to match cluster name and admin url and key?

Not in this PR.

I have two questions.
How to configure multi-clusters ?
How to match cluster name and admin url and key?

Not in this PR. See the issue description, multiple-clusters will be supported in the future, this PR is just one of the prerequisites.

It is difficult to set up a multi-cluster according to the current configuration.
The user modified the configuration according to defaultxxx this time, and the next time the multi-cluster is actually configured, it will be modified again.
It is hoped that this modification can finalize the configuration file structure.

Got it, with ApisixClusterConfig CRD .


schema, bridge: Change vlan.type to vlan.mode

Most references to the vlan access/trunk term are using the
name vlan.mode.
Therefore, type is changed in this context to mode.
As the implementation is not yet in, this change is backward safe.

Coverage increased (+7.6%) to 70.919% when pulling 1cb642437032e0c0d40c219eb7905e850b37c6c3 on EdDev:linux-bridge-port-vlan-mode into b8e180851a9c6f7ed778c93b34656dc80810531f on nmstate:master.

Coverage increased (+7.6%) to 70.919% when pulling 1cb642437032e0c0d40c219eb7905e850b37c6c3 on EdDev:linux-bridge-port-vlan-mode into b8e180851a9c6f7ed778c93b34656dc80810531f on nmstate:master.


FluxSamplerPara can not work with Guf model (Q4, Q8)

I use Q4 Guf and connect to 🔧 Flux Sampler Parameters. It shows error like below. Do you know how to make this work? Thanks
FluxSamplerParams+
mat1 and mat2 shapes cannot be multiplied (462x2048 and 4096x3072)

@andyv181  Was the issue ever solved for the gguf file you are using ? I am using ones from  city96 and they seem to be working.


create a non-root user for cilium-related containers

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
investigate whether it's feasible to run cilium-related containers as a non-root user.

[ ] cilium
[ ] clustermesh-apiserver
[x] hubble-relay: https://github.com/cilium/cilium/pull/23259
[ ] operator-*
[ ] startup-script
[ ] cilium/proxy

ref: https://www.tenable.com/audits/items/CIS_Docker_v1.3.1_L1_Docker_Linux.audit:bdcea17ac365110218526796ae3095b1
Cilium Version
N/A
Kernel Version
N/A
Kubernetes Version
N/A
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Add more item for cilium/proxy image as we recently have this PR merged, thanks to @mhofstetter
https://github.com/cilium/cilium/pull/25081

One related note from upstream
https://github.com/kubernetes/website/pull/45150/commits/7fd32afce4d209775a2a9fe1e03abc3b20896368#r1504714307


Feat: Show Installed apps

Summary

Showing installed wallets for Android & iOS
Added a native module to detect wallets installed in Android
Solved an issue with FlatList and babel imports

Screenshots
iOS
https://user-images.githubusercontent.com/25931366/226438705-3f03215d-5241-48f5-beec-351af8ccbe1e.MP4
Android
https://user-images.githubusercontent.com/25931366/226438733-74051e71-bd87-4ccc-b5da-67f9e4f158d6.mp4

Looking awesome. Reviewed the QRCode + Installed Wallet Parts. After we figure out what wallets to hardcode for iOS we can approve and move onto the next step!


docs: ESM update

Summary
As discussed in #13135, this PR:

documented that there's no hoisting on jest.mock with ESM support activated and added example of putting the mock calls before import statements to properly load the import modules.
updated Using with ES module imports in ManulMocks.md to point out the hoisting mentioned there does not applied when ESM support is activated with link to ECMAScriptModules.md.

Additionally, I've updated the Node version number mentioned in ECMAScriptModules.md to reflect the fact that vm.Module is still considered experimental in 18.8.0.
@mrazauskas I've found that already a link to #10025 in ECMAScriptModules.md, do you want to add some documentations for jest.unstable_mock there as well or you think it's fine just leave it like this for now?

Test plan
No test plan. Only docs are updated.

By the way, jest.unstable_mock() shipped with Jest 27.1.1. See https://github.com/facebook/jest/issues/9430#issuecomment-915109139. It should be documented only in docs of v27 and up.

Thanks!
Yes, it would be very useful to have jest.unstable_mock() documented as well. If a paragraph on jest.mock() is added, then someone trying to mock ESM module might be not happy (;

Okay. One question tho, am I right that jest.mock doesn't work for ESM but CJS modules with ESM support activated? So jest.unstable_mock is meant to mock the ESM?

@mrazauskas I've added a fairly simple example for unstable_mockModule at the bottom of ECMAScriptModules.md. I think that should be enough for now? Maybe we can also put it in JestObjectAPI.md but not sure if you guys want it there as well. I'm assuming it's going to be deprecated soon after jest.mock is able to support ESM`.

Thanks for your patience and collaboration. This is shaping up nicely!

Thanks for your patience and collaboration. This is shaping up nicely!

Yeah we're getting close to finalise it.
I do think this doc is much needed for people migrating to ESM syntax. In fact, a fellow developer using my Electron boilerplate with ESM enabled got confused with this recently. There should be a lot less confusion for the users once we have the updated doc goes live.

thanks!


dmsetup failed

Environment: Mac OS X El Capitan v 10.11.4
This is the second time that I'm installing unik.
The first time I could get it running.
The one difference that I know is that I'm using Docker for Mac right now.
When I'm trying to run the unik daemon, I'm getting the error that dmsetup failed.
The logs are attached in the following file.
logs.txt
Thank you for looking into this.

currently it does not appear that unik will work with Docker for Mac/Windows. Unik uses the Device Mapper feature of the Linux kernel in order to create volumes. It appears that, while the Docker for Mac kernel has some of the Linux kernel functionality, it doesn't have the Device Mapper. We recommend using Docker-Machine for now, until Docker has added the Device Mapper into Docker for Mac/Windows

Thank you for the quick response.

I've opened an issue in the Docker github:
https://github.com/docker/docker/issues/26116

docker for mac is now supported


Server Errors (500) POSTing to Stripe

I have encountered server errors when using a test account locally, as well as on the the demo site.  This occurs after entering a valid test CC number from the stripe docs and submitting the form.
Expected Behavior

Successful transaction should process and (potentially redirect?) send a 200.
Current Behavior

The server responds with a 500, and sends an alert message stating

Unexpected token F in JSON at position 0

The response body appears to be empty, and the following error is emitted in the server logs:
Error during invocation:  TypeError: Cannot read property 'id' of undefined
    at Object.e.exports.handler (/{Project}/functions/orderCreate.js:1:61150)
    at process._tickCallback (internal/process/next_tick.js:68:7)
_http_outgoing.js:470
    throw new ERR_HTTP_HEADERS_SENT('set');
    ^

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
    at ServerResponse.setHeader (_http_outgoing.js:470:11)
    at callback (/{Project}/node_modules/netlify-lambda/lib/serve.js:37:16)
    at p (/{Project}/functions/orderCreate.js:1:61125)
    at n.orders.create (/{Project}/functions/orderCreate.js:1:60996)
    at Timeout._onTimeout (/{Project}/functions/orderCreate.js:1:9248)

Possible Solution

I have tried upgrading gatsby-stripe-starter to v3.0.x per comments in issue 2, but this did not solve the problem.
I am attempting to put some guards around order.id. It's hard to make sense of the line numbers, since the stack trace is claiming line 1 contains the error.
I will also try to remove the headers where the response object is built and see if this helps.  I will report back after I try this.
Steps to Reproduce (for bugs)

Navigate to https://gatsby-starter-stripe.netlify.com/buy/msr-hubba-hubba-nx-2 (or similar URL locally)
Click "add item to cart"
Click "checkout for $xxx"
fill out the payment form with test data
submit the payment
Observe an alert dialog with the aforementioned error message.

Context

This happens for me locally as well as on the demo.  I am trying to handle the success condition to process emails and confirmation screen, but I'm unable to get there.
Your Environment

Version used:
Environment name and version (e.g. Chrome 39, node.js 5.4):
Operating System and version (desktop or mobile):
Link to your project:

OS: OSX 'Mojave' 10.14.5
Node: v10.15.3
Chrome: 74.0.3729.169 (Official Build) (64-bit) Desktop

It looks like the array destructuring order for quantity and skuId might be reversed / and or embedded in another array (see screenshot). I tried modifying that, but the Stripe API rejects the request saying it's an invalid string.
 const items = Object.entries(cart).map(([, cartContents]) => {
      const [skuId, quantity] = cartContents;
      return {
        type: 'sku',
        parent: skuId,
        quantity,
      };
    });

Thanks for reporting this and thank you for the detailed issue! This was, in part, due to the things you pointed out. The latest commit should fix this.

I submitted a PR to stringify the body for the response.  I was receiving an error from the netlify lib's serve.js.

Seems like this is still an issue.  This is on the currently deployed example demo.

Hey, thanks for checking out the starter! I believe this was actually fixed above; however, the deployed demo is an old build which still has this bug. I've not been able to redeploy a new demo as for some reason (???) gatsby-source-stripe is now failing to pull some images on rebuild, though nothing there has changed and I have not been able to determine why. The code in the repo does not have this issue.

We didn't change any of the dependencies lately, correct? I think I upgraded one of my local dependencies, but I thought I rolled it back.
It looks like the version of gatsby-source-stripe was updated with bc17bfb288d29448be1e0e48d505b0b008a5e14f, but I wonder if that is related? Locally I tried upgrading to 3.x.x, but it looks like I downgraded it back to 2.x.x locally for some reason.
"gatsby-source-stripe": "2.x.x",

@zedd45 Yesterday's commit upgraded gatsby-source-stripe to 3.0.2, and many others were also upgraded due to security vulnerabilities.
As for the problem I described above, I've filed an issue at gatsby-source-stripe.


Work around infinite application of tile-and-distribute on linalg_ext.unpack with dynamic tile sizes

Dynamic inner tile sizes currently trigger infinite application of tile-and-distribute on the unpack op, each time calling getTiledImplementation -> getSlice creating more and more extract_slice. As a temporary work-around, we annotate unpack ops with a custom already_tiled attribute to keep track of what's already been tiled.
For some reason this causes errors in non-dynamic-shape cases, but it's not needed there anyway, so we simply check for dynamic inner tiles before applying this tweak.

Can you file an issue assigned to @MaheshRavishankar to evaluate this when he gets back? There may be something structural that can be done instead (it's weird to me that a pattern is doing some expansion but leaving the original in-place) but this is fairly self-contained and will be easy to rework.

Filed #11607

I thought it's the other attribute stuff. I browse the path and found that the workaround I mentioned would impact much more lines. It's nice that you can have this simple workaround work. :)


Imagenet100 Reproduction Accuracy considerably lower than reported.

Describe the bug
Might not be a bug but your guidance is valued:
I tried to reproduce the imagenet100 results using Barlow Twins and get accuracies well below the officially reported figures.

The model trained for 400 epochs with the default configuration on 2 GPUs, but even when just training on 1 GPU the same accuracies can be observed. I suspect that 400 epochs is not enough (as the validation curve keeps increasing), hence I wanted to check if you really trained the models for 400 epochs.
I suspect I am doing something wrong but I am unsure what my mistake is. Thanks for your guidance.
For the dataset: We used these classes as referenced here.
To Reproduce
python /ssl_pm/main_pretrain.py --config-path scripts/pretrain/imagenet-100/ --config-name barlow.yaml
Screenshots

Versions
Latest version (pulled 2 days ago)
--
here you have my barlow.yaml file for reference:
defaults:
  - _self_
  - augmentations: asymmetric.yaml
  - wandb: private.yaml
  - override hydra/hydra_logging: disabled
  - override hydra/job_logging: disabled

# disable hydra outputs
hydra:
  output_subdir: null
  run:
    dir: .

name: "barlow_twins-imagenet100"
method: "barlow_twins"
backbone:
  name: "resnet18"
method_kwargs:
  proj_hidden_dim: 2048
  proj_output_dim: 2048
  scale_loss: 0.1
data:
  dataset: imagenet100
  train_path: "/itet-stor/zdavid/net_scratch/ILSVRC2012_img_train_100" #"/home/david/data/imagenet_100/train"
  val_path: "/itet-stor/zdavid/net_scratch/ILSVRC2012_img_val_100" #"/home/david/data/imagenet_100/val"
  format: "dali"
  num_workers: 4
optimizer:
  name: "lars"
  batch_size: 128
  lr: 0.3
  classifier_lr: 0.1
  weight_decay: 1e-4
  kwargs:
    clip_lr: True
    eta: 0.02
    exclude_bias_n_norm: True
scheduler:
  name: "warmup_cosine"
checkpoint:
  enabled: True
  dir: "trained_models"
  frequency: 1
auto_resume:
  enabled: False

# overwrite PL stuff
max_epochs: 400
devices: [0, 1]
sync_batchnorm: True
accelerator: "gpu"
strategy: "ddp"
precision: 16-mixed

Hi @davidrzs. Thanks for letting us know.
Can you try to run the previous version, with an older pytorch lightning version, etc? This should work as reported in the results table.
The much lower accuracy can be due to some pytorch lightning bug/version change or was introduced by some change that I did in the latest version. The only thing that comes to my mind is that the LARS that I introduced to the repo is messed up.

The curve you posted is really smooth, usually you should have a bit more variance. It's very possibly an optimization problem as Victor suggested. Can you send the loss as well?

I said LARS but actually we didn't change it, we just changed the LR Scheduler. I don't think it introduced any bug, but can you also share the LR plot?

There are two other things that come to my mind. DALI could have changed something. Can you also try without it? Second thing is that training now uses 16-mixed precision instead of the old 16 precision. I noticed that for notebook GPU (1660ti) it simply doesn't work and the outputs of the model are just NaNs. It doesn't give any error or anything like that. Can you try full precision training (this worked for me)?

Here are some screenshots:

I can schedule a 32bit run later tonight or tomorrow with DALI disabled (for the old packages it will take some time as getting these virtual environments to work on the cluster is a real pain due to memory restrictions).
Looking at the learning rate plot, I assume that something is a bit odd there, as we do not have the full progression that we see with other models.
Let me know if any other screenshots or data would be helpful.

@davidrzs Thanks for all the screenshots. Is this the full training? If so, there's definitely an issue with the LR scheduler and most likely that's the cause of the low performance. It can be that lightning is now calling the scheduler in a different way, I don't know.

Yes, this is the full 400 epochs (i.e. the run exited completely).

The batch size in the yaml file is per GPU, that's fine.

Me and @DonkeyShot21 were talking and this can be an issue that happened quite a long time ago with Dali + pytorch lightning. This is the issue https://github.com/NVIDIA/DALI/issues/3865. I'll need some time to check again, but if you see what works from the following, it would be great:

Dali + single GPU
No Dali + single GPU
No Dali and multi GPU
Just need to run for like 20 epochs so we go past the warmup of the LR Scheduler. We shouldn't see any epoch skipping nor issues with the LR plot.

I already tested single GPU and Dali, same behaviour: (here both single GPU and the above multi GPU are plotted):

Will try a "non-Dali" run and report back.

This is from a non-Dali run on two Titan Xp cards:
In the images you can see the previous with Dali run and the new still running non-Dali run:

Do I interpret the epoch chart correctly, that somehow the Dali run gets twice the number of epochs per global trainer step? (which would be weird)

If the issue is the same as before, Dali calls some reset method after each epoch, which then triggers an "epoch skip" from lightning. This looks very much to be the case. I'll check what is the proper fix in the next couple of days.

I let it run overnight (still running, will let it complete) and we are now around epoch 160:
The following are the graphs:

I feel like the accuracy is still lower than where it should be given that soon half of the training will be done?

@davidrzs I checked my old runs, and it seems quite similar to the curves that I have.

I have just scheduled the same model on the cluster with Dali enabled (will report in due time).
The last run (without Dali) terminated after 380 epochs (we have a 48h limit on the cluster) and the following are the plots:

Top1-train seems to be okay, but top1 val seems to be way too low. Do you have any idea why? Will investigate on Wednesday a bit.

Hummm. It can be the change that I added to comply with lightning>=2.0. I'll check that soon (if that's the case, it's only an evaluation issue).

Can confirm that it works with Dali on pytorch lightning==1.9.0. (2gpus)
See screenshots:

Though, the evaluation issue persists. Do you have any idea what could cause the eval issue? That way I could also start to investigate.

@davidrzs I'll push an update that will fix DALI for lightning>2.0. About the accuracy issue, lightning changed how you compute epoch-wise metrics. Beforehand, you just returned it in the validation_step and you would receive it as parameter in another callback. Now, you need to store the metrics in a list and remember manually clean it. I simply forgot to call .clear() on it. The push will also fix this.

Thank you very much for your work on this! Will rerun it after you merged with main and report tomorrow.

Merged it. The modification is really simple so hopefully, it's fine.

Can confirm that it works (learning rate as well as validation accuracies).
Thanks for fixing it!

Though, you might want to update pytorch lightning in the requirements file. (I manually updated and let it run on 2.0.2

Super! It's always annoying to update library versions, but I'm glad it's working now :). I'll also update the requirements file, thought I already had.


[Bug][CDC][E2E] Fix mysql-cdc e2e error #4944

Purpose of this pull request

Check list

[ ] Code changed are covered with tests, or it does not need tests for reason:
[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

[ ] Update the release-note.

@hailin0 @Hisoka-X PTAL

I have fixed it in this pr. https://github.com/apache/seatunnel/pull/4936

Member

I have fixed it in this pr. #4936
+1,
https://github.com/apache/seatunnel/pull/4923

I have fixed it in this pr. #4936
It looks like this Pr fixes the document, how about you make a separate fix cdc or use the current pr to fix the e2e issue?

PTAL: @hailin0 @Hisoka-X @ic4y  thks

deplicate of #4936 , thanks for your contribute.


RHE7 AWS AMI -->  python_runtime error

Hi, I'm using the official AMI RHE7 from AWS and I'm getting the following errors:
Error executing action `install` on resource 'yum_package[python27]'
Error executing action `install` on resource 'poise_languages_scl[python27]'
Error executing action `install` on resource 'python_runtime[2]'

My recipe is simple:
include_recipe 'poise-python'

python_runtime '2'
python_virtualenv "#{node['django1.8']['path']}/venv"

python_package 'Django' do
  version '1.9'
end

package 'git' do
  action :install
end

The weird part is that testing in my local environment with kitchen in a centos7.2 box I don't have any problem.

What's the actual error from yum? You may need to run with chef-client -l debug to see the debug output.

     Chef::Exceptions::Exec
     ----------------------
     poise_languages_scl[python27] (/var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/mixin.rb line 27) had an error: Chef::Exceptions::Exec: yum_package[python27] (/var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/resource.rb line 127) had an error: Chef::Exceptions::Exec: yum -d0 -e0 -y install python27-1.1-20.el7.centos returned 1:
     STDOUT:  You could try using --skip-broken to work around the problem
      You could try running: rpm -Va --nofiles --nodigest

     STDERR: Error: Package: python27-python-devel-2.7.8-3.el7.centos.x86_64 (rhscl-python27-epel-7-x86_64)
                Requires: scl-utils-build

     Cookbook Trace:
     ---------------
     /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
     /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/resource.rb:68:in `action_install'
     /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
     /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

     Resource Declaration:
     ---------------------
     # In /var/chef/cache/cookbooks/poise-python/recipes/default.rb

      20: python_runtime '2' if node['poise-python']['install_python2']

     Compiled Resource:
     ------------------
     # Declared in /var/chef/cache/cookbooks/poise-python/recipes/default.rb:20:in `from_file'

     python_runtime("2") do
       action [:install]
       updated true
       updated_by_last_action true
       retries 0
       retry_delay 2
       default_guard_interpreter :default
       declared_type :python_runtime
       cookbook_name "poise-python"
       recipe_name "default"
       pip_version true
       setuptools_version true
       version "2"
       virtualenv_version true
       wheel_version true
     end

     Platform:
     ---------
     x86_64-linux

Hmm, much have shuffled some requirements in way that makes RHEL sad. I'm working on a revamp of the SCL installer code to better handle the new procedures from SCLo and RHSCL. For now you can use the :system provider which should be 2.7 on EL7 I think?

The system provider is also failing
python_runtime 'myapp' do
   provider :system
   version '2.7'
end

     ```

Error executing action install on resource 'yum_package[python27]'
     Chef::Exceptions::Exec
     ----------------------
     yum -d0 -e0 -y install python27-1.1-20.el7.centos returned 1:
     STDOUT:  You could try using --skip-broken to work around the problem
      You could try running: rpm -Va --nofiles --nodigest

     STDERR: Error: Package: python27-python-devel-2.7.8-3.el7.centos.x86_64 (rhscl-python27-epel-7-x86_64)
                Requires: scl-utils-build

     Cookbook Trace:
     ---------------
     /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
     /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/scl/resource.rb:68:in `action_install'
     /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
     /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:44:in `action_install'

That doesn't sound like the provider is getting set. You can see it still using the SCL code in the stack trace. Make sure you knife upload'd and whatnot? It should just install the package python in the end.

looks weird,
I did the knife upload, this is how my recipe looks like:
include_recipe 'poise-python'

python_runtime 'myapp' do
   provider :system
  version '2.7'
end

python_virtualenv "#{node['django1.8']['path']}/venv"

But I can notice that after the include_recipe 'poise-python' this happens:
Recipe: poise-python::default
* python_runtime[2] action install

And the poise-python::default looks like this:
# Default runtimes, last one will be the default.
python_runtime 'pypy' if node['poise-python']['install_pypy']
python_runtime '3' if node['poise-python']['install_python3']
python_runtime '2' if node['poise-python']['install_python2']

Isn't the include_recipe 'poise-python' automatically calling the python_runtime ?

Ahh, you're crossing the streams. Generally you want either the include_recipe or your own use of python_runtime, not both. Your python_runtime is correct in this case, if you remove the include you should be fine. The issue is the runtime from the recipe is still using the default provider. If you want to use the recipe instead you could set node['poise-python']['provider'] = 'system' via any of the various ways to set attributes (role, env, wrapper, cookbook, etc).

Great , thank's


Create Seperate Gameserver (Part 2)

Updates the Backend and Frontend as well as SEE itself:

Synchronize Code Cities between SEE and the Backend.
Improve communication between server and clients via RPCs.

This MR contains a general overhaul of the initial Backend code as well as important changes to the Frontend and SEE codebases and documentation.

I have adapted some of your changes to make the JSON de/serialization work again. I believe the branch can be merged now, although I have not tested after the last merge of master into this branch.


In tomcat with -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true, the FilterTools.getRelativePath does not works correctly

if a URL contains a %2F, e.g. http://localhost:8080/api/v1/permissionrole/permission/get%2Fabc, the HttpServletRequest.getRequestURI could return the raw uri correctly, but the FilterTools.getRelativePath decoded the %2F in the path. Maybe it's not right.
the correct result should be /api/v1/permissionrole/permission/get%2Fabc, but /api/v1/permissionrole/permission/get/abc returned by FilterTools.getRelativePath.
this PR works for this case, https://github.com/perwendel/spark/pull/553.
Thanks.
https://github.com/perwendel/spark/blob/c697d981268c538dfeeb38786bd9eb0a03ab86f0/src/main/java/spark/servlet/FilterTools.java#L55

I would like to work on this issue

I want to use UTF-8 to solve this problem. When the path in URL has %, the user can not put the URL in the address bar in the browser directly. They should encode URL in UTF-8 .
There are two ways:

url_encode = encodeURL(URL) (in JavaScript language). Then put url_encode into address bar,
open the setting: send url by UTF-8 in browser. And then change the % to %25.


esbuild config function is called with unresolved serverless param

Describe the bug
Hi,
The first time buildOptions getter is called it is in the constructor.
https://github.com/floydspace/serverless-esbuild/blob/2c1d2ac842ac4270c48028d123887193ae957af2/src/index.ts#L85
If I am not wrong then this causes the esbuild config to be called with an unresolved serverless object.

Note: configuration values are only resolved after plugins are initialized. Do not try to read configuration in the plugin constructor, as variables aren't resolved yet. Read configuration in lifecycle events only.
(https://www.serverless.com/framework/docs/guides/plugins/creating-plugins)

Possible moving this logic to the initialize hook might solve this.
To Reproduce
Steps to reproduce the behavior. If you have an example repository that would be even better:

Set : esbuild:    config: "./esbuild.config.js"
Set an ssm import somewhere in custom SECRET: ${ssm.eu-west-1:/domain/${self:custom.stage}/secret}
console.log(serverless) in the config function.
SECRET will still be an un resolved string

Expected behavior
SECRET should be the resolved value from ssm
Versions (please complete the following information):

OS: [ Mac]
Serverless Framework Version: [2.72.3]
Plugin Version: [1.31.0]

Nice one, worth a fix I think. I'll have a look some time over the week

@samchungy had a little time on hand to give this issue a go. Let me know what you think?


current rakudo

Hi, some changes to Rakudo/Parrot caused your module to stop working, I have fixed it so that it works again on the current version of rakudo and added some tests.

Thanks for fixing this, glassflag!
I've left this module alone for a while because I've been planning on making use of NativeCall instead so that it would be faster and not dependent on Rakudo. Unfortunately, I was having problems with zavolaj and have been waiting for future improvements to fix the problems I was experiencing. Though in the meantime, I'd still like it to work so thanks again! :)


api.new.get.thread_data() no longer works with domThread object.

Demonstrated through the following snippet:
gmail.observe.on("view_thread", (domThread) => {
    console.log("Looking at thread:", domThread);
    const threadData = gmail.new.get.thread_data(domThread);
    console.log("Thread data:", threadData); // reports null!
}

Fixed. Closing.


bug: I Cannot add android platform on MacOS

Type: bug
Platform: android
Hi all,
I Cannot add android as a platform on my projects on MacOS.
I've installed lastest versions of Node, cordova and ionic, JDK, and Android-SDK.
When i add IOS works properly, but when i add Android as a platform i have this error:
ANDROID_HOME is not set and "android" command not in your PATH.
Thats is false, if i execute android from the console, the Android SDK Manager executes.
I have installed android-SDK via homebrew. After that, i have added to my .zshrc file the reference to android-SDK:
export ANDROID_HOME=brew --prefix android
Any ideas?
About my system:
OS:  OS X Yosemite 10.10.2
Node: 0.12.0
Ionic: 1.3.10
Cordova: 4.2.0
Java JDK: 1.8.0_31
Thanks!

More info: I've installed Android Studio and i've created an Application. I compiled it and installed into my Nexus 5 this app with no issues.
If i try to add the android platform just using cordova (without Ionic), i have the same error (Android_Home is not set), so it seems that is a problem with cordova.

Hi, it's working now.
If i install only the SDK downloading it or via brew it doesn't work.
After install Android Studio and test an app, i went to the project Structures and check where is the SDK located now.
Then, i went to my .zshrc file and changed to the new reference:
export PATH=${PATH}:/Users/ivanbtrujillo/Library/Android/sdk/platform-tools:/Users/ivanbtrujillo/Library/Android/sdk/tools
Then i closed iTerm and opened a new terminal. Now cordova and ionic can add the Android platform. Hope this issue help others.
Regards!


Add a test for WorkspaceState serialization stability

Check that serialized workspace state indeed has stable ordering for dependencies.
Motivation:
Follow-up from https://github.com/apple/swift-package-manager/pull/3785.
Modifications:
Adds a new test case for WorkspaceState that loads a mis-sorted state file, then re-saves it.
Result:
Because the underlying issue is non-deterministic, this test has a false-negative rate of roughly 50%, i.e. it will pass half the time, even when it should fail.  Repeating the test can improve those odds, but I wan't able to do so reliably from within the test itself vs. multiple swift test invocations.

@swift-ci please smoke test

@swift-ci please smoke test


Cannot configure IPv6 CIDR policy in k8s (CRV rejects it)

We're currently rejecting valid IPv6 CIDR policies because of an over-restrictive CustomResourceValidator in kubernetes cases:
https://github.com/cilium/cilium/blob/e9b182cdf6dde5136e9534a2b03e4fd3bce806a4/pkg/k8s/apis/cilium.io/v2/register.go#L362-L380
Error:
validation failure list:
must validate one and only one schema (oneOf)
spec.egress.toCIDR in body should match '^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\/([0-9]|[1-2][0-9]|3[0-2])$'

/cc @cilium/kubernetes  Was there a particular reason we are avoiding this for now? I believe that the underlying support for this should work, was it just a matter of figuring out a reasonable IPv6 regex pattern?
Adding tags priority/1.1 and stable/needs-backport, we can always remove them if the fix for this doesn't seem appropriate for backport.
Tasks:

[x] Prepare change (#4004)
[x] Test regex (included in #4004)
[x] Test IPv6 CIDR behaviour manually
[ ] Add Ginkgo test for IPv6 CIDR behaviour

aanm likely remembers better. I think we didn't support something IPv6 at the time. It's also possible the regex was buggy :/

aanm likely remembers better. I think we didn't support something IPv6 at the time. It's also possible the regex was buggy :/

@raybejjani It's because there are no tests for IPv6 so I just prevented the users from using IPv6 addresses at all. We need to improve the IPv6 tests. cc @joestringer

@aanm :+1: I manually tested on Friday. I can put together a ginkgo test based on how I tested it.

@joestringer I'm not referring to the CRD tests itself. I was referring to everything around IPv6, from LB to CIDR policy.

@aanm I tested CIDR policies with IPv6, specifically by adding CIDR rules that do not match a CIDR of a known IP accessible within a Minikube environment and seeing that traffic is dropped, then changing the CIDR to match and saw that the IPv6 traffic passed through.
What are you thinking might be missing for LB? Configure a ClusterIP service and see that traffic successfully passes between something external to k8s into the cilium service, gets translated and traffic returns correctly?

What are you thinking might be missing for LB? Configure a ClusterIP service and see that traffic successfully passes between something external to k8s into the cilium service, gets translated and traffic returns correctly?

@joestringer something like that yes, because we don't have e2e tests for IPv6 in kubernetes.