Partial model loading for low-VRAM systems

Summary
This PR updates the model cache behavior with the following improvements:

Added an option to enable partial model loading. If enabled, large models can be partially loaded onto the CUDA device and then streamed to the device during inference. This enables inference of large models on systems with limited VRAM (e.g. full FLUX dev inference on an 8GB GPU).
Updated the RAM and VRAM cache size logic to more effectively use the available memory under the default configs.

Configuration
The following invokeai.yml configs were added / changed:

ram: Now defaults to None (unset), resulting in automatic RAM cache size management. It can still be set to override the default behavior.
vram: Now defaults to None (unset), resulting in automatic VRAM cache size management. It can still be set to override the default behavior.
lazy_offload: This config is deprecated and will be removed in a future migration. It no longer has any effect.
device_working_mem_gb: This is the new recommended config for tuning the balance between working VRAM and cache VRAM. The default of 2gb should work for most use cases.
enable_partial_loading: Defaults to False. Set it to True to get the new partial loading behavior for low-vram operation.

The recommended invokeai.yaml configuration for users wishing to opt-in to the new partial loading behaviour is:
# Comment-out any existing ram/vram overrides:
# ram = 16
# vram = 12

# device_working_mem_gb defaults to 2.
# Uncomment and increase it if you are doing non-standard operations that cause OOM errors.
# (File a bug report if you are hitting unexpected OOM errors)
# device_working_mem_gb: 2

# Enable partial loading behaviour.
enable_partial_loading: True

QA Instructions
I have tested a wide variety of models (SD1, SDXL, FLUX, SD3, LoRAs, quantized/non-quantized) with both enable_partial_loading: True and enable_partial_loading: False.
The scope of this change is large with risk of regressions on particular OSs or HW configurations. We will want to keep a close eye on this change in the RC stage for possible bugs.
Checklist

[ ] The PR has a short but descriptive title, suitable for a changelog
[ ] Tests added / updated (if applicable)
[ ] Documentation added / updated (if applicable)
[ ] Updated What's New copy (if doing a release after this PR)

Running into a few issues.
Starting with only this relevant setting in invokeai.yaml:
enable_partial_loading: true

Restarted the app after each error.
I tried FLUX w/ IP Adapter and got an error: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_addmm)

full error & context
Models:

Loading checkpoint shards: 100%|███████████████████████████████| 2/2 [00:00<00:00,  6.04it/s]
[2024-12-19 16:08:32,570]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '0d07998e-3115-44f2-bfec-5c901cef0b2c:text_encoder_2' (T5EncoderModel) onto cuda device in 4.44s. Total model size: 9334.39MB, VRAM: 9334.39MB (100.0%)
[2024-12-19 16:08:32,745]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '0d07998e-3115-44f2-bfec-5c901cef0b2c:tokenizer_2' (T5Tokenizer) onto cuda device in 0.00s. Total model size: 0.03MB, VRAM: 0.00MB (0.0%)
[2024-12-19 16:08:33,428]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '54a9ad60-0a74-45e4-a13d-37179ab87b45:text_encoder' (CLIPTextModel) onto cuda device in 0.04s. Total model size: 469.44MB, VRAM: 469.44MB (100.0%)
[2024-12-19 16:08:33,503]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '54a9ad60-0a74-45e4-a13d-37179ab87b45:tokenizer' (CLIPTokenizer) onto cuda device in 0.00s. Total model size: 0.00MB, VRAM: 0.00MB (0.0%)
[2024-12-19 16:08:35,182]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '65efbf64-9ea9-4b1d-93c4-e70db9e724d7' (CLIPVisionModelWithProjection) onto cuda device in 0.06s. Total model size: 579.77MB, VRAM: 579.77MB (100.0%)
[2024-12-19 16:08:45,329]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '046c8125-dfc8-4151-be26-f1c5b459bf29:transformer' (Flux) onto cuda device in 9.88s. Total model size: 22700.13MB, VRAM: 21476.13MB (94.6%)
[2024-12-19 16:08:45,371]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model 'ad993f2e-eb70-49d8-b4bb-9ce6bac3d74f' (XlabsIpAdapterFlux) onto cuda device in 0.01s. Total model size: 936.27MB, VRAM: 0.27MB (0.0%)
[2024-12-19 16:08:45,393]::[InvokeAI]::ERROR --> Error while invoking session c1b99b8d-79c6-4aa7-92d6-6a1f717956b7, invocation 01b50de3-ff97-4d72-ab6f-f5be43ca3614 (flux_denoise): Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_addmm)
[2024-12-19 16:08:45,393]::[InvokeAI]::ERROR --> Traceback (most recent call last):
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 129, in run_node
    output = invocation.invoke_internal(context=context, services=self._services)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 300, in invoke_internal
    output = self.invoke(context)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 144, in invoke
    latents = self._run_diffusion(context)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 332, in _run_diffusion
    pos_ip_adapter_extensions, neg_ip_adapter_extensions = self._prep_ip_adapter_extensions(
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 700, in _prep_ip_adapter_extensions
    ip_adapter_extension.run_image_proj(dtype=dtype)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/flux/extensions/xlabs_ip_adapter_extension.py", line 54, in run_image_proj
    self._image_proj = self._model.image_proj(image_prompt_clip_embed)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/ip_adapter/ip_adapter.py", line 59, in forward
    clip_extra_context_tokens = self.proj(embeds).reshape(
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 117, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_addmm)

[2024-12-19 16:08:45,448]::[InvokeAI]::INFO --> Graph stats: c1b99b8d-79c6-4aa7-92d6-6a1f717956b7
                          Node   Calls   Seconds  VRAM Used
             flux_model_loader       1    0.024s     0.000G
             flux_text_encoder       1    7.783s     9.347G
                       collect       2    0.001s     9.343G
                 core_metadata       1    0.001s     9.343G
               flux_ip_adapter       1    0.002s     9.343G
                  flux_denoise       1   11.830s    20.995G
TOTAL GRAPH EXECUTION TIME:  19.640s
TOTAL GRAPH WALL TIME:  19.648s
RAM used by InvokeAI process: 22.89G (+22.031G)
RAM used to load models: 33.22G
VRAM in use: 20.986G
RAM cache statistics:
   Model cache hits: 7
   Model cache misses: 7
   Models cached: 3
   Models cleared from cache: 3
   Cache high water mark: 23.65G

I removed the Global Reference (IP Adapter) layer and tried FLUX w/ the ControlNet Union Pro model. Got a warning [MODEL CACHE] Insufficient GPU memory to load model. Aborting and then an AssertionError.

full error & context
[2024-12-19 16:20:39,308]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '9e901924-ff8c-4de8-8f25-931245b0b0e1:vae' (AutoEncoder) onto cuda device in 0.02s. Total model size: 159.87MB, VRAM: 159.87MB (100.0%)
[2024-12-19 16:20:40,239]::[ModelManagerService]::WARNING --> [MODEL CACHE] Insufficient GPU memory to load model. Aborting
[2024-12-19 16:20:40,240]::[ModelManagerService]::WARNING --> [MODEL CACHE] Insufficient GPU memory to load model. Aborting
[2024-12-19 16:20:40,245]::[InvokeAI]::ERROR --> Error while invoking session 46c5ab42-8a2f-47c4-9f0b-adccdebdce6d, invocation b617ff7e-a6f4-49d6-895a-5afa5894370f (flux_denoise):
[2024-12-19 16:20:40,245]::[InvokeAI]::ERROR --> Traceback (most recent call last):
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/model_cache/model_cache.py", line 213, in lock
    self._load_locked_model(cache_entry, working_mem_bytes)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/model_cache/model_cache.py", line 265, in _load_locked_model
    model_bytes_loaded = self._move_model_to_vram(cache_entry, vram_available)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/model_cache/model_cache.py", line 284, in _move_model_to_vram
    return cache_entry.cached_model.partial_load_to_vram(vram_available)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/model_cache/cached_model/cached_model_with_partial_load.py", line 97, in partial_load_to_vram
    cur_state_dict[key] = param.to(self._compute_device, copy=True)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 108.00 MiB. GPU 0 has a total capacity of 23.63 GiB of which 20.44 MiB is free. Including non-PyTorch memory, this process has 23.42 GiB memory in use. Of the allocated memory 22.95 GiB is allocated by PyTorch, and 12.01 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 129, in run_node
    output = invocation.invoke_internal(context=context, services=self._services)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 300, in invoke_internal
    output = self.invoke(context)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 144, in invoke
    latents = self._run_diffusion(context)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 289, in _run_diffusion
    controlnet_extensions = self._prep_controlnet_extensions(
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 553, in _prep_controlnet_extensions
    model = exit_stack.enter_context(controlnet_model)
  File "/usr/lib/python3.10/contextlib.py", line 492, in enter_context
    result = _cm_type.__enter__(cm)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/load_base.py", line 60, in __enter__
    self._cache.lock(self._cache_record.key, None)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/model_cache/model_cache.py", line 219, in lock
    cache_entry.unlock()
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/model_cache/cache_record.py", line 27, in unlock
    assert self._locks >= 0
AssertionError

Ok, maybe I need to set device_working_mem_gb: 6.0 (was previously unset). Now I get KeyError: '046c8125-dfc8-4151-be26-f1c5b459bf29:transformer'.

full error & context
Loading checkpoint shards: 100%|███████████████████████████████| 2/2 [00:00<00:00,  6.13it/s]
[2024-12-19 16:21:31,326]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '0d07998e-3115-44f2-bfec-5c901cef0b2c:text_encoder_2' (T5EncoderModel) onto cuda device in 4.27s. Total model size: 9334.39MB, VRAM: 9334.39MB (100.0%)
[2024-12-19 16:21:31,503]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '0d07998e-3115-44f2-bfec-5c901cef0b2c:tokenizer_2' (T5Tokenizer) onto cuda device in 0.00s. Total model size: 0.03MB, VRAM: 0.00MB (0.0%)
[2024-12-19 16:21:32,167]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '54a9ad60-0a74-45e4-a13d-37179ab87b45:text_encoder' (CLIPTextModel) onto cuda device in 0.04s. Total model size: 469.44MB, VRAM: 469.44MB (100.0%)
[2024-12-19 16:21:32,239]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '54a9ad60-0a74-45e4-a13d-37179ab87b45:tokenizer' (CLIPTokenizer) onto cuda device in 0.00s. Total model size: 0.00MB, VRAM: 0.00MB (0.0%)
[2024-12-19 16:21:33,371]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '9e901924-ff8c-4de8-8f25-931245b0b0e1:vae' (AutoEncoder) onto cuda device in 0.02s. Total model size: 159.87MB, VRAM: 159.87MB (100.0%)
[2024-12-19 16:21:35,146]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model 'd830a8f2-0db8-4504-aad9-ebf7b6930560' (InstantXControlNetFlux) onto cuda device in 1.59s. Total model size: 6297.98MB, VRAM: 6297.98MB (100.0%)
[2024-12-19 16:21:35,150]::[InvokeAI]::ERROR --> Error while invoking session e1fc9e22-df57-4fd2-8687-269b290f9da6, invocation fa2a3d36-0e7f-4bdf-9302-d8750b2d7752 (flux_denoise): '046c8125-dfc8-4151-be26-f1c5b459bf29:transformer'
[2024-12-19 16:21:35,150]::[InvokeAI]::ERROR --> Traceback (most recent call last):
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 129, in run_node
    output = invocation.invoke_internal(context=context, services=self._services)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 300, in invoke_internal
    output = self.invoke(context)
  File "/home/bat/Documents/Code/InvokeAI/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 144, in invoke
    latents = self._run_diffusion(context)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/app/invocations/flux_denoise.py", line 299, in _run_diffusion
    (cached_weights, transformer) = exit_stack.enter_context(transformer_info.model_on_device())
  File "/usr/lib/python3.10/contextlib.py", line 492, in enter_context
    result = _cm_type.__enter__(cm)
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/load_base.py", line 75, in model_on_device
    self._cache.lock(self._cache_record.key, working_mem_bytes)
  File "/home/bat/Documents/Code/InvokeAI/invokeai/backend/model_manager/load/model_cache/model_cache.py", line 203, in lock
    cache_entry = self._cached_models[key]
KeyError: '046c8125-dfc8-4151-be26-f1c5b459bf29:transformer'

[2024-12-19 16:21:35,200]::[InvokeAI]::INFO --> Graph stats: e1fc9e22-df57-4fd2-8687-269b290f9da6
                          Node   Calls   Seconds  VRAM Used
             flux_model_loader       1    0.017s     0.000G
             flux_text_encoder       1    7.593s     9.347G
                       collect       2    0.001s     9.343G
                 core_metadata       1    0.001s     9.343G
               flux_controlnet       1    0.000s     9.343G
                  flux_denoise       1    2.855s     9.352G
TOTAL GRAPH EXECUTION TIME:  10.467s
TOTAL GRAPH WALL TIME:  10.473s
RAM used by InvokeAI process: 9.64G (+8.784G)
RAM used to load models: 38.05G
VRAM in use: 6.322G
RAM cache statistics:
   Model cache hits: 7
   Model cache misses: 7
   Models cached: 2
   Models cleared from cache: 1
   Cache high water mark: 22.17G

Tried with it set to 9.0, same exact error.
Just trying FLUX without any extras, full model. With 1536x1536 and device_working_mem_gb unset, I get a OOM during VAE decode. Setting device_working_mem_gb to a larger value allows it to complete.

It seems the handling of OOMs has changed.
I expected OutOfMemoryErrors to bubble up the stack and be recorded as the error type for the node, but we get the AssertionError instead.  This occurs when we OOM regardless of the enable_partial_loading setting. The user-facing result is that we generic Server Error toasts instead of more descriptive toasts saying an OOM occurred.

Looks like we unlock one too many times when a move to VRAM fails during a call to ModelCache.lock() - once in the exception handler in _move_model_to_vram(), then again in the top-level exception handler for ModelCache.lock().
We also get two warnings - one at each of these two levels.
I'll defer to you to fix it - but if I do this, at least the OOMs bubble up correctly.
diff --git a/invokeai/backend/model_manager/load/model_cache/model_cache.py b/invokeai/backend/model_manager/load/model_cache/model_cache.py
index 760ec5f348..d4f552e094 100644
--- a/invokeai/backend/model_manager/load/model_cache/model_cache.py
+++ b/invokeai/backend/model_manager/load/model_cache/model_cache.py
@@ -209,18 +209,8 @@ class ModelCache:
             # Models don't need to be loaded into VRAM if we're running on CPU.
             return

-        try:
-            self._load_locked_model(cache_entry, working_mem_bytes)
-            self._logger.debug(
-                f"Finished locking model {key} (Type: {cache_entry.cached_model.model.__class__.__name__})"
-            )
-        except torch.cuda.OutOfMemoryError:
-            self._logger.warning("Insufficient GPU memory to load model. Aborting")
-            cache_entry.unlock()
-            raise
-        except Exception:
-            cache_entry.unlock()
-            raise
+        self._load_locked_model(cache_entry, working_mem_bytes)
+        self._logger.debug(f"Finished locking model {key} (Type: {cache_entry.cached_model.model.__class__.__name__})")

         self._log_cache_state()

Some notes after reviewing the issues that @psychedelicious found:

I tried FLUX w/ IP Adapter and got an error: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_addmm)

The root cause is that the IP-Adapter model has multiple entrypoints. Currently, auto-device-casting only works when a model is called via its torch.nn.module.forward(...) method. In the future, we should be able to improve the autocast context management to support this case, but for now I have added XlabsIpAdapterFlux to a list of models that opt-out of partial loading (in 7fddb06). There is a risk that other models have a similar problem, but we just haven't encountered it yet.

Related to the other issues, the following fixes are needed:

[ ] The working memory required for VAE decode needs to be calculated dynamically based on the size of image being decoded.
[ ] Fix up the logic for handling exceptions during model load / unload. The model should be kicked out of the cache, and we should make sure that all lock tracking gets updated correctly.
[ ] I'm not yet sure what is causing the OOM error during partial load of the ControlNet Union Pro model. I need to investigate this some more. @psychedelicious, if you are able to reproduce it with debug logs turned on, that would be helpful.
[ ] When loading a model that does not support partial loading, locked models that do support partial loading should be partially unloaded to make room for it. We don't currently do this.
[ ] There is a bug in flux_denoise where we load a FLUX model, don't lock it, load other models, and then expect the FLUX model to still be in the cache. There are probably other places in our code base where we do this, since the model loading APIs don't do a good job of discouraging it. This error was possible before, but the prevalence seems to have gone up - I need to think some more about why that is.

Marked as draft to prevent accidental merge.

Closing in favor of smaller PRs that will be easier to test.


Bugfix - pow(Eigen, Eigen) forcing Matrix return

Summary
As identified here the changes to pow introduced by #2546 have broken the extended distribution tests by forcing the return to always be an Eigen::Matrix input, even when Eigen::Array inputs have been passed.
This PR updates the return type deduction to correctly return Eigen::Array inputs where appropriate.
Tests
Extended distribution tests have been temporarily enabled to ensure that this fixes the issue
Side Effects
N/A
Release notes
Fix bug with vectorised pow() incorrectly forcing Eigen::Matrix return instead of Eigen::Array
Checklist

[x] Math issue N/A

[x] Copyright holder: Andrew Johnson
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[x] the basic tests are passing

unit tests pass (to run, use: ./runTests.py test/unit)
header checks pass, (make test-headers)
dependencies checks pass, (make test-math-dependencies)
docs build, (make doxygen)
code passes the built in C++ standards checks (make cpplint)

[x] the code is written in idiomatic C++ and changes are documented in the doxygen

[x] the new changes are tested

Extended distribution tests have been temporarily enabled to ensure that this fixes the issue

This does not have to be hardcoded. You can enable the extended tests buy ticking the "withRowVectors" in Jenkins: https://jenkins.flatironinstitute.org/job/Stan/job/Math/job/PR-2793/build?delay=0sec
I already started them https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FMath/detail/PR-2793/4/pipeline
If all goes green I will approve. Thanks again!

Oh neat, thanks!


Question about the reimplementation of the result of original CLIP

Hi there, Congrats for the great work!
In table 1, I ve noticed you also include the result of Original CLIP model

Could you please share the setting of this experiments? Cuz my reimplementation based on your code shows lots of differences than yours. Like

the size of CLIP (Base large huge?)
The fixed text prompts you used. ( The "encode_text_with_prompt_ensemble" method you implemented?)
any modifications on the vision side ? like DPAM

Thank you so much for your patience!

Hi,
1、We use the same large backbone as AnomalyCLIP。
2、For the original CLIP, we use two single prompts (refer to Appendix Baselines for details). 3、We do not modify the vision side.

Hi, 1、We use the same large backbone as AnomalyCLIP. 2、For the original CLIP, we use two single prompts (refer to Appendix Baselines for details). 3、We do not modify the vision side.

Thx a lot!


Moved DataPrepper to core package

Signed-off-by: Steven Bayer<EMAIL_ADDRESS>Description
Move DataPrepper class to a core package. Part of large refactoring to moving all core classes to a new core package
Issues Resolved
n/a
Check List

[x] New functionality includes testing.
[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #927 (778e45a) into main (3b31f27) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main     #927   +/-   ##
=========================================
  Coverage     92.21%   92.21%
  Complexity      660      660
=========================================
  Files            80       80
  Lines          1940     1940
  Branches        164      164
=========================================
  Hits           1789     1789
  Misses          110      110
  Partials         41       41

Impacted Files
Coverage Δ

...ava/com/amazon/dataprepper/DataPrepperExecute.java
0.00% <ø> (ø)

.../java/com/amazon/dataprepper/core/DataPrepper.java
96.15% <ø> (ø)

...n/dataprepper/parser/model/MetricRegistryType.java
66.66% <ø> (ø)

...dataprepper/pipeline/server/DataPrepperServer.java
89.83% <ø> (ø)

...aprepper/pipeline/server/ListPipelinesHandler.java
100.00% <ø> (ø)

...n/dataprepper/pipeline/server/ShutdownHandler.java
100.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b31f27...778e45a. Read the comment docs.


Feather Icon Theme: Pinned Icons don't change color

Sidebar, Filter, etc.

Will this issue be updated?  😶🌫
This pair of icons should meet the repair requirements.
https:icones.js.org/collection/fe?s=star

I'm working on it. It's part of a larger update for Roam Studio coming this year.


-o isn't working properly

Describe the bug
-o is outputting (error: Found argument 'Kernel' which wasn't expected, or isn't valid in this context)
To Reproduce
Steps to reproduce the behavior:

run ❯ macchina --theme Beryllium -o Host Kernel Distribution Packages DesktopEnvironment Memory ProcessorLoad Shell Terminal WindowManager Battery
errors out with:

error: Found argument 'Kernel' which wasn't expected, or isn't valid in this context
Usage: macchina [OPTIONS]
For more information try '--help'

Expected behavior
proper output
Screenshots

macchina --doctor output
❯ macchina --doctor
Let's check your system for errors... Here's a summary:

We've collected 18 readouts, including 2 failed read(s) and 1 read(s) which resulted in a warning.
Readout "LocalIP" failed with message: Unable to get local IPv4 address.
Readout "Battery" failed with message: Dictionary does not contain information about the battery. Are you using a third-party battery?

1 of the 3 unsuccessful read(s) resulted in a warning:
Readout "Distribution" threw a warning with message: Since you're on macOS, there is no distribution to be read from the system.

System Information
You don't have to provide this information if you're not comfortable doing so, but it'll help us solve the issue a lot faster.

Operating System: macos
Terminal: iTerm2
Macchina's version

❯ macchina --version
macchina     6.1.8
libmacchina  6.3.5

Fixed, thanks for reporting!

@grtcdr
Fixed, thanks for reporting!

Just built from source, still seems to be broken

That's really weird, it seems to work for me (casing is different however, this issue should be addressed separately).

That's really weird, it seems to work for me (casing is different however, this issue should be addressed separately).

Comma seperation along with leaving no spaces work, thanks !

I forgot to mention that multiple arguments to a flag require a delimiter. Glad to hear it's working.


Add setting to change the value of the particles in the clientmods settings tab

I am opening this issue mostly for remembering to research about this. I would like to make this into a PR later. Furthermore, I also want to learn from this, so I would like if you don't do this yourself. >:)
Description: Add a setting in the client mods setting to change the value of the intensity of the particles.

I am opening this issue mostly for remembering to research about this. I would like to make this into a PR later. I also want to learn from this, so I would like if you don't do this yourself. >:)

I wrote half of it before realizing you wanted to do it, anyways I'll save my code cause I want to see what you come up with, gl

I wrote half of it before realizing you wanted to do it, anyways I'll save my code cause I want to see what you come up with, gl

Oh lol. It'll take time for me since I am not totally free for it. I'll have to plan my day out and see if I can manage today or tomorrow :D

Closing this Issue since I am not able to find time for it, and am busy. I don't want to disrupt your CSM's dev by keeping this issue open. Although I will be using your repository for future reference and I will be working on this locally for educational purposes :+1:

Closing this Issue since I am not able to find time for it, and am busy. I don't want to disrupt your CSM's dev by keeping this issue open. Although I will be using your repository for future reference and I will be working on this locally for educational purposes 👍

Got you, I'll go ahead and merge my settings code


ClrDatagridStateInterface needs page number in order to support GitHub style pagination

Summary
Our APIs follow the pagination principles as set out by GitHub API and we add RFC 5988 (Link header).
Currently ClrDatagridStateInterface page state has from, to and size.
If ClrDatagridStateInterface state also had the page number we could page with our APIs. 'https://my-api/users?page=2&size=100'

What is the change? ClrDatagridStateInterface state
Why should it go in Clarity? Support a common pagination style
Does this change impact existing behaviors? If so how? No, just extra field for page number
If this change introduces a new behavior, is this behavior accessible? Yes

Use case
Provide some specific details about how this change would improve your application and use case.
Adds abililty to use Datagrid pagination without having to change API pagination technique.
Examples
Are there any examples in other projects that demonstrate this feature or enhancement?
Many project use this pagination technique. Spring Boot is one example:
https://docs.spring.io/spring-data/commons/docs/current/api/org/springframework/data/domain/PageRequest.html#of-int-int-org.springframework.data.domain.Sort-
Workarounds
Is there a workaround for this feature at the moment, even if it is less than ideal?
Not sure if there is a workaround to get the page number when ClrDatagridStateInterface state changes?

CLosing I got this working by binding to [(clrDgPage)]


ADCS: Repeatedly poll and save ADCS telemetry

Now that the main bulk of ADCS command functions are complete, the Commissioning Manual requires that certain ADCS telemetry requests be repeatedly polled every few seconds over the course of between several minutes to one full orbit (see Sections 2.2 and 2.3 of the Commissioning Manual for more details).
In total there are 30 telemetry request functions that must be called in various combinations, of which there are 18 combinations which we care about for commissioning.
This function will need to take inputs:

Which functions to poll (either the specific functions to poll)
The period to poll them over (in seconds; either 1 second or 10 seconds)
How long to poll them for (either time in minutes/hours or else enable/disable; need to check what's most appropriate for the "1 orbit" condition)

Note that all of these inputs can be replaced with a single input: the numbered commissioning step to execute, in exchange for less freedom of operation.
What it will need to do:

Check the ACP Execution State telemetry to ensure that Time Since Iteration Start > 500 ms (see Section 2.3)
Call the appropriate telemetry request functions and load them into their structs (the Current ADCS State telemetry request may also be necessary)
Save the structs to memory (preferably with timestamp, which is another ADCS telemetry request or can be from the OBC)
Do this all periodically until the time of polling is complete, then downlink the data when next possible.

This downlink will be for ground processing, and varies depending on the commissioning step.

Thanks! That's a good point, actually -- this functionality could be pretty useful more generally.
As for the one-second interval... as far as I can tell, all the ADCS data should be logged within one second from the 500 ms Time Since Iteration Start so that it's all on the same frame.

What does "frame" mean in this case? Is it some sort of time partition/sampling the ADCS uses? Or more related to data partitioning?

The commissioning manual seems to use "frame" to refer to a single update of all the ADCS telemetry... so it would be a time partition.

It looks like there's a built-in method to log telemetry data to the ADCS SD card, which generates log files directly. I think it will be easier to do this for the ADCS purpose specifically, because then the ADCS can do it automatically when we request a given commissioning step.


Cannot update auto-publishing time

For one of our resources we have an auto-publication set to Wednesday at 12:00:

Try to update the time to 14:00 and save
I see the resource overview page
I go and open the auto-publication page
The time is still set to 12:00 and was not updated.

Note: if I update the day from Wednesday to `Tuesday, that is saved. It is only the time part that does not work.

Thank you Peter, I'll fix

Note: even if I remove the auto-publishing and re-enable it, I cannot set the timestamp to anything other than 12:00 😄


Accordion at-least-one-open option needed (feature request)

Need an option to make sure that at least one accordion group is open at a time. This is different from close-others which helps in keeping one group open. But that open group can be closed. An option to keep one group open always will be great.

I agree that this can be a useful feature. Feel free to send a PR this way.

For anyone intending on trying to tackle this, this should be user configurable for which accordion group remains open by default if all of them are closed, and default to the first group otherwise.

@wesleycho I am not sure if I follow you. Shouldn't this option prevent a group to be closed ?

Right, but on initialization, it should also pick which group is opened by default, no? Could we have a case where we would want all to be initialized as closed but on opening of one, one always remains open at all times?

So I can see that on initialization you tell it that you want to open the first / second / ... one. But what you mean with the "but on opening of one, one always remains open at all times". Don't follow that.

I want to tackle this, but I still don't know what to do.
So we put the at-least-one-open on the uib-accordion that will prevent a group the be closed if the rest are closed. We can agree on that.
Now, if the accordion starts with all closed, which one to open by default? Configure as a parameter to at-least-one-open? something along the lines:
<uib-accordion at-least-one-open="0">
  ...
</uib-accordion>

and also if there is no parameter, using a default one from the config?
After initialization, I don't think we need to worry about anything else. I mean, if you have group 3 open, you can't close it so we always force one group (any) always open.
Thoughts @wesleycho ?

I think both letting the user optionally specify which one to open to be open and using the first one if the user does not specify is appropriate behavior.

@icfantv "which one to open to be open" I don't follow that, but I think you agree with my idea?

Whoops.  I didn't check my grammar.  I meant to say that giving the user the option to select which accordion to be open, or if they choose not to, to pick a sensible default, is the correct behavior - so yes, agreeing with your idea.

:+1: Will do it probably tomorrow then.

Alright, @icfantv @wesleycho. Need opinion.
The problem with having one opened by default, is that at-least-one-open will be evaluated at startup when there is no group added. That means that there is no sane way (that I can think of) to implement it.  I would need lot of loops to check wether one has been opened, de-activate the checking after the initial ones are there...
A sane way (but would add more complexity) is to add a flag to the group which is going to be the default, something along the lines uib-default-group. Even that would need looping and extra checking so thinking it twice, it could not be sane either.
Ideas?

Wait, I might have an idea to do it right without extra directive.

I am not sure whether we should support this - the complexity would be immense, and we are already running into a similar issue with active tabs. It should be noted that this can be implemented on the users' side through a $watch

Closing due to lack of activity and there is a way for the user to do it on their end.  Additionally, this is not a feature that's high in demand.

@wesleycho Would you mind giving us extra details on how it could be implemented with a $watch? That would be very helpful. Thanks in advance!


implement easier counter commands

Resolves https://github.com/rGunti/FloppyBot/issues/293

Also solved #75


Training from scratch fails

I am trying to train from scratch but when I start training I get the following error
Traceback (most recent call last):
File "facenet_train.py", line 257, in
main(parse_arguments(sys.argv[1:]))
File "facenet_train.py", line 47, in main
pairs = lfw.read_pairs(os.path.expanduser(args.lfw_pairs))
File "/home/kalvik/facenet/facenet/src/lfw.py", line 64, in read_pairs
with open(pairs_filename, 'r') as f:
IOError: [Errno 2] No such file or directory: '../data/pairs.txt'

I commented out the validation part with lfw, since I didn't have the data. you can do that to, otherwise try giving absolute path

But isn't the pairs.txt supposed to be generated by the triplet selection function?

The pairs file is only used for evaluating the performance on LFW. It should be perfectly fine to just comment out the lines related to LFW evaluation.
If you want to use the LFW evaluation you could follow the steps here to download the pairs.txt file etc.

Could you please provide the link for pairs.txt. In section 5 you didn't provide a link.

The pairs file can be found here.

Perfect Thank you.


add infrastructure for time travel

Pull Request Test Coverage Report for Build<PHONE_NUMBER>6
Details

12 of 12 (100.0%)  changed or added relevant lines in 5 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.006%) to 99.677%

Totals

Change from base Build<PHONE_NUMBER>9:
0.006%

Covered Lines:
12552

Relevant Lines:
12553

💛  - Coveralls


Using the skeleton and components of sphinx-basic-ng

This is just an experimentation PR, meant to showcase the feasibility of swapping the skeleton.
Although this is not a good first step to switching to sphinx-basic-ng as the base, we should start with inheriting only the components. But nevertheless, I experimented with it a bit. This PR is an update on the work, and I will update the description shortly with the things done.

I'm going through all PRs and see if I can bring them up to date.
This has not been touched in 1+ year, is still in draft, and has a lot of conflict, so just in order to keep the list of open PR smaller I'm going to close it for now, but we can reopen if needed.
Thanks,


[WIP] Adds readme for web directory

Loosely related to #559.
WIP because GitHub is choosing not to render the svg tag.

@orta suggestions for the best way to display these colours? This is what I'm going for:

My only known hack here doesn't work http://www.tizag.com/htmlT/bgcolor.php - https://gist.github.com/orta/62febeb573df8e52965a5af7ddf5ba17

Images are the best way to go

          1 Warning

:warning:
PR is classed as Work in Progress

SwiftLint found issues\n\n#### Warnings

File
Line
Reason

Moya.swift
395
Opening braces should be preceded by a single space and on the same line as the declaration.

Moya.swift
390
There should be no space before and one after any comma.

Moya.swift
390
There should be no space before and one after any comma.

Moya.swift
390
There should be no space before and one after any comma.

Moya.swift
390
There should be no space before and one after any comma.

Moya.swift
390
There should be no space before and one after any comma.

Moya.swift
395
There should be no space before and one after any comma.

Moya.swift
395
There should be no space before and one after any comma.

Moya.swift
395
There should be no space before and one after any comma.

Moya.swift
395
There should be no space before and one after any comma.

Moya.swift
395
There should be no space before and one after any comma.

Moya.swift
188
Function body should span 50 lines or less excluding comments and whitespace: currently spans 59 lines

Moya.swift
390
Return arrow and return type should be separated by a single space or on a separate line.

Moya.swift
395
Return arrow and return type should be separated by a single space or on a separate line.

Moya.swift
389
Lines should not have trailing whitespace.

Moya.swift
394
Lines should not have trailing whitespace.

Moya.swift
420
Else and catch should be on the same line, one space after the previous declaration.

Moya.swift
188
Function should have complexity 10 or less: currently complexity equals 14

Moya.swift
520
File should contain 400 lines or less: currently contains 520

Observable+Moya.swift
62
Force unwrapping should be avoided.

  Generated by :no_entry_sign: danger

though the purple is massive, and the rest are the right sized, would recommend throwng in image optim too

Okay, i think this is ready for review 👍

Better fix all them swiftlintsssssssssssssssssssss

Yup, working on it now. It's also running more than once.

but nah, everything looks good from me.
One option for improvement though. You could do all the images at 2x size, then set the width/height to be half that. Then you have retina images. This is how I do it in most of my web projects.

with

instead of this at full size:

okaaaaay

It's a free TIL ;)

Yep - that should do it

I've opened https://github.com/Moya/Moya/pull/561 to address the SwiftLint problems.


run-length-encoding: Fix incorrect description

The description incorrectly stated that white spaces
would be passed through unchanged (details #613).

According to the consistency case, the spaces should go through unchanged.
   "consistency": {
      "cases": [
          {
            "description": "encode followed by decode gives original string",
            "input": "zzz ZZ  zZ",
            "expected": "zzz ZZ  zZ"
          }
      ]
   }


"Go to Sheet" API

Use Case
I've created a Dashboard Extension and would like the user to be redirected to different dashboard after clicking different sections in the extension.
Is it possible to expose the "Go to Sheet" API like the current "Tableau Go to Sheet Action"? Otherwise, is there any work around to achieve this?
Many thanks.

Hey @guangningyu. Yeah this is a great idea. We've thought about it in the past and have considered building it. One thing we're worried about is that, because the Extension lives on a dashboard, when you switched tabs you'd be killing the Extension and you would no longer be able to make API calls unless you also had a new Extension on the new dashboard. Do you think that would be a problem for you?

Hi @Kovner, thanks for your reply.
I don't think it would be a problem for me since redirection is the only thing I need. I can always add an extension/button in the target dashboard which can take me back to the previous one.
While I think you've made a very good point and it may cause problem for other use cases.
By the way, do you think there is any work around way to achieve the similar functionality?

Hi @Kovner, is it possible to include this request into the future development plan?
In my opinion, navigation between dashboards can be a quite common request. Taking my current work as an example, I have a dashboard used as first screen which shows the overall status of multiple work streams. The users would like to click on different numbers and be directed to corresponding dashboard which contains more detailed stats of that specific work stream.
What do you think?
Thanks again!


unable to get download file

public void getLabel(@QueryParam("orderItemsId") String orderItemsId,HttpServletRequest httpServletRequest,HttpServletResponse response) {
        String dataDirectory = httpServletRequest.getSession().getServletContext().getRealPath("/WEB-INF/files/label.pdf");
        Path file = Paths.get(dataDirectory);
        if (Files.exists(file))
        {
            response.setContentType("application/pdf");
            response.addHeader("Content-Disposition", "attachment; filename=\"label.pdf\"");
            try
            {
                Files.copy(file, response.getOutputStream());
                response.getOutputStream().flush();
            }
            catch (IOException ex) {
                ex.printStackTrace();
            }
        }
    }`

i am using @ApiResponses(value = {             @ApiResponse(code = 200, message = "Perfect") })
without content-Disposition header swagger is unable to sync output file and whereas is with this header is get a link in ResponseBody which also corrupts my file.
I am not sure what i am missing ?

You're not declaring what the response will be.  You need to add a @Produces annotation like such:
@Produces("application/pdf")

to your method.  Also specify your response type:
@ApiResponses(value = { @ApiResponse(code = 200, message = "Perfect", response = byte.class) })

Right now, you're saying "nothing is returned", so swagger-ui doesn't know what to do.

@RequestMapping(value = "/orders/labels", method = RequestMethod.GET, produces = MediaType.APPLICATION_OCTET_STREAM)
I am already using octet_stream. just have added response = byte.class . Still i am receiving corrupted pdf file. Also tried @Produces("application/pdf") even this didn't worked.
application-pdf-labels-orderItemsId=ff-blob-http---localhost-8080-f1e0b4e0-7b63-439b-8eeb-1a281524bfa1.pdf

OK if you submit an issue again, please give all the details the first time.
Sounds to me like you need to open this on swagger-ui but to confirm, what version are you using?

I am using springfox-swagger2 - version 2.5.0 with springfox-swagger-ui - version 2.5.0.

I think you'll have to open the issue in the springfox repo--it's not clear what version 2.5.0 maps to.  You can test using http://petstore.swagger.io and load your URL to see if the version is the issue.

it points to swagger-annotation version 1.5.9 and yes just tried using petstore.swagger.io

The question is about the swagger-ui version, not swagger-annotation.

Iissue persist if i populate my swagger.json in http://petstore.swagger.io (Not sure which version petsore link uses). Above screenshot was taken with petstore only.  Moreover springfox 2.5.0 uses swagger-ui version 2.2.6 (just confirmed)
PS - I have started shifting to swagger ui. I am also working to fix this issue

Any way around this ?


Adjust spacing of widget input-groups to better utilize limited space

This PR works in tandem with angularportal-js PR #514.

Coverage remained the same at 33.499% when pulling 552fcc94a657959adc000e14fb769d31bc1e4472 on thevoiceofzeke:widgets into 11940dc20263b9c818ddbe3153ee2b07b39d20d0 on UW-Madison-DoIT:master.

Coverage remained the same at 33.499% when pulling 6f11ea66496f2fc9761475fa25fc9448f2b5565f on thevoiceofzeke:widgets into 11940dc20263b9c818ddbe3153ee2b07b39d20d0 on UW-Madison-DoIT:master.

Coverage remained the same at 33.499% when pulling e49130012e6c34e44ef7a2ce713fd6ae147d47fd on thevoiceofzeke:widgets into 11940dc20263b9c818ddbe3153ee2b07b39d20d0 on UW-Madison-DoIT:master.

Coverage remained the same at 33.499% when pulling 5e7ac794390ad3fb2884c424375e0691ff74957e on thevoiceofzeke:widgets into 11940dc20263b9c818ddbe3153ee2b07b39d20d0 on UW-Madison-DoIT:master.


chore: remove search.js example

The example is violating the TOS of google.com
Fix #1557

LGTM. I was thinking we'd just replace the demo with something similar that shows grabbing page content. This example is also nice b/c it shows a couple of the APIs together (keyboard input and click).

@ebidel yeah we'll probably make another one with our own protocol viewer

What about DuckDuckGo?

https://chromedevtools.github.io/devtools-protocol/ is the protocol viewer @aslushnikov is referring to. I think it's probably safer to use that for an example than a site that our team does not maintain. It has an input with search results...so very similar to the original demo.


extend and server only part of the class

I wanted to use .extend function make part of Astronomy class secure, and to work only server side.
It does not seem to work fully. so far I have noticed two issues:

if I have field that is only defined server side, then in browser console there are warnings of type
Trying to set a value of the "secretField" field that does not exist in the "Falfsecret" class warnings I could of course ignore, but it is not good that browser even knows that such field exists. It must be sent somehow to the client after all. Otherwise, serverside updates for this field work.

if I add a transform function via extend only in server side, it does not work. If I move it to client/server side class definition, it starts to work.

My usecase is that I want to use Astronomy for simplicity, yet I want part of information only ever be available in server side. Each document that is retrieved from db, would have sensitive information cut and transformed, before document is sent to the client.
Maybe there is some other ways how to achieve this to still be able to use Astronomy?

When you add some fields on the server and you publish them to the client then all fields are sent. You would have to define the fields option in the find method. It will solve a problem. However, as I think about it, it would be good to do it automatically. I can make use of the Class.find() and Class.findOne() methods and add the fields option automatically.

I will test it out. However, why do you need a different transform function on the server? It's because of the problem in the point number 1?

besically I want part of the class to be operated only on the server. There would be transform function that takes several sensitive fields and produce non-sensitive field from them.
and opposite direction as well (though I am not sure how to do pre-insert or pre-update hook, yet.
In the documention .extend is mentioned in the context of having some stuff only on the server - if those fields are not actually only on the server, then what's the point of extend? what are the usecases you were thinking about when writing about server-only extends?
maybe I am wrong to us Astronomy this way at all? Maybe there is some other way how to do different server-only hooks, outside Astro classes?

The problem is not a class definition but publications. When you publish some documents even if they differ you have to make sure that you are not publishing them to the client. However I can automate that process.
You don't have to use a transform function for that. You can use event like afterInit. You can initialize compute values of some fields in this event.

how can I control publications within astronomy? I haven't thought about that part yet. And also where to do security checks.
I will try events like afterInit later tonight. That sounds good.

You don't control publications from the level of astronomy. You do it in Meteor using the Meteor.publish() method. There are many articles and tutorials about that.

I tried using event afterInit in server side extend. It was triggered, when I did server side upsert.
However, it seems that it is not triggered when I do findOne on client side.
Also, in documentation there is afterFind event mentioned (but not documented), but it was also not triggered on client side findOne.

Your problem is more complicated than it seems to be. It's because how Meteor works. You can have extra fields on the server and publish only subset of these fields to the client. However, it doesn't work the other way. You can't have extra fields on the client.
Ok, I lied here a little bit. You can have extra fields but they should be computed from the already published fields. I don't know if you can publish not existing fields. In theory it should be possible to create such workaround but I haven't tested it.
Let's take a look at the example:
Items = new Mongo.Collection('items');

Item = Astro.Class({
  name: 'Item',
  collection: Items,
  fields: {
    serverAndClient: 'string',
    computed: 'string'
  }
});

Meteor.methods({
  'add': function(doc) {
    doc.save();
  }
});

if (Meteor.isServer) {
  Item.extend({
    fields: {
      server: 'string',
    },
    events: {
      beforeSave: function() {
        this.set('computed', 'computed');
      }
    }
  });

  Meteor.publish('all', function() {
    return Items.find({}, {
      fields: {
        server: 0
      }
    });
  });
}

if (Meteor.isClient) {
  Meteor.startup(function() {
    Meteor.subscribe('all');
  });
}

As you can see, a field being computed has to be defined for both client and server. There is also the server only field. In the beforeSave method (on the server) you should compute a value of the computed field and save it into the database.

beforeSave I get, but how to do the same other way around? calculate value after reading server field from the database, then publish computed to the client, while keeping server field secret and only on server?
I was thinking about maybe doing it within Meteor.publish function, as I couldn't get it to work with afterInit or afterFind events (as described in previous comment), but I have not yet figured out how to do it.

You just have to publish all fields except the server side fields. It's possible in the publish method as I made in the example above. Pasting it here, one more time.
Meteor.publish('all', function() {
  return Items.find({}, {
    fields: {
      server: 0
    }
  });
});

It means that all fields will be published except the server field. I will try to do it automatically in future Astronomy releases.

yeah, not publishing secret field is simple. but how to do computed field, on server side, before returning data to client?

let me write a bit of pseudocode to ilustrate what I want to achieve:
client side:
Item = Astro.Class({
  name: 'Item',
  collection: Items,
  fields: {
    serverAndClient: 'string',
    computed: 'string'
  }
});

server side:
Item.extend({
    fields: {
      secret: 'string',
    }
  });

Item.transformAfterFind(doc) {
   doc.computed = encrypt(doc.secret);
}

You have to compute a field value on a document save, on the server. It's not possible to compute this field on find.

Oh, I though that that's what the transform() function is for?

You can use the transform function to add extra fields but not in your case and only on the client. The computed field has to be computed from the server field. It's not send to the client, so the transform function won't be able to compute a value for the computed field. If you think that the transform function will be executed in the publication, then you are wrong. The transform function will be executed only on the client. In the publication you have to return a cursor which does not call the transform function. Only fetching a document executes the transform function.
Summing everything up. The only way to solve it is adding an extra server only field on the server. Add the computed field for both client and server. Compute a value of the computed field during the save operation on the server. Publish all fields except server only fields. That's it. May sound complicated but it's the only proper way I know.

Yeah. I think I will then have to save the computed field in db on save. Second option is to leave the field completely out of db model, and create it ad-hoc by calling server method. that of course discards all reactivity for particular field.
anyway, thanks a lot for your help. Things are more clear now!

Yes. It's definitely better to choose the first option.
No problem :)

sorry for reopening this issue, but I have one more question.
In your example you write
Meteor.methods({
  'add': function(doc) {
    doc.save();
  }
});

I have tried it and it works well, when I do Meteor.call('add',itemInstance); from client side. But maybe it  would feel more natural to use, if it would be possibile to just call itemInstance.save() on the client and let Astronomy itself to figure out that there are server side only Class extension containing extra fields and event handlers, that need to be invoked as well?
Currently just doing save() completely ignores all serverside class extension, which did not feel immediately obvious (for me that is)

To understand it you have to know how calling the save method on the client works in Meteor. In fact, it's calling the Meteor method. It takes the plain JavaScript object as an argument. I can't send Astronomy class. And on the server it does not know that it's Astronomy document.
The workaround for that is overriding the Mongo.Collection class. Which is not a good idea but I've decided that I will do it in Astronomy 2.0. I also hope that MDG will introduce collection hooks that would not force me to override the Collection class.
In addition, it's not good pattern to use the save() method directly. Many Meteor developers do not recommend using it. It's hard to secure your application using allow/deny rules. The methods approach is much better.


Pre-compiled module is missing<EMAIL_ADDRESS>

Got this warning today:
WARN Pre-compiled module is missing<EMAIL_ADDRESS>Env details
INFO Tessel Environment Versions:
INFO t2-cli: 0.1.22
INFO t2-firmware: 0.2.0
INFO Node.js: 8.11.3

Are you installing the "usb" module directly in your Tessel project?

Yes, I would use that to connect to some USB devices.
Its working with the old fw and Node 6, it’s been a year I’ve had to touch
the code.
On 2020. Jan 7., Tue at 18:31, Rick Waldron<EMAIL_ADDRESS>wrote:

Are you installing the "usb" module directly in your Tessel project?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tessel/t2-cli/issues/1720?email_source=notifications&email_token=AADARBU6YJBQ2SYJLFECBNTQ4S4ANA5CNFSM4KC3R3GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIJUVNA#issuecomment-571689652,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADARBXM5RWGKEE45WPASCTQ4S4ANANCNFSM4KC3R3GA
.

--
_Nec
<EMAIL_ADDRESS>

Similar issue like this was https://github.com/tessel/node-usb/issues/231


Transform outgoing address to try to find a resource match

Fixes https://github.com/dotnet/aspire/issues/1919
SQL server:

Cosmos

Microsoft Reviewers: Open in CodeFlow

@davidfowl What's the deal with the request to http://<IP_ADDRESS>/metadata/instance?api-version=2020-06-01 by Cosmos?

@davidfowl What's the deal with the request to http://<IP_ADDRESS>/metadata/instance?api-version=2020-06-01 by Cosmos?

Not sure. I've seen DefaultAzureCredential make a ton of requests as it tries to resolve the managed identity. Not sure this is the same.
cc @Pilchie

Cosmos calls that to determine if it's running in an Azure VM and collect some diagnostic info if so.  See https://docs.microsoft.com/azure/virtual-machines/windows/instance-metadata-service and https://github.com/Azure/azure-cosmos-dotnet-v3/pull/3100


The divs aren't hidden and the stylesheets are leaking out of them overwriting each other

Why does this happen? I don't know send help

Turns out that <style scoped> is a lie


Maybe a more succinct way to describe my concern here is that two independent Go sources (ProgA does not have any dependency path through Go code with ProgB) can nonetheless have version bumps through a CUE only module, due to a common Go dependency and MVS.

Overall I don't share that concern. In fact the opposite. If the bump occurs because Go and CUE are co-versioned, then there is, or at least may be, a dependency. But we're going in circles here.

have been using MVS dependency management with both CUE and Go in the same repos, successfully, without commingling. (hof has a reference implementation that uses MVS without commingling, that includes sumdb calculations, without the SUMDB remote, no PROXY needed) There are many organizations with polyglot repositories that get along just fine without mixing language dependencies and systems.

I have been using MVS dependency management with both CUE and Go in the same repos, successfully, without commingling.

Not saying that is not possible, but there are certainly cases where not commingling will result in badness.
Do you have a documentation of how MVS in hof works?

I can't find my good docs at the moment... I think they are somewhere in a git history
The code is pretty much limited to this directory: https://github.com/hofstadter-io/hof/tree/_dev/lib/mod

at least may be, a dependency

It's the case where there is not direct dependency, or thus an overlap between two MVS DAGs in GO, but only because CUE has bridged this gap by mixing the dependencies, that they become intertwined ( creates an MVS overlap ) A a common dependency at two versions (a leaf in the MVS DAG) must now be the same, for two independent DAGs (as far as MVS is concerned).

I understand the practical reasons for wanting to use a go.mod directory, but that effectively makes Cue a satellite of Go, now and forever. It sends the message that a Cue developer is a special specialization of Go developers; and that unless you are a Go developer, you are not a first class citizen in the Cue ecosystem.
 But it is quite different from the stated goal which is to make a universal data language.
In short there are far-reaching implications in terms of the positioning of the project, and defining its indended audience.

To echo @shykes's point, were I not already involved deeply with Go development, if I came to CUE freshly and saw this use of Go tooling, I would turn away. If, instead, I found CUE using, say, Node.js tooling, or Java tooling, I would turn away, thinking, "I came for the configuration and data manipulation, not for the integration with a single ecosystem that I don't want to use."

@myitcv it doesn't seem like the proposal as written extends Go Modules to avoid the social behavior we see emerging in the Go ecosystem, in response to the limitations of Semantic Import Versioning and the constraints they put on Module authors.
I'm talking specifically about the emerging behavior where more and more module authors are refusing to ever release a v1 of their module, due to challenges with tooling and not because they intend to always offer an unstable API. This pattern of behavior has emerged from those authors feeling like the SIV provides too much of barrier and support burden to use semantic versions.
Is that a behavioral pattern, and subsequently a sociotechnical risk, that we're comfortable also adopting here?

Consider also this article: https://github.com/KateGo520/Hero/issues/1
A lot of go users find go modules to be difficult to use and polarizing. Are you sure this is the right example to follow?

Everyone: please note that this proposal DOES NOT require Go tooling to be installed.
Not to say there are other points, which we are addressing in a new proposal.
@theckman: as I said earlier, I'm not wedded to semver at all, and even don't like it all that much. But one needs to have something, no? Any alternatives that do not suffer from these issues?  @verdverm is suggesting another approach using MVS, which I reckon in your opinion would suffer from the same issues?
We need a hermitic reproducible way to manage modules. Also, it is imperative to unique packages versions across imports. Neither SAT solving nor a Nix-style approach seem to solve this. MVS is a good solution for this. I'm not aware of any other algorithm that achieves this.

So in advance of the new/revised proposal, here is a description of the commingling issue. The large groups/yellow blocks represent VCS repositories. At the top is a repo (Base) that has both Go and CUE files. The CUE files define, say, the set of options or a declarative detailed API description of the Go package in that module.
Repo G isn't technically necessary here, but the graph renders more nicely with it.
At the bottom is a repo (MyApp) that creates a Go app from the top packages, and also uses the CUE definition in that repo to define its own API, verify inputs what have you. It, however, also mixes in definitions from repo C.
Now C, in turn, requires v1.1.0 of the CUE package. Let's assume we are resolving Go and CUE independently. We now have a problem at hand: the CUE definitions resolves to a newer version than anticipated, and thus will indicate a more permissive API than can be expected from the Go app created in MyApp, and thus will not correctly represent the features and API.
This issue can be avoided by ensuring that there is only one version of repo Base. Commingling Go and CUE resolution, in this instance, would solve that.
To emphasize, this is not specifically a Go+CUE issues but really an issue for any language or system where CUE is included as a description of an API or other form of contract. The general solution seems ensuring that any resolution within one repo will select only one version of a repo across languages, satisfying all other constraints.
There may be other ways to ensure that repos are unique across different languages. But co-resolving versions seems like a possibility. We have some ideas how to make this more pluggable and even allow resolving multiple languages at once (within limits). But we would love to hear about other solutions that can solve this issue.
@verdverm: can you explain why you think this is not an issue and how you would solve this in your (independently resolved) MVS approach?

@theckman: as I said earlier, I'm not wedded to semver at all, and even don't like it all that much. But one needs to have something, no? Any alternatives that do not suffer from these issues? @verdverm is suggesting another approach using MVS, which I reckon in your opinion would suffer from the same issues?

@mpvl So the first thing I think I should call out is that Go Modules, as it exists today, marries you to semantic versioning, and makes it extremely difficult to use any other versioning scheme (including calendar based). I think we can address that issue, and the one I raised earlier around the risks of people not releasing v1 modules, by either making Semantic Import Versioning (SIV) optional or removing it entirely. This issue, which is a proposal for optional SIV in Go, may help provide more context: https://github.com/golang/go/issues/44550
The idea would be to not include the major version in the name of the Module (e.g., example.org/module/v2), and to instead make use of the declared version in the .mod file.

I'd agree with @theckman, I'm not a fan of the required /vX in the import path which does not match the underlying code base. I think that if a module wants to support importing 2 major versions, it could do this within its own codebase using folders and ensuring it works on their own.
MVS could certainly support using SemVer or Calendar versions, per module, such that both are supported and a module is free to choose either. A module would not be able to use both or switch once it is chosen (I believe).
@mpvl A picture is worth a 1000 words. This is definitely a problem, an in this case, commingling would force my upgrade to Go v1.1.0. Knowing that SemVer is not strictly adhered to, I could be forced into making changes to my own code. This would be more irritating when it happens in downstream dependencies and my Go only program is updating a Go only module, but some other dep breaks things, because it's dep chain was forced to bump versions due to a CUE only module. That being said, these situation look to only happen when choosing versions initially or updating direct dependency versions, and the downstream impacts therein.
My issue is for Go only users (or other languages) who have no attachment to CUE. In this case my own app depends on a (series of) module(s) in some commingled MVS chain, and that a Go version is now required to increase because of a CUE only module, far removed. Supporting N languages together seems like it would result in compounding dependencies, as one multi-language module can force me to version align with and download a huge number of modules in N other languages which are irrelevant to my own application. I will make a picture as well, so that this is clearer.
Generally I think mixing in more languages is going to make this problem even worse and that CUE should only worry about CUE and not N languages. If there are compatibility issues, the user should be left to deal with this. CUE forcibly injecting itself into other languages dependency systems is seen as not playing nice with other ecosystems. I think this will hurt adoption (as @seh related). Personally I will have to think about how I would proceed, with the current choices being maintaining a fork to disable this behavior or replacing with something else. I can say I will not use a CUE that has this approach to commingled dependency management.
RE: Go tooling, I think we are talking more generally than in simply requiring the Go binary to be available. I consider using the Go libraries from a CUE binary to achieve the same thing using Go tooling ( reading/writing go.mod, parsing Go code to infer used imports, talking to GOPROXY/GOSUMDB, and in general inspecting and processing Go for dependency reasons).

We need a hermitic reproducible way to manage modules. Also, it is imperative to unique packages versions across imports. Neither SAT solving nor a Nix-style approach seem to solve this. MVS is a good solution for this. I'm not aware of any other algorithm that achieves this.

Leaving aside Flakes, a Nix style approach would require that every requirement of a package be made explicit. For instance, my current CUE-related workspace is built using an expression that includes:
    cue = { stdenv, buildGoModule, fetchFromGitHub, lib }:
    buildGoModule rec {
      pname = "cue";
      version = "v0.3.0-beta.6";

      src = fetchFromGitHub {
        owner  = "cuelang";
        repo   = "cue";
        rev    = version;

        sha256 = "02fafsl5krvkxc9p5acz38hb9zbp1gz5zvz1cvgr7g5dd9h2j8fv";
      };

      vendorSha256 = "0drnmf8gfj3k74z5zh0032g5kg3nzji5ji1m3qbxfzisp5cvba7m";

      #doCheck = false;

      CGO_ENABLED=0;
      buildFlagsArray = ["-ldflags='-s -w -X cuelang.org/go/cmd/cue/cmd.version=${version}'"];
    };

(oops! I need to update to 0.3.0! :smile: )
Specifically, the rev, sha256 and vendorSha256 fields specify a revision in Git, as well digests of the source code and the go modules fetched to satisfy the go.sum there. How Nix interacts with different module ecosystems varies, but in the Go case, it delegates versioning to the Go system.
On the one hand, I think Nix does provide hermetic, reproducible module collection. If not, maybe I don't understand what you're envisioning as that requirement.
The irritation with Nix (and honestly, it's not a huge issue) is that collecting those digests is frustrating. The pragmatic approach, generally, is to put in a fake digest (nixpkgs includes a lib.fakeSha256 constant...) and then copy the real digests in from the build errors. It's a kind of poor man's TOFU system.
The other thing about Nix is that dependencies tend to be resolved distro-wide. In some way's that's reasonable: no OS distribution really ships multiple versions of glibc, say. Nix is nice in that different versions of the distribution can co-exist, and different applications can use different glibcs if needed.
For language-based distribution systems, the domain of version resolution is different. CUE, I think, wants an application domain, right? Everything is resolved together, and the way unification works puts me in mind of a hypothetical issue with old package.json style NPM resolution: if my app has an indirect dependency to another package via two paths, it's possible the code gets a value from one place (and therefore version X of my dep) and passes it to another that uses version Y. I think it's a subtle enough problem that it may have occurred and not been recognized.
My point is that, a whole-application SAT solution (and here I think of Bundler from Ruby), can solve this issue well. People complain of "dependency hell" (and build NPM), but the upshot is that versions need to be resolved at the application domain in Ruby, and therefore you really can only have one version of any module. Bundler produces a lockfile with exact versions (which are held as source packages rather that VCS revisions), which also gets a lot of the way to reproducibility, and solves the fiddly and error prone process of resolving exact versions manually.

With regards to semver: I think that's a really interesting point for CUE. As defined, semver talks about changes in interface, which I understand as being related to the functions a module exports, basically. CUE is a different kind of language than the ones semver was developed to describe, and I wonder if it really is appropriate to describe a CUE module in terms of its interface.

@nyarly yes, nix is indeed hermetic, but it indeed doesn't solve the other problem you mention of enforcing only a single version different instances of the same import.
Semver is still somewhat relevant to CUE, but as long as the import paths distinguish major versions, one could argue that only minor versions are relevant. Then again, the same could be argued for programming languages.
I don't find SAT solvers give satisfactory results and dependency resolution at scale is a nightmare.
CUE being different also opens up possible solutions. It requires all imports of the same package to be the same version, but only barely. For instance, in the above problem, we could expect users to manually "downcast" (#454) versions of there is a possibility for discrepancies. But this doesn't obviate the need to align versions used in one language with an import in CUE (although in this case it would be a simpler exercise).
Also, the commingling issue is to some extent also something that is particular to CUE, or any language that is included alongside some system to describe its API for that matter. Some of the solutions may also come from guidelines and best practices. For instance, the above issue would more or less resolve itself if all CUE that depends on a commingled setup is itself managed within the same package management system. These could still depend on independent CUE repositories, but not vise versa. I'll try to write something up about this, but I believe with such restrictions it actually becomes doable to achieve consistency in a multi-lingual setup.
@verdverm we certainly don't plan to offers integrations for all languages. But I think a solution should be pluggable enough to make such integrations relatively straightforward.
@theckman Yes I know the Go modules depends on Semver. Although, I'm not a fan of semver, I am a fan of standardization, and ultimately it doesn't matter that much. The point of people sticking with v0.x is an interesting one, though. Hard to avoid if one allows for a standard of specifying a ramp-up of backwards incompatible changes, but probably having such a mechanism have more of a wall-of-shame feel to it would help. I think in the end eco-system incentives are a good way to go there, though: automatically detect that the semver guidelines are followed and demote packages that are not at v1 or break semver rules. I know this is not fool proof either.

BTW, for those unfamiliar, Rich Hickey's Spec-ulate is an excellent talk about this topic: https://www.youtube.com/watch?v=oyLBGkS5ICk. An interesting thing to keep in mind when watching this is that CUE can actually be used as a "spec" definition, describing and validating backwards compatibility. This gives an idea how CUE could be different.

This seem relevant and related: https://github.com/golang/go/issues/36460 (cmd/go: lazy module loading, changes to how Go will process and manage transitive dependencies)

This issue has been migrated to https://github.com/cue-lang/cue/issues/851.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


Removed unnecessary await keywords

Hi,
I think that async/await keywords unnecessary.
removing them can provide minor performance improvement

@emrekara37  Thank you for your PR. I shall check and merge if everything is okay.


add_field_layer Example Error

I'm modeling some code off of the example add_field_layer, and I've been running into some issues. I decided to run the example file (minus actually adding the fields) to see if it was on my end or not. Turns out that it produces the same error that I was getting with my code (I saved the code as test.py):
updateUserProfileDisabled  - attribute not implemented in Portal class.
No JSON object could be decoded
Traceback (most recent call last):
  File "C:\Python27\ArcGISx6410.3\Lib\site-packages\Pythonwin\pywin\framework\scriptutils.py", line 326, in RunScript
    exec codeObject in __main__.__dict__
  File "C:\Users\shihadehm\Desktop\test.py", line 29, in <module>
    initialize=True)
  File "C:\Python27\ArcGIS10.3\Lib\site-packages\arcrest\agol\layer.py", line 123, in __init__
    self.__init()
  File "C:\Python27\ArcGIS10.3\Lib\site-packages\arcrest\agol\layer.py", line 137, in __init
    for k,v in json_dict.iteritems():
AttributeError: 'str' object has no attribute 'iteritems'

The line of code throwing the exception (Line 29) is
fl = FeatureLayer(
            url=url,
            securityHandler=agolSH,
            proxy_port=proxy_port,
            proxy_url=proxy_url,
            initialize=True)

I tried using multiple feature layers with the same results, and my URLs are definitely correct. Any help is much appreciated.

@mshihadeh what version of ArcREST are you using?


Todo Improvement

Input what items we have in inventory, then the todo will automatically mark what items has been collected.

Hello!
Is this currently in progress or something you want help with? I would gladly take this

Hey. I'm available to help too.


feat: (POC) handle multiple paths

Description:
This PR is a Proof Of Concept changing the behaviour of gitleaks cli to match pre-commit run and pre-commit run -a.
Close #657.
The PoC consists on adding a new option to the gitleaks cli, named --all-files. If the user uses this flag, gitleaks will create disregard any other flag and will create a "no git scanner". The list of files to scan will be created from the --all-files argument and all the arguments at the end.
Example:
We execute gitleaks --all-files file1 file2 file3 ... fileN.

Gitleaks will disregard any argument when using --all-files
A no git scanner will be created to scan each file individually
Gitleaks will receive file1 and [file2 file3 ... fileN]. We will create the list of files by concatenating both things [file1 ... fileN]

Things to consider about this PoC that should be improved:

The use of the reflect library to compare structs
Handle what happens if one file of the list does not exist, cannot be open, ...

Checklist:

[ ] Does your PR pass tests?
[ ] Have you written new tests for your changes?
[ ] Have you lint your code locally prior to submission?

Thank you for opening this PR. Unfortunately I will not be merging since v8 has been released and was a rewrite of the project.


Effects eat exceptions

We recently upgraded our version of ngrx from 2 to 4 and noticed that exceptions no longer bubble out of effects.  Is this expected?
We have a root ErrorHandler that remotely logs any exceptions we get in our client, but it no longer triggers because ngrx effects catch everything in order to log errors to the console.  The exception still breaks the effect, so it's not like it's being handled in a recoverable fashion, so why doesn't it re-throw?
We can work around it, for sure, but it means we need to update all of our effects to catch errors in order to log them one-by-one instead of generically using a catch-all ErrorHandler.  Is there some other way we should be doing this?

This must be a defect. It has been reported earlier. I am surprised no one is looking into it !!

This post should be in ngrx 4. Maybe someone will look..


Adjust position of module?

Hello,
currently the modul is placed at the left side.
Is there a possibility to place the module in the center of the screen or anywhere else?
Regards, Christoph

Hi,
I achieved this by adding position: "top_center" after module: "MMM-GoogleMapsTraffic"
Thanks
Steve

position: 'bottom_right',

take a look here, pretty helpful
https://forum.magicmirror.builders/topic/286/regions


New package: Oracle.JDK.19 version <IP_ADDRESS>

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[ ] Is there a linked Issue?

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

This package installs and launches normally on a Windows 10 VM.

Verify:

The package doesn't add itself to the system path.


Fixing using NearestNeighborSearcher instead KNearestSearcher

It seems after the refactoring NearestNeighborSearcher was not replaced by KNearestSearcher.
I wonder why the tests did not pick this up.

Thanks! I think I released breaking changes in GeoStatsBase.jl using a patch version :/ That is why the downstream packages were broken without notice. Merging it right away and submitting a patch release.


Dice loss branch

Added dice loss
Brief Description of the PR:
dice loss can be used as a loss function for training segmentation models
Fixes # (issue)
Type of change

i have added the dice loss file to tensorflow_addons/losses
i have added the test file for dice loss to tensorflow_addons/losses/tests

How Has This Been Tested?
i have used unittest to test this code
*

@googlebot i signed it

/cc @yarri-oss I think we already have a dice loss working also on 3d volumes /cc @fyangf :
https://github.com/tensorflow/models/blob/master/official/vision/beta/projects/volumetric_models/losses/segmentation_losses.py
So in the next release of:
pip install tf-models-official
We also had another dice loss at:
https://github.com/tensorflow/models/blob/master/research/object_detection/core/losses.py

/cc @yarri-oss I think we already have a dice loss working also on 3d volumes /cc @fyangf :
https://github.com/tensorflow/models/blob/master/official/vision/beta/projects/volumetric_models/losses/segmentation_losses.py
So in the next release of:
pip install tf-models-official
We also had another dice loss at:
https://github.com/tensorflow/models/blob/master/research/object_detection/core/losses.py

so should i close the pull request

The dice loss under https://github.com/tensorflow/models/blob/master/official/vision/beta/projects/volumetric_models/losses/segmentation_losses.py is tailored for 3D volumetric model use cases. I'm ok with having a more generic implementation outside of that.

@gamenerd457 Please keep this open in the meantime as we would discuss this recurrent topic internally.
Another good place where this PR could be submitted is https://github.com/tensorflow/models/tree/master/official/vision/keras_cv/losses

i opened this pr in https://github.com/tensorflow/models/tree/master/official/vision/keras_cv/losses but it is still  not reviewed

As https://github.com/tensorflow/models/pull/10249 was rejected cause:

keras_cv is ramping down in this repo now. We do not accept changes to this folder.

we are talking with @qlzh727 and @fchollet about the roadmap.

@gamenerd457  If you are still interested in this please comment at https://github.com/keras-team/keras-cv/issues/296#issuecomment-1096952066

A PR now is at https://github.com/keras-team/keras-cv/pull/371 you can contribute there. Thanks


[Perf] Enhance formatting of fields in perf report

This commit applies few PrettyTable() formatting directives to improve the readability and formatting of few columns in the perf-report.
Prior to this fix, the report would look like:
agurajada-Linux-Vm:[600] $ ./scripts/perf_report.py \
                 --file /tmp/test.sh.run-all-client-server-perf-tests.out

    **** Performance comparison for NumClients=5, NumOps=5000000 (5 Million) ****
+--------------------------+-------------------+----------+-------------------+----------+
|         Run-Type         | Server throughput | S:%-Drop | Client throughput | C:%-Drop |
+--------------------------+-------------------+----------+-------------------+----------+
|  Baseline - No logging   | ~100.14 K ops/sec |   0.0    |  ~22.91 K ops/sec |   0.0    |
|   L3-logging (no LOC)    |  ~99.77 K ops/sec |  -0.36   |  ~22.69 K ops/sec |  -0.96   |
| L3-fast logging (no LOC) | ~102.04 K ops/sec |   1.89   |  ~24.74 K ops/sec |   7.98   |
|  L3-logging default LOC  |  ~99.12 K ops/sec |  -1.02   |  ~22.46 K ops/sec |   -2.0   |
|    L3-logging LOC-ELF    |  ~95.97 K ops/sec |  -4.16   |  ~22.86 K ops/sec |  -0.23   |
+--------------------------+-------------------+----------+-------------------+----------+

Updated report with these changes will look like:
    **** Performance comparison for NumClients=5, NumOps=5000000 (5 Million) ****
+--------------------------+-------------------+----------+-------------------+----------+
| Run-Type                 | Server throughput | Srv:Drop | Client throughput | Cli:Drop |
+--------------------------+-------------------+----------+-------------------+----------+
| Baseline - No logging    | ~100.14 K ops/sec |  0.00 %  |  ~22.91 K ops/sec |  0.00 %  |
| L3-logging (no LOC)      |  ~99.77 K ops/sec | -0.36 %  |  ~22.69 K ops/sec | -0.96 %  |
| L3-fast logging (no LOC) | ~102.04 K ops/sec |  1.89 %  |  ~24.74 K ops/sec |  7.98 %  |
| L3-logging default LOC   |  ~99.12 K ops/sec | -1.02 %  |  ~22.46 K ops/sec | -2.00 %  |
| L3-logging LOC-ELF       |  ~95.97 K ops/sec | -4.16 %  |  ~22.86 K ops/sec | -0.23 %  |
+--------------------------+-------------------+----------+-------------------+----------+

Note:

Run-type is left-aligned.
%age-drop value is properly justified, with '%' units-specifier.

FYI; @gregthelaw - for the review. Am checking this in ...


[Feature Request] When the type of el-date-picker is daterange, can I quickly switch the year and month by clicking on the year and month? This feature is supported when the type of el-date-picker is date.

Existing Component
是
Component Name
el-date-picker
Description
When the type of el-date-picker is daterange, can I quickly switch the year and month by clicking on the year and month? This feature is supported when the type of el-date-picker is date.


Save SVG to path

I want to create many figures and save them to a folder. This does not seem to be possible currently, without interacting with the IGV browser and manually saving SVGs. Could saving SVGs to a path be implemented? Thanks!

This is difficult, in fact I haven't figured out a way to implement this yet.  The SVG is created in the browser,  I don't know of a way to fetch it from python.   It might be possible, but its not easy, if you have any suggestions please post them here.


Removing reference to "line of best fit" in initial rushing yards over expected model

As outlined by NeuroNisl in the Discord:

The graph shows the “line of best fit” between Actual Rushing Yards and Expected Rushing Yards. Those running backs that outperformed the model’s expectations (that is, more actual rushing yards than expected rushing yards) fall below the line, while those that underperformed the model are above the line of best fit.
Wouldn't this only be true if you plotted the linear function y=x instead of the linear regression result which doesn't necessarily need to be this function?

He is correct. It is not a correlation (such in the linear regression chapter) and this needs a slight edit to remove the terminology.

Completed in the book's page proofs.


Bundlr support

Hey,
It would be great to provide a Bundlr integration meaning peoples bundles are guaranteed to get on chain. AR and SOL are already supported.

Will do

Yep. SOL is a lot friendlier in that manner.
FYI we lowered the required confirmations on an AR transfer to 20.
If you need any support on the integration. Feel free to message me on Discord.

What's the pricing?

I was going to start on bundlr. I looked through your js client and it seems I need to:

Provide a way to fund a bundrl account - can this be done with any of the supported currencies?
Post data items - upload folder is sequence of data item posts with logging and manifest
Withdraw bundlr funds

It looks like there is now a rust client for the bundlr network so closing this.


Please mention which conditions are the default on Linux

It would be helpful for people hosting on Linux

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 945c28b7-6cc7-d4ed-3b96-a401eef42f59
Version Independent ID: eb45fbf9-29fb-c67b-2f50-454380a502dd
Content: Data Protection key management and lifetime in ASP.NET Core
Content Source: aspnetcore/security/data-protection/configuration/default-settings.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

I did the legwork for someone else to add this to the docs. Surfacing this info to the public would be a very welcome addition :)
https://github.com/aspnet/AspNetCore/blob/master/src/DataProtection/DataProtection/src/Repositories/DefaultKeyStorageDirectories.cs#L24

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog manageable. If you believe there is a concern which hasn't been addressed, please file a new issue.


fix(#11283) bug with switch case

What kind of change does this PR introduce?
bugfix
fixes #11283

Did you add tests for your changes?
yes

Does this PR introduce a breaking change?
no

What needs to be documented once your changes are merged?
nothing

Thanks