Documentation unclear about sidecar customization (could be bug in injector)

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description

As you know there are annotations to control the pods memory and cpu requests and limits. We use them
For the specific purpose of optionally setting up a prestop hook for the proxy, we also embed using the methodology discussed in https://istio.io/latest/docs/setup/additional-setup/sidecar-injection/#customizing-injection

{{- define "mesh-container" }}
{{- if .Values.mesh.enabled -}}
{{- if .Values.mesh.prestop.enabled -}}
- name: istio-proxy
  image: auto
  lifecycle:
    preStop:
      exec:
        command: [ "{{ join "\",\"" .Values.mesh.prestop.commands }}" ]
{{- end }}
{{- end }}
{{- end }}

That works great
Except then we found effectively that memory and cpu annotations are ignored. My guess is the template takes precedence and sees no clause so just uses default values?

Version
istio 1.20.2

Additional Information
i have two concerns

This isn't very clear in the docs. There's comments on the same page like: "Additionally, certain fields are configurable by annotations on the pod, although it is recommended to use the above approach to customizing settings. Additional care must be taken for certain annotations:

If sidecar.istio.io/proxyCPU is set, make sure to explicitly set sidecar.istio.io/proxyCPULimit. Otherwise the sidecar’s cpu limit will be set as unlimited.
If sidecar.istio.io/proxyMemory is set, make sure to explicitly set sidecar.istio.io/proxyMemoryLimit. Otherwise the sidecar’s memory limit will be set as unlimited."
but it doesn't mention they won't even WORK in this case.

creating a complete alternative template would be pretty hard to do and preserve function. In fact arguably it would be a deployment nightmare.

Given the above, is this

A bug in istio injector not merging the two correctly (there's no conflict per se)
A bug in the docs, and the only option is to drop the prestop or try to copy everything into a new template, which is rarely feasible.

I'd be happy to try to submit clearer docs, but it's not obvious to me the current behavior is as designed.

hmm...

hello

Hey, sorry for the (very long) delay here. I don't really get what you tried.
Where is the define mesh-container specified? Where is mesh-container used?
How are referencing the custom template?
We have a unit test around this I tried to reproduce in, but I couldn't grok how to set it up:
diff --git a/pkg/kube/inject/testdata/inject/custom-template.iop.yaml b/pkg/kube/inject/testdata/inject/custom-template.iop.yaml
diff --git a/pkg/kube/inject/testdata/inject/custom-template.iop.yaml b/pkg/kube/inject/testdata/inject/custom-template.iop.yaml
index f68cf9accb..efb2586a3a 100644
--- a/pkg/kube/inject/testdata/inject/custom-template.iop.yaml
+++ b/pkg/kube/inject/testdata/inject/custom-template.iop.yaml
@@ -8,30 +8,15 @@ spec:
     sidecarInjectorWebhook:
       templates:
         custom: |
-          metadata:
-            annotations:
-              # Disable the built-in transformations. In the future we may want a template-level API
-              prometheus.istio.io/merge-metrics: "false"
-              sidecar.istio.io/rewriteAppHTTPProbers: "false"
-              foo: bar
-          spec:
-            containers:
-            {{- range $index, $container := .Spec.Containers }}
-            - name: {{ $container.Name }}
-              env:
-              - name: SOME_ENV
-                value: "true"
-              - name: SOME_FILE
-                value: /var/lib/data/foo.json
-              volumeMounts:
-              - mountPath: /var/lib/data/foo.json
-                subPath: foo.json
-                name: some-injected-file
-            {{- end}}
-            volumes:
-            - name: some-injected-file
-              downwardAPI:
-                items:
-                - path: foo.json
-                  fieldRef:
-                    fieldPath: "metadata.annotations['foo']"
+          {{- define "mesh-container" }}
+          {{- if .Values.mesh.enabled -}}
+          {{- if .Values.mesh.prestop.enabled -}}
+          - name: istio-proxy
+            image: auto
+            lifecycle:
+              preStop:
+                exec:
+                  command: [ "{{ join "\",\"" .Values.mesh.prestop.commands }}" ]
+            {{- end }}
+            {{- end }}
+            {{- end }}
\ No newline at end of file
diff --git a/pkg/kube/inject/testdata/inject/custom-template.yaml b/pkg/kube/inject/testdata/inject/custom-template.yaml
index 0d1914bacf..5bfbb54e3b 100644
--- a/pkg/kube/inject/testdata/inject/custom-template.yaml
+++ b/pkg/kube/inject/testdata/inject/custom-template.yaml
@@ -9,7 +9,8 @@ spec:
   template:
     metadata:
       annotations:
-        inject.istio.io/templates: "custom"
+        inject.istio.io/templates: "sidecar,custom"
+        sidecar.istio.io/proxyCPU: "123m"
       labels:
         app: hello
     spec:

is what I had, if it helps

The "sidecar.istio.io/proxyCPU is set, make sure to explicitly set sidecar.istio.io/proxyCPULimit" part makes sense and I agree could use some improved docs BTW

We have a chart to deploy our applications.
It always specifies proxy cpu/memory requests and limits.  Similar to

sidecar.istio.io/proxyMemory: {{ .memory | quote }}
sidecar.istio.io/proxyMemoryLimit: {{ .memory | quote }}
sidecar.istio.io/proxyCPU: {{ .cpu | quote }}
sidecar.istio.io/proxyCPULimit: {{ .cpu | quote }}

In addition if mesh.prestop,enabled is set by the user, this part is injected

{{- define "mesh-container" }}
{{- if .Values.mesh.enabled -}}
{{- if .Values.mesh.prestop.enabled -}}
- name: istio-proxy
  image: auto
  lifecycle:
    preStop:
      exec:
        command: [ "{{ join "\",\"" .Values.mesh.prestop.commands }}" ]
{{- end }}
{{- end }}
{{- end }}

This follows the advice in https://istio.io/latest/docs/setup/additional-setup/sidecar-injection/#customizing-injection
(I'm open to other methods, the desired outcome is if some triggers exist in the helm values deploy file the proxy container adds a pre-stop hook.)

This works perfectly!
Except now the proxy cpu and memory annotations are ignored. You basically set proxy's cpu and memory to unbounded.
I believe this is an order of merge operations from the default injection template combined with this one provided in the deployment pod template spec

Part of the confusion in this ticket @howardjohn is we have two completely different templates. We have the go-template injection by the injection controller and we have the helm template that we as application deployed use. I hope that corrects the confusion.

To further clarify. Here's fhe first part of our template pod template spec. I've edited it down
  template:
    metadata:
      annotations:
        app.kubernetes.io/version: {{ required ".Values.image.tag is required " .Values.image.tag | quote }}
        deployed: {{ date "20060102150405" now   | quote }}
      {{- range $name, $value := .Values.annotations }}
        {{ $name }}: "{{ $value }}"
      {{- end }}
      {{- include "mesh-annotations" . | nindent 8 }}
      labels:
      {{- range $name, $value := .Values.labels }}
        {{ $name }}: "{{ $value }}"
      {{- end }}
        identity: {{ include "identity.service-account" . | quote }}
        version: {{ include "identity.version" . | quote }}
        app: {{ .Release.Name }}
      {{- end}}
        {{- include "mesh-labels" . | nindent 8 }}

This isn't too interesting by itself, but it includes "mesh-labels" and "mesh-annotations:
"mesh-labels" determines sidecar injection
{{- define "mesh-labels" }}
{{- if .Values.mesh.enabled }}
sidecar.istio.io/inject: "{{ .Values.mesh.enabled }}"
{{- end }}
{{- end }}

while "mesh-annotations" is the resource stuff (I'll redact most of it)
sidecar.istio.io/proxyMemory: {{ .memory | quote }}
sidecar.istio.io/proxyMemoryLimit: {{ .memory | quote }}
sidecar.istio.io/proxyCPU: {{ .cpu | quote }}
sidecar.istio.io/proxyCPULimit: {{ .cpu | quote }}

Finally mesh sidecar container definition. This is ONLY injected if mesh.prestop.enabled is on, otherwise we let the injection controller handle this.
{{- define "mesh-container" }}
{{- if .Values.mesh.enabled -}}
{{- if .Values.mesh.prestop.enabled -}}
- name: istio-proxy
  image: auto
  lifecycle:
    preStop:
      exec:
        command: [ "{{ join "\",\"" .Values.mesh.prestop.commands }}" ]
{{- end }}
{{- end }}
{{- end }}

So IIUC the first part is the Deployment spec, so the fact its templated is invisible to Istio.
Here is what I think the end result you have:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: shell
spec:
  selector:
    matchLabels:
      app: shell
  template:
    metadata:
      annotations:
        sidecar.istio.io/proxyCPU: 123m
        sidecar.istio.io/proxyCPULimit: 200m
        sidecar.istio.io/proxyMemory: 123Mi
        sidecar.istio.io/proxyMemoryLimit: 250Mi
      labels:
        app: shell
    spec:
      containers:
      - image: auto
        lifecycle:
          preStop:
            exec:
              command:
              - sleep
              - "12"
        name: istio-proxy
      - args:
        - /bin/sleep
        - infinity
        image: howardjohn/shell
        name: shell

something like that?
In my testing that works fine. Is there a difference for yours?

Archive.zip
Yes. Attached are rendered deployments and the final injected pods. Note memory and cpu use the "default" with prestop, and use the annotations without

Do you have some controller mutating Deployment configs to add resourcces? Or maybe even to pods, running before Istio's injector? While the deployment object has no resources, the pod gets:
proxy.istio.io/overrides:  |
{
  "containers": [
    {
      "name": "istio-proxy",
      "resources": {
        "limits": {
          "cpu": "100m",
          "memory": "256Mi"
        },
        "requests": {
          "cpu": "100m",
          "memory": "256Mi"
        }
      },
      "volumeMounts": [
        {
          "name": "kube-api-access-g9qnt",
          "readOnly": true,
          "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
        }
      ],
      "lifecycle": {
        "preStop": {
          "exec": {
            "command": [
              "sleep",
              "12"
            ]
          }
        }
      },
      "terminationMessagePath": "/dev/termination-log",
      "terminationMessagePolicy": "File",
      "imagePullPolicy": "Always"
    }
  ]
}

So this is what the Istio webhook actually sees as the container spec by the time it reaches the webhook. This suggests something is actually adding these defaults.
I see kubernetes.io/limit-ranger: 'LimitRanger plugin set: memory request for container, which suggests its the limit ranger adding these.
So the order of operations is:
Base pod < templates (where annotation logic lives) < container overrides
The problem is something mutates the 'container overrides' before the Istio webhook, making it logically:
Base pod < templates (where annotation logic lives) < container overrides < limit-ranger mutation

That makes sense! But why does it only kick in when the skeleton option exists? I'm guessing because the yaml is wholly absen in that case
So without our "template"
deployment -> Limit Ranger - doesn't see "bad" container" so no-op -> istio-injector creates pod yaml -> pod...

with our template
deployment -> Limit Ranger - sees the template skeleton with no request/liimit and mutates-> istio-injector creates pod yaml and sees these take precendence so ignores annotations -> pod...

This makes sense but short of removing limitrange (because it's a lame crd and you can't omit minimums IIRC)
what can we do? Is there another way to achieve the prestop desired?

Yep, exactly.
There is probably a few approaches:

Don't use limit ranger (understandably unacceptable)
Add the resources into the istio-proxy container spec
Instead of in-lining the config into the pod (like https://istio.io/latest/docs/setup/additional-setup/sidecar-injection/#customizing-injection), make a new template and reference that. See https://github.com/istio/istio/blob/f2f6f265d9c3ffab144fd49df4ec9a287a7fe121/pkg/kube/inject/testdata/inject/custom-template.yaml#L12 (you want custom,sidecar so they both apply -- else you will have to fully replace it which you don't want). Then define the custom template like so https://github.com/istio/istio/blob/f2f6f265d9c3ffab144fd49df4ec9a287a7fe121/pkg/kube/inject/testdata/inject/custom-template.iop.yaml#L10. Note you will need to read some values from the pod, so you would probably have your deployment templating insert something like an annotation, etc to program it

Those are all good ways. But seem inflexible. I'm not complaining per se - the design issues is fundamental to order of operations of admissions controllers, but very much affects this. Lets go through this list

Don't use limit ranger (understandably unacceptable)

Yeah we need this.

Create your own limit range controller as a replacement

It might be feasible to fork our own controller or use the CEL based as a replacement, but ultimately the issue is if you use istios "pod skeleton" feature, some mutator is is likely to have issues

Repeat the literal cpu/memory settings in the pod skeleton, instead of just as annotation.

_You didn't mention this but its actually the easiest thing - controlled in the same chart. The catch is if someone else changes something via annotation that mutating controller goes astray with. So I think this is better than the first option but has some future proofing.

Instead of in-lining the config into the pod (like https://istio.io/latest/docs/setup/additional-setup/sidecar-injection/#customizing-injection), make a new template and reference that.

This is injected as permanent template in the injector CM I assume from what I read? That seems like that's not terribly flexible since you'd need to have 1 million templates (exaggeration) to cover every permutation of memory and cpu ;)

This is injected as permanent template in the injector CM I assume from what I read? That seems like that's not terribly flexible since you'd need to have 1 million templates (exaggeration) to cover every permutation of memory and cpu ;)

No, your template would look like
containers:
- name: istio-proxy
  lifecycle:
    preStop:
      exec:
        command: [ "sleep", "{{index .ObjectMeta.Annotations `some-annotation/sleep-duration`}}" ]

then when you want to use it, annotate on your pod inject.istio.io/templates: "custom,sidecar" and some-annotation/sleep-duration: 12s

Ok, that clarifies things and I've tested it and it works. Small correction below
      prestop: |
        {{ if (isset .ObjectMeta.Annotations `mesh.foo.com/sleep-duration`) -}}
        spec:
          containers:
          - name: istio-proxy
            lifecycle:
              preStop:
                exec:
                  command: [ "sleep", "{{index .ObjectMeta.Annotations `mesh.foo.com/sleep-duration`}}" ]
        {{- end}}

Anyway now it's just finding the time to redeploy everywhere. Thanks @howardjohn


Hyfetch changes the OS info from Neofetch

Description
Hyfetch substitutes different information for OS than does Neofetch. Can Hyfetch be altered to show the info as Neofetch does?
Neofetch:
OS: Ubuntu 22.04.4 LTS x86_64
Hyfetch:
OS: Ubuntu bookworm/sid (jammy) x86_64
Neofetch version: 7.1.0
Hyfetch version: 1.4.11

You opened the issue in the wrong place. In the future, please open hyfetch-related issues in the hyfetch repo, thanks!
This change is intentional, introduced in https://github.com/hykilpikonna/hyfetch/commit/ff8f9b28792430365459121032adf522f083bd1a. It reads /etc/debian_version for debian-based distros that are not ubuntu.

However, it's strange that your Ubuntu was detected as debian instead of ubuntu.
Can you show me the output of lsb_release -si?

Thanks for  directing me to correct forum.
@.:~$ lsb_release @.:~$ *
About a month or two ago I had tried debian, after a few hours of it I
restored ubuntu.
On Mon, Apr 1, 2024 at 12:21 PM Azalea @.***> wrote:

You opened the issue in the wrong place. In the future, please open
hyfetch-related issues in the hyfetch repo
https://github.com/hykilpikonna/hyfetch, thanks!
This change is intentional, introduced in @.***
https://github.com/hykilpikonna/hyfetch/commit/ff8f9b28792430365459121032adf522f083bd1a.
It reads /etc/debian_version for debian-based distros that are not ubuntu.
image.png (view on web)
https://github.com/dylanaraps/neofetch/assets/22280294/dd1066ca-7d18-4d14-8985-98de1677d0eb
However, it's strange that your Ubuntu was detected as debian instead of
ubuntu.
Can you show me the output of lsb_release -si?
—
Reply to this email directly, view it on GitHub
https://github.com/dylanaraps/neofetch/issues/2459#issuecomment-2030289631,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA22VOLYPEOM4JYLS24NNDDY3GQRPAVCNFSM6AAAAABFQYI5N2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQGI4DSNRTGE
.
You are receiving this because you authored the thread.Message ID:
@.***>


[Feature][ui] Add 'About' menu to top-right dropdown menu

Add 'About' menu to top-right dropdown menu
Description
Currently if a regular DS user want to know the core version of DS, there's no entry for him to get this information, unless he ask the system administrator to log on the server to lookup the jars, which is quite inconvenient.
Possible Solution
We can simply add an 'About' menu to top-right dropdown menu, after clicking on it, there will be a popup window or a new page which contains version info and others if necessary.
添加‘关于’菜单到右上角下拉菜单
描述
在当前版本的DS中,如果普通用户想获取当前DS的版本号,系统中没有相应的入口去获取这些信息,只能通过寻找系统管理员登陆服务器,查看jar版本得知,较为不便.
解决方案
在右上角下拉菜单中添加‘关于’菜单,点击后通过弹出狂或新页面展示版本信息,和其他必要信息如API文档地址、帮助文档等.

Related issue: #3247

[WeOpen Star] I'd like to work on it


ADLS Support

Hi team,
Does cobrix support reading from Azure Blob storage? I'm having some issues doing the following:
copybook_path =<EMAIL_ADDRESS>file_path =<EMAIL_ADDRESS>df = ( spark.read.format("za.co.absa.cobrix.spark.cobol.source")
.option("copybook", copybook_path)
.load(file_path)
)

Hi,
Cobrix uses Hadoop Filesystem abstraction, so as long as the fs is supported by Hadoop, it should support the os.
Do other data sources work for you (spark-csv, for instance) with abfss?

Thanks for the quick response Ruslan.
Yes, if I do something like this:
file_path =<EMAIL_ADDRESS>df2 = ( spark.read.format("csv")
             .load(file_path)
     )

I can read the CSV into a DataFrame directly with abfss.
Here is the full trace for the error I'm getting:
Py4JJavaError: An error occurred while calling o509.load.
: Failure to initialize configuration
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.SimpleKeyProvider.getStorageAccountKey(SimpleKeyProvider.java:51)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getStorageAccountKey(AbfsConfiguration.java:529)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.initializeClient(AzureBlobFileSystemStore.java:1528)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:207)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:116)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
	at za.co.absa.cobrix.spark.cobol.source.parameters.CobolParametersValidator$.validatePath$1(CobolParametersValidator.scala:72)
	at za.co.absa.cobrix.spark.cobol.source.parameters.CobolParametersValidator$.validateOrThrow(CobolParametersValidator.scala:95)
	at za.co.absa.cobrix.spark.cobol.source.DefaultSource.createRelation(DefaultSource.scala:51)
	at za.co.absa.cobrix.spark.cobol.source.DefaultSource.createRelation(DefaultSource.scala:47)
	at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:385)
	at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:424)
	at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:391)
	at scala.Option.getOrElse(Option.scala:189)
	at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:391)
	at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:278)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)
	at py4j.Gateway.invoke(Gateway.java:295)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:251)
	at java.lang.Thread.run(Thread.java:748)

I see. It seems it tries to use the filesystem, but for some reason fails to initialize it.
Sorry, we can't help you with it at the moment since we don't have access to an Azure account and also no one from our team has experience with it.
If you figure out what is wrong and what can be done from Cobrix's side to make it work, we'll gladly include the fix. Or you could contribute a PR, that would be fantastic and the most welcome.
Again, I'm really sorry. I want to help, but just don't know how.

You can mount ADLS on Databricks and use it as local storage. That is the right way to use ADLS in Databricks.

You can mount ADLS on Databricks and use it as local storage. That is the right way to use ADLS in Databricks.

@vijayinani  not true. this is dependent on security guidelines. mounting a storage account will expose data to all users, but if user is unauthorized then that's a problem.
@alexdesroches i think the stack trace gives you the clue. you're using python but cobrix is assembled in scala and most likely cannot access the authentication you set in pyspark with spark.conf.set
We can access the underlying hadoop configuration as per this link sc._jsc.hadoopConfiguration().set('my.mapreduce.setting', 'someVal')
Can you try changing spark.conf.set to sc._jsc.hadoopConfiguration().set and report back the results?


Add support for sec and ref scenes

Add comprehensive check to parse and manage secondary and reference layers from embedded tropo and SET layers.

For SET and tropo layers which contain corresponding reference and secondary scenes, the program organizes individual epochs within a dates subdirectory below the parent directory where differential fields are computed and deposited, as so:
(ARIA-tools) [ssangha@leffe just_SET2]$ ls
DEM  productBoundingBox  solidEarthTide  user_bbox.json
(ARIA-tools) [ssangha@leffe just_SET2]$ ls solidEarthTide/
20211120_20211108  20211120_20211108.aux.xml  20211120_20211108.vrt  20211120_20211108.xml  dates
(ARIA-tools) [ssangha@leffe just_SET2]$ ls solidEarthTide/dates/
20211108  20211108.aux.xml  20211108.vrt  20211108.xml  20211120  20211120.aux.xml  20211120.vrt  20211120.xml

@mgovorcin and @bbuzz31 next week I could really use your help to vet and expand upon this as we work to interface with and update prep_aria.
There are currently two sets of sample products for which the SET and tropo reference/secondary scenes have been inserted, respectively.
For SET refer to the products here: /u/leffe-data/ssangha/solid_earth_test/A064/products
And for testing purposes here is a minimum commanline input: ariaExtract.py -l 'solidEarthTide' -nt 20 -f "products/*.nc" -d Download -of ISCE --nc_version 1c -w just_SET
For tropo refer to the products here: /u/leffe-data/ssangha/solid_earth_test/temp_tropo_test/D071/products And for testing purposes here is a minimum commanline input: ariaExtract.py -l 'troposphereTotal' -nt 20 -f "products/*.nc" -d Download -of ISCE --nc_version 1c -w just_tropo`
Note @bbuzz31 that I've encountered the following error in this test stack you shared that I didn't see before. Could you please advise? Thanks:
>>> from osgeo import gdal
>>> gdal.BuildVRT('/u/leffe-data/ssangha/solid_earth_test/temp_tropo_test/D071/just_tropo/troposphereWet/HRES/dates/20200112', ['NETCDF:"/u/leffe-data/ssangha/solid_earth_test/temp_tropo_test/D071/products/S1-GUNW-D-R-071-tops-20200130_20200112-135156-34956N_32979N-PP-dd0c-v2_0_4.nc":/science/grids/corrections/external/troposphere/HRES/reference/troposphereWet'])
Warning 6: gdalbuildvrt does not support ungeoreferenced image. Skipping NETCDF:"/u/leffe-data/ssangha/solid_earth_test/temp_tropo_test/D071/products/S1-GUNW-D-R-071-tops-20200130_20200112-135156-34956N_32979N-PP-dd0c-v2_0_4.nc":/science/grids/corrections/external/troposphere/HRES/reference/troposphereWet

Note @bbuzz31 that I've encountered the following error in this test stack you shared that I didn't see before. Could you please advise? Thanks:

Sounds like the CRS was not embedded at the same level.


tooltip position class

Currently it was not possible to style the container of the tooltip according to the actual position of the tooltip.
Added css class defining the actual position of the tooltip with the tooltip container. This would allow some scenarios of styling (which I needed) e.g. make the border of the tooltip where the arrow is currently to have different color and width.

Thanks. I'll have to check but I think v4 already has this.

ohh I haven't noticed you had v4 in progress, I would have used that instead :(.
Maybe a small note in the README file would help someone else :).
It seems to be there in v4
https://github.com/iamceege/tooltipster/blob/v4/js/jquery.tooltipster.js#L1592

Yes, sorry, I have to take the time this month to work on this. Thank you for your support to the project.


[BUG] Print footer cuts off XML form content

Description
The footer overlaps with form content and can block legibility, as it current floats above the display area. One potential fix is shrinking the content area so that it doesn't overlap with form content.
Steps to Reproduce

Navigate to DOT Beta MeF viewer.
Open Consolidated Group XML provided by Joe
Open the Itemized Other Costs Schedule, scroll to the bottom of the page

Acceptance Criteria
Fix is merged and deployed to the beta site for DOT testing

https://github.com/newjersey/taxation-mef-viewer/commit/1ef5da6eae6774e1cdcb960b430aec8111572159

Fix merged and live at https://main.d213b2tcid8n28.amplifyapp.com/


Filtering "get children" on type "user" does not work

Hi,
I am trying out version 1.0.0 and I think I have found an issue. Filtering get children on user does not work. Other types are still outputted.
While computers, for example test3 and RODC1, should not be shown using this filter I am not sure what is correct for the three other marked examples.

CHILD2$ is the sAMAccountName of a child domain (this is not shown when filtering on computer)
smsa1 is a standalone managed service account (this is also shown when filtering on computer)
gmsa1 is a group managed service account (this is also shown when filtering on computer)

Thanks!

Hi @jsdhasfedssad, as usual thank you for the interest you have for the tool and to raise me issues when you see something wrong.
I choose to filter by objectClass attribute and it can contain several class per object. In AD, computers do have a "user" objectClass, manged service accounts do have a "computer" objectClass.
This is not precise enough and I will think about a better way to filter it.
I don't understand why you don't have users outputed when you filter with "user", it's like they don't have the "user" objectClass.

Could you show me a "get object SOME_USER --attr objectClass" and "get object SOME_USER --attr objectCategory"?
Same for your child domain? (i'm very surprised it has a sAMAccountName btw)
And finally "get object DC=adlab,DC=local --attr msDS-Behavior-Version"

The command I used above does actually identify users as well. In the above screenshots branchuser1, rodc_admin1, fTYYlViSVK, krbtgt_13330 and delegation1 are regular user accounts (if krbtgt accounts are counted as that).
To my knowledge, forests and domains having accounts in the AD is as it should be. All my forests and domains have one. These accounts can for example be abused to get trust keys.
Here is the output from your commands:

I just noticed, this also affects the get writable command when the filter --otype is used.

I will think about a better way to filter but unfortunately it risks to not be as straightforward. I let this open until I fix it.

@jsdhasfedssad can you confirm children --otype useronly fix your issue?


When the camera is inside the boat, it will see the water.

Hi! When the camera is inside the boat, it will see the water. Is it possible to change the water inside the floats to be transparent or to cut off the surface mesh inside the floats? Thanks. 😀

Hi theres a bit of a history to this, its not for the faint hearted!
Mask out the water: https://github.com/huwb/crest-oceanrender/issues/77
Show the water through a window or such: https://github.com/huwb/crest-oceanrender/issues/163
Mostly speculation and unshipped i would guess!
I'll close this as i'm guessing any conversation belongs in one of those issues.

Thanks, my fault, I haven't noticed that before. 🤐


Mailer with defoverridable

defmacro __using__(opts) do
    quote bind_quoted: [opts: opts] do
      alias Swoosh.Mailer

      @otp_app Keyword.fetch!(opts, :otp_app)
      @mailer_config opts

      @spec deliver(Swoosh.Email.t(), Keyword.t()) :: {:ok, term} | {:error, term}
      def deliver(email, config \\ [])

      def deliver(email, config) do
        Mailer.deliver(email, parse_config(config))
      end

      defoverridable deliver: 2
...

It is possible?

There is no point doing this. You can simply define a different function that wraps deliver, call it wrapped_deliver or whatever. And use that function in your own codebase.

If you really like the name "deliver", you can name the module that contains "use" "DefaultMailer" and define a different module that defines your own "deliver" that does your extra things and call DefaultMailer.deliver when necessary.


Customizable icon

This pull request introduces flexibility on navigation icon.
Default is still heroicon-o-language
navigation_icon => heroicon-o-language is added in filament-translation-manager config file.
This will allow to customize the navigation icon when config file is published.

Thanks! I will merge


:book: Add Argo CD Agent integration solution

Summary
Add a solution doc for integration with the Argo CD Agent project.
Related issue(s)
https://github.com/open-cluster-management-io/addon-contrib/pull/23

/lgtm


Cannot delete orphaned app

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
arn:aws:amplify:ap-southeast-2:032312983640:apps/d9o5hf4erei4n
Region
ap-southeast-2
Amplify Console feature
No response
Describe the bug
I am attempting to delete an App via the Amplify Console, it deleted the first environment and then has been stuck for almost 24hrs now.
"Removing backend envs for app d9o5hf4erei4n. This operation will take a few minutes."
I have then attempted to delete either the app or the backend environments via the Amplify CLI, but I receive the following error:
"An error occurred (NotFoundException) when calling the DeleteBackendEnvironment operation: App d9o5hf4erei4n not found."
I have then manually tried to delete the resources used, ie. deleted the lambda functions, the stack via Cloudfront, etc. I believe the only thing left is the app itself, which everytime I open it up shows that it is still removing the backend envs for the app.
Expected behavior
I should be able to delete an Amplify App
Reproduction steps

In the API Conosle I selected Actions | Delete App.
I then get a message in blue saying it is invoking remove all backends
It turns red and states: Invoking remove all backends execution failed
I try the Amplify CLI: aws amplify delete-backend-environment --app-id d9o5hf4erei4n --environment-name dev
I get an error: An error occurred (NotFoundException) when calling the DeleteBackendEnvironment operation: App d9o5hf4erei4n not found.

Build Settings
No response
Additional information
No response

Hello, thanks for reporting this issue. We delete things in certain order, I suspect that in this case the backend environment (CFN stack) was deleted before the app so it may be in this stuck state. Can you please try deleting it directly with the AWS CLI?
aws amplify delete-app --app-id 123

Let us know if this helps


Fix pyarrow failures in cluster testing

This PR resolves the independent cluster test failures related to pyarrow by adding libstdcxx-ng>=12.1.0 to the conda environment, which gives us the required GLIBCXX version to use the latest pyarrow builds.
It also makes the pyarrow minimum version consistent across testing and packaging, matching up with Dask's documented min version of 1.0.0. Note that by installing libstdcxx-ng>=12.1.0 in the cluster, we are effectively always installing pyarrow 7 or 8; might be worth following up with work to more explicitly test our minimum version requirements.
cc @ayushdg @galipremsagar

Codecov Report

Merging #553 (895dd9b) into main (ff4a8a5) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #553      +/-   ##
==========================================
- Coverage   82.71%   82.68%   -0.03%
==========================================
  Files          69       69
  Lines        3500     3500
  Branches      707      707
==========================================
- Hits         2895     2894       -1
+ Misses        516      513       -3
- Partials       89       93       +4

Impacted Files
Coverage Δ

dask_sql/_version.py
32.27% <0.00%> (-0.29%)
:arrow_down:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


[WIP][Docs] Getting Started With TVM Tutorial

This patch refactors a number of tutorials, and adds additional documents, to create a Getting Started with TVM Tutorial. The aim of the tutorial is to guide a new user of TVM through the features and architecture of the project. It starts with a basic overview of TVM, followed by some guidance on how to install TVM.
It then works through examples of how to optimize a ResNet-50v2 models, using TVMC and the Python interface. Having given a high level example of what TVM is capable of, it then takes a bottom-up approach of describing TE using vector addition and matrix multiplication, then continues to build on the matrix multiplication example to describe templates, AutoTVM, and Auto Scheduling. Finally, some additional information about targeting GPUs and RPCs is described.

New patch set for introduction and installation: https://github.com/apache/tvm/pull/7638

New patch set for tvmc https://github.com/apache/tvm/pull/7640

Auto Tuning with Python https://github.com/apache/tvm/pull/7641
Tensor Expressions https://github.com/apache/tvm/pull/7642
AutoTVM and matmul https://github.com/apache/tvm/pull/7643
Auto Scheduler and matmul https://github.com/apache/tvm/pull/7644

Closed as this work is covered in the PRs indicated above.


bug: multiplayer crashes unexpectedly

Multiplayer crashes if we access (192.168.x.x:2308) from external clients (like browser, ...). It crashes with the following errors:
thread '<unnamed>' panicked at src/server/game_server.rs:139:53:
called `Result::unwrap()` on an `Err` value: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }

Or,
thread 'main' panicked at src/utils.rs:83:37:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Can you specify what you mean by browser so I can try to reproduce ?

Follow following steps:

open Multiplayer in HOSTING mode
choose color
then now you are at page waiting for opponent connection using your IP address (192.168.x.x:2308)
now visit 192.168.x.x:2308 form any web browser to see error

This is kind of normal since 192.168.x.x:2308 is opening a tcp stream, connecting with a browser will send an http request to the endpoint which is not handled at all right now. The only correct way to play a multiplayer game is to use the join method from another cli with chess-tui


Is the sheet_to_json XSS attack safe?

I read through the code about sheet_to_json. I see there is a safe_format_cell. Can you confirm that the code is designed for XSS attack. You know the user can enter anything in the excel. We can't trust their input.

XSS isn't really something that should be solved at the data layer; it's specifically about injecting HTML where plaintext is expected (that is later rendered into a HTML page).
This means that in a JSON string, there's not really such a thing as "XSS" - after all, HTML has no special meaning in the context of a JSON string, and it's just bytes. It would make no sense to assume that it's going to be used in HTML somewhere.
The problem of XSS should therefore be solved at the display layer - for example, if you render JSON into a HTML page, then the code that does the rendering should be escaping values (or, ideally, use a library or function that does it for you, such as React or jQuery's .text). The data in the JSON should remain untouched and unescaped.

safe refers to the fact that decode_range assumes a bounded range like A1:C3.  It does not handle the more general case of ranges like A:C or 1:3.   Ultimately, a more robust implementation of the character approach will replace decode_range and safe_decode_range will be eliminated.


BORDER SWITCH REQUIRED, what does it mean?

I have added the grabber-v4l2 part to my config and now I get the message: BORDER SWITCH REQUIRED!! CURRENT BORDER TYPE: unknown=0 hor.size=0 vert.size=0.
What does it mean and where does it come from? (Is there any way to search in the source code online on github?)
The update rate of my leds is very very low, like once every two seconds.
When i don't start the grabber via config, but manually, everything works fine.

Ok Googling "Border Switch required" site:https://github.com/tvdzwan/hyperion gives me the hint that it has something to do with the blackborder detector. Does it make a difference in which order things are defined in the config file?

The border switch is nothing to worry about. It means that the automatic detected a different black-border then before and is switching the removal algorithm to compensate.
Your second issue seems related to starting the 'service'. It could be that it is using a different configuration file. How are you starting manually? The default installation takes the config-file from '/etc/hyperion.config.json'.

i think we are starting via config file.
as i understand this isnt manually?!
the /etc/hyperion.config.json  ist just a link to the main config file folder /opt/hyperion/config ?
we replaced our config there with "frame grabber" section excluded. and the extra v4l2grabber included.
should we replace the "link" with the same main file from opt folder ?
or can we ignore this message

The link in /etc is only created if there is no file present and will then link to /opt/hyperion/config. The idea is that the user can put its own configuration in /etc and this will not be overwritten by running the update script.
The order in the json file does not matter as long as it is proper json (easy to get a komma wrong).

@Fabi1080 kannst ma schliessen bitte :D danke


hostname mapped to loopback causes hang and bad physical node detection

Original issue: https://charm.cs.illinois.edu/redmine/issues/1031

Setting /etc/hosts to map the hostname of the machine to the loopback address <IP_ADDRESS> at a minimum confuses physical node detection, probably breaks net layers, and may also break verbs.

Original date: 2016-04-12 14:20:13

This is almost handled in util/sockRoutines.c but falls back to skt_lookup_ip(hostname) if a host has multiple IP addresses (e.g., ethernet and infiniband) even if skt_lookup_ip(hostname) returns a loopback address.  It is probably safer to just return the first up non-loopback interface.  Working on a fix.
 skt_ip_t skt_my_ip(void)
 {
   char hostname[1000];
 #if CMK_HAS_GETIFADDRS
   skt_ip_t ip = _skt_invalid_ip;
     /* Code snippet from  Jens Alfke
  *     http://lists.apple.com/archives/macnetworkprog/2008/May/msg00013.html */
   struct ifaddrs *ifaces=0;
   int ifcount = 0;
   if( getifaddrs(&ifaces) == 0 ) {
         struct ifaddrs *iface;
         for( iface=ifaces; iface; iface=iface->ifa_next ) {
             if( (iface->ifa_flags & IFF_UP) && ! (iface->ifa_flags & IFF_LOOPBACK) ) {
                 const struct sockaddr_in *addr = (const struct sockaddr_in*)iface->ifa_addr;
                 if( addr && addr->sin_family==AF_INET ) {
                     ifcount ++;
                     memcpy(&ip, &addr->sin_addr, sizeof(ip));
                 }
             }
         }
         freeifaddrs(ifaces);
   }
   /* fprintf(stderr, "My IP is %d.%d.%d.%d\n", ip.data[0],ip.data[1],ip.data[2],ip.data[3]); */
   if (ifcount==1) return ip;
 #endif

   if (gethostname(hostname, 999)==0)
       return skt_lookup_ip(hostname);

   return _skt_invalid_ip;
 }

Original date: 2016-04-12 22:44:36

Fixed as ~~https://charm.cs.illinois.edu/gerrit/#/c/1139/~~ https://github.com/UIUC-PPL/charm/commit/2cf4c81bfba1a3726423bf1c07a4da707ff03909


Feat/hard delete button

What does this pull request change?
Adds a delete button with pop-up dialogue that required confirmation on delete.
Can be used on mulitple deleting stuff. Must have a wrapper that defines what happens onConfirmDelete

Is an x instead of the old trash can
Has a tooltip with "delete + namePath".

Why is this pull request needed?
Issues related to this change

https://medium.com/@wdeamonica/x-to-delete-or-x-for-error-misunderstanding-x-icon-in-user-interface-1fd57baa2f5b

I think it's a valid point, that in this context the x is less intuitive.
It might also signify harder "Delete whole attribute".
I will go through form today, and look at how to make it more clear.
I think moving the X to the right corner instead of so close to the attribute might make it much more clear that it's a delete.


Merge changelog from release branch back to main

Containerregistry was released from release branches and changelog was updated there.

API change check
API changes are not detected in this pull request.


Lack of Confirmation Message After Saving Restaurant Location

Description:
Users are encountering a significant issue on the website when inputting a restaurant's location. After making changes to the restaurant's location details and saving them, the platform fails to display a confirmation message. This lack of feedback leaves users uncertain about whether their changes have been successfully saved, potentially causing confusion and the need for users to double-check their actions.
Steps to Reproduce:

Log in to the website on a compatible device.
Access the restaurant's location settings or editing section.
Make changes to the restaurant's location details.
Save the changes.
Observe the screen for any confirmation message.

Expected Result:
After making changes to the restaurant's location details and saving them, users should receive a confirmation message indicating that their changes have been successfully saved. This feedback is important to ensure that users are aware of the status of their actions.
Actual Result:
Users find that, after saving changes to the restaurant's location, the platform does not display any confirmation message. This issue leaves users uncertain about whether their changes were saved, potentially leading to confusion and the need for additional verification.
Impact:
This issue has a moderate negative impact on the user experience, as it can lead to uncertainty and potential double-checking of actions. Users should have clear feedback regarding the successful application of changes to the restaurant's location.

I'm pleased to inform you that the issue you reported regarding the missing confirmation message for restaurant location updates has been successfully resolved.
Thank you for bringing this issue to our attention, and we appreciate your patience. If you encounter any further problems or have additional feedback, please don't hesitate to let us know.
Best regards,
@beKashif


Rust - With Simple Function, LLDB Debugger variables duplicated and random?

OS:  Windows 10 Pro Build 19043
VSCode version:   1.62.3
CodeLLDB version:  1.6.10
Compiler:  rustc 1.56.1 with cargo run
Debuggee:  I honestly don't know. Prompt me further if it's necessary information.

TLDR: Debugger presenting very abnormal information.
Mirrored from Stackoverflow: https://stackoverflow.com/questions/70001003/rust-with-simple-function-debugger-variables-duplicated-and-random
use std::fmt::Debug;
use std::ops::Add;
fn main() {

    fn calculate<T:Debug + Add::<Output=T>, const N: usize>(data_set: [T;N]) -> (i32,i32) {

        // Key Code
        let _var = data_set.get(0);
        println!("Output_1: {:?}",data_set.get(0));

        // Ignore
        return (0,0)
    }

let data = [1509,1857,1736,1815,1576];
let result = calculate(data);
println!("Output_2: {:?}",result);
}

I have a very straightforward rust function which just simply wants to get the first value of the list that is passed into it.
I have been trying to debug it in vs code with lldb, but the local variables in the debug panel are being shown to be duplicated 3 times over which I don't understand why is happening.
Memory Stack/Pile/Heap before function:
data: {1509,1857,1736,1815,1576}

Memory Stack/Pile/Heap immediately after entering function:
data_set: {1509,1857,1736,1815,1576}
data_set: {1509,1857,1736,1815,1576}
data_set: {1509,1857,1736,1815,1576}

I have deduced that the problem is caused by the line println!("Output_1: {:?}",data_set.get(0));  because by removing it, the duplicate data_set's disappear. However, I frankly have no understanding why it causes the problem.
Furthermore, whilst executing the line println!("Output_1: {:?}",data_set.get(0)); , the values go random and extreme, changing the memory stack/heap to
data_set: {1509,1857,1736,1815,1576}
data_set: {-445645856,223,-445645856,223,-445645856} // numbers are different each execution.
data_set: {1509,1857,1736,1815,1576}

I am new to rust and don't think i've done any special modifications to the extensions.
Why is this happening and is this normal behaviour?
If someone would be able to take 3 minutes and confirm whether this is reproducable behaviour that would be excellent as well.

Testing has revealed that the duplication issue was introduced in either version 1.6.9 or 1.6.10. I can not confirm which version the error specifically arose in due to other smaller abstract bugs.

Probably a consequence of fixing #568.   CodeLLDB used to suppress multiple instances of a variable on the same tree level.  Normally, though, this only happens when there are multiple variables declared that that shadow one another.
You didn't say what target you are building for, but it's probably ...-msvc?  I wouldn't be surprised if there's a bug in handling of PDB debug info in LLDB, which causes it to produce multiple instances of the same variable.  See https://github.com/vadimcn/vscode-lldb/wiki/Windows.

Just wanted to point out. That I have the same problem.

I reproduced it here
win 10 x64
rust - 1.57 msvc
vscode - 1.63,2
codelldb - v1.6.10

May someone test with gnu toolchain?

May someone test with gnu toolchain?

At my test, that was the result.
OS: Linux manjaro 5.4.163-1-MANJARO
Rustc: 1.56.1
Code: 1.62.3
CodeLLDB: 1.6.10

With gnu toolchain on windows it doesn't happens. Seems to be something with msvc toolchain or pdb file as mention by @vadimcn

Should be fixed in v1.7.0.
Note that this is is done by de-duplicating local variables on name, the underlying LLDB issue is still there.


Lack of subject update Changing nats filter subjects

Problem description
Hello team, I am having a issue trying to update the subject of a consumer with nack, so the issue is related with the change of one subject to a multiple subjects filter and also when I want to add another simple subject filter into the array of subjects.
The only way to apply the changes is deleting the consumer and then restarting nack so it is reacreating the consumer with the correct subject.
Any idea or plan to fix this issue? do you have a better workaround than delete the consumer and restart the nack pod?
issue 1:

issue 2:

Nack logs: E1102 10:34:19.371115       1 controller.go:460] failed to process consumer: failed to update consumer "workload" on stream "MAIN": consumer subject filters cannot overlap (10138)
What version amI using?
nack version: natsio/jetstream-controller:0.13.0
nats version:  2.10.3
yml version: jetstream.nats.io/v1beta2
Is this defect reproducible?
I think yes
Given the capability you are leveraging, describe your expectation?
a kick fix and final fix in the next version of nack

The steps I've used (these are even worse, so your steps would be much better though still not great!):

Delete the consumer in NATS itself
kubectl delete consumers.jetstream.nats.io <consumername>
kubectl apply

@lukaso thanks for the reply, the deletion of the consumer works fine but very difficult to do that when you consumer is running in productions with a lot of traffic, instead of that as a Workaround you can edit the subject consumer using the command with the Nats cli anyways also there is a bug trying to use the Nats cli with the Args, but you can do using consumer edit with json configuration, look the this issue in the Cli repo for more details  https://github.com/nats-io/natscli/issues/918

The steps I've used (these are even worse, so your steps would be much better though still not great!):

Delete the consumer in NATS itself
kubectl delete consumers.jetstream.nats.io <consumername>
kubectl apply

@lukaso thanks for the reply, the deletion of the consumer works fine but very difficult to do that when you consumer is running in productions with a lot of traffic, instead of that as a Workaround you can edit the subject consumer using the command with the Nats cli anyways also there is a bug trying to use the Nats cli with the Args, but you can do using consumer edit with json configuration, look the this issue in the Cli repo for more details  https://github.com/nats-io/natscli/issues/918

Hey.
Thanks for reporting the issue.
I'm looking into it.

I found a bug in jsm and issued a fix.
https://github.com/nats-io/jsm.go/pull/501

Thanks a lots @Jarema 🎉🎉🎉

Hello, any more updates on this? I believe jsm needs a new release and then nack needs to reference that new version. Is there anything more?

No. It's just that. Do we have a schedule for releasing jsm @ripienaar ?

Not a schedule as such but there will be a release before mid jan

Looks like a jsm release hasn't been cut yet.

Here's a gist I'm using to work around this problem. Remember this is code that works for me but no warranty (since it is mutating your system). https://gist.github.com/lukaso/347c8d9afdc6b21ce8fbd15456594480

Can we get the latest ESM release out?

Created this PR to fix this issue: https://github.com/nats-io/nack/pull/170


Extract_geographical_watershed_properties.ipynb: fix run failure in new Jupyter env

Fixes https://github.com/Ouranosinc/raven/issues/392 using hints from https://github.com/Ouranosinc/raven/issues/388. Closes #388.
Note I am not sure what I am doing as I do not understand the code.  I simply try to get around the errors and produce similar graph.  The old graph had spatial_ref in the title, the new one don't.  Wonder if it's something critical.
With the new Jupypter env for latest RavenPy 0.5.2 from PR https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/75 Extract_geographical_watershed_properties.ipynb fails with the following error below.
23:00:58  _ raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 7 _
23:00:58  Notebook cell execution failed
23:00:58  Cell 7: Cell execution caused an exception
23:00:58
23:00:58  Input:
23:00:58  # Note that geojson, and rasterio/rioxarray need to be installed for this to work.
23:00:58  # In the next birdy release, the tiff grid will be automatically converted to a DataArray.
23:00:58  # Here we need to manually convert it.
23:00:58  features, statistics, grid = stats_resp.get(asobj=True)
23:00:58
23:00:58  with rasterio.io.MemoryFile(grid[0]) as memfile:
23:00:58      with memfile.open() as dataset:
23:00:58          grid = rio.open_rasterio(dataset)
23:00:58
23:00:58  grid.plot()
23:00:58
23:00:58  Traceback:
23:00:58
23:00:58  ---------------------------------------------------------------------------
23:00:58  ValueError                                Traceback (most recent call last)
23:00:58  <ipython-input-8-8b167a3ecdc9> in <module>
23:00:58        4 features, statistics, grid = stats_resp.get(asobj=True)
23:00:58        5
23:00:58  ----> 6 with rasterio.io.MemoryFile(grid[0]) as memfile:
23:00:58        7     with memfile.open() as dataset:
23:00:58        8         grid = rio.open_rasterio(dataset)
23:00:58
23:00:58  /opt/conda/envs/birdy/lib/python3.7/site-packages/rasterio/io.py in __init__(self, file_or_bytes, dirname, filename, ext)
23:00:58      103         """
23:00:58      104         super(MemoryFile, self).__init__(
23:00:58  --> 105             file_or_bytes=file_or_bytes, dirname=dirname, filename=filename, ext=ext)
23:00:58      106
23:00:58      107     @ensure_env
23:00:58
23:00:58  rasterio/_io.pyx in rasterio._io.MemoryFileBase.__init__()
23:00:58
23:00:58  /opt/conda/envs/birdy/lib/python3.7/site-packages/xarray/core/common.py in __bool__(self)
23:00:58      127
23:00:58      128     def __bool__(self: Any) -> bool:
23:00:58  --> 129         return bool(self.values)
23:00:58      130
23:00:58      131     def __float__(self: Any) -> float:
23:00:58
23:00:58  ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
23:00:58
23:00:58  _ raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 10 _
23:00:58  Notebook cell execution failed
23:00:58  Cell 10: Cell execution caused an exception
23:00:58
23:00:58  Input:
23:00:58  unique, counts = numpy.unique(grid.sel(band=1), return_counts=True)
23:00:58  display(unique, counts)
23:00:58
23:00:58  # Pixels values at '127' are NaN and can be ignored.
23:00:58  plt.bar(unique[:-1], counts[:-1])
23:00:58  plt.show()
23:00:58
23:00:58  grid.where(grid != 127).sel(band=1).plot.imshow(cmap="tab20")
23:00:58  plt.show()
23:00:58
23:00:58  Traceback:
23:00:58
23:00:58  ---------------------------------------------------------------------------
23:00:58  AttributeError                            Traceback (most recent call last)
23:00:58  <ipython-input-11-a26c52126e9c> in <module>
23:00:58  ----> 1 unique, counts = numpy.unique(grid.sel(band=1), return_counts=True)
23:00:58        2 display(unique, counts)
23:00:58        3
23:00:58        4 # Pixels values at '127' are NaN and can be ignored.
23:00:58        5 plt.bar(unique[:-1], counts[:-1])
23:00:58
23:00:58  AttributeError: 'list' object has no attribute 'sel'
23:00:58
23:00:58  _ raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 11 _
23:00:58  Notebook cell execution failed
23:00:58  Cell 11: Cell execution caused an exception
23:00:58
23:00:58  Input:
23:00:58  import cartopy.crs as ccrs
23:00:58
23:00:58  # Set a CRS transformation:
23:00:58  crs = ccrs.LambertConformal(
23:00:58      central_latitude=49, central_longitude=-95, standard_parallels=(49, 77)
23:00:58  )
23:00:58
23:00:58  ax = plt.subplot(projection=crs)
23:00:58  grid.name = "Land Use Categories"
23:00:58  grid.where(grid != 127).sel(band=1).plot.imshow(ax=ax, transform=crs, cmap="tab20")
23:00:58  plt.show()
23:00:58
23:00:58  Traceback:
23:00:58
23:00:58  ---------------------------------------------------------------------------
23:00:58  AttributeError                            Traceback (most recent call last)
23:00:58  <ipython-input-12-f9fdb6f0e562> in <module>
23:00:58        7
23:00:58        8 ax = plt.subplot(projection=crs)
23:00:58  ----> 9 grid.name = "Land Use Categories"
23:00:58       10 grid.where(grid != 127).sel(band=1).plot.imshow(ax=ax, transform=crs, cmap="tab20")
23:00:58       11 plt.show()
23:00:58
23:00:58  AttributeError: 'list' object has no attribute 'name'

I'll take a look at this this afternoon!

Alternatively, use make docs/source/notebooks/Extract_geographical_watershed_properties.ipynb.refresh to re-run against localhost but all localhost will be replaced for you automatically.

I think the method of file path parsing needs some tweaking. Failing with:
FileNotFoundError: [Errno 2] No such file or directory: 'docs/source/notebooks/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb'

Will try re-running manually for now.

Alternatively, use make docs/source/notebooks/Extract_geographical_watershed_properties.ipynb.refresh to re-run against localhost but all localhost will be replaced for you automatically.

I think the method of file path parsing needs some tweaking. Failing with:
FileNotFoundError: [Errno 2] No such file or directory: 'docs/source/notebooks/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb'

@Zeitsperre How did you get this error?  What was your $PWD when you launch make?

Will try re-running manually for now.

It works, passing Jenkins for all Raven notebooks http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/new-docker-build-for-ravenpy/18/console, thanks !
@richardarsenault given all Raven notebooks working now, if we deploy a new Jupyter env, is it okay for your summer school?

@tlvu Yes it's all good for the summer school, thanks!

FileNotFoundError: [Errno 2] No such file or directory: 'docs/source/notebooks/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb'

@Zeitsperre How did you get this error? What was your $PWD when you launch make?

/my_user/git/raven/. Here's the full stack:
Refreshing notebook docs/source/notebooks/Extract_geographical_watershed_properties.ipynb
make[1]: Entering directory '/home/my_user/git/raven'
bash -c 'WPS_URL="http://<IP_ADDRESS>:9099" FINCH_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/finch/wps" FLYINGPIGEON_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/flyingpigeon/wps" jupyter nbconvert --to notebook --execute --ExecutePreprocessor.timeout=240 --output "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"; sed -i "s@http://<IP_ADDRESS>:9099/outputs/@https://pavics.ouranos.ca/wpsoutputs/@g" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"'
[NbConvertApp] Converting notebook docs/source/notebooks/Extract_geographical_watershed_properties.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] Writing 144740 bytes to docs/source/notebooks/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb
Traceback (most recent call last):
  File "/home/my_user/miniconda3/envs/raven-dev1/bin/jupyter-nbconvert", line 11, in <module>
    sys.exit(main())
  File "/home/my_user/miniconda3/envs/raven-dev1/lib/python3.8/site-packages/jupyter_core/application.py", line 254, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/my_user/miniconda3/envs/raven-dev1/lib/python3.8/site-packages/traitlets/config/application.py", line 845, in launch_instance
    app.start()
  File "/home/my_user/miniconda3/envs/raven-dev1/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 350, in start
    self.convert_notebooks()
  File "/home/my_user/miniconda3/envs/raven-dev1/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 524, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "/home/my_user/miniconda3/envs/raven-dev1/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 490, in convert_single_notebook
    write_results = self.write_single_notebook(output, resources)
  File "/home/my_user/miniconda3/envs/raven-dev1/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 450, in write_single_notebook
    write_results = self.writer.write(
  File "/home/my_user/miniconda3/envs/raven-dev1/lib/python3.8/site-packages/nbconvert/writers/files.py", line 126, in write
    with io.open(dest, 'w', encoding='utf-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'docs/source/notebooks/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb'
make[1]: Leaving directory '/home/my_user/git/raven'

Could it be that this particular make command doesn't work in zsh? I can't think of why else it would fail.

Could it be that this particular make command doesn't work in zsh? I can't think of why else it would fail.
EDIT: Tried from bash. No difference in output.

This is really odd.  This works for me on my machine.
Can you try this command direct in your shell?
bash -c 'WPS_URL="http://<IP_ADDRESS>:9099" FINCH_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/finch/wps" FLYINGPIGEON_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/flyingpigeon/wps" jupyter nbconvert --to notebook --execute --ExecutePreprocessor.timeout=240 --output "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"; sed -i "s@http://<IP_ADDRESS>:9099/outputs/@https://pavics.ouranos.ca/wpsoutputs/@g" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"'

Can you try this command direct in your shell?
bash -c 'WPS_URL="http://<IP_ADDRESS>:9099" FINCH_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/finch/wps" FLYINGPIGEON_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/flyingpigeon/wps" jupyter nbconvert --to notebook --execute --ExecutePreprocessor.timeout=240 --output "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"; sed -i "s@http://<IP_ADDRESS>:9099/outputs/@https://pavics.ouranos.ca/wpsoutputs/@g" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"'

https://gist.github.com/Zeitsperre/e430dda569c6e93ff9903d7c323020e3

So it looks like this works with nbconvert 6.0.7. --output must only be the name of the file. --output-dir is to specify the new location, but by default it will use the same location as source.:
WPS_URL="http://<IP_ADDRESS>:9099" FINCH_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/finch/wps" FLYINGPIGEON_WPS_URL="https://pavics.ouranos.ca/twitcher/ows/proxy/flyingpigeon/wps" jupyter nbconvert
--to notebook --execute --ExecutePreprocessor.timeout=240 --output "Extract_geographical_watershed_properties.ipynb" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"; sed -i "s@http://<IP_ADDRESS>:9099/outputs/@https://pavics.ouranos.ca/wpsoutputs/@g" "docs/source/notebooks/Extract_geographical_watershed_properties.ipynb"

Merged anyways since the build error not related to the notebook I changed:
tests/test_wps_hydrobasins_shape_selection.py .
/home/runner/work/_temp/09d6c889-d3e4-47f8-a805-4f001914160b.sh: line 4:  4210 Segmentation fault      (core dumped) env RAVENPY_TESTDATA_PATH=$GITHUB_WORKSPACE/raven-testdata pytest tests


Performance comparison with FastNoiseSIMD in readme

It would be useful if the readme would also include FastNoiseSIMD in the performance comparison table, currently it only has "FastNoise Lite", "FastNoise (Legacy)" and "FastNoise2 (AVX2)", but no "FastNoiseSIMD (AVX2)".
I'm interested in knowing if there have been any significant performance improvements between FastNoiseSIMD and FastNoise2 that would make updating an existing codebase from FastNoiseSIMD to FastNoise2 worth it.

Here is where the original data come from for those tables, it has FastNoiseSIMD values. https://docs.google.com/spreadsheets/d/1f_ldJeuYvOjc_MxbzNKcURLDcvKIt1L2TuC--Kme-7w/edit?usp=drivesdk
Compare using the median, items per second values

Thanks for that table, that's good to have.
I assume the FastNoiseSIMD values in that table are AVX2, and the table does not contain any AVX512 comparisons?
So for 3D Simplex noise, FastNoiseSIMD AVX2 can do 199.90 "Million points of noise generated per second", and FastNoise2 AVX2 268.44. That's a significant improvement, nice.

They are AVX2 numbers, you can run the AVX512 numbers yourself if you have a CPU that supports it. Here is the benchmark tool I made, https://github.com/Auburn/NoiseBenchmarking


Add cupy.roots for Hermitian or symmetric matrix

Jenkins, test this please.

Jenkins CI test (for commit 5fa543bf5a796df7a76c486b1b6f543f374c029a, target branch master) failed with status FAILURE.

Jenkins, test this please.

Jenkins CI test (for commit cbdbeb7a1281b4b3af80bfd9f348e3b41532c2fb, target branch master) succeeded!

Jenkins, test this please.

Jenkins CI test (for commit f1db85bcbeae35d50794977984b9ea8026f15bb2, target branch master) failed with status FAILURE.

Jenkins, test this please.

Jenkins CI test (for commit f1db85bcbeae35d50794977984b9ea8026f15bb2, target branch master) failed with status FAILURE.

Jenkins, test this please.

Jenkins CI test (for commit f1db85bcbeae35d50794977984b9ea8026f15bb2, target branch master) succeeded!

LGTM.


Outputs fix

Path logic was changed. The flow outputs were printed based on what the PATH key was. It was expecting "0/0" but now the actual value is 0. EXEC_START_PATH constant inside SyncTriggerEventListener class has to be changed.

@tudorlesan is the issue already fixed? Can we close it?

@oritstone yes it can be closed.


Weird behavior for top-level-await with karma runner

While working on a project recently I observed a weird behavior which was difficult to debug but lead to karma runner having a weird behavior for the top-level-await.
Here is a module:
let moduleValue = "uninitialized";

await new Promise((resolve) => {
  setTimeout(() => {
    moduleValue = "initialized";
    resolve();
  }, 200);
});

export default moduleValue;

You will notice that there is a top level await in this module. If following module is imported as following:
import moduleValue from './myModule';

This import works fine within react app and chrome browser. But same import does not work with the karma test with chrome browser. One other weird thing is if we change 200 down to 3 mili seconds then it works fine with Karma.
Here is the webpack configuration which is shared among both:
const webpack = require('webpack');

module.exports = {
  experiments: {
    topLevelAwait: true,
  },
};

So the questions are:

Is there a special configuration or profile which is used to launch the browser with karma, which could impact this behavior?
Should not a regular browser and code with karma behave the exactly same?
Is there any known discrepancies which highlight difference of real browser and a browser launch with Karma?

For reference here is the github repo which have a real use case. Try yarn start and yarn test:karma to compare.
https://github.com/nazarhussain/karma-top-level-await

Having the same problem, investigating too :-)

I did reproduce the problem on both firefox and chromium...


⚠ Bofrost.es has degraded performance

In 8fd86c0, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1396 ms

Resolved: Bofrost.es performance has improved in de9dd06 after 6 minutes.


Handle multiline properties & update css list

Fixes issues with the no-misspelled-properties rule. It currently doesn't work very well with multiline properties and only does partial string matches without trying to understand the context which in turn means it would let spelling errors through.
e.g.
.foo {
  colors: ... // would pass because of -moz-border-bottom-COLORS being a real property
}

I've also moved away from our own maintained list of properties to a more coherent, full and more importantly maintained list known-css-properties. Any properties that aren't featured there can either be requested to be added there OR added to the extra properties setting of this rule. This will lead to a lot less frustration with our poorly maintained list.
fixes #1101
fixes #1024
fixes #1045
fixes #951
fixes #805
fixes #720
closes #1062
closes #767
I'll update the vendor prefixes rule on top of this too and then we can removed our properties file.
<DCO 1.1 Signed-off-by: Dan Purdy<EMAIL_ADDRESS>

Coverage decreased (-0.009%) to 97.509% when pulling 018679b6b3d2a792936d6c775a584b0f150e2836 on feature/update-misspelled-properties into 3f1bc3556833fcfe24041532040297f7435dd561 on develop.

Coverage decreased (-0.01%) to 97.508% when pulling b9243f2240c1319728adc52c23276181acbd47bb on feature/update-misspelled-properties into 3f1bc3556833fcfe24041532040297f7435dd561 on develop.

Coverage decreased (-0.01%) to 97.508% when pulling ec7e238800e3ae0cceac21eea2253c48400731f1 on feature/update-misspelled-properties into 3f1bc3556833fcfe24041532040297f7435dd561 on develop.

Coverage decreased (-0.01%) to 97.508% when pulling ecc7d1b0a4d545a9fe64fb983f5637459e79b457 on feature/update-misspelled-properties into 2225ed5649bef847b23cd03282aa2d8da17b60a2 on develop.

Coverage decreased (-0.009%) to 97.506% when pulling 8a7f07e61a8683d7b826611b75e779ac9817f75c on feature/update-misspelled-properties into ebbc426c537fab6332911914f0d889e8646a1590 on develop.

Coverage decreased (-0.009%) to 97.506% when pulling d157eb86081ab040db7905bc8b528a8aacaeadaa on feature/update-misspelled-properties into ebbc426c537fab6332911914f0d889e8646a1590 on develop.


Use api view and rest response.

Allowing it to work like the rest of your api. Allowing for other parsers and renderers.

Thank you for your contribution !
I'm waiting for @oudeismetis about issue #28 as this might be related.

Just saw this. Only thing different in the PR I just opened is I added the line permission_classes = (AllowAny,)

Merged manually in branch v1.0.3


Set test coverage right.

Coverage remained the same at 13.876% when pulling 6ee343f6f94a5bc85e249322ab73c83e31825c89 on tests into eb19715f990821825759244a7543815a3ebdc8c8 on master.


Internal progress bar

After first run, app requests permission, after accept permission show up internal loading. After restart app all ok.
Try disable permission and run app.

Are you  running the release apk from codemagic or you cloned/built the project?

Are you running the release apk from codemagic or you cloned/built the project?

From codemagic

I've got the exact same problem. Fresh install from Codemagic the app-release.apk after giving storage permission the app hangs with CircularProgressIndicator in the center of the screen

If I kill the app and restart all work correctly.

Any fixes for this issue?

On debugging the issue comes as context != null is not true

Any updates on this?

@yashbansal042 This issue is fixed now. Pull the code from the master branch

@JideGuru Thanks a lot for the update.


GSD Request

--- GSD JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from  to before v4.9.300",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=9e5c40b5706d8aae2cf70bd7e01f0b4575a642d0"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "9e5c40b5706d8aae2cf70bd7e01f0b4575a642d0",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "ASoC: ops: Reject out of bounds values in snd_soc_put_volsw_sx()\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v4.9.300 by commit 9e5c40b5706d8aae2cf70bd7e01f0b4575a642d0. For more details please see the references link."
}
--- GSD JSON ---

This issue has been assigned GSD-2022-1000089


Sample without dnx451

Anyone coming from a non-Microsoft background (NodeJs, etc) or looking at developing a solution that can be deployed anywhere is likely going to want a pure dnxcore50 setup.  It would be helpful to see a sample that didn't include dnx451 to help work through issues via a known good setup.

@ShaneCourtrille many of the samples work on both DNX451 and DNXCORE50, so it's just a matter of deleting the parts of the sample you don't want and you'll have a sample for either system. E.g. in https://github.com/aspnet/Mvc/tree/dev/samples/MvcSample.Web you can edit the project.json file to remove the dnx451 target, as well as delete any part of the source code that mention it, and it should be a 100% functioning sample for dnxcore50. We don't intend to create any samples that necessarily target just one or the other target because most things work the same on both.


Backported Fix for the Creation missmatching calculation

Hi @berendt
the last  PR does not fix the reported Bug
the read as following  must be patched:
https://github.com/ceph/ceph/pull/39270/files

At this point, we rely on releases from the upstream. Backports for Ceph within this container are not provided.

Fixed in Upstream
https://ceph.io/releases/v14-2-17-nautilus-released/

@matfechner I updated the base images 8 days ago (https://github.com/osism/docker-image-ceph-daemon/blob/master/.github/workflows/build-docker-image.yml#L32). Please check if we are now on the right Ceph version.

@berendt, thanks a lot,  yes it is the right version now and it works fine


Display timer to track solving time

To enhance the user experience for speed solving enthusiasts. This feature will allow users to track their solving time and compete against themselves or others.

I want to work on this. please assign me


Remove: CABS.SalesPrix version 6.16.0

## Newer version in repo

[X] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.5 schema?

Microsoft Reviewers: Open in CodeFlow

Hi @Trenly,
This manifest removes Apps and Features entries that are present in previous PR versions. Should these entries be added to this version also?
(Automated response - build 513.)


Improve the ergonomics of the TranspileLayout class

Summary
This commit updates the TranspileLayout class to make it easier to work with. The TranspileLayout class was orginally just a container class that took the information returned from the transpiler needed to reverse the transpiler layout permutation (in the absence of ancillas) so that QuantumCircuit.from_circuit() could compare Operator instances across a transpile(). However, the internal data structures used by the transpiler are hard to work with, and don't map well to the normal reasoning around how the transpiler transforms the circuit. To improve the usability of the class this commit adds 4 new methods to the class:

initial_layout_list() and final_layout_list() which compute a list form of the initial_layout and final_layout respectively
full_layout() which generates a list that maps the input circuits qubit positions in the input circuit to the final position at the end of the circuit (which is what most people think of when they hear "final layout")
permute_sparse_pauli_op() which takes in a SparsePauliOp object and permutes it based on the full layout. This is especially useful when combined with the Estimator primitive

To implement these functions a few extra pieces of information are needed to fully implement them. The first is we need to know the number of original circuit qubits. This is used to account for any ancillas that are added in the circuit, once we have the input circuit count we can use the original_qubit_indices attribute to discern which qubits in the initial layout are from the original circuit and which are ancillas. The second piece of information needed is the list of qubits in the output circuit. as this is needed to look up the position of the virtual qubit in the final_layout. These are both added as optional private attributes to the TranspileLayout class and there are some small changes to the pass manager and QPY to accomodate them.
Similarly the change in the TranspileLayout class requires a new QPY version to include the missing details that were not being serialized in QPY and weren't representable in the previous payload format.
Details and comments
Fixes #10826
Fixes #10818

This looks like it is more or less what is needed here.  However, I am not a big fan of calling the "final layout" the "full layout" as it seems off.  Namely in the initial_layout is what goes in, irrespective of the need for swap mapping.  The final_layout should be the actual "final layout" (here called "full"), with the post-routing layout called something like post_routing_layout  or the like.  If most people think this way, myself included, why not try to reach that end state?

This looks like it is more or less what is needed here. However, I am not a big fan of calling the "final layout" the "full layout" as it seems off. Namely in the initial_layout is what goes in, irrespective of the need for swap mapping. The final_layout should be the actual "final layout" (here called "full"), with the post-routing layout called something like post_routing_layout or the like. If most people think this way, myself included, why not try to reach that end state?

Well the path to changing final_layout to being the equivalent of full_layout in this PR is pretty lengthy because to change the old behavior of the final_layout symbol and migrate the existing functionality to a new name it would be a multi-release process:

add the alternative in this release
deprecate final_layout to warn users in the next release it's going away
remove the attribute after the deprecation period
add a new final_layout() method that matches full_layout() here.

I don't have an issue changing the names of anything added in this PR, full_layout was just the best name I could come up with. We can also add a post_routing_layout alias to start that migration process here. But at least in the medium term we'll need to have a name like full_layout (it can be anything except for final_layout, initial_layout, or input_qubit_mapping) for the method.

One question too is what happens to a layout if the swap mapper decides to use an ancilla qubit for routing? The active number of qubits is no longer the same as the original circuit. Does this just track the permutations of the original set?

For full_layout and permute_sparse_pauli_op it only tracks the permutations of the qubits in the original circuit. I realized I didn't add an explicit test case of this for full_layout to point to (only via permute_sparse_pauli_op).
initial_layout_list has a filter_ancilla option so you can either get the full width or just for the qubits in the original circuit. The final_layout_list method is only full width though, although I guess I could add a filter_ancilla option to it too it just wasn't as natural a fit to me for that given the permutation it tracks.

Building on what's in this PR, maybe something like this works as a full set of coherent operations?  I've judiciously used the word "virtual" to avoid naming clashes and distinguish bi-maps Qubit: int from List[int], but mostly these are just the first names that came to mind:
class TranspileLayout:
    # Bi-map of initial positions (post-layout, pre-routing) of virtual `Qubit`
    # to physical `int`
    def initial_virtual_layout(self, filter_ancillas: bool) -> Layout: ...
    # Map of initial positions virtual list indices to physical `int`
    def initial_index_layout(self, filter_ancillas: bool) -> List[int]: ...
    # Bi-map of final positions (post-routing) of virtual `Qubit` to physical `int`.
    def final_virtual_layout(self, filter_ancillas: bool) -> Layout: ...
    # Map of final positions virtual list indices to physical `int`.
    def final_index_layout(self, filter_ancillas: bool) -> List[int]: ...

    # Potentially this too (I can't see much meaning for `filter_ancillas` here.)
    def final_permutation(self) -> List[int]: ...

    # ... and now a bunch of helper methods that could also live elsewhere.
    def permute_to_initial(self, operator: OperatorLike) -> OperatorLike: ...
    def permute_to_final(self, operator: OperatorLike) -> OperatorLike: ...

For comparison:

initial_virtual_layout is like the current initial_layout attribute, except it gains a filter_ancillas argument
initial_index_layout is the PR's initial_layout_list
final_virtual_layout has no counterpart
final_index_layout is like the PR's full_layout except it gains a filter_ancillas argument
final_permutation is like the current final_layout attribute, except it more accurately uses a List[int] rather than mapping.
permute_to_initial has no counterpart
permute_to_final is the PR's permuate_sparse_pauli_op

I know the argument for full_layout is to avoid having a name too close to the current final_layout attribute, but here I'm with Paul - I think the current attribute has the wrong name, and we it'd be better not to constrain the new API based on it, especially if we document that people should avoid using the attributes and should instead always use the methods.

I should also say: I like the general direction of the PR, and I think this will be a very nice ergonomic improvement for users of the transpiler across the board - it's not exposing any information they couldn't already calculate themselves, but it's definitely making it easier to work with.

I think my issue with the existing final_layout attribute is that, to me, it's not a layout at all. Imo (by the typing/terminology of the class) Layouts must be a mapping of the virtual space to the physical space, while this is a mapping of the physical space to itself linked by which hidden virtual qubit is assigned to the physicals at the start/end.  That's why I had it only as routing_permutation, and then only using integer indices; I think that particular object is quite different to the returns from the methods that I called *_layout.
I think the closest coherent object we could put out for the current final_layout which makes the virtual Qubit instances explicit would be a mapping of something like Qubit: (int, int), which  be computed as
initial = transpile_layout.initial_virtual_layout(filter_ancillas=True)
final = transpile_layout.final_virtual_layout(filter_ancillas=True)
{
   virtual: (initial[virtual], final[virtual])
   for virtual, initial_physical in initial.get_virtual_bits().items()
}

I'm not super sure what use such an object could have though, which is why I didn't include it.
In terms of methods vs attributes: personally, I'd just make everything public here a method, even if (as an internal detail), it just returns a copy of an attribute.  The cost of copying a layout is small compared to any operation using layouts, and users shouldn't really be able to mutate a TranspileLayout anyway; it wouldn't mean anything and would just make the circuit/layout correspondence invalid.

I reworked the API per the suggestions in https://github.com/Qiskit/qiskit/pull/10835/commits/3f2542b79e65d79acea1c037cbe1786803ffd977 .  The only thing I haven't touched is the permute_sparse_pauli_op method. Mostly because I'm not sure which operator types I should support. I didn't see a common interface that would let me do the permutation in the same manner on something like BaseOperator in a consistent efficient manner. So we should figure out which set of operator classes we want to support for the new methods (and optionally where we should put the functionality)

I will improve BaseEstimator after this PR. Some operator classes beyond SparsePauliOp are being discussed internally, so it can be a future issue.

I played around with this and it works quite well.

I've updated this PR to not have the permute_sparse_pauli_op method anymore. I pushed a follow-up PR #10947 that adds an apply_layout method to SparsePauliOp that has the same functionality. It is a better fit on the operator class, and it also enables us to add similar methods to the other operators in the future if needed.


Support custom initializer in links.CRF1d

Currently, the parameter of CRF1d is initialized by zero.
But this parameter could be initialized randomly.
There are some example using normal distribution (1) and uniform distribution (2 and 3).
So, in this PR, I change links.loss.CRF such that it accept the attribute initialW.

Could you add to the docs of __init__ the introduced argument? It would be great if you could document the default zero behavior too.

Could you add to the docs of __init__ the introduced argument? It would be great if you could document the default zero behavior too.

Sorry, I added.

Thanks for you quick response. I wrote some minor comments. Also, do you have any inputs on this one https://github.com/chainer/chainer/pull/5807/files#r240513089 ?

Thanks for comments, I updated:pray:.

Thanks for comments, I updated.

jenkins, test this please.

Jenkins CI test (for commit b4c06a3593675c95d6980758d8d6c535f8824f95, target branch master) succeeded!

Jenkins CI test (for commit b4c06a3593675c95d6980758d8d6c535f8824f95, target branch master) succeeded!

@hvy I'm not sure whether it is correct, I added a test for links.CRF1d.
Currently, I only checked whether cost is initialized with zeros or not.

Thanks again! I left some comments, if you could take a look. Maybe I didn't communicate well enough but I really meant that you could implement a forward only check similar to https://github.com/chainer/chainer/blob/master/tests/chainer_tests/links_tests/activation_tests/test_maxout.py

I apologize that I made a test based on not test_maxout.py but test_black_out.py :bow:.
Thanks all, I'll revise it.

I update PR.
I tried to check following two things.

check whether forward works correctly over initializers
check whether initialize works correctly over initializers

Please review them:bow:

jenkins, test this please.

Jenkins CI test (for commit 1fb36e5083501437cc546b9b0bfbe0c7940f187b, target branch master) failed with status FAILURE.
(For contributors, please wait until the reviewer confirms the details of the error.)

Sorry for repeated comments. Could you take another look you think?

Thanks for the fixes. I'll take another look.

jenkins, test this please.

Added some minor comments. Could you take a look?

Jenkins CI test (for commit dca1cfbc1611bd3c865bbf521e50fc57b44cb47f, target branch master) failed with status FAILURE.
(For contributors, please wait until the reviewer confirms the details of the error.)

It basically LGTM but I added a minor comment since it causes the test to fail. Please take a look.

I removed the unnecessary branch, thanks:bow:.

jenkins, test this please.

Jenkins CI test (for commit 4f4c929f81b794251bb63abccc9a728ed99ad10d, target branch master) failed with status FAILURE.
(For contributors, please wait until the reviewer confirms the details of the error.)

It seems like test_forward_gpu test are failing for float16. Could take a look at it, maybe relax the tolerances a bit?

It is caused because NumPy(1.9.3) can't take cupy.core.core.ndarray as an input.
I avoid this error by calling chainer.cuda.to_cpu before numpy.exp.
@hvy What do you think about this workaround?

Thanks, I'll retrigger the CI.

jenkins, test this please.

Jenkins CI test (for commit ee31d0215f8f6d06366eb788fb365e79f99dacaf, target branch master) failed with status FAILURE.
(For contributors, please wait until the reviewer confirms the details of the error.)

The CI failure above are unrelated to the changes in this PR (it's a random tolerance failure). It LGTM so I'll merge it once other CIs pass!

Again, thanks for the PR and all the debugging. LGTM!

I sincerely appreciate your kind review :pray:!!


[VR920 - SensoApp] Cannot change the temperature (quick veto) on a room

How to reproduce:

Select a room
Change the temperature of the room (Quick Veto)

Expected behavior
The temperature of the room should change for 3 hours
Current behavior
I receive an error and nothing happens

Issue fixed with the PR #200


option to create without destination manifest / just detination manifest

For DCDM's to save time can we have an option to option to create ~without destination manifest  ~ just detination manifest?

Hey, thanks for raising this as well - we need to do this. I didn't think of adding that option, I think that's actually the best/easiest way to do it for now.
We can probably either alter move.py, or create a verify.py script to do the destionation manifest check if needs be.

Also, doing a manual check with md5deep is very simple if it comes to it
md5deep -ler drivename > manifest.md5 - then we can compare afterwards.

also lol I only fixed this after @AnjaMahler is not even with the IFI anymore! :(


cuDNN error: CUDNN_STATUS_EXECUTION_FAILED when training models from recipes

Dear all,
I would like to use SpeechBrain. I have started by creating models when based on the LibriSpeech recipe.
I am trying to create LM model. I have gor the bellow errer despite i have two GPU and 128 Go Ram
speechbrain.utils.epoch_loop - Going into epoch 1
0%|                                                                                | 0/10172552 [00:02<?, ?it/s]
speechbrain.core - Exception:
Traceback (most recent call last):
File "/srv/nas_data/atrabelsi/speechbrain/recipes/LibriSpeech/LM/train.py", line 207, in
lm_brain.fit(
File "/srv/nas_data/atrabelsi/speechbrain/speechbrain/core.py", line 1129, in fit
self._fit_train(train_set=train_set, epoch=epoch, enable=enable)
File "/srv/nas_data/atrabelsi/speechbrain/speechbrain/core.py", line 985, in _fit_train
loss = self.fit_batch(batch)
File "/srv/nas_data/atrabelsi/speechbrain/recipes/LibriSpeech/LM/train.py", line 54, in fit_batch
predictions = self.compute_forward(batch, sb.Stage.TRAIN)
File "/srv/nas_data/atrabelsi/speechbrain/recipes/LibriSpeech/LM/train.py", line 39, in compute_forward
logits = self.hparams.model(tokens_bos)
File "/home/jupyter-atrabels/.conda/envs/speech_brain/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/srv/nas_data/atrabelsi/speechbrain/speechbrain/lobes/models/RNNLM.py", line 112, in forward
x, hidden = self.rnn(x, hx)
File "/home/jupyter-atrabels/.conda/envs/speech_brain/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/srv/nas_data/atrabelsi/speechbrain/speechbrain/nnet/RNN.py", line 271, in forward
output, hn = self.rnn(x)
File "/home/jupyter-atrabels/.conda/envs/speech_brain/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jupyter-atrabels/.conda/envs/speech_brain/lib/python3.9/site-packages/torch/nn/modules/rnn.py", line 661, in forward
result = _VF.lstm(input, hx, self._flat_weights, self.bias, self.num_layers,
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED
Best regards

Hi, this typically is due to OOM. Reduce the batch size and increase the gradient_accumulation to compensate.

Hello,
Any news on this issue, please? Thanks!

Hello,
it seems that there has been no activity for a long time, so I am closing this issue.
Feel free to reopen it if you still encounter the problem. Thank you. 🙂


Tests for Proxy Functionality

This is intended to provide some tests to prove that the functionality of proxy works and to prevent regressions.

@mhjort: Since you're looking at this already, can you take a look at this updated one and suggest some further tests?  I've got to handle some day-job stuff so I don't have time to think until later tonight.
@davidkazlauskas: please DO NOT MERGE until we have some consensus on whether testing is complete enough.

Had some time in a meeting to think:
Still needed:

[ ] add negative test (should fail w/ exception if cannot connect out)
[ ] add with-http-fake test (?)

I decided not to add the with-http-fake tests because that should be handled/added by http-fake.
httpkit.fake seems to as expected (it ignores the proxy), so I didn't add it.
Unless someone sees something I need to add/change, I think this is ready to merge.

Added a test demonstrating how to use basic-auth with this proxy implementation.
@ptaoussanis @mhjort look good?
@davidkazlauskas I'm happy enough with this to merge.


Can't deploy: Error ENOTFOUND

Can't deploy this.
I get a cryptic error message saying:
Sun Jul 15 2018 22:19:18 GMT+0000 (UTC) – fetching
> [0] events.js:183
> [0]       throw er; // Unhandled 'error' event
> [0]       ^
> [0]
> [0] Error: getaddrinfo ENOTFOUND HB1iBOkiV3PKDlmNp2ip5eS1

How to fix this?
Thanks a lot!

HI. I'm having the same issue. Did anyone manage to resolve it?

Hi, I did, I had to manually modify the code, I changed bin/slackin to bin/slacking.js, I changed the start  scripts on package.json to
"scripts": {
    "start": "cd bin && node slackin"
  }

I removed the post-install and build commands because for some reason they were executed on the server but the start command will never be called so I had to remove them.
Before you upload make sure that you have built the code (running gulp) and don't forget to include the 'dist' folder on the now.sh, files since there is no more 'build' command anymore.
"now": {
    "type": "npm",
    "files": [
      "bin",
      "lib",
      "dist",
      "gulpfile.babel.js"
    ],

I'm still getting same issue even after following above any other solutions ?

I forgot to meantion that i deleted everything on the slackin.js and only left:
slackin(flags).listen(port, hostname, function(err) {
  if (err) throw err;
  if (!flags.silent)
    console.log("%s – listening on %s:%d", new Date(), hostname, port);
});

Of course I added the keys into the flags object.

I'm having this same issue. I've previously deployed it on Now with no problems but something must have changed since then. If anyone else has ideas on how to get it working again it'd be much appreciated!


Open generics registration

PR for #1286
Well, I took a glance through source code and found out that FluentValidation takes care of registration. Therefore it could be its responsibility to handle open generics, as it does with regular types through .AddFluentValidation(fv => fv.RegisterValidatorsFromAssemblyContaining<Startup>()) extension. Said that, I made some changes to the FluentValidation and FluentValidation.DependencyInjectionExtension projects and seems like it does a trick. The only thing I need is check and verify if these changes are valid, because I didn't have a much time to check everything out and honestly it is a first time a looked at source code. Thanks)

Thanks for the suggestion, but this is not in scope for the project, so isn’t something that will be included. Handling open generics by default is not as simple as the change you’ve made - Please see my comment on the other issue for more details. Thanks.


Implement Constructable inventory features

This adds support for the Constructable inventory features, addressing issue #171. Currently it adds the following data from the results of the ManagedCluster API call:

client_config (from c.spec.managedClusterClientConfigs)
annotations (except for last-applied-configuration)
labels

With this, you could for example add the following to an inventory.yml file:
compose:
  openshift_api: client_config.url
keyed_groups:
  - prefix: env
    key: labels.env

Let me know if you'd like me to continue working on this!

@philipsd6 thanks for this, will enable this today

My latest changes are quite extensive and opinionated! It should be backwards compatible, but now you can choose to NOT name your cluster_groups, and instead rely on Constructable features to group.


OLS-721: Script to generate packages to prefetch in Cachi2

Description
Script to generate packages to prefetch in Cachi2
This script performs several steps:
1. removes torch+cpu dependency from project file
2. generates requirements.txt file from pyproject.toml + pdm.lock
3. removes all torch dependencies (including CUDA/Nvidia packages)
4. downloads torch+cpu wheel
5. computes hashes for this wheel
6. adds the URL to wheel + hash to resulting requirements.txt file
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library
[ ] Bump-up library or tool used for development (does not change the final image)
[x] Helper script

Related Tickets & Documents

Related Issue #OLS-721
Closes #

/lgtm
/approve

/retest


Support generic data classes

data class Response<T : Parcelable>(
  val code: Int,
  val message: String,
  val body: T
) : AutoParcelable

hey! where should I start looking if I'd want to implement this?

Closing the issue as we don't use the library anymore and we don't have plans on supporting it.


vikas.html

It is a payment form created using HTML CSS.. actually ma'am i am beginner here ..so i just thought to contribute in your repo by this code..kindly merge it ...it would be a great help.

Thanks @vikastripathi707 for your payment form suggestion. For now, as this doesn't directly connect to any of the current lessons, the best place to share your creation would be in https://github.com/microsoft/Web-Dev-For-Beginners/discussions/categories/show-and-tell.
Closing this PR so you can share your payment form within Discussions. When we build out the ability to onboard additional projects to our repo for extra practice and exercises, we will announce to the community. Thank you!

Thanks @vikastripathi707 for your payment form suggestion. For now, as this doesn't directly connect to any of the current lessons, the best place to share your creation would be in https://github.com/microsoft/Web-Dev-For-Beginners/discussions/categories/show-and-tell.
Closing this PR so you can share your payment form within Discussions. When we build out the ability to onboard additional projects to our repo for extra practice and exercises, we will announce to the community. Thank you!