[k3s] no additonal interface in pod

Hi,
I've installed multus-cni but it seems that i've something missed...
I'm on a 2 master kubernetes build with k3sup:
$ uname -a:
Linux master-100 5.4.0-48-generic #52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Linux master-101 5.4.0-48-generic #52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

$ kubectl get nodes -o wide:
NAME         STATUS   ROLES         AGE   VERSION        INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
master-100   Ready    etcd,master   18m   v1.19.2+k3s1   <IP_ADDRESS>   <none>        Ubuntu 20.04.1 LTS   5.4.0-48-generic   containerd://1.4.0-k3s1
master-101   Ready    etcd,master   17m   v1.19.2+k3s1   <IP_ADDRESS>   <none>        Ubuntu 20.04.1 LTS   5.4.0-48-generic   containerd://1.4.0-k3s1

I've cloned the repo with git clone https://github.com/intel/multus-cni.git and modfied the multus-daemonset.yml the argsby adding -  "--multus-log-level=debug"
Run
$ kubectl apply -f multus-cni/images/multus-daemonset.yml
#customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io created
clusterrole.rbac.authorization.k8s.io/multus created
clusterrolebinding.rbac.authorization.k8s.io/multus created
serviceaccount/multus created
configmap/multus-cni-config created
daemonset.apps/kube-multus-ds-amd64 created
daemonset.apps/kube-multus-ds-ppc64le created
daemonset.apps/kube-multus-ds-arm64v8 created

Take a look into the the multus pod...
kubectl logs -n kube-system kube-multus-ds-amd64-qxlq8
2020-10-04T23:34:33+0000 Generating Multus configuration file using files in /host/etc/cni/net.d...
2020-10-04T23:34:33+0000 Attemping to find master plugin configuration, attempt 0
2020-10-04T23:34:39+0000 Attemping to find master plugin configuration, attempt 5
2020-10-04T23:34:44+0000 Attemping to find master plugin configuration, attempt 10
2020-10-04T23:34:49+0000 Attemping to find master plugin configuration, attempt 15
2020-10-04T23:34:54+0000 Attemping to find master plugin configuration, attempt 20
2020-10-04T23:34:59+0000 Attemping to find master plugin configuration, attempt 25

witch tried to find the config... so I've created one:
$ cat /etc/cni/net.d/10-multus.conf
{
   "name":"multus-cni-network",
   "type":"multus",
   "logFile":"/var/log/multus.log",
   "logLevel":"debug",
   "capabilities":{
      "portMappings":true
   },
   "delegates":[
      {
	 "cniVersion":"0.3.1",
	 "name":"default-cni-network",
	 "plugins":[
	    {
	       "type":"flannel",
	       "name":"flannel.1",
	       "delegate":{
	          "isDefaultGateway":true,
	          "hairpinMode":true
	       }
	    },
	    {
	       "type":"portmap",
	       "capabilities":{
	          "portMappings":true
	       }
	    }
	 ]
      }
   ],
   "kubeconfig":"/etc/cni/net.d/multus.d/multus.kubeconfig"
}

So the pod created a new config:
2020-10-04T23:35:58+0000 Using /host/etc/cni/net.d/10-multus.conf as a source to generate the Multus configuration
2020-10-04T23:35:59+0000 Config file created @ /host/etc/cni/net.d/00-multus.conf
{ "cniVersion": "0.3.1", "name": "multus-cni-network", "type": "multus", "logLevel": "debug", "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig", "delegates": [ { "name":"multus-cni-network", "type":"multus", "logFile":"/var/log/multus.log", "logLevel":"debug", "capabilities":{ "portMappings":true }, "delegates":[ { "cniVersion":"0.3.1", "name":"default-cni-network", "plugins":[ { "type":"flannel", "name":"flannel.1", "delegate":{ "isDefaultGateway":true, "hairpinMode":true } }, { "type":"portmap", "capabilities":{ "portMappings":true } } ] } ], "kubeconfig":"/etc/cni/net.d/multus.d/multus.kubeconfig" } ] }
2020-10-04T23:35:59+0000 Entering sleep (success)...

Now I've created a new definition:
$ cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf-1
spec:
  config: '{
	    "cniVersion": "0.3.1",
	    "type": "macvlan",
	    "master": "eno1",
	    "mode": "bridge",
	    "ipam": {
	        "type": "host-local",
	        "ranges": [
	            [ {
	                 "subnet": "<IP_ADDRESS>/16",
	                 "rangeStart": "<IP_ADDRESS>",
	                 "rangeEnd": "<IP_ADDRESS>",
	                 "gateway": "<IP_ADDRESS>"
	            } ]
	        ]
	    }
	}'
EOF

And create a pod, which using it:
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: pod-case-01
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-conf-1
spec:
  containers:
  - name: pod-case-01
    image: docker.io/centos/tools:latest
    command:
    - /sbin/init
EOF

But quering the pod for the interfaces still only 1(2) interfaces listed:
$ kubectl exec pod-case-01 -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: eth0@if27: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
    link/ether 26:fc:32:bf:a1:2b brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::24fc:32ff:febf:a12b/64 scope link
       valid_lft forever preferred_lft forever

Nothing shown in kubernetes events, no further log in the multus pods and no /var/log/multus.log file

Ahhh ha! I think you found it, where it's reading: Attemping to find master plugin configuration
One thing that you need to do is install a "default network CNI first" -- you'll see in the requirements in the quickstart: https://github.com/intel/multus-cni/blob/master/doc/quickstart.md#prerequisites
Typically, what I'd do at least for testing is to install Flannel.
I'm unfamiliar with k3sup, but, looking at this link: https://github.com/alexellis/k3sup#-setup-a-kubernetes-server-with-k3sup
I notice there's a --flannel-backend flag, and also, you could install it by just applying the flannel yaml specs @ https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml
Once you have any file (that's not a multus config) in /etc/cni/net.d -- then Multus can be installed. Multus uses the default network's configuration to configure itself, so it can use that configuration to create your default interfaces (the eth0)

k3s comes with flannel default, so the --flannel-backend flag is only required, if the backend should be changed (e.g. to host-gw)
The /etc/cni/net.d directory is empty, so k3s doesn't write a config in there...
Since i now know from the flannel cni gitlab repo, that the config is a .conflist file it was easy to find the file on the host system. k3s stores it under /var/lib/rancher/k3s/agent/etc/cni/net.d/10-flannel.conflist.
So my idea was to modify the multus-daemonset.yml changing:
https://gist.github.com/tburschka/155c0eba756672505cd31582c2875fca
while detection of config now works, testing the new configuration leads to another error:
default           0s          Normal    Scheduled                pod/web-dhcp                                                          Successfully assigned default/web-dhcp to master-101
default           0s          Warning   FailedCreatePodSandBox   pod/web-static                                                        Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "13ce07fb1325d9f92eb28308173604fa4c8f358998edf4b76635e19ca2b84433": failed to find plugin "multus" in path [/var/lib/rancher/k3s/data/07bf5246a6ab2127234428fbf3023ce85b0376a7b946bbdaee2726b7d9a6fad8/bin]

in this directory are a lot of files symlinked (filtered here for files with cni):
lrwxrwxrwx 1 root root    3 Oct 15 21:53 bridge -> cni
-rwxr-xr-x 1 root root 3.2M Sep 21 17:00 cni
lrwxrwxrwx 1 root root    3 Oct 15 21:53 flannel -> cni
lrwxrwxrwx 1 root root    3 Oct 15 21:53 host-local -> cni
lrwxrwxrwx 1 root root    3 Oct 15 21:53 loopback -> cni
lrwxrwxrwx 1 root root    3 Oct 15 21:53 portmap -> cni

so my last idea was to symlink multus as well (tried cni and /opt/cni/bin/multus) which causes different errors:
default       0s          Warning   FailedCreatePodSandBox   pod/web-static                   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1a8c051849cb3034d067feca5fd44f5062c0ebd94cf735fc1e06aafaf89da8e9": unexpected end of JSON input

default       0s          Warning   FailedCreatePodSandBox   pod/web-static                   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "5fb36c8f9ca50131f915f203a1e38c3cee179be00f7b77a38a3458bdb856cb59": Multus: [default/web-static]: error getting pod: Unauthorized

(This seems more to be an k3s issue, that multus isn't symlinked)
Little bit off-topic... i've got an alternative working, but the flannel variant would be nice since less overhead: https://gist.github.com/tburschka/4bc938c6ca733cbd366a24e301753685 )

Sorry for slow response!
I see that it uses /var/lib/rancher/k3s/agent/etc/cni/net.d/10-flannel.conflist -- I think you need to configure the Multus daemonset to say "Hey you need to look in a specific directory for the configuration"
You can find the configuration parameters here @ https://github.com/intel/multus-cni/blob/master/doc/how-to-use.md#entrypoint-parameters and then modify the Multus daemonset as provided by the quickstart.
...If they can get the right parameters, we might consider adding this to the reference deployments @ https://github.com/k8snetworkplumbingwg/reference-deployment -- to have a reference deployment against k3s

I followed these instructions (mostly) and they worked for me: https://www.reddit.com/r/rancher/comments/ilzdp7/howto_set_up_k8s_or_k3s_so_pods_get_ip_from_lan/
With this addition beforehand (you need CNI plugins):
wget https://github.com/containernetworking/plugins/releases/download/v0.9.0/cni-plugins-linux-amd64-v0.9.0.tgz
mkdir -p /opt/cni/bin
tar -C /opt/cni/bin -xvzf cni-plugins-linux-amd64-v0.9.0.tgz

Basically, don't let k3s install flannel for you if you want to use Multus, deploy flannel by hand instead.

I had the same issue, the 00-multus.conf points to the wrong kubconfig file.
Fixed it by setting "kubeconfig": "/var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig" in the multus-cni-config configmap. I also changed the name of the mounted configmap to 00-multus.conf, so that it's the first file which gets loaded by kubelet.
I also discovered that you can set cnibin to /var/lib/rancher/k3s/data/current/bin, which then points to the correct data directory.
Here my configmap and daemonset:
kind: ConfigMap
apiVersion: v1
metadata:
  name: multus-cni-config
  namespace: kube-system
  labels:
    tier: node
    app: multus
data:
  # NOTE: If you'd prefer to manually apply a configuration file, you may create one here.
  # In the case you'd like to customize the Multus installation, you should change the arguments to the Multus pod
  # change the "args" line below from
  # - "--multus-conf-file=auto"
  # to:
  # "--multus-conf-file=/tmp/multus-conf/70-multus.conf"
  # Additionally -- you should ensure that the name "70-multus.conf" is the alphabetically first name in the
  # /etc/cni/net.d/ directory on each node, otherwise, it will not be used by the Kubelet.
  cni-conf.json: |
    {
        "cniVersion": "0.3.1",
        "name": "multus-cni-network",
        "type": "multus",
        "kubeconfig": "/var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig",
        "delegates": [
            {
                "name": "cbr0",
                "cniVersion": "0.3.1",
                "plugins": [
                    {
                        "type": "flannel",
                        "delegate": {
                            "hairpinMode": true,
                            "forceAddress": true,
                            "isDefaultGateway": true
                        }
                    },
                    {
                        "type": "portmap",
                        "capabilities": {
                            "portMappings": true
                        }
                    }
                ]
            }
        ]
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-multus-ds-amd64
  namespace: kube-system
  labels:
    tier: node
    app: multus
    name: multus
spec:
  selector:
    matchLabels:
      name: multus
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        tier: node
        app: multus
        name: multus
    spec:
      hostNetwork: true
      nodeSelector:
        kubernetes.io/arch: amd64
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: multus
      containers:
      - name: kube-multus
        image: docker.io/nfvpe/multus:stable
        command: ["/entrypoint.sh"]
        args:
        - "--multus-conf-file=/tmp/multus-conf/00-multus.conf"
        - "--cni-version=0.3.1"
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
          limits:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: true
        volumeMounts:
        - name: cni
          mountPath: /host/etc/cni/net.d
        - name: cnibin
          mountPath: /host/opt/cni/bin
        - name: multus-cfg
          mountPath: /tmp/multus-conf
      volumes:
        - name: cni
          hostPath:
            path: /var/lib/rancher/k3s/agent/etc/cni/net.d
        - name: cnibin
          hostPath:
            path: /var/lib/rancher/k3s/data/current/bin
        - name: multus-cfg
          configMap:
            name: multus-cni-config
            items:
            - key: cni-conf.json
              path: 00-multus.conf

While i'm using calico (for the moment) i read the comments and updated my flannel config based on the comments:
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: multus-cni-config
  namespace: kube-system
  labels:
    tier: node
    app: multus
data:
  # NOTE: If you'd prefer to manually apply a configuration file, you may create one here.
  # In the case you'd like to customize the Multus installation, you should change the arguments to the Multus pod
  # change the "args" line below from
  # - "--multus-conf-file=auto"
  # to:
  # "--multus-conf-file=/tmp/multus-conf/70-multus.conf"
  # Additionally -- you should ensure that the name "70-multus.conf" is the alphabetically first name in the
  # /etc/cni/net.d/ directory on each node, otherwise, it will not be used by the Kubelet.
  cni-conf.json: |
    {
      "name": "multus-cni-network",
      "type": "multus",
      "capabilities": {
        "portMappings": true
      },
      "delegates": [
        {
          "cniVersion": "0.3.1",
          "name": "default-cni-network",
          "plugins": [
            {
              "type": "flannel",
              "name": "flannel.1",
                "delegate": {
                  "isDefaultGateway": true,
                  "hairpinMode": true
                }
              },
              {
                "type": "portmap",
                "capabilities": {
                  "portMappings": true
                }
              }
          ]
        }
      ],
      "kubeconfig": "/var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig"
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-multus-ds-amd64
  namespace: kube-system
  labels:
    tier: node
    app: multus
    name: multus
spec:
  selector:
    matchLabels:
      name: multus
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        tier: node
        app: multus
        name: multus
    spec:
      hostNetwork: true
      nodeSelector:
        kubernetes.io/arch: amd64
      tolerations:
        - operator: Exists
          effect: NoSchedule
      serviceAccountName: multus
      containers:
        - name: kube-multus
          image: docker.io/nfvpe/multus:stable
          command: ["/entrypoint.sh"]
          args:
          - "--multus-conf-file=auto"
          - "--cni-version=0.3.1"
          resources:
            requests:
              cpu: "100m"
              memory: "50Mi"
            limits:
              cpu: "100m"
              memory: "50Mi"
          securityContext:
            privileged: true
          volumeMounts:
            - name: cni
              mountPath: /host/etc/cni/net.d
            - name: cnibin
              mountPath: /host/opt/cni/bin
            - name: multus-cfg
              mountPath: /tmp/multus-conf
      volumes:
        - name: cni
          hostPath:
            path: /var/lib/rancher/k3s/agent/etc/cni/net.d
        - name: cnibin
          hostPath:
            path: /usr/lib/cni
        - name: multus-cfg
          configMap:
            name: multus-cni-config
            items:
              - key: cni-conf.json
                path: 70-multus.conf

Now i have the issue, that k3s can't find multus:
default       0s          Warning   FailedCreatePodSandBox   pod/web-dhcp                     Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b8f95d23e47b2ea1cec335d42dbc2a32b0f69d9f3934a7466890a0a24ef07578": failed to find plugin "multus" in path [/var/lib/rancher/k3s/data/986d5e8cf570f904598f9a5d531da2430e5a6171d22b7addb1e4a7c5b87a47d0/bin]
default       0s          Warning   FailedCreatePodSandBox   pod/web-static                   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e146b306829ac251670a027846aae78341d3e6b570d28f2b35205c0f3f1188d3": failed to find plugin "multus" in path [/var/lib/rancher/k3s/data/986d5e8cf570f904598f9a5d531da2430e5a6171d22b7addb1e4a7c5b87a47d0/bin]

Since the host system is an ubuntu and the cni plugins are located in /usr/lib/cni the error message is correct, but i couldn't find a way (yet) to fix this issue (and symlink files is a possible workaround but it would be nice, if i don't need to to this).

While i'm using calico (for the moment) i read the comments and updated my flannel config based on the comments:
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: multus-cni-config
  namespace: kube-system
  labels:
    tier: node
    app: multus
data:
  # NOTE: If you'd prefer to manually apply a configuration file, you may create one here.
  # In the case you'd like to customize the Multus installation, you should change the arguments to the Multus pod
  # change the "args" line below from
  # - "--multus-conf-file=auto"
  # to:
  # "--multus-conf-file=/tmp/multus-conf/70-multus.conf"
  # Additionally -- you should ensure that the name "70-multus.conf" is the alphabetically first name in the
  # /etc/cni/net.d/ directory on each node, otherwise, it will not be used by the Kubelet.
  cni-conf.json: |
    {
      "name": "multus-cni-network",
      "type": "multus",
      "capabilities": {
        "portMappings": true
      },
      "delegates": [
        {
          "cniVersion": "0.3.1",
          "name": "default-cni-network",
          "plugins": [
            {
              "type": "flannel",
              "name": "flannel.1",
                "delegate": {
                  "isDefaultGateway": true,
                  "hairpinMode": true
                }
              },
              {
                "type": "portmap",
                "capabilities": {
                  "portMappings": true
                }
              }
          ]
        }
      ],
      "kubeconfig": "/var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig"
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-multus-ds-amd64
  namespace: kube-system
  labels:
    tier: node
    app: multus
    name: multus
spec:
  selector:
    matchLabels:
      name: multus
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        tier: node
        app: multus
        name: multus
    spec:
      hostNetwork: true
      nodeSelector:
        kubernetes.io/arch: amd64
      tolerations:
        - operator: Exists
          effect: NoSchedule
      serviceAccountName: multus
      containers:
        - name: kube-multus
          image: docker.io/nfvpe/multus:stable
          command: ["/entrypoint.sh"]
          args:
          - "--multus-conf-file=auto"
          - "--cni-version=0.3.1"
          resources:
            requests:
              cpu: "100m"
              memory: "50Mi"
            limits:
              cpu: "100m"
              memory: "50Mi"
          securityContext:
            privileged: true
          volumeMounts:
            - name: cni
              mountPath: /host/etc/cni/net.d
            - name: cnibin
              mountPath: /host/opt/cni/bin
            - name: multus-cfg
              mountPath: /tmp/multus-conf
      volumes:
        - name: cni
          hostPath:
            path: /var/lib/rancher/k3s/agent/etc/cni/net.d
        - name: cnibin
          hostPath:
            path: /usr/lib/cni
        - name: multus-cfg
          configMap:
            name: multus-cni-config
            items:
              - key: cni-conf.json
                path: 70-multus.conf

Now i have the issue, that k3s can't find multus:
default       0s          Warning   FailedCreatePodSandBox   pod/web-dhcp                     Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b8f95d23e47b2ea1cec335d42dbc2a32b0f69d9f3934a7466890a0a24ef07578": failed to find plugin "multus" in path [/var/lib/rancher/k3s/data/986d5e8cf570f904598f9a5d531da2430e5a6171d22b7addb1e4a7c5b87a47d0/bin]
default       0s          Warning   FailedCreatePodSandBox   pod/web-static                   Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e146b306829ac251670a027846aae78341d3e6b570d28f2b35205c0f3f1188d3": failed to find plugin "multus" in path [/var/lib/rancher/k3s/data/986d5e8cf570f904598f9a5d531da2430e5a6171d22b7addb1e4a7c5b87a47d0/bin]

Since the host system is an ubuntu and the cni plugins are located in /usr/lib/cni the error message is correct, but i couldn't find a way (yet) to fix this issue (and symlink files is a possible workaround but it would be nice, if i don't need to to this).

I followed these instructions (mostly) and they worked for me: https://www.reddit.com/r/rancher/comments/ilzdp7/howto_set_up_k8s_or_k3s_so_pods_get_ip_from_lan/
With this addition beforehand (you need CNI plugins):
wget https://github.com/containernetworking/plugins/releases/download/v0.9.0/cni-plugins-linux-amd64-v0.9.0.tgz
mkdir -p /opt/cni/bin
tar -C /opt/cni/bin -xvzf cni-plugins-linux-amd64-v0.9.0.tgz

Basically, don't let k3s install flannel for you if you want to use Multus, deploy flannel by hand instead.

Any way to uninstall the default flannel installed by k3s and deploy it by hand without uninstalling/re-installing the entire k3s cluster?


registerAsync Error version 0.8.4

I get an error  Nest can't resolve dependencies of the controller (service, ?).... Please make sure that the argument at index [1] is available when using BullModule.registerAsync, but it works fine when using BullModule.register().  I use a shared module, and import it in another module.
nest-bull version 0.8.4

@jayakusumah Thanks for reporting this
I'll try to look into this issue in the next few days. It'd be great if, in the meantime, you could provide some code reproducing this error, or even submit a PR with failing tests.

I got the same problem: controllers in others module cannot inject queue defined in another module.
I tried exports processors, import queue defined module but no luck.
https://github.com/smonv/bull-queue-test here is code for quick setup demo the problem

I found a workaround to solve this problem. Instead of directed inject queue inside controller, indirect inject queue by a service.
Create a task service inside QueueModule so naturally that service can inject queue. Export task service and let other controllers import it.

Same issue, in my case I need to use registerAsync so that I can load my ConfigService to get redis credentials from.

I think I've found the issue, when you use registerAsync it doesn't respect the name from the options and registering the ff.

@smonv The queue can only be injected in the scope where BullModule has been registered. That's why you can directly inject it in another module without wrapping it in a service that you export. This is the expected behavior.

I'm experiencing this same problem. Is there a work around at this moment?

Yes, you'll need to create a module to reexport the BullModule
Here's an example:
import { BullModule, BullModuleOptions } from 'nest-bull'
import { Module } from '@nestjs/common'

import { ConfigModule } from 'src/config/config.module'
import { ConfigService } from 'src/config/config.service'

function redisOptions(configService: ConfigService) {
  return {
    host: configService.getString('REDIS_HOST'),
    port: configService.getInt('REDIS_PORT'),
  }
}

const BullQueueModule = BullModule.registerAsync([
  {
    name: 'email',
    imports: [ConfigModule],
    useFactory: (configService: ConfigService): BullModuleOptions => {
      return {
        name: 'email',
        options: {
          redis: redisOptions(configService),
        },
      }
    },
    inject: [ConfigService],
  },
  {
    name: 'cron',
    imports: [ConfigModule],
    useFactory: (configService: ConfigService): BullModuleOptions => {
      return {
        name: 'cron',
        options: {
          redis: redisOptions(configService),
        },
      }
    },
    inject: [ConfigService],
  },
])

@Module({
  imports: [BullQueueModule],
  exports: [BullQueueModule],
})
export class QueueModule { }

had the same issue.
really glad i found this thread here, otherwise there is no documentation yet.
It would be really great if it could be added to the readme. Thanks!

Yes, you'll need to create a module to reexport the BullModule
Here's an example:
import { BullModule, BullModuleOptions } from 'nest-bull'
import { Module } from '@nestjs/common'

import { ConfigModule } from 'src/config/config.module'
import { ConfigService } from 'src/config/config.service'

function redisOptions(configService: ConfigService) {
  return {
    host: configService.getString('REDIS_HOST'),
    port: configService.getInt('REDIS_PORT'),
  }
}

const BullQueueModule = BullModule.registerAsync([
  {
    name: 'email',
    imports: [ConfigModule],
    useFactory: (configService: ConfigService): BullModuleOptions => {
      return {
        name: 'email',
        options: {
          redis: redisOptions(configService),
        },
      }
    },
    inject: [ConfigService],
  },
  {
    name: 'cron',
    imports: [ConfigModule],
    useFactory: (configService: ConfigService): BullModuleOptions => {
      return {
        name: 'cron',
        options: {
          redis: redisOptions(configService),
        },
      }
    },
    inject: [ConfigService],
  },
])

@Module({
  imports: [BullQueueModule],
  exports: [BullQueueModule],
})
export class QueueModule { }

Hi @captainjapeng ,
Could you please share full example with usage, I tried re-export but still got same error.

@maxymshg you'll need to import the QueueModule to the module you want to use it and then also setup your Processor on the Providers
email.module.ts
@Module({
  imports: [
    forwardRef(() => QueueModule),
  ],
  providers: [EmailService, CronProcessor],
  exports: [EmailService],
})
export class EmailModule {}

cron.processor.ts
@Injectable()
@Processor({ name: 'cron' })
export class CronProcessor {
    private readonly logger = new Logger('EmailCron')

    constructor(
        @InjectQueue('email')
        private queueService: Queue<EmailJob>,
        @InjectQueue('cron')
        private cron: Queue,
    ) {
        this.logger.log('Intializing Cron Processor')
        cron.add(null, {
            repeat: { cron: '0 0 1-2 * * *' },
            jobId: 'resume-email',
        })
    }

    @Process()
    async process(job: Job) {
        this.logger.log('Resuming Email Service')
        await this.queueService.resume()
    }
}

Here's an example of a cron job that resume the email queue every 1-2 AM UTC if ever the quota has been reached.

Got it, thanks for sharing this.

Hi, I tried it, but i have the same problem.

Let's track this here https://github.com/nestjs/bull/issues/171


Swap homebrew-pypi-poet for new Homebrew tooling

See https://github.com/tdsmith/homebrew-pypi-poet/issues/74

Won't do (at least for now): Homebrew tooling still requires me to use the PyPI API to find the sdist URL.


Nintendo Joy Con Usage vs. Gestures

Hello,
There are some community claims that a specific version of ALVR can support joy cons. Is this the case? I don't yet own Joy Cons so I'm testing things with my XBox controller for now.
When I enable gestures and launch VRChat, I cannot use my current xbox controller's thumbsticks reliably. I'm assuming because figner tracking thinks I'm doing one thing and the xbox joystick think another so they're conflicting.
When I disable gestures and launch vrchat, nothing about my controller acts as a pinch gesture or VR controller trigger click and therefore I cannot click/select anything in either SteamVR or VRChat.
What are my options here?

yes, visionOS ALVR has support for Joy-Con connected to the headset, but you must turn off gestures for it to work. You can also use a DS4 or Pro Controller technically, but the bindings might be funky. Luckily the settings app allows rebinding per-app for now.

It seems the support is specifically for Joy Con and not other controllers? My XBox controller has no button that will operate as a VR trigger.

the controllers have to be connected to the headset, not the PC, so we're at the mercy of whatever Apple happens to support for now, unfortunately.

The XBox controller is connected directly to the headset. I will purchase a Joy Con to test if it is somehow working diferrently than my current XBox controller.

oh if it's actually connected and shows the rebinding stuff in settings, you should be able to use it in ALVR fine. Just change your streamer settings from Controllers -> Gestures -> Only touch to Off so that it uses the controller inputs

Hello, trying HL Alyx for the first time with joycons, got the game loaded, but I can't move. It says move left joystick to move, it doesn't do anything. And if I hold down, it opens the menu to quit game, adjust settings, etc. Any ideas?
I'm using some joycon knock offs from Amazon, I am assuming that's not my problem. All the other buttons seem to work as expected. e.g. Triggers let me grab that antenna on the radio, turn the radio knob, etc.
Thanks!


Alarm Bot Documentation Missing Details

http://docs.botframework.com/sdkreference/csharp/dialogs.html
The Alarm Bot section is missing some details like adding the Chronic Nuget package and it doesn't explain what to do in the MessageController or how to interact with the bot.

Thanks for the feedback. We'll have someone look into it.

@bryantlikes if you open the main solution, it has the SimpleAlarmBot  project that has all the nuget dependencies required to build and run the alarm bot. You can start that project and use Bot Framework Emulator to interact with it locally.

Please re-open if this is still causing you grief.


Tables without borders

Hello,
The markdown tables are rendered without borders.
How can it be modifed?

Hi @Petrichor31 , I added borders (see example here. Feel free to customise  it in sass here

Cool thank you!


Dicom API changes for Image replacement

Updated Dicom conformance statements for Upsert feature

@mmitrik
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

#sign-off


document building against different CUDA Toolkit versions

Signed-off-by: Rong Ou<EMAIL_ADDRESS>

build


feat: update file constants

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

3 of 3 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 87.26%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
486

Relevant Lines:
544

💛  - Coveralls


[feat/aut178] SmartPeak Documentation

Docs on

how to build smartpeak,
how to run tests and examples,
how to install and use smartpeak gui (basics)

Here is a preview for this PR: https://smartpeak.readthedocs.io/en/feat-aut178/index.html

That's great work! nice short videos. that's a lot of work. we will need to split and list the remaining things.


ERROR WHILE RUNNING RUN_AUGMENTOOLKIT FILE

Hi, im trying to run run_augmentoolkit.py but getting this error:
Response:
-----

No response to print
2024-11-21 14:31:57,173 - ERROR - Above prompt resulted in error, probably the model's fault: Error code: 401 - {'detail': 'User is not authorized to access this resource'}
Traceback (most recent call last):
  File "/home/bilal/Downloads/augmentoolkit/augmentoolkit/generation_functions/generation_step_class.py", line 142, in generate
    response, timeout = await self.engine_wrapper.submit_chat(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bilal/Downloads/augmentoolkit/augmentoolkit/generation_functions/engine_wrapper_class.py", line 113, in submit_chat
    stream = await self.client.chat.completions.create(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bilal/anaconda3/envs/augtk/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1661, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/bilal/anaconda3/envs/augtk/lib/python3.11/site-packages/openai/_base_client.py", line 1839, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bilal/anaconda3/envs/augtk/lib/python3.11/site-packages/openai/_base_client.py", line 1533, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/bilal/anaconda3/envs/augtk/lib/python3.11/site-packages/openai/_base_client.py", line 1634, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'detail': 'User is not authorized to access this resource'}
Generation step failed -- too many retries!
Traceback (most recent call last):
  File "/home/bilal/Downloads/augmentoolkit/augmentoolkit/generation_functions/pipeline_step_class.py", line 89, in generate_data
    result, full_output = await generator.generate(**processed_data, **self.static_arguments)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bilal/Downloads/augmentoolkit/augmentoolkit/generation_functions/generation_step_class.py", line 181, in generate
    raise Exception("Generation step failed -- too many retries!")
Exception: Generation step failed -- too many retries!
cannot unpack non-iterable NoneType object
Traceback (most recent call last):
  File "/home/bilal/Downloads/augmentoolkit/augmentoolkit/generation_functions/pipeline_step_class.py", line 133, in run
    result, full_output = await self.generate_data(processed_data, engine_wrapper)
    ^^^^^^^^^^^^^^^^^^^
TypeError: cannot unpack non-iterable NoneType object
100%|███████████████████████████████████████████| 20/20 [00:05<00:00,  3.89it/s]
Converting generations to training data
entering saving mode
Converting ./output/judge_paragraph_generations/intermediate_generations to a dataset
Traceback (most recent call last):
  File "/home/bilal/Downloads/augmentoolkit/original/processing.py", line 436, in <module>
    asyncio.run(main())
  File "/home/bilal/anaconda3/envs/augtk/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/bilal/anaconda3/envs/augtk/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bilal/anaconda3/envs/augtk/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/bilal/Downloads/augmentoolkit/original/processing.py", line 270, in main
    steps.convert_logging_to_dataset(input_pth=os.path.join("judge_paragraph_generations", "intermediate_generations"), output_pth="judge_paragraph_generations")
  File "/home/bilal/Downloads/augmentoolkit/original/steps.py", line 96, in convert_logging_to_dataset
    raise Exception("ERROR!! Trying to convert a logging directory to a dataset, when that directory does not exist!")
Exception: ERROR!! Trying to convert a logging directory to a dataset, when that directory does not exist!

Configurations:
API:
  LARGE_MODEL: meta-llama/Meta-Llama-3.1-8B-Instruct-Reference
  LARGE_API_KEY: MY_HIDDEN_API
  LARGE_BASE_URL: https://api.together.xyz
  LARGE_MODE: api
  SMALL_MODEL: meta-llama/Meta-Llama-3.1-8B-Instruct
  SMALL_BASE_URL: https://api.deepinfra.com/v1/openai
  SMALL_API_KEY: key-here
  SMALL_MODE: api
HUGGINGFACE:
  HUB_PATH: yourusername/your-path-here
  PRIVATE: False
  PUSH_TO_HUB: False
PATH:
  DEFAULT_PROMPTS: ./prompts
  INPUT: ./input
  OUTPUT: ./output
  PROMPTS: ./prompts
PHASE:
  PHASE_INDEX: 3
  WORK_IN_PHASES: False
SKIP:
  ANSWER_RELEVANCY_CHECK: False
  REPAIR_QA_TUPLES: False
  FILTER_CHUNKS: False
  QUESTION_CHECK: False
  CONVERSATION_GENERATION: False
SYSTEM:
  CHUNK_SIZE: 1900
  COMPLETION_MODE: False
  CONCURRENCY_LIMIT: 50
  CONVERSATION_INSTRUCTIONS: For this conversation, you are generating a chat between
    a generalist, generic AI assistant, and a human.
  DOUBLE_CHECK_COUNTER: 1
  DO_NOT_USE_SYSTEM_PROMPTS: False
  FINAL_ASSISTANT_PROMPTS_NO_RAG: [
  'You are a helpful AI assistant.',
  'You are A VASTLY intelligent ARTIFICIAL INTELLIGENCE with DOMAIN-EXPERT KNOWLEDGE from a variety of fields.

  USE your knowledge to be helpful and truthfully answer questions about the world.',
  "u are ai asstant plz answr questions"] # a wide variety of system prompts helps the AI learn better. What, you expect your users to spell things right?
  FINAL_ASSISTANT_PROMPTS_RAG: [
  'You are a helpful AI assistant.  Answer questions',
  'You are an AI with vast knowledge. Here is some potentially-relevant context:

  {data}

  Answer questions according to your knowledge.']
  STOP: True
  SUBSET_SIZE: 20
  USE_FILENAMES: False
  USE_SUBSET: True
  RAG_FAILURE_PERCENTAGE: 0.1 # How much of the RAG data has the wrong chunk retrieved deliberately? To train it to answer correctly even if wrong facts are shown to it. We will need another dataset thing for making data where the question asks something that is not present and the rag retrieves something irrelevant obbviously and it is supposed to say "I don't know" or something.
SCRAPING:
  USE_GUTENBERG: False
  START_URL: "https://www.gutenberg.org/ebooks/bookshelf/57"
  MAX_BOOKS: 5
  MAX_FAILURES: 5

P.S: The idea is good but the repo needs maintainers as it takes a lot of time to get answers. Thanks

Check the traceback, you're getting a 401. This seems to be because you have set the api key and model for the large model, but not the small one (which is still using deepinfra defaults) in your config. If you fix that it should work.
I'd like consistent maintainers haha, tough balancing this and all the work I have. Where can I recruit them?

I tried and it worked (not with together ai but with deepinfra). Thanks
API:
  LARGE_MODEL: meta-llama/Meta-Llama-3-8B-Instruct
  LARGE_API_KEY: CREARE_YOUR_OWN_FREE_API
  LARGE_BASE_URL: https://api.deepinfra.com/v1/openai
  LARGE_MODE: api
  SMALL_MODEL: meta-llama/Meta-Llama-3-8B-Instruct
  SMALL_BASE_URL: https://api.deepinfra.com/v1/openai
  SMALL_API_KEY: CREARE_YOUR_OWN_FREE_API
  SMALL_MODE: api
HUGGINGFACE:
  HUB_PATH: yourusername/your-path-here
  PRIVATE: False
  PUSH_TO_HUB: False
PATH:
  DEFAULT_PROMPTS: ./prompts
  INPUT: ./input
  OUTPUT: ./output
  PROMPTS: ./prompts
PHASE:
  PHASE_INDEX: 3
  WORK_IN_PHASES: False
SKIP:
  ANSWER_RELEVANCY_CHECK: False
  REPAIR_QA_TUPLES: False
  FILTER_CHUNKS: False
  QUESTION_CHECK: False
  CONVERSATION_GENERATION: False
SYSTEM:
  CHUNK_SIZE: 1900
  COMPLETION_MODE: False
  CONCURRENCY_LIMIT: 50
  CONVERSATION_INSTRUCTIONS: For this conversation, you are generating a chat between
    a generalist, generic AI assistant, and a human.
  DOUBLE_CHECK_COUNTER: 1
  DO_NOT_USE_SYSTEM_PROMPTS: False
  FINAL_ASSISTANT_PROMPTS_NO_RAG: [
  'You are a helpful AI assistant.',
  'You are A VASTLY intelligent ARTIFICIAL INTELLIGENCE with DOMAIN-EXPERT KNOWLEDGE from a variety of fields.

  USE your knowledge to be helpful and truthfully answer questions about the world.',
  "u are ai asstant plz answr questions"] # a wide variety of system prompts helps the AI learn better. What, you expect your users to spell things right?
  FINAL_ASSISTANT_PROMPTS_RAG: [
  'You are a helpful AI assistant.  Answer questions',
  'You are an AI with vast knowledge. Here is some potentially-relevant context:

  {data}

  Answer questions according to your knowledge.']
  STOP: True
  SUBSET_SIZE: 20
  USE_FILENAMES: False
  USE_SUBSET: True
  RAG_FAILURE_PERCENTAGE: 0.1 # How much of the RAG data has the wrong chunk retrieved deliberately? To train it to answer correctly even if wrong facts are shown to it. We will need another dataset thing for making data where the question asks something that is not present and the rag retrieves something irrelevant obbviously and it is supposed to say "I don't know" or something.
SCRAPING:
  USE_GUTENBERG: False
  START_URL: "https://www.gutenberg.org/ebooks/bookshelf/57"
  MAX_BOOKS: 5
  MAX_FAILURES: 5

Check the traceback, you're getting a 401. This seems to be because you have set the api key and model for the large model, but not the small one (which is still using deepinfra defaults) in your config. If you fix that it should work.
I'd like consistent maintainers haha, tough balancing this and all the work I have. Where can I recruit them?

Just two more questions:
Q1. If I want to change the format of the created dataset to alpaca is there any configuration for that?
Q2. I provided in the input folder a pdf of 100 smth pages (all text). while the qna pairs I got were not much as per pdf text.


Data Migration - Password Conversion Issue

The passwords are not usable once our existing data has been migrated over from the current DCC instance to the new Roadmap codebase.
Could be related to the Devise gem, Ruby or Rails ... all were upgraded during the creation of Roadmap.

config/initializers/devise_example.rb gets read by Rails along with devise.rb instead of just the latter, causing legacy passwords from migrated databases to be invalid. Propose renaming the file to devise.rb.example to alleviate problem.
Incidentally, the same thing happens with config/initializers/recaptcha_example.rb


Remove setopt localoptions

The setopt localoptions inside the zplug() function reverts any change made by any other plugins or themes. This is caused by #111, and the problem of #110 can be solved by quoting arguments of __zplug::clone function.
I tested with this .zshrc:
setopt shwordsplit

source $HOME/.zplug/zplug

zplug "b4b4r07/hello_bitbucket", \
    as:command, \
    from:bitbucket, \
    do:"chmod 755 *.sh", \
    of:"*.sh"
zplug "yous/lime"

zplug check || zplug install
zplug load

Note that yous/lime calls setopt prompt_subst to enable parameter expansion, but zplug() reverts that after zplug load, prompt will display only $(prompt_lime_render).
With this patch, zplug installs b4b4r07/hello_bitbucket with no error messages and the chmod of hello.sh has changed to 755. Also setopt will include prompt_subst.

I have confirmed. Thank you @yous


[$1000]  Web - The copy to clipboard is seen on the sign in page too on public room

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to staging dot on web chrome
Create a new room in public mode
Click on room header and copy the URL
Open an incognito window and paste the URL (male sure you're not signed on staging before)
Right click on 'here' so that 'Copy URL' option opens up
Click on the Copy URL option and notice that the 'Copy URl' button is still showing on the sign in page for a period of time

Expected Result:
The copy to clipboard should not be seen on the sign in page on public room
Actual Result:
The copy to clipboard is seen on the sign in page too
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.26-4
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/90a7beaf-c5a8-49eb-8134-8f59855e6976
https://github.com/Expensify/App/assets/93399543/1834ceaa-2492-4c8b-88e4-6dc22030e992
Expensify/Expensify Issue URL:
Issue reported by: @priya-zha
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1686739229581989
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0123e154d1a63d268a
Upwork Job ID:<PHONE_NUMBER>761204736
Last Price Increase: 2023-06-22

Thank you for finding the dupe, @fedirjh! I didn't use the right keywords to surface that.


[FEATURE] Elo rating system snippet

Description

https://en.wikipedia.org/wiki/Elo_rating_system
What does your PR belong to?

[ ] Website
[x] Snippets
[ ] General / Things regarding the repository (like CI Integration)

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (non-breaking improvement of a snippet)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have checked that the changes are working properly
[x] I have checked that there isn't any PR doing the same
[x] I have read the CONTRIBUTING document.

This is useful in multiplayer competitive leagues using ELO but ELO is now considered an out of date rating system for video games due to the extreme de-bouncing needed. While still used, it is instead preferred to use a more metric based approach to your games(truskill, skillz, other rating systems that take into account minute metrics to determine ratings).
ELO is still a valid rating system for games that use strategy for complexity. Such games are Chess, Go, or any perfect information symmetrical game. While a cool example, I'm not certain it has much use here since if it isn't one of these few games and it has minute differences in skill like 'headshot' or 'objective data' then it is an extremely inaccurate rating.

Can you explain further about the debouncing? Is that to do with rating inflation or something?
Do you think it's accurate for a 1v1 game of Call of Duty where it's either win or loss? I've researched a bit and came across Glicko 2 and TrueSkill but their implementations are too complex for a snippet (I think).

I'm merging as this has practical uses, even if there are better ranking systems than this one. We can add more snippets like this anyways and someone might need this, so why not?

Debouncing and ELO rating are required to clamp gains based on differences in the ratings and how often you defeat opponents of the same rating as you, and opponents of a lower rating than you


Issues for Installation

Hi SingleCellNet Team,
When I try to install the package, I got the following error and not sure what to deal with:
Error in i.p(...) :
(converted from warning) installation of package ‘C:/Users/YUHENG~1/AppData/Local/Temp/Rtmp0S7yMI/file1128ee055cf/singleCellNet_0.1.0.tar.gz’ had non-zero exit status
Would you guys have any idea what may this happened and how to fix this? Really appreciate all your help.
Best

Hi,
Could you send me your code and the full error message? i am unable to replicate your error. And did you install via install_github("pcahan1/singleCellNet") ?

Thanks, I realized it was some r issue that caused the error. I removed the package and installed successfully.


Add the search functionality to the Project List Component

Goal
Use the Reusable Search Bar Component to add the search functionality to the Project List Component.
Note: Before Starting this issue, #22 issue should be completed.

can i try?

Sure @tranjenny

Are you using the Gatsby framework?

Is this what you want @wdevon99 ? I have implemented whole functionality on sample data, should I generate a PR?
Search functionality is working well with filtering projects by their name.

Is this issue still open ?

Already completed in #178


core getter does not honor NAPALM_TIMEOUT value

Environment

Nautobot version (Docker tag too if applicable): 1.4.2
Python version: 3.9.13
Database platform, version: postgres 12.8
Middleware(s):

Steps to Reproduce

set NAPALM_TIMEOUT NAPALM_TIMEOUT = int(os.getenv("NAUTOBOT_NAPALM_TIMEOUT", 120))
click on device configuration tab
blank screen returns with no errors and no config after 30 seconds
set "conn_timeout" in platform NAPALM arguments resulting in no change in behavior

Expected Behavior
configuration to appear (or an error)

Observed Behavior
no configuration or error

NOTE
LLDP Neighbors getter does populate (which returns data in less than 30 seconds)
I had to modify nautobot_plugin_nornir to increase timeout for device so nautobot_golden_config backup would succeed on device - it certainly will not return data within the default 30 seconds
neighboring devices with smaller configurations all populate properly as long as they return before the timeout
older version of nautobot (built in same infra) retrieves configuration after about 50 seconds of clicking the device configuration tab

Thanks for the submission @eneiford. This is being passed into the derived NAPALM driver relevant to the specified platform.
Can you let us know which device platform you're connecting to? Also, can you find out if your browser console is receiving any errors as well?

driver is nxos_ssh.  I will review the browser console when I am able.  I am not getting any logs at all in the container.
Sent from my iPhone

On Sep 13, 2022, at 14:07, Bryan Culver @.***> wrote:

Thanks for the submission @eneiford. This is being passed into the derived NAPALM driver relevant to the specified platform.
Can you let us know which device platform you're connecting to? Also, can you find out if your browser console is receiving any errors as well?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.

i am not getting any browser console notifications when the configuration
fails to return.  I can flip to the LLDP Neighbors tab and the return
values will populate, but the configuration will not.  Calls to the
endpoint /api/dcim/devices/{mydev.id}/napalm/?method=get_config return the
following error:
{
"get_config": {
"error": "Method get_config failed: \nPattern not detected:
'{router-name}#' in output.\n\nThings you might try to fix this:\n1.
Explicitly set your pattern using the expect_string argument.\n2. Increase
the read_timeout to a larger value.\n\nYou can also look at the Netmiko
session_log or debug log for more information.\n\n"
}
}
On Tue, Sep 13, 2022 at 4:51 PM Eric @.***> wrote:

driver is nxos_ssh.  I will review the browser console when I am able.  I
am not getting any logs at all in the container.
Sent from my iPhone
On Sep 13, 2022, at 14:07, Bryan Culver @.***> wrote:

Thanks for the submission @eneiford https://github.com/eneiford. This
is being passed into the derived NAPALM driver relevant to the specified
platform.
Can you let us know which device platform you're connecting to? Also, can
you find out if your browser console is receiving any errors as well?
—
Reply to this email directly, view it on GitHub
https://github.com/nautobot/nautobot/issues/2393#issuecomment-1245844898,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEWPC37PJJSPGAHOPTOZYRDV6DGIPANCNFSM6AAAAAAQIJT5WA
.
You are receiving this because you were mentioned.Message ID:
@.***>

@eneiford #2574 made related changes to this, which was released in 1.4.6. Have you checked out a recent release of Nautobot to see if this has been addressed?


[Suggestion] Add backup champ picks in arena mode in case main pick is banned or picked

Great project! Many thanks, this works well in Arena mode. Just wondering if it's possible to get an arena mode version?

This is just a duplicate of #15


Zero config running & debugging doesn't work for python files in packages

Environment data

VS Code version: Code 1.53.2 (622cb03f7e070a9670c94bae1a45d78d7181fbd4, 2021-02-11T11:45:54.515Z)
Extension version (available under the Extensions sidebar): 2021.2.582707922
OS and version: Darwin x64 18.7.0
Python version (& distribution if applicable, e.g. Anaconda): 3.9.1 via poetry & pyenv
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): poetry, in workspace directory .venv
Relevant/affected Python packages and their versions: N/A
Relevant/affected Python-related VS Code extensions and their versions: (See details at end of report for full IDE dump)
Value of the python.languageServer setting: Pylance

Overview
Please see https://github.com/sryabkov/python_modules_vscode
It shows that there are two incompatible ways to structure your code.
Option 1: no packages: Go To Definition doesn't work.
Option 2: python packages: Debug File doesn't work.
Please fix Option 2 so that I don't have to create debug configs and instead can run a file that's in a module with no config.
Expected behaviour
"Start debugging" or "Run without debugging" works whether or not my python code is in a module or not.
Or there's a way to have it automatically run this file if it's in a module and discover all the config needed for the module config.   Ideally I wouldn't have to manage many Run Configs for each different file in a module, I'd just be able to always "run this file".
Actual behaviour
 /usr/bin/env /private/tmp/python_modules_vscode/venv/bin/python3.9 /Users/dmankin/.vscode/extensions/ms-python.python-2021.2.582707922/pythonFiles/lib/python/debugpy/launcher 55126 -- /private/tmp/python_modules_vscode/code/option2/sut_test.py
source /private/tmp/python_modules_vscode/venv/bin/activate
$  /usr/bin/env /private/tmp/python_modules_vscode/venv/bin/python3.9 /Users/dmankin/.vscode/extensions/ms-python.python-2021.2.582707922/pythonFiles/lib/python/debugpy/launcher 55126 -- /private/tmp/python_modules_vscode/code/option2/sut_test.py
Traceback (most recent call last):
  File "/private/tmp/python_modules_vscode/code/option2/sut_test.py", line 5, in <module>
    from .sut import some_method_that_returns_string
ImportError: attempted relative import with no known parent package
$ source /private/tmp/python_modules_vscode/venv/bin/activate
(venv) $

Steps to reproduce:

git clone https://github.com/sryabkov/python_modules_vscode.git
cd python_modules_vscode
python3.9 -mvenv venv
code .
Choose venv as your virtual environment
Open up option2/sut_test.py
Choose Run > Run without debugging
Choose "python file" as your configuration

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Nothing relevant

Output from Python terminal

 /usr/bin/env /Users/dmankin/dev/experimental/ep-fax-control-plane-py-spike/.venv/bin/python3.9 /Users/dmankin/.vscode/extensions/ms-python.python-2021.2.582707922/pythonFiles/lib/python/debugpy/launcher 54910 -- /Users/dmankin/dev/experimental/ep-fax-control-plane-py-spike/src/sourcesvc/main.py
source /Users/dmankin/dev/experimental/ep-fax-control-plane-py-spike/.venv/bin/activate
$  /usr/bin/env /Users/dmankin/dev/experimental/ep-fax-control-plane-py-spike/.venv/bin/python3.9 /Users/dmankin/.vscode/extensions/ms-python.python-2021.2.582707922/pythonFiles/lib/python/debugpy/launcher 54910 -- /Users/dmankin/dev/experimental/ep-fax-control-plane-py-spike/src/sourcesvc/main.py
Traceback (most recent call last):
  File "/Users/dmankin/dev/experimental/ep-fax-control-plane-py-spike/src/sourcesvc/main.py", line 5, in <module>
    from ..fax_control_plane.api_objects import *
ImportError: attempted relative import with no known parent package
$ source /Users/dmankin/dev/experimental/ep-fax-control-plane-py-spike/.venv/bin/activate
$

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz (12 x 2200)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 3

Memory (System)
32.00GB (0.07GB free)

Process Argv
-psn_0_544901 --crash-reporter-id a5006729-690f-48ab-9c1c-d60db2dd7dd8

Screen Reader
no

VM
0%

Extensions (33)

Extension
Author (truncated)
Version

atlascode
atl
2.8.5

code-gnu-global
aus
0.2.2

better-toml
bun
0.3.2

vscode-markdownlint
Dav
0.39.0

go
gol
0.22.1

vscode-graphql
Gra
0.3.15

terraform
has
2.7.0

Python-autopep8
him
1.0.2

intellij-idea-keybindings
k--
1.3.0

MagicPython
mag
1.1.0

vscode-clang
mit
0.2.3

theme-monokai-pro-vscode
mon
1.1.18

vscode-docker
ms-
1.10.0

python
ms-
2021.2.582707922

vscode-pylance
ms-
2021.2.3

jupyter
ms-
2021.2.576440691

remote-containers
ms-
0.158.0

cpptools
ms-
1.2.1

java
red
0.75.0

vscode-yaml
red
0.15.0

jinjahtml
sam
0.16.0

code-spell-checker
str
1.10.2

opa
tsa
0.7.0

vsnadel
tsr
0.0.1

vscodeintellicode
Vis
1.2.11

vscode-java-debug
vsc
0.31.0

vscode-java-dependency
vsc
0.18.0

vscode-java-pack
vsc
0.12.1

vscode-java-test
vsc
0.28.0

vscode-maven
vsc
0.27.1

HCL
who
0.0.5

material-theme
zhu
3.9.15

vscode-proto3
zxh
0.5.3

(4 theme extensions excluded)

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
pythonvspyt617cf:30253240
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstry914:30256638
pythonvsdeb440:30248342
pythonvsded773:30248341
pythonvspyt875:30259475
dockersubsetcf:30259262

Unfortunately we can't make this use-case work without a configuration because you are launching from a place that isn't the top-level for imports, thus breaking relative imports in your code. You need to tell the debugger where to run Python from, else you get the ImportError you're seeing.

Could we default to top level of the workspace or something that would work for many cases?
I can run the same file with cd code/ && python3 -m option2.sut_test.  option2 being the highest parent of sut_test.py that has an __init__.py, and code/ being the path to the directory containing option2.
Or is there something I could put in PYTHONPATH that would mean it would work without any further configuration?
The big problem is that if I have many modules to run then I have to have many debug configs each manually maintained and chosen.  I'd be okay with a single one that sets up PYTHONPATH or similar.

You can configure a .env file if you want to set PYTHONPATH.


Update head-objects.md

headless
Changes

Checks
By submitting your pull request for review, you agree to the following:

[ ] This contribution was created in whole or in part by me, and I have the right to submit it under the terms of this repository's open source licenses.
[ ] I understand and agree that this contribution and a record of it are public, maintained indefinitely, and may be redistributed under the terms of this repository's open source licenses.
[ ] To the best of my knowledge, all proposed changes are accurate.

new head

new head


Fix compat typo

I accidentally set compat to
[compat]
DataFrames = "0.19, 0.20, 0.21, 22"

When I tagged the last version. Sorry for the complication :facepalm:

Do I also need to do something on my repo end? I already updated the Project.toml, but do I need to change which commit that General points to?


Multiple tokens and logout

Does this library support multiple tokens for a single user? Let's say, a user authenticate from Web and mobile app at the same time. Does the user get the same token?
How would a logout procedure look like? Deleting Token? If so, with single Token per user, the user is logged out of all active sessions (web & mobile) which is not desired.

Multiple tokens can be possible by implementing a custom JWT_PAYLOAD_HANDLER. By default user's wouldn't get an exact same token, since they'll have different expiration times.
logout would just be removing the token from the client. Tokens are not stored anywhere in the backend, they are completely stateless. Each request will be authenticated if the token is present and is valid, if it's not present requests won't be authenticated.
Users will be able to login on the web and on a mobile app, those tokens are independent and don't affect each other.
You might want to read a bit more on authentication with JSON Web Tokens.

Thanks a lot.


roblox doesnt open

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.
[X] I have thoroughly looked through the available Wiki articles and could not find a solution to my problem.
[X] I am using the latest version of Bloxstrap.
[ ] I did not answer truthfully to all the above checkboxes.

What problem did you encounter?
bloxstrap exception error

Bloxstrap Log
System.AggregateException: One or more errors occurred. (Deserialised application settings is null!)
[Inner Exception]
System.Exception: Deserialised application settings is null!

https://github.com/bloxstraplabs/bloxstrap/wiki/"Deserialised-application-settings-is-null!"-when-launching-Roblox


Battle not starting on Windows 11 and 10 after installing docker linux subsystem

Describe the bug
I'm using robocode tankroyale in my classroom in windows and linux. Now my students with windows 10 and 11 can't start battle.
They recently installed the docker with linux subsystem.
To Reproduce
Steps to reproduce the behavior:

Start robocode
Add bots
Start battle
nothing happens

Error info
no error
Expected behavior
start battle with selected bots
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

Windows 11 and 10

Java info:

Version: some studets with 23, other 18...
Vendor: OpenJDK

Additional context
We tried with 24.4 and 26.1

@martinezpenya Thank you for reporting this issue.
Is the problem occurring outside Docker or within Docker? Or is is "just" happening after having installed Docker?

It's happening after having installed Docker.
Before this it works great.

OK. This should be easy to reproduce. I will have a look. :+1:

@martinezpenya I have just installed Docker Desktop on Windows 11 running on WSL 2, and I can reproduce the issue.
I don't know what causes it yet, but I will find out.

You're Super! thank you for your time!

I discovered, that this "only" occurs for the Java bots, but not the .Net bots.
And the issue is within the Bot API for Java, when a bot needs to send a message to the server via its socket. The first message is "ready" when the bot is about to participate in the battle. The server waits for the signal from the bots, but times out as it does not receive a "ready" signal from the bots.

If I can help you in any way, let mi know.

I found the bug in the Bot API for Java, and fixed it.
The fix will be available with the next version of Robocode coming soon. 😊

Fixed with version 0.27.0, just released.

@martinezpenya Can you confirm the fix work for your students as well?

Yes of course I will... My next class with them is on thursday, i'll come back and confirm that 0.27.0 solves the issue. Thank you so much!.

Hi @flemming-n-larsen everything works like a charm again. Thank you for your time.
Is there any way to support your work? buy-me-a-coffe or something like this?
Thank you!

@martinezpenya I am very happy to hear that! Thank you for reporting back! 😊
I haven't thought of buy-me-a-coffe. I will consider that.
Currently, I make Robocode for fun, and make sure to fix any issue that arise.


Add g-cloud services to extract-model-data.py

CCS have requested a CSV of all current G9 services, including the service id, supplier, name of service and lot. This is so they can confirm there spend data before publishing it. The data that this will give them will allow them to cross reference against their own datasets and ensure they have the correct figures.

No description given :disappointed:
Why do we need this? Why G8 and G9? The change is probably fine, but without context it's hard to say if it's the right thing/a thing that we need.

Sorry Kev, have added more info.

 CCS have requested a csv file of all current G9 service IDs. This would include Service ID, Supplier, name of service and Lot. (Request has come from Milan to get done(.


No precompiled kernels found for MI200

Hi,
I was trying to run this script on MI200 machine, which has gfx90a architecture. While I was not able to install the package and got error:
using rocminfo at /usr/bin/rocminfo
sudo apt install -y miopenkernels-gfx90a-110kdb
Hit:1 http://azure.archive.ubuntu.com/ubuntu focal InRelease
Get:2 http://azure.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://azure.archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:4 http://azure.archive.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:5 https://packages.microsoft.com/ubuntu/20.04/prod focal InRelease
Hit:6 https://repo.radeon.com/amdgpu/22.10.1/ubuntu focal InRelease
Hit:7 https://repo.radeon.com/rocm/apt/5.1.1 ubuntu InRelease
Fetched 336 kB in 1s (634 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
34 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package miopenkernels-gfx90a-110kdb

Does this precompiled kernel package support MI200 gfx90a architecture? Or which package should I install?
Thanks.

there is no kdb
http://repo.radeon.com/rocm/apt/5.1.1/pool/main/m/miopenkernels-gfx90a-110/

@xiaoyu-work could you please try the following instead?
sudo apt install -y miopenkernels-gfx90a-110pkg
The issue is fixed in develop branch and ready for the next release.

Hi @junliume , apt install -y miopenkernels-gfx90a-110 doesn't work for new released ROCm 5.2 image: https://hub.docker.com/layers/dev-ubuntu-20.04/rocm/dev-ubuntu-20.04/5.2-complete/images/sha256-4030c8af0c06c286174758523dabe4b3850bf72d4a8c1ef275d3ec69aa475f65?context=explore. Can you fix this?

@xiaoyu-work
could you please try:
apt install -y miopenkernels-gfx90a-110kdb

we will fix the inconsistency in the script. CC: @JehandadKhan

Thanks @junliume . This package name works. Which package name will be the final one?

#1678  dealt with this issue


Fix OS names/architectures for Linux Prerequisites

Update the Linux Prerequisites Guide to clean up OS names and architectures
Here is the links to official docs from each source that use an alternative instead of x64:

RHEL: https://access.redhat.com/products/red-hat-enterprise-linux/evaluation
CentOS: http://isoredirect.centos.org/centos/7/isos/x86_64/CentOS-7-x86_64-DVD-1611.iso
Oracle: https://docs.oracle.com/cd/E52668_01/E54695/html/ol7-install-media.html
Fedora: https://getfedora.org/en/server/download/
Debian: https://www.debian.org/distrib/ (also uses amd64 in other places)
Ubuntu: https://www.ubuntu.com/download/server and https://www.ubuntu.com/download/desktop
Mint: https://www.linuxmint.com/download.php
openSUSE: https://software.opensuse.org/distributions/tumbleweed
SLES: https://www.suse.com/products/server/download/

I also split off Ubuntu and Linux Mint because they have a separate versioning system. Linux Mint is based on Ubuntu but Linux Mint 17 (for example) is not based on Ubuntu 17.04 or 17.10.
The note "Ubuntu / Linux 16.10" makes no sense because Ubuntu is a Linux distribution. If "Linux" refers to Linux Mint here, then a version for Linux Mint is missing

@omajid,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

Thanks @omajid. @JRAlexander can you please review this? @leecow, this would also impact other places where we mention Linux versions such as https://github.com/dotnet/core/blob/master/release-notes/2.0/2.0-supported-os.md, no?


HHH-16125 PostgreSQL enum types + reimplementation of enum support

First try was #6232.
This time things went somewhat more smoothly, except that I ran into a problem with enum types in foreign keys.
I'm now concerned that we'll have to abandon the whole specialiedTypeDeclaration thing I added to let converters have a vote on the generated DDL types. In place of that I would probably introduce SqlTypes.ENUM and pass the enum type along with the Size class. So there would be a real DdlType for native enums.

Superseded by #6480.


MSYS-570 : update error message

https://github.com/chef/knife-ec2/issues/500 - Update error message to "Verify that you have public internet access." for vpc private IP.

@btm, Travis was failing as bundler is unable to resolve dependency: user requested 'io-console (= 0.4.3). They have fixed it in bundler (1.15.0). So I am using bundler --pre version (1.15.0) instead of (1.14.6). We'll remove pre release when they launch bundler 1.15.
FYI: https://github.com/rubygems/rubygems/issues/1911


Clarity around IAntiforgery and ValidateAntiForgeryToken

Hi everyone, I hope you're well.
I've spent the last two nights messing around with the anti-forgery request functionality. Specifically, I've been trying to figure out why I keep getting a 'Bad Request' response.
Nothing struck me as being overtly incorrect when using Chrome's developer tooling to examine the request and response headers (i.e., cookies were being set correctly, and the X-XSRF-TOKEN header was appended to POST requests).
The gotchas

For quite a while I was sending GET requests and wondering why Angular wasn't appending the aforementioned X-XSRF-TOKEN header. This one is my bad. 😄

When I figured out gotcha number 1, I was still perplexed as to why I got the 'Bad Request' response.

The problem code

Send a GET request to a 'login' API route. The code in the method did the following:

Sign out the IdentityConstants.ExternalScheme
Sign in a user
Append the XSRF-TOKEN cookie
Return a 200 OK response.

[AllowAnonymous]
[Route("login")]
public async Task<IActionResult> Login()
{
    await HttpContext.SignOutAsync(IdentityConstants.ExternalScheme);

    var result = await<EMAIL_ADDRESS>"Password", false, lockoutOnFailure: false);

    var tokens = antiforgery.GetAndStoreTokens(HttpContext);
    HttpContext.Response.Cookies.Append("XSRF-TOKEN", tokens.RequestToken, xsrfCookieOptions);

    return Ok(result);
}

Send a POST request, with X-XSRF-TOKEN header, to a 'logout' API route. This code did the following:

Invoke SignOutAsync
Return a 200 OK response.

[HttpPost]
[ValidateAntiForgeryToken]
[Route("logout")]
public async Task<IActionResult> Logout()
{
    await signInManager.SignOutAsync();

    return Ok();
}

Reproducing the issue

Exercise the 'login' endpoint, and review the cookies that get set.

Outcome: Identity cookie set, Antiforgery cookie set, XSRF-TOKEN cookie set.

Exercise the 'logout' endpoint.

Expected outcome: Identity cookie unset, 200 OK response
Actual outcome: 400 Bad Request response.

The correct ASP.NET Core code
This code should be invoked by a client in the following sequence:

Send GET request to /login endpoint
Send GET request to /renew-xsrf-token endpoint
Send POST request to /logout endpoint.

This sequence results in the Antiforgery functionality working as expected when /logout is invoked, rather than the 400 Bad Request response.
[AllowAnonymous]
[Route("renew-xsrf-token")]
public IActionResult RenewXsrfToken()
{
    var tokens = antiforgery.GetAndStoreTokens(HttpContext);
    HttpContext.Response.Cookies.Append("XSRF-TOKEN", tokens.RequestToken, xsrfCookieOptions);

    return Ok();
}

[AllowAnonymous]
[Route("login")]
public async Task<IActionResult> Login()
{
    await HttpContext.SignOutAsync(IdentityConstants.ExternalScheme);

    var result = await<EMAIL_ADDRESS>"Password", false, lockoutOnFailure: false);

    return Ok(result);
}

[HttpPost]
[ValidateAntiForgeryToken]
[Route("logout")]
public async Task<IActionResult> Logout()
{
    await signInManager.SignOutAsync();

    return Ok();
}

The correct Angular code
Note the following sequencing in this TypeScript code:

Perform the login
In a separate GET request, renew the XSRF-TOKEN cookie
Perform the logout (now successful).

public login() {
  this.http.get('/login').subscribe(tokenRenewResult => {
    this.http.get('/renew-xsrf-token').subscribe(loginResult => console.log(loginResult));
  });
}

public logout() {
  this.http.post('/logout', {}).subscribe(result => console.log(result));
}

Root cause
Setting an identity by invoking a PasswordSignInAsync method and then immediately setting an XSRF-TOKEN cookie in the same request does not work.
It appears that the XSRF-TOKEN, even when set after invoking the PasswordSignInAsync method, does not recognise the newly signed in user immediately.
Demarcating this process by spanning it across two separate HTTP requests (i.e., the first request sets the user identity using the PasswordSignInAsync method, and a second request sets the XSRF-TOKEN cookie in the context of this newly signed in user) resolves the issue.
This issue was the give-away for me: https://github.com/aspnet/Antiforgery/issues/155
Specifically this quote:

After I login using my service the anti-forgery token returned is not valid as it was created based on a null user. I've tried setting ClaimsPrincipal after my PasswordSignInAsync and regenerating the anti-forgery token (see below) but that still does not work. Any ideas?

Proposal
Could you consider updating the documentation such that it makes mention of the importance of:

Setting an XSRF-TOKEN cookie in the context of a logged in identity
Having a clear demarcation between first logging the user in, and then renewing their XSRF-TOKEN in the context of that newly logged-in user (i.e., demarcation via separate HTTP request, rather than trying to set all cookies in a single request).

I'd love to hear what you all think.
Kind regards,
Daniel Loth

@Lothy Let's say I'm an attacker and when you visit my page with your cookie I call renew-xsrf-token.  That will give me a XSRF token.  Can't I then go ahead and make requests?  The attacker shouldn't be able to generate XSRF tokens.

This fundamental design flaw/issue has been around since the early days of MVC on ASP.NET. In short, the anti forgery token contains identity information, so if you get an anti forgery token and the identity of the user changes (anonymous to logged in, logged in to anonymous, user A to user B) then subsequent validations will fail.

@brockallen Right, that's what I'm doing with my JWT, but my point is that the XSRF token needs to be generated and returned with the login request, not on a subsequent request.  I'm having a hard time figuring out how to do that.
After my JWT is generated I call another controller function
[HttpGet]
[Route("init")]
[Authorize]
public IActionResult Initialization()
{
       return Ok();
 }

Which essentially logs the user in so when my XSRFCookieFilter fires at the end
var tokens = antiforgery.GetAndStoreTokens(context.HttpContext);
context.HttpContext.Response.Cookies.Append("XSRF-TOKEN", tokens.RequestToken, new CookieOptions() { HttpOnly = false });

context.HttpContext.User  is set.  But when I go ahead and use that XSRF token in the next request I get a 400 error, and I leave the route init open, which I just said don't do.  I guess I could only generate XSRF from my login function?

So I was able to figure out how to do it in one request.  After the JWT is generated, you need to validate that JWT and it will return a ClaimsPrincipal which can then be set for the current user.
// Generate JWT
// Validate JWT
ClaimsPrincipal principal = validator.ValidateToken(jwt, validationParameters, out validatedToken);
this.HttpContext.User = principal;

I've found a workaround without the need for an extra roundtrip (/renew-xsrf-token). Inspired by https://github.com/aspnet/Antiforgery/issues/155#issuecomment-318271448
I'm using netcoreapp2.1
public class AccountController : Controller
{
    private readonly UserManager<User> _userManager;
    private readonly SignInManager<User> _signInManager;
    private readonly IUserClaimsPrincipalFactory<User> _principalFactory;

    public AccountController(
        UserManager<User> userManager,
        SignInManager<User> signInManager,
        IUserClaimsPrincipalFactory<User> principalFactory)
    {
        _userManager = userManager;
        _signInManager = signInManager;
        _antiforgeryprincipalFactory = principalFactory;
    }

    [HttpPost]
    [AllowAnonymous]
    public async Task<IActionResult> Login(LoginModel model)
    {
        var result = await<EMAIL_ADDRESS>"Password", false, lockoutOnFailure: false);

        await ForceSetXsrsTokens();

        return Ok(result);
    }

    private async Task ForceSetXsrsTokens()
    {
        // set principle on HttpContext.User as m_SignInManager.PasswordSignInAsync doesnt do it
        // username is automatically part of tokens

        var user = await _userManager.FindByEmailAsync("test@test.com");

        var principle = await _principalFactory.CreateAsync(user);
        HttpContext.User = principle;

        var tokens = _antiforgery.GetAndStoreTokens(HttpContext);
        Response.Cookies.Append(
            "XSRF-TOKEN",
            tokens.RequestToken,
            new CookieOptions()
            {
                HttpOnly = false,
                IsEssential = true
            });
    }
}

Thanks for contacting us.
@javiercn, can you please look into this? Thanks!

Thank you for clarifying that.
It's very useful to have that information available.

Closing this issue as the question has been answered and there's no more action to be taken here.


passing down select props

🤔 This is a ...

[ ] New feature
[x] Bug fix
[ ] Demo update
[ ] Component style update
[ ] TypeScript definition update
[ ] Bundle size optimization
[ ] Performance optimization
[ ] Refactoring
[ ] Code style optimization
[ ] Test Case
[ ] README update
[ ] Other (about what?)

🔗 Related issue link

💡 Background and solution
Material UI props are not being passed down.
☑ Self Check before Merge
⚠ Please check all items below before review. ⚠

[ ] Demo in storybook is updated/provided or not needed
[ ] TypeScript definition is updated/provided or not needed
[ ] README API section is updated or not needed

Sorry I missed this PR somehow


Documentation: Operator commutativity

Matrix dot products and tensor products (aka Kronecker products) are generally not commutative, as claimed in the first paragraph of the "Operators" section. Please correct.

Thanks!


Removing /v1 from the links e.g. self in JSON results

I am using HTTP headers to add versioning capabilities in my API. I am accessing my API endpoint as such...
http://localhost:3000/api/posts
However, for the links in relationship like comments. It shows something like this...
http://localhost:3000/api/v1/posts/1/relationships/comments
What I want to do is to have this...
http://localhost:3000/api/posts/1/relationships/comments
I read the suggestion from #257 but I could not figure out where to put the monkey patch file (apologies for that.)
Thank you!
PS: This is connected to these issues: #591 #257 #159

To monkey patch, you can create an initializer like this:
# config/initializers/jsonapi_resources_patch.rb

module JSONAPI
  class LinkBuilder
    def formatted_module_path_from_class(klass)
      '/api/'
    end
  end
end


Wrong quaternions in MotionEngine Output Packet, Format 1

Hi. When the Format 1 MotionEngine output is selected it seems to return wrong quaternions that can be observed even in MotionStudio (when you switch between Fromat 0 and Format 1 with angular position checkbox checked the values are different).
Is there a solution?

Format 1 is based on Windows 8's Sensor API spec which specifies a different frame of reference than Format 0 and also takes the conjugate of that quaternion.  This means that the rotation is expressed in a different frame than Format 0.  I believe that would explain the differences that you are seeing.  Thanks.


As a user I should be able to create scene with default asset setting.

Currently current setting is not respecting defaultAsset.
Expected This action should respect defaultAsset setting.

Working as expected.


Issue 41 - support for SQ 5.4+

Changes Unknown when pulling 0e59b437549a2b51f63e4a3f417de6757f25f92d on issue-41 into ** on master**.


Segfault when root package is not defined

I get the segfault shown below when a root package is not defined in my manifest.yaml for deploying dependent packages.
For an example - this produces the segfault:
project:
  name: WskDeployWithDeps
  version: 0.0.2
  packages:
    myrediskv:
      license: Apache-2.0
      version: 0.0.2
      namespace: my-namespace
      dependencies:
        poc-common-func:
          location: https://github.com/my-org/my-project
          version: 0.0.1
          inputs:
            redisURL:
              type: string
              value: my-url-goes-here
              description: Redis database URL

The following works properly because deployers/servicedeployer.go:472 expects there to be a root package defined:
project:
  name: WskDeployWithDeps
  version: 0.0.2
  packages:
    myrediskv:
      license: Apache-2.0
      version: 0.0.2
      namespace: my-namespace
      dependencies:
        poc-common-func:
          location: https://github.com/my-org/my-project
          version: 0.0.1
          inputs:
            redisURL:
              type: string
              value: my-url-goes-here
              description: Redis database URL
package:
  name: rediskv

Here is the segfault:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x13bea4a]

goroutine 1 [running]:
github.com/apache/incubator-openwhisk-wskdeploy/deployers.(*ServiceDeployer).DeployDependencies(0xc420102240, 0x0, 0x0)
	/Users/aervedio/go/src/github.com/apache/incubator-openwhisk-wskdeploy/deployers/servicedeployer.go:472 +0xa4a
github.com/apache/incubator-openwhisk-wskdeploy/deployers.(*ServiceDeployer).deployAssets(0xc420102240, 0x0, 0x14c0677)
	/Users/aervedio/go/src/github.com/apache/incubator-openwhisk-wskdeploy/deployers/servicedeployer.go:374 +0x50
github.com/apache/incubator-openwhisk-wskdeploy/deployers.(*ServiceDeployer).Deploy(0xc420102240, 0x0, 0x0)
	/Users/aervedio/go/src/github.com/apache/incubator-openwhisk-wskdeploy/deployers/servicedeployer.go:339 +0x2c9
github.com/apache/incubator-openwhisk-wskdeploy/cmd.Deploy(0x0, 0x0)
	/Users/aervedio/go/src/github.com/apache/incubator-openwhisk-wskdeploy/cmd/root.go:243 +0x4c5
github.com/apache/incubator-openwhisk-wskdeploy/cmd.RootCmdImp(0x171fac0, 0xc4201fc340, 0x0, 0x4, 0x0, 0x0)
	/Users/aervedio/go/src/github.com/apache/incubator-openwhisk-wskdeploy/cmd/root.go:55 +0x22
github.com/spf13/cobra.(*Command).execute(0x171fac0, 0xc4200a0100, 0x4, 0x4, 0x171fac0, 0xc4200a0100)
	/Users/aervedio/go/src/github.com/spf13/cobra/command.go:753 +0x475
github.com/spf13/cobra.(*Command).ExecuteC(0x171fac0, 0x1a, 0x0, 0xc4200d1f70)
	/Users/aervedio/go/src/github.com/spf13/cobra/command.go:843 +0x334
github.com/spf13/cobra.(*Command).Execute(0x171fac0, 0x10037bf, 0x100371e)
	/Users/aervedio/go/src/github.com/spf13/cobra/command.go:791 +0x2b
github.com/apache/incubator-openwhisk-wskdeploy/cmd.Execute()
	/Users/aervedio/go/src/github.com/apache/incubator-openwhisk-wskdeploy/cmd/root.go:69 +0x40
main.main()
	/Users/aervedio/go/src/github.com/apache/incubator-openwhisk-wskdeploy/main.go:26 +0x20

The actual line that is failing is this one:
qName, err := utils.ParseQualifiedName(depServiceDeployer.RootPackageName, depServiceDeployer.Deployment.Packages[depServiceDeployer.RootPackageName].Package.Namespace)

I traced the code and can see that depServiceDeployer.Deployment.Packages[depServiceDeployer.RootPackageName] returns a nil (also depServiceDeployer.RootPackageName is empty) and so subsequent property dereferencing blows up with a nil pointer.

And, now that I am looking closer, it looks like I am wrong about - at least - the work around. When I add the package definition to the bottom, the tool ignores the dependency and ONLY tries to deploy the package. So, while it no longer segfaults, it also does not deploy the dependencies I defined... The first config that segfaults at the end at least deploys the dependencies okay (and can undeploy them).

@allen-servedio is it possible to open granular issues for each case?  To me, it still seems like the non-deprecated, canonical schema (as described in spec.) works.  Then it is a matter of emitting the proper warnings and/or errors for those that follow deprecated (or incorrect) schema.

Okay. I created #742 for it ignoring the packages section. So the scope of this defect is the segfault issue.
In case it helps, the manifest.yaml for the dependency looks like this:
project:
  name: PocCommonFunc
  version: 0.0.1
  packages:
    rediskv:
      license: Apache-2.0
      inputs:
        redisURL:
          type: string
          description: Redis database URL
      actions:
        kvget:
          function: retrieve/index.js
          runtime: nodejs
        kvput:
          function: store/index.js
          runtime: nodejs

@allen-servedio Running the first manifest (above) that causes you a segfault works for me and correctly reports on error (Unix/Mac):
./incubator-openwhisk-wskdeploy -i -v -m ~/manifest.yaml
Info: Configuration:
      > WithinOpenWhisk: [false]
      > ApiHost: []
      > Auth: []
      > Namespace: []
      > ApiVersion: []
      > CfgFile: [/Users/Matt/.wskprops]
      > CliVersion: [unset]
      > CliBuild: [unset]
      > Verbose: [true]
      > ProjectPath: [.]
      > DeploymentPath: []
      > ManifestPath: [/Users/Matt/manifest.yaml]
      > UseDefaults: [false]
      > UseInteractive: [true]
      > Strict: [false]
      > Key: []
      > Cert: []
      > Managed: [false]

Info: The API host is [openwhisk.ng.bluemix.net], from .wskprops.
Info: The auth key is set, from .wskprops.
Info: The namespace is [_], from .wskprops.
[ploy/utils.ParseOpenWhisk]:116:[Inf] Unmarshal OpenWhisk info from internet.
Error: manifestreader.go [103]: [ERROR_YAML_FILE_FORMAT_ERROR]: File: [manifest.yaml]:
==> manifestreader.go [147]: [ERROR_YAML_FILE_FORMAT_ERROR]: File: [poc-common-func]:
==> ==> zip: not a valid zip file
==>

Sorry, I should have specified, the code I am running is based on this pull request: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/738
For you to get the segfault, it is going to have to point to a valid repo.

@allen-servedio  I am confused.  You submitted a PR to support dependencies on private repos that if we accept it results in an error elsewhere in the code?  Why does the PR not address this further issue as well as add a testcase for it?

@allen-servedio how do we automate a testcase for this (i.e., with someones private repo. credentials)?

Okay, sorry @mrutkows for wasting your time with a terrible example. I have pushed a public repo that you can use to reproduce the error. I used the the version of wskdeploy that is in your current master branch.
Go ahead and do this:
git clone git@github.com:allen-servedio/wskdeploy-defect-740.git
cd wskdeploy-defect-740/defect-740
wskdeploy -v -m manifest.yaml

And you will get this failure:
Info: action [hellopkg/hello] has been successfully deployed.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2deb87]

goroutine 1 [running]:
github.com/apache/incubator-openwhisk-wskdeploy/deployers.(*ServiceDeployer).DeployDependencies(0x10983730, 0x0, 0x0)
	/home/travis/gopath/src/github.com/apache/incubator-openwhisk-wskdeploy/deployers/servicedeployer.go:472 +0x8d7
github.com/apache/incubator-openwhisk-wskdeploy/deployers.(*ServiceDeployer).deployAssets(0x10983730, 0x0, 0x0)
	/home/travis/gopath/src/github.com/apache/incubator-openwhisk-wskdeploy/deployers/servicedeployer.go:374 +0x3b
github.com/apache/incubator-openwhisk-wskdeploy/deployers.(*ServiceDeployer).Deploy(0x10983730, 0x0, 0x0)
	/home/travis/gopath/src/github.com/apache/incubator-openwhisk-wskdeploy/deployers/servicedeployer.go:358 +0x3de
github.com/apache/incubator-openwhisk-wskdeploy/cmd.Deploy(0x1, 0x1097f7ac)
	/home/travis/gopath/src/github.com/apache/incubator-openwhisk-wskdeploy/cmd/root.go:241 +0x45d
github.com/apache/incubator-openwhisk-wskdeploy/cmd.RootCmdImp(0x565720, 0x109c9a40, 0x0, 0x3, 0x0, 0x0)
	/home/travis/gopath/src/github.com/apache/incubator-openwhisk-wskdeploy/cmd/root.go:54 +0x14
github.com/spf13/cobra.(*Command).execute(0x565720, 0x108ac388, 0x3, 0x3, 0x565720, 0x108ac388)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:632 +0x2df
github.com/spf13/cobra.(*Command).ExecuteC(0x565720, 0x3, 0x0, 0x0)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:722 +0x285
github.com/spf13/cobra.(*Command).Execute(0x565720, 0x0, 0x0)
	/home/travis/gopath/src/github.com/spf13/cobra/command.go:681 +0x1b
github.com/apache/incubator-openwhisk-wskdeploy/cmd.Execute()
	/home/travis/gopath/src/github.com/apache/incubator-openwhisk-wskdeploy/cmd/root.go:68 +0x30
main.main()
	/home/travis/gopath/src/github.com/apache/incubator-openwhisk-wskdeploy/main.go:26 +0x11

I then run wsk package list and wsk action list and can see that it actually deployed the hellopkg into my namespace and the hellopkg/hello action into it.
You can then use wskdeploy undeploy -v -m manifest.yaml to clean up your environment.
Next time I'll give you a better example - thank you for your patience.

Please refer to my previous comment, (fixed with PR #744), closing this.


Decouple connector interface

This release has number of [no-backward-compatibility] flags. But these incompatible changes related
only to the "under the hood" files. It means, if you operate only with high-level instances, like
Connector and Rest\/Checkout/Payments/OrderManagement/etc resources (you have the code,
like in the example files) - you are safe for upgrade.
If you have your own implementation of REST API Services, HTTP Transport or Base Resource instance,
you need to check the code before go live.
Changed

Decouple the HTTP Transport Connector interface. Remove Guzzle hardcoded dependency  [no-backward-compatibility]
Decouple ConnectorException. Remove Guzzle hardcoded dependency [partial-backward-compatibility]

Change the constructor signature. Replace RequestException exception with int $code
getResponse method is not longer return Guzzle ResponseInterface.
This method marked as deprecated and return null

Rename Klarna/Rest/Transport/Connector.php to Klarna/Rest/Transport/GuzzleConnector.php to
make the name explicit. [backward-compatible]
The Klarna/Rest/Transport/Connector.php still exists, but marked as deprecated
Change the structure and return values for all the methods in GuzzleConnector (ex Connector)
to follow the new Connector interface. [no-backward-compatibility]
The exception throwing behavior: [partial-backward-compatibility]

GuzzleConnector (ex Connector) not longer throws Guzzle related RuntimeException exceptions.
The Connector throws only RuntimeException
The only two types of exceptions can be thrown by SDK now:

RuntimeException
ConnectorException

Added

Add new HTTP Transport CURLTransport. This transport does not require any package dependencies
Add PHP SDK SPL Autoloader
Add ApiResponse class to unify an HTTP Transport reponse
Extend ResponseValidator with expectSuccessfull and isSuccessfull methods. The methods allows
to validate the reponse code and parse the Klarna Error Message if possible

Coverage decreased (-20.07%) to 78.788% when pulling d01c174fe0fb9a29663ab39922c7e11cd455cc19 on decouple_connector_interface into c2974541764386be91ac0d9d45f3476869f91693 on v4.1.

Coverage decreased (-20.07%) to 78.788% when pulling d01c174fe0fb9a29663ab39922c7e11cd455cc19 on decouple_connector_interface into c2974541764386be91ac0d9d45f3476869f91693 on v4.1.

Coverage decreased (-20.07%) to 78.788% when pulling d01c174fe0fb9a29663ab39922c7e11cd455cc19 on decouple_connector_interface into c2974541764386be91ac0d9d45f3476869f91693 on v4.1.

@freethan any other additions?

Good to go!


perf: buffer IO for Keystore (de)serialization

What ❔
Use BufReader and BufWriter rather than serializing to and
deserializing from memory, reducing one of the memory usage peaks caused
by temporaries.
For one particular case this avoids a 13GiB allocation.
Why ❔
The CPU prover experiences high memory usage that can bring down some instances.
This is an ongoing effort to reduce sailing and peak memory usage.
Checklist

[x] PR title corresponds to the body of PR (we generate changelog entries from PRs).
[ ] Tests for the changes have been added / updated.
[ ] Documentation comments have been added / updated.
[x] Code has been formatted via zk fmt and zk lint.
[ ] Spellcheck has been run via zk spellcheck.
[ ] Linkcheck has been run via zk linkcheck.

I tried testing the PR, but that's not possible as we're in the middle of 1.5.0 upgrade. Will follow up once the upgrade is done.

If possible, comment when that's done if you need any update to the code. I check Github notifications so I'll see it.
Re: warnings: of course, I'll fix those soon.

Rebased and fixed the warnings.