Xmas 2022 updates

Upgrading bats-core/bats-assert from v0.3.0 to v2.1.0
Upgrading bats-core/bats-core (bats/bats) from v1.7.0 to v1.8.2 (no-faccessat2)
Upgrading buildkite-plugins/docker-compose from v4.5.0 to v4.9.0
Upgrading docker-compose spec from v2.0 to v3.8
Upgrading the test.bats and splitting the tests into:

test-success.bats
test-failure.bats

Resolves https://github.com/buildkite-plugins/buildkite-plugin-tester/issues/56

Just a note: I am using test-success.bats and test-failure.bats to show variations in how you can do the BATS test setup and test cases with bats-assert. Trying to demonstrate what's on offer and also a simple way to help people to have something to reference.

Another note: This upgrade also fixes the dind (docker-in-docker) issue with using the buildkite-plugins/docker-buildkite-plugin with the buildkite/plugin-tester docker image.
Example
Running the following code:
  - label: ":shell: Test using Buildkite's own Bats docker box"
    command: "bats tests/"
    plugins:
      - docker#v5.3.0:
          image: buildkite/plugin-tester:latest
          workdir: /plugin
    agents:
      queue: elastic

Would produce this error:
Error: BATS_TMPDIR (/tmp) is not writable🚨 Error: The command exited with status 1
This is not related to forgetting to chmod a file or directory or the lack of permissions/access or wrong user.
This is caused by an underlying issue with the new system call (faccessat2) which was introduced in Linux 5.8. It returns a different error code, from ENOSYS to EPERM.
As a result, it has wide-ranging effects on both linux and docker, affected by the underlying dependencies/libraries that are used (glibc, seccomp...etc.)
References:

Issue raised in bats-core: https://github.com/bats-core/bats-core/issues/564
Issue raised in runc (used by docker): https://github.com/opencontainers/runc/issues/2151
Issue raised in 'moby' (used by docker): https://github.com/moby/moby/issues/42154
Issue raised in WSL: https://github.com/microsoft/WSL/issues/6562
Bug Report in Red Hat: https://bugzilla.redhat.com/show_bug.cgi?id=1900021
Bug Report in Arch Linux: https://bugs.archlinux.org/index.php?do=details&task_id=69563
Release Notes for Alpine 3.14: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.14.0

Release Notes Extract:

faccessat2
Use of the faccessat2 syscall has been enabled in musl. Due to runc issue 2151, new system calls incorrectly returned EPERM instead of ENOSYS when invoked under a Docker with libseccomp predating their release. Therefore, Alpine Linux 3.14 requires at least one of the following:

runc v1.0.0-rc93

if using Docker's Debian repositories, this is part of containerd.io 1.4.3-2
if using Docker Desktop for Windows or Mac, this is part of Docker Desktop 3.3.0

Docker 20.10.0 (which contains moby commit a181391) or greater, AND libseccomp 2.4.4 (which contains backported libseccomp commit 5696c89) or greater. In this case, to check if your host libseccomp is faccessat2-compatible, invoke scmp_sys_resolver faccessat2. If 439 is returned, faccessat2 is supported. If -1 is returned, faccessat2 is not supported. Note that if runc is older than v1.0.0-rc93, Docker must still be at least version 20.10.0, regardless of the result of this command.
As a workaround, in order to run under old Docker or libseccomp versions, the moby default seccomp profile should be downloaded and on line 2, defaultAction changed to SCMP_ACT_TRACE, then --seccomp-profile=default.json can be passed to dockerd, or --security-opt=seccomp=default.json passed to docker create or docker run. This will cause the system calls to return ENOSYS instead of EPERM, allowing the container to fall back to faccessat.

Note also that when using nested Docker, every layer must meet one of the above requirements, since if any layer improperly denies the use of faccessat2, Alpine Linux 3.14 will not function correctly.

Just another note:
These changes have been tested with the following:

Local Testing with bk local run
Local Testing with docker-compose run --rm tests

Local Testing with bk local run
>>> Starting local agent 🤖
>>> Starting build 👟
>>> Executing initial command: buildkite-agent pipeline upload
>>> Executing command step :docker: 🔨

~~~ Preparing plugins
# Plugin "github.com/buildkite-plugins/docker-compose-buildkite-plugin" will be checked out to "/var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-393153070/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-0"
# Switching to the temporary plugin directory
$ cd /var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-393153070/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-03444664275
$ git clone -v --recursive -- https://github.com/buildkite-plugins/docker-compose-buildkite-plugin .
Cloning into '.'...
POST git-upload-pack (193 bytes)
POST git-upload-pack (gzip 3820 to 1927 bytes)
remote: Enumerating objects: 3676, done.
remote: Counting objects: 100% (856/856), done.
remote: Compressing objects: 100% (339/339), done.
remote: Total 3676 (delta 576), reused 740 (delta 515), pack-reused 2820
Receiving objects: 100% (3676/3676), 900.53 KiB | 1.58 MiB/s, done.
Resolving deltas: 100% (2116/2116), done.
# Checking out `v4.9.0`
$ git checkout -f v4.9.0
Note: switching to 'v4.9.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at bf6b5a1 Merge pull request #364 from buildkite-plugins/toote_separator_issue-363
# Moving temporary plugin directory to final location
$ cd /private/var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-build-367920459
$ cd /Users/frank.peng/Repos/experiments/git-forks/buildkite-plugin-tester

~~~ Running plugin docker-compose command hook
$ /var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-393153070/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-0/hooks/command
2022-12-28 14:28:19 WARN   POST http://<IP_ADDRESS>:52540/jobs/f308c8b4-d47c-47d1-abfd-80d001c5bc48/data/exists: 404 (Attempt 1/10 Retrying in 5s)
2022-12-28 14:28:19 FATAL  Failed to see if meta-data exists: POST http://<IP_ADDRESS>:52540/jobs/f308c8b4-d47c-47d1-abfd-80d001c5bc48/data/exists: 404

~~~ :docker: Building Docker Compose Service: tests
⚠ No pre-built image found from a previous 'build' step for this service and config file. Building image...
$ docker-compose -f docker-compose.yml -p buildkitef308c8b4d47c47d1abfd80d001c5bc48 build --pull tests
[+] Building 7.4s (2/3)
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 32B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/bats/bats:latest@sha256:c3db40  7.3s
Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER set in environment
[2022-12-28T03:28:26.916490000Z][docker-credential-desktop][F] user: Current requires cgo or $USER set in environment
[common/pkg/paths.Home()
[	common/pkg/paths/paths.go:105 +0x58
[common/pkg/paths.Container()
[	common/pkg/paths/user_darwin.go:37 +0x1c
[common/pkg/paths.Data()
[	common/pkg/paths/paths_darwin.go:27 +0x1c
[+] Building 7.8s (2/3)
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 32B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/bats/bats:latest@sha256:c3db40  7.8s
Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER set in environment
[2022-12-28T03:28:27.397693000Z][docker-credential-desktop][F] user: Current requires cgo or $USER set in environment
[common/pkg/paths.Home()
[	common/pkg/paths/paths.go:105 +0x58
[common/pkg/paths.Container()
[	common/pkg/paths/user_darwin.go:37 +0x1c
[common/pkg/paths.Data()
[	common/pkg/paths/paths_darwin.go:27 +0x1c
[+] Building 7.9s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 32B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/bats/bats:latest@sha256:c3db40  7.8s
 => [1/8] FROM docker.io/bats/bats:latest@sha256:c3db40540a4942497a8c7ca1  0.0s
 => CACHED [2/8] RUN apk --no-cache add ncurses curl jq                    0.0s
 => CACHED [3/8] RUN mkdir -p /usr/local/lib/bats/bats-support     && cur  0.0s
 => CACHED [4/8] RUN mkdir -p /usr/local/lib/bats/bats-assert     && curl  0.0s
 => CACHED [5/8] RUN mkdir -p /usr/local/lib/bats/bats-mock     && curl -  0.0s
 => CACHED [6/8] RUN mkdir -p /usr/local/lib/bats/bats-file      && curl   0.0s
 => CACHED [7/8] RUN if [[ -e /bin/bash ]]; then echo "/bin/bash already   0.0s
 => CACHED [8/8] WORKDIR /plugin                                           0.0s
 => exporting to image                                                     0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:f64b45590a88492ba53dcc7c799081108caac7ccdd43c  0.0s
 => => naming to docker.io/library/buildkitef308c8b4d47c47d1abfd80d001c5b  0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

~~~ :docker: Starting dependencies
$ docker-compose -f docker-compose.yml -p buildkitef308c8b4d47c47d1abfd80d001c5bc48 up -d --scale tests=0 tests
[+] Running 1/0
 ⠿ Network buildkitef308c8b4d47c47d1abfd80d001c5bc48_default  Created 0.0s

+++ :docker: Running  in service tests
$ docker-compose -f docker-compose.yml -p buildkitef308c8b4d47c47d1abfd80d001c5bc48 run --name buildkitef308c8b4d47c47d1abfd80d001c5bc48_tests_build_1 --rm tests
test-failure.bats
 ✓ bats intentional failure - expects Not AOK
 ✓ bats-mock should throw error
 ✓ bats intentional failure - alerts error
test-success.bats
 ✓ bats works AOK
 ✓ bats-mock works

5 tests, 0 failures

~~~ Running plugin docker-compose pre-exit hook
$ /var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-393153070/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-0/hooks/pre-exit

~~~ :docker: Cleaning up after docker-compose
$ docker-compose -f docker-compose.yml -p buildkitef308c8b4d47c47d1abfd80d001c5bc48 kill
no container to kill
$ docker-compose -f docker-compose.yml -p buildkitef308c8b4d47c47d1abfd80d001c5bc48 rm --force -v
No stopped containers

$ docker-compose -f docker-compose.yml -p buildkitef308c8b4d47c47d1abfd80d001c5bc48 down --volumes
[+] Running 1/0
 ⠿ Network buildkitef308c8b4d47c47d1abfd80d001c5bc48_default  Removed 0.1s

>>> Command succeeded in 18.326503709s
>>> Build finished in 18.326511167s

Local Testing with docker-compose run --rm tests
➜  buildkite-plugin-tester git:(xmas-2022-updates) docker-compose run --rm tests
WARN[0000] The "BUILDKITE_TAG" variable is not set. Defaulting to a blank string.
test-failure.bats
 ✓ bats intentional failure - expects Not AOK
 ✓ bats-mock should throw error
 ✓ bats intentional failure - alerts error
test-success.bats
 ✓ bats works AOK
 ✓ bats-mock works

5 tests, 0 failures

Sorry, looks like I missed a commit for the no faccessat2 bats/bats image. Re-running the tests now.

Local Testing with bk local run
➜  buildkite-plugin-tester git:(xmas-2022-updates) ✗ bk local run
>>> Starting local agent 🤖
>>> Starting build 👟
>>> Executing initial command: buildkite-agent pipeline upload
>>> Executing command step :docker: 🔨

~~~ Preparing plugins
# Plugin "github.com/buildkite-plugins/docker-compose-buildkite-plugin" will be checked out to "/var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-402104290/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-0"
# Switching to the temporary plugin directory
$ cd /var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-402104290/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-03143097272
$ git clone -v --recursive -- https://github.com/buildkite-plugins/docker-compose-buildkite-plugin .
Cloning into '.'...
POST git-upload-pack (193 bytes)
POST git-upload-pack (gzip 3820 to 1927 bytes)
remote: Enumerating objects: 3676, done.
remote: Counting objects: 100% (856/856), done.
remote: Compressing objects: 100% (338/338), done.
remote: Total 3676 (delta 575), reused 742 (delta 516), pack-reused 2820
Receiving objects: 100% (3676/3676), 901.39 KiB | 300.00 KiB/s, done.
Resolving deltas: 100% (2115/2115), done.
# Checking out `v4.9.0`
$ git checkout -f v4.9.0
Note: switching to 'v4.9.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at bf6b5a1 Merge pull request #364 from buildkite-plugins/toote_separator_issue-363
# Moving temporary plugin directory to final location
$ cd /private/var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-build-134158543
$ cd /Users/frank.peng/Repos/experiments/git-forks/buildkite-plugin-tester

~~~ Running plugin docker-compose command hook
$ /var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-402104290/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-0/hooks/command
2022-12-29 17:10:20 WARN   POST http://<IP_ADDRESS>:57394/jobs/3aca7323-43ed-4a81-83d8-577905cf76df/data/exists: 404 (Attempt 1/10 Retrying in 5s)
2022-12-29 17:10:20 FATAL  Failed to see if meta-data exists: POST http://<IP_ADDRESS>:57394/jobs/3aca7323-43ed-4a81-83d8-577905cf76df/data/exists: 404

~~~ :docker: Building Docker Compose Service: tests
⚠ No pre-built image found from a previous 'build' step for this service and config file. Building image...
$ docker-compose -f docker-compose.yml -p buildkite3aca732343ed4a8183d8577905cf76df build --pull tests
[+] Building 1.1s (2/3)
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 2.25kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/bats/bats:latest-no-faccessat2  1.0s
Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER set in environment
[2022-12-29T06:10:22.905251000Z][docker-credential-desktop][F] user: Current requires cgo or $USER set in environment
[common/pkg/paths.Home()
[	common/pkg/paths/paths.go:105 +0x58
[common/pkg/paths.Container()
[	common/pkg/paths/user_darwin.go:37 +0x1c
[common/pkg/paths.Data()
[	common/pkg/paths/paths_darwin.go:27 +0x1c
[+] Building 1.5s (2/3)
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 2.25kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/bats/bats:latest-no-faccessat2  1.5s
Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER set in environment
[2022-12-29T06:10:23.321075000Z][docker-credential-desktop][F] user: Current requires cgo or $USER set in environment
[common/pkg/paths.Home()
[	common/pkg/paths/paths.go:105 +0x58
[common/pkg/paths.Container()
[	common/pkg/paths/user_darwin.go:37 +0x1c
[common/pkg/paths.Data()
[	common/pkg/paths/paths_darwin.go:27 +0x1c
[+] Building 6.8s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 2.25kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/bats/bats:latest-no-faccessat2  1.5s
 => [1/8] FROM docker.io/bats/bats:latest-no-faccessat2@sha256:ab8b147d6f  0.0s
 => CACHED [2/8] RUN apk --no-cache add ncurses curl jq                    0.0s
 => CACHED [3/8] RUN mkdir -p /usr/local/lib/bats/bats-support     && cur  0.0s
 => [4/8] RUN mkdir -p /usr/local/lib/bats/bats-assert     && curl -sSL h  1.7s
 => [5/8] RUN mkdir -p /usr/local/lib/bats/bats-mock     && curl -sSL htt  1.5s
 => [6/8] RUN mkdir -p /usr/local/lib/bats/bats-file      && curl -sSL ht  1.6s
 => [7/8] RUN if [[ -e /bin/bash ]]; then echo "/bin/bash already exists"  0.3s
 => [8/8] WORKDIR /plugin                                                  0.0s
 => exporting to image                                                     0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:8c0d7ada8cef9d5dd5c37244e1f8215dfdb2a3a1a3d97  0.0s
 => => naming to docker.io/library/buildkite3aca732343ed4a8183d8577905cf7  0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

~~~ :docker: Starting dependencies
$ docker-compose -f docker-compose.yml -p buildkite3aca732343ed4a8183d8577905cf76df up -d --scale tests=0 tests
[+] Running 1/0
 ⠿ Network buildkite3aca732343ed4a8183d8577905cf76df_default  Created 0.0s

+++ :docker: Running  in service tests
$ docker-compose -f docker-compose.yml -p buildkite3aca732343ed4a8183d8577905cf76df run --name buildkite3aca732343ed4a8183d8577905cf76df_tests_build_1 --rm tests
test-failure.bats
 ✓ bats intentional failure - expects Not AOK
 ✓ bats-mock should throw error
 ✓ bats intentional failure - alerts error
test-success.bats
 ✓ bats works AOK
 ✓ bats-mock works

5 tests, 0 failures

~~~ Running plugin docker-compose pre-exit hook
$ /var/folders/g5/gldb9rfd6xv_6s5fh1x4w3840000gp/T/buildkite-plugins-402104290/github-com-buildkite-plugins-docker-compose-buildkite-plugin-v4-9-0/hooks/pre-exit

~~~ :docker: Cleaning up after docker-compose
$ docker-compose -f docker-compose.yml -p buildkite3aca732343ed4a8183d8577905cf76df kill
no container to kill
$ docker-compose -f docker-compose.yml -p buildkite3aca732343ed4a8183d8577905cf76df rm --force -v
No stopped containers

$ docker-compose -f docker-compose.yml -p buildkite3aca732343ed4a8183d8577905cf76df down --volumes
[+] Running 1/0
 ⠿ Network buildkite3aca732343ed4a8183d8577905cf76df_default  Removed 0.1s

>>> Command succeeded in 19.404386959s
>>> Build finished in 19.40439675s

Local Testing with docker-compose run --rm tests
➜  buildkite-plugin-tester git:(xmas-2022-updates) ✗ docker-compose run --rm tests
WARN[0000] The "BUILDKITE_TAG" variable is not set. Defaulting to a blank string.
test-failure.bats
 ✓ bats intentional failure - expects Not AOK
 ✓ bats-mock should throw error
 ✓ bats intentional failure - alerts error
test-success.bats
 ✓ bats works AOK
 ✓ bats-mock works

5 tests, 0 failures

While we are at it, I would also suggest updating bats-mock to the (newly released) 2.1.0 version ;)

While we are at it, I would also suggest updating bats-mock to the (newly released) 2.1.0 version ;)

I'll look into it, lols! 🤣

Just for the record, upgraded buildkite-plugins/bats-mock from v2.0.1 to v2.1.0 as suggested.
Looking to revert back from:

(Multi Architecture Image) bats/bats:1.8.2-no-faccessat2@sha256:ed084f4b241c7e43422ff0a1d624a3a9609ef804ac8953449d2da63d6b8246e0
TO:
(Linux/AMD64 Architecture Image) bats/bats:1.8.2-no-faccessat2@sha256:ab8b147d6fd604a25872580db473a61d446370af00addfe0a0d8b4394eae0f6b

Reason: Faced with what looks to be a Docker Buildkit error
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: rpc error: code = Unknown desc = error getting credentials - err: exit status 1, out: ``

Excerpt of Output for Error:
...
...
[+] Building 2.5s (3/3) FINISHED
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 32B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => ERROR [internal] load metadata for docker.io/bats/bats:1.8.2-no-facce  2.5s
------
 > [internal] load metadata for docker.io/bats/bats:1.8.2-no-faccessat2@sha256:ed084f4b241c7e43422ff0a1d624a3a9609ef804ac8953449d2da63d6b8246e0:

------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: rpc error: code = Unknown desc = error getting credentials - err: exit status 1, out: ``
🚨 Error: The command exited with status 17
^^^ +++
^^^ +++
user command error: The plugin docker-compose command hook exited with status 17
...
...

Although docker build and docker buildx build work and able to build image, but not sure if practical, due to changes that might be required for the build pipeline, when releasing.

Should be ready for merge, let me know if you have any other changes in mind. :)


Store user emails in encrypted format instead of plain text.

Hi guys,
I have simple question/request ?
Is it possible to store the emails in the database in encrypted format instead of plain text ?
I've tried to pass the email as encrypted string by bypassing the UserValidation with custom UserValidator, but when i try to get the user ( Login ) the authentication fails.
Best Regards,
Dobromir Ivanov

Link to #836 and #1341, and close in favour of the more recent ones.


Check If Font is Proprietary

Page 1 CSS font includes Roboto. Please check if this font is open to public use.

CSS is redesigned by another team. This is no longer an issue.


Vite integration

Could you guys make it support Vite

Hello, @toeydevelopment ! Thanks for using Horizon UI. We'll take into consideration your suggestion, and we'll let you know if the whole community decides it's a necessary update. Regards, Vlad.


Separate destination folder image and css

Hi, I was wondering how I can set a different destination folder for the generated image and the css. Now they are both in the same folder. Ty!

Separate folders can be managed by using the separate image and CSS
streams. See the "Continuing the pipeline" example in the README as an
example. You won't need the buffer and optimization streams, only the split
dests:
https://github.com/twolfson/gulp.spritesmith/tree/6.4.0#continuing-the-pipeline
On Mar 30, 2017 1:46 AM, "Murtaza Yurtsev"<EMAIL_ADDRESS>wrote:

Hi, I was wondering how I can set a different destination folder for the
generated image and the css. Now they are both in the same folder. Ty!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/twolfson/gulp.spritesmith/issues/119, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AA3FWB0UD9J7OHpT2wJTdq-nRwg5DY9Uks5rq2v8gaJpZM4MuC87
.


Improve int/buffer and boolean/buffer [v2]

Changelog:

stretch test timeouts a bit
use node->node_type->default_options instead of hardcoded values

@otaviobp please verify if now make check-fbp pass

Tests are working fine now. +1

Merged


Sorting tables by "modified" is broken

The decorator should point to the column name (changed_on).
Currently:

With this fix:

Works both for ascending and descending sorting.

Codecov Report

Merging #6027 into master will increase coverage by 1.16%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #6027      +/-   ##
==========================================
+ Coverage    64.6%   65.77%   +1.16%
==========================================
  Files         446      446
  Lines       23814    25109    +1295
  Branches     2639     2639
==========================================
+ Hits        15386    16515    +1129
- Misses       8415     8581     +166
  Partials       13       13

Impacted Files
Coverage Δ

superset/models/helpers.py
85.97% <100%> (ø)
:arrow_up:

superset/connectors/sqla/models.py
81.13% <0%> (+0.75%)
:arrow_up:

superset/db_engine_specs.py
55.7% <0%> (+0.87%)
:arrow_up:

superset/viz.py
82.28% <0%> (+1.24%)
:arrow_up:

superset/sql_lab.py
73.93% <0%> (+2.5%)
:arrow_up:

superset/views/core.py
78.52% <0%> (+4.4%)
:arrow_up:

superset/cli.py
57.26% <0%> (+7.26%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9344f1...5b26313. Read the comment docs.

👍


Selecting multiple lines with the Option key (Feature request)

This example is from Sublime:

The default shortcut for square selection is Shift+Alt+Mouse Left Drag on Windows, should also be a equivalence on macOS.

This feature already exists and is documented here: https://code.visualstudio.com/docs/getstarted/tips-and-tricks#_multi-cursor-selection


luci-app-adblock result[0].trim is not a function

https://github.com/openwrt/luci/blob/master/applications/luci-app-adblock/htdocs/luci-static/resources/view/adblock/overview.js#L125

Just update your luci and rpcd packages.

Just happened to me.
~# opkg update
...
# opkg upgrade luci rpcd
Upgrading luci on root from git-20.136.49537-fb2f363-1 to git-20.142.51595-0c6fd62-1...
Downloading http://downloads.openwrt.org/releases/19.07.3/packages/arm_cortex-a15_neon-vfpv4/luci/luci_git-20.142.51595-0c6fd62-1_all.ipk
Configuring luci.

The package is freshly installed, not even present in the menu. Maybe something is missing in the data?

After a fresh install, a reboot seems to be required.

same issue. reboot not helping


scipy as native package

Hi, I'm trying to build scikit learn for my device and scikit uses scipy. Your layer helps me a lot to cross compile scipy, however, when trying to build scikit learn, I still got some errors such as 'no scipy module' ... I believe I need to build scipy as native package too, and then I can cross compile scikit too.
Did you try to build scikit ? Or maybe do you have any idea how to build scipy as native package ? Just adding BBCLASSEXTEND+="native" gives me some complications.

Unfortunately, I did not try to build scikit (or native scipy) as that is not a requirement on the project for which I had to build scipy.
This recipe is at a point that satisfies my project requirements, so I will not personally be making any further changes to it (except to fix bugs I might come across). However, this repository serves as a community effort for all OpenEmbedded scipy solutions, so I am happy to accept contributions and pull requests. If you do get scikit/native scipy to work, please consider contributing your changes!

Thanks to this repo i was able to get scikit-learn working on NXP LS1043A, i'll put some info here soon.

`SRC_URI += "file://OEToolchainConfig.cmake \
        file://environment.d-cmake.sh \

        file://0001-CMakeDetermineSystem-use-oe-environment-vars-to-load.patch \

        file://0001-FortranCInterface-Fix-broken-search-for-test-exe-whe.patch \

        file://0005-Disable-use-of-ext2fs-ext2_fs.h-by-cmake-s-internal-.patch \

        "

`
I had to add this patch

dunfell bbappend for lapack required these patches to build:
https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4404/diffs?commit_id=8e2683cf6db078b33cf0d5162c1a72b6f553b0be

On Dec 7, 2020, at 7:29 PM, Joel Winarske<EMAIL_ADDRESS>wrote:

dunfell bbappend for lapack required these patches to build:
https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4404/diffs?commit_id=8e2683cf6db078b33cf0d5162c1a72b6f553b0be
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

That patch is already included on the 'dunfell' branch.
https://github.com/gpanders/meta-scipy/blob/dunfell/recipes-devtools/cmake/cmake-native/0001-FortranCInterface-Fix-broken-search-for-test-exe-whe.patch

I missed your dunfell branch!

Hi. I'm also trying to use dunfell branch to build scikit and for taht scipy-native should be build. But I'm getting following problem:
building 'dfftpack' library
| compiling Fortran sources
| Fortran f77 compiler: /home/ubuntu/projects/test/build/tmp/hosttools/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops -march=armv8-a+crc
| Fortran f90 compiler: gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -march=armv8-a+crc
| Fortran fix compiler: gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -march=armv8-a+crc
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy/fftpack
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy/fftpack/src
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy/fftpack/src/dfftpack
| compile options: '-I/home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/core/include -c'
| gfortran:f77: scipy/fftpack/src/dfftpack/dcosqi.f
| gfortran:f77: scipy/fftpack/src/dfftpack/dcosqb.f
| gfortran:f77: scipy/fftpack/src/dfftpack/dcost.f
| f951: Error: bad value (‘armv8-a+crc’) for ‘-march=’ switch

Does anybody have a fix for that issue (seems that not correct arch is propagated to x86_64 build)? Thanks.

Hi. I'm also trying to use dunfell branch to build scikit and for taht scipy-native should be build. But I'm getting following problem:
building 'dfftpack' library
| compiling Fortran sources
| Fortran f77 compiler: /home/ubuntu/projects/test/build/tmp/hosttools/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops -march=armv8-a+crc
| Fortran f90 compiler: gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -march=armv8-a+crc
| Fortran fix compiler: gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops -march=armv8-a+crc
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy/fftpack
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy/fftpack/src
| creating /home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/build/temp.linux-x86_64-3.8/scipy/fftpack/src/dfftpack
| compile options: '-I/home/ubuntu/projects/test/build/tmp/work/x86_64-linux/python3-scipy-native/1.3.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/numpy/core/include -c'
| gfortran:f77: scipy/fftpack/src/dfftpack/dcosqi.f
| gfortran:f77: scipy/fftpack/src/dfftpack/dcosqb.f
| gfortran:f77: scipy/fftpack/src/dfftpack/dcost.f
| f951: Error: bad value (‘armv8-a+crc’) for ‘-march=’ switch

Does anybody have a fix for that issue (seems that not correct arch is propagated to x86_64 build)? Thanks.

Thanks to this repo i was able to get scipy and scikit-learn working on NXP LS1043A (aarch64 cross compile), i'll put some info here soon.

Do you have somewhere your work pushed for sharing? Thanks.

Thanks to this repo i was able to get scipy and scikit-learn working on NXP LS1043A (aarch64 cross compile), i'll put some info here soon.

Do you have somewhere your work pushed for sharing? Thanks.

OK after combining this layer + https://github.com/tuxable-ltd/meta-scikit-learn I have scikit learn on dunfell compiled. I need to check if it will on target (my target is 64 bit) and then I'll share. Thanks.

OK after combining this layer + https://github.com/tuxable-ltd/meta-scikit-learn I have scikit learn on dunfell compiled. I need to check if it will on target (my target is 64 bit) and then I'll share. Thanks.

Please see #15

Please see #15


pico pins should be name "GP..." to match online description

Currently called P... which is just enough different from GP... to be confusing.

fixed


Qcodes notebook update for Ithaco

Itacho notebook update
Created a folder in the driver_examples to host files needed to be imported in the notebooks.

@jenshnielsen
Could you review this?

Left a few comments inline. I have created a pr to resolve the issue that prevents you from measureing curr dirrectly.
Other than that I suggest that we rename files to static_files to make it more clear what this folder is.
It would be nice if one could see from the image where the wires go and what is connected to what.

Thanks @jenshnielsen  for the correction and the related PR. Then, I will make all amendments based on your PR and other comments and push it


cannot access attribute AND value at the same time for the element

using spark-xml 0.1.1-s_2.11 on Apache Spark 2.3.1, Scala 2.11
I uploaded a sample XML from github site:
<one>
     <two myTwoAttrib="BBBBB">two</two>
      <three>three</three>
</one>

and the schema produced doesn't include the attribute:
I get:
root
|-- one: struct (nullable = true)
|    |-- three: string (nullable = true)
|    |-- two: string (nullable = true)

When I create my own schema:
customSchema = StructType([
  StructField("one",
              StructType([
                StructField("three",StringType(),True),
                StructField("two",StructType([
                          StructField("myTwoAttrib",StringType(),True),
                          StructField("_VALUE", StringType(),True),
                        ]), True)
              ]),True)])

I get the attribute, but cannot get the value "two" anymore
So I am able to get the attribute OR the value but not both at the same time
I am using following options:
option("valueTag","_VALUE").option("excludeAttribute","false").option("attributePrefix","_")
Can you help please?
Thanks a lot in advance!

Is there any workaround/fix for this? I am still having trouble accessing both an attribute and a value for the same element.

I can't reproduce this on 0.6.0. It may be something long since fixed (this reported vs 0.1.1?)
    val df = spark.read
      .option("rowTag", "one")
      .option("inferSchema", true)
      .xml("/tmp/test.xml")
    df.show()
    df.printSchema()

...

+-----+------------+
|three|         two|
+-----+------------+
|three|[two, BBBBB]|
+-----+------------+

root
 |-- three: string (nullable = true)
 |-- two: struct (nullable = true)
 |    |-- _VALUE: string (nullable = true)
 |    |-- _myTwoAttrib: string (nullable = true)

If there's a current reproduction, let me know.


Prop Default Values

Is there a way to set default values for components through props that will be passed on submission?
I'm able to set default values visually through using inputProps={{defaultValue: value}} or through checkBoxProps={{ defaultChecked: value }} but on submissions the values don't get submit so I'm wondering if theres a way so I can avoid making a default values variable and passing it to the useForm hook.
I want to use inProp default values because I'm generating dynamic components.

@vawogbemi this is a reasonable ask and seems simple enough to implement. Give me a few days and I'll get the library updated and published

This should be resolved under 1.0.2 https://github.com/crhistianramirez/react-hook-form-chakra/releases/tag/1.0.2


Added clarification on number of validators prompt

Hey there - thanks for all the work on this tool, it's great.  While testing it out recently, I noticed that the prompt for the number of keys to generate is unclear (mostly when running with an existing mnemonic).  It currently reads, for both new and existing mnemonics, as:
"Please choose how many validators you wish to run"
This makes sense for creating from a new mnemonic, but when creating from an existing mnemonic it is unclear if that prompt is asking you for the total validators you'd like to run (including those you already have keys for) or just the number of new ones.  After testing, it is the latter - you are supposed to specify the number of new validators you'd like to generate keys for.
This PR simply changes the above from "validators" to "new validators" which I feel makes it more clear. I understand the same prompt is use for both new and existing flows, and I think will both benefit from the clarity.
Thanks in advance for taking a look!

Yes! Was confused by that wording too :)

@hwwhww @CarlBeek just wanted to make sure you saw this, let me know if you have any questions!

Thanks for opening this PR, @colfax23! 🙏 I created a new PR which solves this same issue, but rebased off of dev which has had a bunch of internationalisation changes.
This change will then be merged into the releases from there.

Awesome!  Of course :) Thanks for taking a look and including.

closing as this has been included in the new internationalised versions of the launchpad

Please confirm me as a validator
On Wed, Feb 2, 2022, 16:49 Carl Beekhuizen, @.***>
wrote:

Closed #204 https://github.com/ethereum/eth2.0-deposit-cli/pull/204.
—
Reply to this email directly, view it on GitHub
https://github.com/ethereum/eth2.0-deposit-cli/pull/204#event-5998852202,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKSH745HGA46LH6BLOV3VJ3UZGRINANCNFSM46DF4SVQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
com>


Create Ferma library AST Transformations to work with OrientDB Graph API

Ferma lib is here: https://github.com/Syncleus/Ferma
Some discussion with Ferma contributor goes here:
https://groups.google.com/a/syncleus.com/forum/#!topic/ferma-list/NfWMIAzbfEQ

after some research i found that no, ferma lib is not nice choice for this. I can use the same scheme for OrientVertex classes


Reproduction Issues of Code Llama

Hi, thx for the work.
I just tried to reproduce the results from the codellama paper. Here's the log file:
{
  "humaneval": {
    "pass@1": 0.018292682926829267
  },
  "config": {
    "prefix": "",
    "do_sample": false,
    "temperature": 0.2,
    "top_k": 0,
    "top_p": 0.95,
    "n_samples": 1,
    "eos": "<|endoftext|>",
    "seed": 0,
    "model": "/mnt/models/CodeLlama-7b-Python",
    "modeltype": "causal",
    "peft_model": null,
    "revision": null,
    "use_auth_token": false,
    "trust_remote_code": true,
    "tasks": "humaneval",
    "instruction_tokens": null,
    "batch_size": 1,
    "max_length_generation": 512,
    "precision": "bf16",
    "load_in_8bit": false,
    "load_in_4bit": false,
    "limit": null,
    "limit_start": 0,
    "postprocess": true,
    "allow_code_execution": true,
    "generation_only": false,
    "load_generations_path": null,
    "load_data_path": null,
    "metric_output_path": "/root/ch/bigcode-evaluation-harness/outputs/metrics/humaneval_codellama_7b_python.json",
    "save_generations": true,
    "save_generations_path": "/root/ch/bigcode-evaluation-harness/outputs/generations/humaneval_codellama_7b_python.json",
    "save_references": false,
    "prompt": "prompt",
    "max_memory_per_gpu": "auto",
    "check_references": false
  }
}

The result is far away from the 38.4% in the original paper. In issue#158, I noticed that for the instruction model, humanevalsynthesize-python should be used, while I only got only 1.8% too.
Would anyone be able to help reproduce the codellama results?

Quite strange, I got 40.2% with the same script using another conda environment.
{
  "humaneval": {
    "pass@1": 0.4024390243902439
  },
  "config": {
    "prefix": "",
    "do_sample": false,
    "temperature": 0.2,
    "top_k": 0,
    "top_p": 0.95,
    "n_samples": 1,
    "eos": "<|endoftext|>",
    "seed": 0,
    "model": "/mnt/models/CodeLlama-7b-Python",
    "modeltype": "causal",
    "peft_model": null,
    "revision": null,
    "use_auth_token": true,
    "trust_remote_code": true,
    "tasks": "humaneval",
    "instruction_tokens": null,
    "batch_size": 1,
    "max_length_generation": 512,
    "precision": "bf16",
    "load_in_8bit": false,
    "load_in_4bit": false,
    "limit": null,
    "limit_start": 0,
    "postprocess": true,
    "allow_code_execution": true,
    "generation_only": false,
    "load_generations_path": null,
    "load_data_path": null,
    "metric_output_path": "/root/ch/bigcode-evaluation-harness/outputs/metrics/humaneval_codellama_7b_python.json",
    "save_generations": true,
    "save_generations_path": "/root/ch/bigcode-evaluation-harness/outputs/generations/humaneval_codellama_7b_python.json",
    "save_references": false,
    "prompt": "prompt",
    "max_memory_per_gpu": "auto",
    "check_references": false
  }
}

What happened? The former environment is created using the requirement.txt with python=3.7.10, while the next one has python of 3.10.0

I haven't tried execution with python 3.7 but it could be that the execution of the generated solution failed in that env. Try evaluationg the generations you got with python 3.10 in your python 3.7 env using evaluation only mode by providing --load_generations_path /root/ch/bigcode-evaluation-harness/outputs/generations/humaneval_codellama_7b_python.json flag (see)

Thx @loubnabnl, I've figured it out :>


Update instructions

Hi, couldn't find any update instructions on what files to keep when you release new updates. What files do I need to retain when I drag/drop to update?

Any changes that you want to make to the config should be inside config_custom.cfg
So when you update the config files, just do not overwrite config_custom.cfg file and you should be good
I realized I removed the part in the README that explains this, I'll add it back soon
Cheers!

Thank you!


Snapshot_merge: 'WHEN MATCHED' cannot appear more than once in a 'UPDATE' clause of a MERGE statement

Describe the feature
The SQL generated for updating records in merge statements generated by snapshots does not work on SQL Server.
Describe alternatives you've considered
I want it to work on SQL Server as well as all other databases.
Additional context
This applies to SQL Server, but it could be better written anyway to not confuse developers.
Who will this benefit?
SQL Server dbt users
Are you interested in contributing this feature?
I can make a PR solving this

https://github.com/fishtown-analytics/dbt/blob/a4280d7457c32dce072d3224190a4779abc77d77/core/dbt/include/global_project/macros/materializations/snapshot/snapshot_merge.sql#L14-L24
could be rewritten as
    when matched
     and DBT_INTERNAL_DEST.dbt_valid_to is null
     and DBT_INTERNAL_SOURCE.dbt_change_type in ('update', 'delete')
        then update
        set dbt_valid_to = DBT_INTERNAL_SOURCE.dbt_valid_to

https://github.com/fishtown-analytics/dbt/blob/a4280d7457c32dce072d3224190a4779abc77d77/core/dbt/include/global_project/macros/materializations/snapshot/snapshot_merge.sql#L14-L24
could be rewritten as
    when matched
     and DBT_INTERNAL_DEST.dbt_valid_to is null
     and DBT_INTERNAL_SOURCE.dbt_change_type in ('update', 'delete')
        then update
        set dbt_valid_to = DBT_INTERNAL_SOURCE.dbt_valid_to

Agree that the more concise version is preferable, anyway!

Agree that the more concise version is preferable, anyway!


[BUG] VaultException: Configured token cannot be verified. It is most likely expired or invalid.

Description
Had a working Salt / Vault setup using the old integration.
Started to get deprecation warnings so have been attempting to switch to the new vault extension.
Have been trying for hours and I can't get past:
Error running 'vault.read_secret': Failed to read secret! VaultException: Configured token cannot be verified. It is most likely expired or invalid.
I get this if I try issuing from the master or supply locally on the client.
The token is definitely valid (tested it direct on vault and tried different ones). Even tried the root token.
If I change the URL, it fails with a different error, so definitely seems to be attempting to connect to valult.
Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)
/etc/salt/master.d/vault.conf:
vault:
  auth:
    method: token
    token: xxx
  server:
    url: https://vault.myinstance.com
  cache:
    backend: disk
  issue:
    type: token
    token:
      params:
        explicit_max_ttl: 30  # Tokens will be valid for 30s
        num_uses: 10          # Tokens will be limited to 10 uses
  policies:
    assign:
      - 'salt/minions'
      - 'salt/minion/{minion}'

Please be as specific as possible and give set-up details.

[ ] on-prem machine
[ X ] VM (Virtualbox, KVM, etc. please specify) - KVM/Ganeti
[ ] VM running on a cloud service, please be explicit and add details
[ ] container (Kubernetes, Docker, containerd, etc. please specify)
[ ] or a combination, please be explicit
[ ] jails if it is FreeBSD
[ ] classic packaging
[ X ] onedir packaging
[ ] used bootstrap to install

Steps to Reproduce the behavior
salt-call vault.read_secret salt/minion/myminion.domain.com/something

Expected behavior
Should return the secret.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3007.1

Python Version:
        Python: 3.10.14 (main, Apr  3 2024, 21:30:09) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.16.0
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4

Salt Extensions:
 saltext.vault: 1.0.0

Salt Package Information:
  Package Type: onedir

System Versions:
          dist: debian 12.5 bookworm
        locale: utf-8
       machine: x86_64
       release: 6.1.0-21-amd64
        system: Linux
       version: Debian GNU/Linux 12.5 bookworm

To verify the token, the client issues a lookup request to the server and checks the status code of the response. If it is not 200, it interprets this as the token being invalid.
It would be very interesting to see what the response is exactly. Unfortunately, there is no logging at that line. Can you add it and report back what it says? You should find the relevant file at /opt/saltstack/salt/extras-3.10/saltext/vault/utils/vault/factory.py:
diff --git a/src/saltext/vault/utils/vault/factory.py b/src/saltext/vault/utils/vault/factory.py
index 9978efb..ccad692 100644
--- a/src/saltext/vault/utils/vault/factory.py
+++ b/src/saltext/vault/utils/vault/factory.py
@@ -621,6 +621,7 @@ def _fetch_token(config, opts, token_cache, unwrap_client, force_local=False, em
                 # lookup and verify raw token
                 token_info = unwrap_client.token_lookup(embedded_token, raw=True)
                 if token_info.status_code != 200:
+                    log.error(f"Token lookup failed! status: {token_info.status_code} text: {token_info.text}")
                     raise VaultException(
                         "Configured token cannot be verified. It is most likely expired or invalid."
                     )

You might need to restart the daemon for the change to take effect.

@ichilton Since I did not hear back from you, did you find the cause? If not, you can try the new release, which at least logs the API response to aid in debugging.

Hi @lkubb,
Sorry, I completely missed these replies!
I just came back to this in the hope that the problem might have gone away / been fixed, but seeing the same.
I upgraded to v1.1.1 as you suggested and I see the same error, but with an indication that it's getting a 503 response.
root@salt-master:~# salt-pip  list | grep vault
saltext.vault      1.0.0

root@salt-master:~# salt-pip install --upgrade saltext.vault
Requirement already satisfied: saltext.vault in /opt/saltstack/salt/extras-3.10 (1.0.0)
Collecting saltext.vault
  Downloading saltext.vault-1.1.1-py2.py3-none-any.whl.metadata (5.4 kB)
[snip requirement already satisfied stuff]
Downloading saltext.vault-1.1.1-py2.py3-none-any.whl (97 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.8/97.8 kB 5.9 MB/s eta 0:00:00
Installing collected packages: saltext.vault
  Attempting uninstall: saltext.vault
    Found existing installation: saltext.vault 1.0.0
    Uninstalling saltext.vault-1.0.0:
      Successfully uninstalled saltext.vault-1.0.0
Successfully installed saltext.vault-1.1.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.3.2 -> 24.2
[notice] To update, run: /opt/saltstack/salt/bin/python3.10 -m pip install --upgrade pip
root@salt-master:~# salt-pip list | grep vault
saltext.vault      1.1.1
root@salt-master:~# systemctl restart salt-master
root@salt-master:~# systemctl restart salt-minion
root@salt-master:~# salt-call vault.read_secret salt/global/test
[ERROR   ] Token lookup failed! status code: 502
Error running 'vault.read_secret': Failed to read secret! VaultException: Configured token cannot be verified. It is most likely expired or invalid.

Any ideas?
Thanks,
Ian

Hi,
Ok, tracked down the 503 error - i've got nginx in front of vault, for SSL termination and as a proxy.
I had this:
  location / {
    proxy_pass http://localhost:8200/;
  }

Vault is only listening on v4:
listener "tcp" {
  address       = "<IP_ADDRESS>:8200"
  tls_disable	= 1
}

So I was getting this in the nginx error.log:
2024/07/30 20:43:56 [error] 3350#3350: *162 connect() failed (111: Connection refused) while connecting to upstream, client: <redacted>, server: _, request: "GET /v1/auth/token/lookup-self HTTP/1.1", upstream: "http://[::1]:8200/v1/auth/token/lookup-self", host: "<redacted>"

So I fixed it with:
  location / {
    proxy_pass http://<IP_ADDRESS>:8200/;
  }

Now I get a 403 error...
root@salt-master:~# salt-call vault.read_secret salt/global/test
[ERROR   ] Token lookup failed! status code: 403
Error running 'vault.read_secret': Failed to read secret! VaultException: Configured token cannot be verified. It is most likely expired or invalid.

<redacted> - - [30/Jul/2024:20:45:31 +0100] "GET /v1/auth/token/lookup-self HTTP/1.1" 403 80 "-" "python-requests/2.31.0"

Any ideas? - or is there a way of seeing why vault would be doing that?
If I do vault token lookup <token from /etc/salt/master.d/vault.conf>, it works fine and doesn't expire until a future date.
Thanks,
Ian

Hi @lkubb,
If I do a request to that URL from the salt-master, it does succeed.
VAULT_TOKEN environment var is set with the value from:
vault:
  auth:
    method: token
    token: <TOKEN>

root@salt-master:~# curl -I --request GET --header "X-Vault-Token: $VAULT_TOKEN" https://vault.fqdn.com/v1/auth/token/lookup-self
HTTP/1.1 200 OK
Server: nginx/1.22.1
Date: Tue, 30 Jul 2024 20:18:16 GMT
Content-Type: application/json
Content-Length: 685
Connection: keep-alive
Cache-Control: no-store
Strict-Transport-Security: max-age=31536000; includeSubDomains

You're not doing a HTTP HEAD request are you?
root@salt-master:~# curl -I --header "X-Vault-Token: $VAULT_TOKEN" https://vault.fqdn.com/v1/auth/token/lookup-self
HTTP/1.1 403 Forbidden

Thanks,
Ian

tcpdump'ing port 8200 on the vault server, I can see the raw HTTP requests hitting vault.
I can see that the token used in the token-lookup request is not the token from the config file (interestingly it does have the same first few characters - but then is otherwise different).
I was then thinking it must be the token for the minion that vault issues by calling auth/token/create with the configured token.
However, there is no sign of any such call to vault - the only call is the /v1/auth/token/lookup call in question.
So where is it getting the token from??
Thanks,
Ian

Sorry, I completely missed these replies!

No worries!

Ok, tracked down the 503 error

Awesome :)

Now I get a 403 error... Any ideas? - or is there a way of seeing why vault would be doing that?

Permission denied on this endpoint means the token the minion uses is invalid for that server.

I can see that the token used in the token-lookup request is not the token from the config file.

The token lookup is only done when using local configuration, not when a minion receives a freshly issued token from the master. I would guess this is where the issue can be found:
Since you're using salt-call (even on the master node), you're calling the minion, which apparently has a local configuration itself. Just remove the vault configuration, especially the outdated token, from /etc/salt/minion* configuration files (or replace it with a valid one).

Soooo, I just found some old vault config hidden in a file under /etc/salt/master.d, which explains the incorrect token!
but now I get this:
root@salt-master:~# salt-call vault.read_secret salt/global/test
[ERROR   ] Failed to get Vault connection from master! An error was returned: VaultInvocationError: child policies must be subset of parent
Error running 'vault.read_secret': Failed to read secret! CommandExecutionError: {'error': 'VaultInvocationError: child policies must be subset of parent'}

Config is now:
vault:
  auth:
    method: token
    token: "token"
  server:
    url: https://vault.fqdn.com
  cache:
    backend: disk
  issue:
    type: token
    token:
      params:
        explicit_max_ttl: 300  # Tokens will be valid for 300s
        num_uses: 100          # Tokens will be limited to 100 uses
  policies:
    assign:
      - salt/minions
      - salt/minion/{minion}

Now where am I going wrong? :)
Thanks,
Ian

https://salt-extensions.github.io/saltext-vault/topics/basic_configuration.html#prerequisites
When using token issuance, you need to ensure your master is allowed to issue roles different from itself (or add all roles you want to assign to minions to it). There are two ways to do that:

Give it sudo capabilities on auth/token/create (strongly discouraged)
Create a token role

Thank you - will read up and try again later!

@ichilton In case it's still relevant, I added a more detailed description of Token Roles and their purpose to the docs: https://salt-extensions.github.io/saltext-vault/topics/basic_configuration.html#credential-issuance
Since the cause of the original issue turned out to be external and the new logging helped the diagnosis, I'm going to close this. Feel free to ask further questions in here or in a discussion. :) Thanks!


terminal: dont block indefinitely on macaroon channel

In this commit, we ensure that when we wait on the macaroon channel for the macaroon from the integrated LND, that we also listen to other exit signals such as the interceptor, lndQuit chanel and the error channel. This is to prevent shutdown from blocking forever in the case where there was an issue preventing LND from sending the initial macaroon.
The issue was caught in this comment: https://github.com/lightninglabs/lightning-terminal/pull/541#discussion_r1285486027

@viktortigerstrom: review reminder

@viktortigerstrom: review reminder


How to check out updated nested submodules if changing top-level submodule commit

Hello,
Disclaimers: (1) Sorry for the possibly incomprehensible title. (2) I understand development code is unsupported.
Background
@lizziel and I are developing an update to GEOS-Chem chemistry within CAM. The new update has both new submodules under CAM, and updates to old submodules. Something like this:
Old
ESCOMP/CESM @ cesm3_0_beta01
-- components/cam: ESCOMP/CAM @ cam6_3_162
---- components/cam/src/chemistry/geoschem/geoschem_src: geoschem/geos-chem @ 14.1.2
---- components/cam/src/hemco: ESCOMP/HEMCO @ hemco-cesm1_2_1_hemco3_6_3_cesm_rmebranch

New
ESCOMP/CESM @ cesm3_0_beta01
-- components/cam: geoschem/CAM @ feature/geoschem_14.4
---- components/cam/src/chemistry/geoschem/geoschem_src: geoschem/geos-chem @ feature/geoschem_14.4_in_cesm
---- components/cam/src/hemco: ESCOMP/HEMCO @ dev/hemco-cesm1_4_0_hemco3_9_0
---- components/cam/src/chemistry/hetp: ** new external **

What has happened?
I want to checkout ESCOMP/CESM at cesm3_0_beta01, and update to use our fork of CAM.
Analogous to usage of manage_externals, I change .gitmodules section on cam to be:
[submodule "cam"]
        path = components/cam
        url = https://www.github.com/geoschem/CAM
        fxDONOTUSEurl = https://www.github.com/ESCOMP/CAM
        fxtag = feature/geoschem_14.4
        fxrequired = ToplevelRequired

Then run ./bin/git-fleximod update.
The expected result is that components/cam is checked out at feature/geoschem_14.4, and the externals in CAM, i.e., geoschem is checked out at feature/geoschem_14.4_in_cesm, hemco is checked out at dev/..., and the new external hetp is checked out.
What actually happens is that feature/geoschem_14.4 is checked out in components/cam, but the underlying submodules use what's specified in cam6_3_162:
Recursively checking out submodules of cam

...
e             geoschem not checked out, aligned at tag 14.1.2
e                hemco not checked out, aligned at tag hemco-cesm1_2_1_hemco3_6_3_cesm_rme
...

                 cam updated to feature/geoschem_14.4

If components/cam is navigated you can see that git status shows the geoschem and hemco submodules have been checked out at their cam6_3_162 commits and not the ones specified in feature/geoschem_14.4.
Why?
The problem is that Git doesn't actually store the submodule commit hash in .gitmodules, unlike how it is stored by manage_externals in Externals.cfg. This is, of course, standard Git behavior, but a manage_externals user might not realize this, and the presence of fxtag in .gitmodules could imply that git-fleximod will help replicate old behavior.
The correct way to checkout an updated CAM external after checking out ESCOMP/CESM, is to update the Git index:
git update-index --cacheinfo 160000,a126198071fe4e831cd1465d44e6eb0dd3f9cb1a,components/cam

To point components/cam to the commit corresponding to feature/geoschem_14.4; then, .gitmodules can be updated so fxtag is the same, then ./bin/git-fleximod update will work as expected.
Questions
What is the standard git-fleximod approach to switching a external to a different commit correctly (both in terms of convention and also handling the submodules within it correctly)?
Is git-fleximod considering to mimic the behavior of manage_externals by handling the case where fxtag specified in .gitmodules is out-of-sync with the git index?
Thanks!

git-fleximod update only updates the sandbox - to update the submodule reference in the remote you need to add and commit the submodule - you can do this with
git commit -a -m'update submodule'

Thanks @jedwards4b, what if I haven't checked out components/cam yet in the sandbox and I want to update the reference so it checks out a fork instead (which contains updated externals)?
Even after I git-fleximod update, go into components/cam and checkout feature/geoschem_14.4, whether or not I commit the updated ref into the sandbox will not change the behavior of git-fleximod update; it seems to ignore any new/updated externals under components/cam.

Minimal example:
git clone git@github.com:ESCOMP/CESM
cd CESM
sed -i "s/ESCOMP\/CAM/geoschem\/CAM/g" .gitmodules
sed -i "s/cam6_3_162/feature\/geoschem_14.4/g" .gitmodules
./bin/git-fleximod update

Expected: components/cam/src/chemistry/geoschem_src should be at feature/geoschem_14.4_in_cesm (commit 002a25d). The remote tree has the right submodule reference, see: https://github.com/geoschem/CAM/tree/feature/geoschem_14.4/src/chemistry/geoschem
Actual: components/cam/src/chemistry/geoschem_src is at 14.1.2, which is what's pointed from cam6_3_162: https://github.com/ESCOMP/CAM/tree/cam6_3_162/src/chemistry/geoschem

I also wonder if this would work?

clone cesm
run fleximod in cesm clone
cd components/cam
add remote for geoschem/cam
checkout cam branch
run fleximod in cam

This is what you would do with the offline GEOS-Chem if the top-level did not have a commit of the submodule you want, except instead of running fleximod you do git submodule update --init --recursive.

If that does work, why can't fleximod be run recursively in the submodules specified in .gitmodules?

Hi @lizziel, this doesn't work. I think it's similar to what my process is doing, git-fleximod update is checking out components/cam in the end and doesn't recurse to reinitialize the submodules.
What I'm doing:

clone ESCOMP/CESM
./bin/git-fleximod update to checkout a clean tree

cd components/cam
git remote add geoschem git@github.com:geoschem/CAM.git
git fetch geoschem
git checkout feature/geoschem_14.4

In accordance to git submodule behavior, the submodules are not updated at this stage:
➜  cam git:(ab476f9b) git checkout feature/geoschem_14.4
Updating files: 100% (25/25), done.
M       src/chemistry/geoschem/geoschem_src
M       src/hemco
Previous HEAD position was ab476f9b Merge pull request #1052 from jedwards4b/rme/cam6_3_161
branch 'feature/geoschem_14.4' set up to track 'geoschem/feature/geoschem_14.4'.
Switched to a new branch 'feature/geoschem_14.4'
➜  cam git:(feature/geoschem_14.4) ✗ git status
Refresh index: 100% (2198/2198), done.
git sOn branch feature/geoschem_14.4
Your branch is up to date with 'geoschem/feature/geoschem_14.4'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   src/chemistry/geoschem/geoschem_src (new commits)
        modified:   src/hemco (new commits)

no changes added to commit (use "git add" and/or "git commit -a")
u➜  cam git:(feature/geoschem_14.4) ✗ git submodule
-2ff978f92a5ac9a6ab243e5c14d06a7e2d2f5799 ccs_config
 f923081508f4264e61fcef2753a9898e55d1598e chem_proc (chem_proc5_0_06)
-422ddaa770a3cea6e83a60c9700ebce77acaceed cime
-7476950699909813d1938a34bd8d71bf5bfbf1e9 components/cdeps
-bdf6ea04d6133434fcaa4de5336de106f01290d0 components/cice
-c05dd5c4fc85327e76523aaea9cfe1e388748928 components/cism
-a9433779f0ae499d60ad118d2ec331628f0eaaa8 components/clm
-6384ff4e4a6bc82a678f9419a43ffbd5d53ac209 components/cmeps
-81c720c7ee51f9c69f2934f696078c42f4493565 components/mizuRoute
-8c682b1b7f15d146816de302e0d359da3e957056 components/mosart
-88503adbc275fb2ccbb6b598e460deaeb140e515 components/rtm
-270433531d33c64da7944d80564fe39a84917d26 libraries/FMS
-82b0071e69d14330b75d23b0bc68543ebea9aadc libraries/mct
-f52ade075619b32fa141993b5665b0fe099befc2 libraries/parallelio
-4b9dc4871a259f00f35bb47708d876cb7dcdf75c share
 4944547f04b1457d78bf7d3c0becddcfe0deabb9 src/atmos_phys (atmos_phys0_02_006)
-4bad0b7a722f70c01e7314aacf32eec5596ca068 src/chemistry/cloud_j
+28345ee76e5631d6d14869a36dc73e9dd6e0ce1e src/chemistry/geoschem/geoschem_src (preFlexChem-6085-g28345ee76)
-2a99b24625ed26cf87ae88697ddd6cf8bbdec812 src/chemistry/hetp
 df3550b0f6a835778f32ccc8c6291942e0413f62 src/dynamics/fv3 (fv3int_053124)
-b8c33daa4b124f116b0ba1b1386968c4c15b42e4 src/dynamics/mpas/dycore
+3a6d999ab0dbee9f03ab6b9a13dd3b6d9670eb54 src/hemco (hemco-cesm1_2_1_hemco3_6_3_cesm_rme)
 825e7f20e2dd368b95b1e3cb2562ab571318bb4d src/physics/ali_arms (ALI_ARMS_v1.0.1)
 bf165cd84ef94087d9a5669a5ad47838ab24c0ef src/physics/carma/base (carma4_01)
-50cee042e588fefd3fe58c2a1d638affec8c0389 src/physics/clubb
-34d8eef3d231a87c0f73e565f6b5d548876b294a src/physics/cosp2/src
 84f27d804207e79e344e8deec98b471207f9b1f0 src/physics/pumas (pumas_cam-release_v1.36)
 be3cad3a12d25918f5016b509b15057f84aab608 src/physics/pumas-frozen (pumas_cam-release_v1.17_rename)
 df02975ab93165b34a59f0d04b4ae6148fe5127c src/physics/rrtmgp/data (v1.8)
 4d8c5df4c63434aaab854afd1b02f5986d41dfb3 src/physics/rrtmgp/ext (v1.7)

But even the top-level git-fleximod does not reinit the submodules:
cd ../..
./bin/git-fleximod update

This gives the output:
➜  CESM git:(master) ✗ ./bin/git-fleximod update
            ccs_config at tag ccs_config_cesm0.0.109
                 share at tag share1.0.19
                  cime at tag cime6.0.246
                   mct at tag MCT_2.11.0
            mpi-serial at tag MPIserial_2.5.0
s                  cam cam6_3_162-41-ga1261980 a126198071fe4e831cd1465d44e6eb0dd3f9cb1a is out of sync with .gitmodules cam6_3_162
                   ww3 at tag ww3i_0.0.2
                   rtm at tag rtm1_0_79
                pysect at tag 3.2.2
                mosart at tag mosart1_0_49
             mizuroute at tag cesm-coupling.n02_v2.1.2
                   fms at tag fi_240516
            parallelio at tag pio2_6_2
                 cdeps at tag cdeps1.0.37
                 cmeps at tag cmeps0.14.63
                  cice at tag cesm_cice6_5_0_9
                  cism at tag cismwrap_2_2_001
                   clm at tag ctsm5.2.007
                   mom at tag mi_240522
          ccs_config up to date.
               share up to date.
Recursively checking out submodules of cime

e    CIME/non_py/cprnc has no fxtag defined in .gitmodules, module at v1.0.6
                cime up to date.
                 mct up to date.
          mpi-serial up to date.
Recursively checking out submodules of cam

...
              geoschem at tag 14.1.2
                 hemco at tag hemco-cesm1_2_1_hemco3_6_3_cesm_rme
...
Recursively checking out submodules of hemco

e                HEMCO has no fxtag defined in .gitmodules, module at 3.6.3-cesm
                 cam up to date.

The underlying submodules in src/chemistry are never updated.

@jimmielin, did you also do a final step of running fleximod within CAM? I would think this would then update the submodules. Or are you saying you did this and it doesn't work?

Yes, running fleximod within components/cam gave the same result. So somehow fleximod will not recurse and reinitialize git submodules unlike git submodule update --init --recursive. Maybe there is a flag for this behavior that I am not aware.

Okay - the minimal example helps - and you are correct, that isn't the expected behavior.  Here is what I think happened:
It looks like you updated the cam/.gitmodules file to reflect the new modules that you want for cesm but when you committed that branch you didn't update the corresponding submodules.  You can see this with a standalone checkout of your cam branch:
 git clone https://www.github.com/geoschem/CAM -b feature/geoschem_14.4 cam_geoschem_14.4
Cloning into 'cam_geoschem_14.4'...
warning: redirecting to https://github.com/geoschem/CAM.git/
remote: Enumerating objects: 39126, done.
remote: Counting objects: 100% (231/231), done.
remote: Compressing objects: 100% (79/79), done.
remote: Total 39126 (delta 159), reused 207 (delta 152), pack-reused 38895
Receiving objects: 100% (39126/39126), 46.56 MiB | 18.75 MiB/s, done.
Resolving deltas: 100% (28561/28561), done.
Updating files: 100% (2198/2198), done.
derecho7: ~/sandboxes
:) cd cam_geoschem_14.4/
derecho7: ~/sandboxes/cam_geoschem_14.4
:) git status
On branch feature/geoschem_14.4
Your branch is up to date with 'origin/feature/geoschem_14.4'.

nothing to commit, working tree clean
derecho7: ~/sandboxes/cam_geoschem_14.4
:) ./bin/git-fleximod status
e            chem_proc not checked out, aligned at tag chem_proc5_0_06
e                carma not checked out, aligned at tag carma4_01
e                pumas not checked out, aligned at tag pumas_cam-release_v1.36
e         pumas-frozen not checked out, aligned at tag pumas_cam-release_v1.17_rename
e             ali_arms not checked out, aligned at tag ALI_ARMS_v1.0.1
e           atmos_phys not checked out, aligned at tag atmos_phys0_02_006
e                  fv3 not checked out, aligned at tag fv3int_053124
e             geoschem not checked out, out of sync at tag None, expected tag is geoschem_14.4_in_cesm
e              cloud_j not checked out, out of sync at tag None, expected tag is feature/set_levels_with_cloud_from_parent_model
e                 hetp not checked out, aligned at tag geoschem_hetp_1.0
e                hemco not checked out, out of sync at tag None, expected tag is dev/hemco-cesm1_4_0_hemco3_9_0
e           rte-rrtmgp not checked out, aligned at tag v1.7
e          rrtmgp-data not checked out, aligned at tag v1.8
e                 mpas not checked out, aligned at hash b8c33daa
e                cosp2 not checked out, aligned at tag v2.1.4cesm
e                clubb not checked out, aligned at tag clubb_4ncar_20231115_5406350
e                 cism not checked out, aligned at tag cismwrap_2_2_001
e                  rtm not checked out, aligned at tag rtm1_0_79
e               mosart not checked out, aligned at tag mosart1_0_49
e            mizuRoute not checked out, aligned at tag cesm-coupling.n02_v2.1.2
e           ccs_config not checked out, aligned at tag ccs_config_cesm0.0.106
e                 cime not checked out, aligned at tag cime6.0.246
e                cmeps not checked out, aligned at tag cmeps0.14.63
e                cdeps not checked out, aligned at tag cdeps1.0.34
e                share not checked out, aligned at tag share1.0.19
e                  mct not checked out, aligned at tag MCT_2.11.0
e           parallelio not checked out, aligned at tag pio2_6_2
e                 cice not checked out, aligned at tag cesm_cice6_5_0_9
e                  clm not checked out, aligned at tag ctsm5.2.007
e                  fms not checked out, aligned at tag fi_240516
    testfails = 3, local mods = 0, needs updates 30

To try to solve this I created a copy of your branch in my fork  https://www.github.com/jedwards4b/CAM
but when I try to checkout geoschem I am getting an error:
'geoschem_14.4_in_cesm' did not match any file(s) known to git'
Looking in you .gitmodules file I realize that the fxtag should be feature/geoschem_14.4_in_cesm
Okay - so I made that correction, ran git-fleximod update geoschem cloud_j hemco to get the submodules you have changed and then git commit -a to update the submodule pointers in git.   I then pushed that branch back to my fork.
You can see in my fork that these submodules now point to the correct hashes:
eg: geoschem
Going back to cesm though I see that that still hasn't solved the problem.   I'll need to dig into it more and get back to you.

Thanks @jedwards4b. Yes, everything works if we git clone https://github.com/geoschem/CAM, and use CAM as the top-level. But when CESM is the top level it doesn't recurse down to update the submodules.
Yes, geoschem_14.4_in_cesm is misspelled in the .gitmodules file in geoschem/CAM, it should be feature/geoschem_14.4_in_cesm. Thanks for noticing - I will let @lizziel know. This is a temporary fork used for development, in any case.

Thank you for reporting this issue and providing wonderful details.  I believe that #51 will solve this issue.  Please let me know if you can test and confirm.

Thanks @jedwards4b! I have tested with the latest version you released (0.8.0) and the issue is resolved. I appreciate the quick resolution, I'll close this issue now.


AWS multi-account support

What type of suggestions?
Missing Information
What page is concerned?
No response
What version of the docs?
2.6+
What is the issue?
Add information about multi-account support for AWS. Probably need to be in the Overview page as summary of what the AWS support means, and somewhere else in the docs to details how to use it.
I also believe that kubefirst features should be in parity in all clouds to give the same experience to everyone no matter which cloud they are using, but the reality is that it's not always possible: either because of engineering time we start with specific cloud and release right away, specific needs that have to be fulfilled right away or sometimes it's not a possibility cause it's related to a specific feature of a cloud. To help the users navigate toward all these, I have a known limitation section on all main cloud pages listing what is different from the usual experience, like at https://kubefirst.konstruct.io/docs/aws/overview#known-limitations . I usually approach this as "what is missing in cloud X compared to the other clouds" vs "what is available that is not elsewhere" as I feel it limit the confusion is someone saw the multi-account feature and look for it in Google or Civo. Could be good to also highlight this missing feature on other clouds.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

AWS docs have been updated to:

Remove note about workloads not being supported
Added a section under Advanced about Multi-account


Expand glob patterns.

Fixes https://github.com/caskroom/homebrew-cask/issues/12991
These are my first Ruby code lines so please be forgiving 😄

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.
[x] Have you successfully run brew tests with your changes locally?

@reitermarkus Fixed. If it will pass the review I'll squash commits into one.

@reitermarkus Fixed.

@jawshooah @reitermarkus Fixed.

@mxl, did you have a specific Cask in mind which would benefit from this? If so, could you create a PR for it over at https://github.com/caskroom/homebrew-cask, so we can test this on a real-world example?

@reitermarkus https://github.com/caskroom/homebrew-cask/pull/30206
Also https://github.com/caskroom/homebrew-cask/blob/master/Casks/ableton-live.rb is already using glob patterns and I suspect that there may exist other casks that uses glob patterns.

Would be great if you could put the empty_directory-related lines in a separate commit.

Thanks, @mxl!


Inspector - AdminUsersWithNoMFAEnforced.ps1 - Returns Error

Describe the bug
AdminUsersWithNoMFAEnforced.ps1 returns the following error:
WARNING: Error message: Cannot bind argument to parameter 'DirectoryRoleId' because it is an empty string.
To Reproduce
Steps to reproduce the behavior:

Run the script
View HTML results
Scroll to: 1: Administrative Users with No Multi-Factor Authentication Enforced
See returned value points to log files
Log files contain error information

Expected behavior
A list of accounts with admin roles assigned that do not have MFA enabled.
Screenshots

Desktop

OS: Win11
Browser Chrome
Version 115.0.5790.171

Additional context
Error Log:

Error time: 08/11/2023 11:13:35
Error message: Cannot bind argument to parameter 'DirectoryRoleId' because it is an empty string.
Error exception: System.Management.Automation.ParameterBindingValidationException: Cannot bind argument to parameter 'DirectoryRoleId' because it is an empty string.
at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
Failed script: [REDACTED]\Security\Office365\Test\365Inspect-main\inspectors\AdminUsersWithNoMFAEnforced.ps1
Failed at line number: 29
Failed at line:                     $roleMembers = Get-MgDirectoryRoleMember -DirectoryRoleId $RID
Powershell command path: [REDACTED]\Security\Office365\Test\365Inspect-main\inspectors\AdminUsersWithNoMFAEnforced.ps1
Position message: [REDACTED]\Security\Office365\Test\365Inspect-main\inspectors\AdminUsersWithNoMFAEnforced.ps1
Stack trace: at Inspect-AdminUsersWithNoMFAEnforced, [REDACTED]\Security\Office365\Test\365Inspect-main\inspectors\AdminUsersWithNoMFAEnforced.ps1: line 29
at , [REDACTED]\Security\Office365\Test\365Inspect-main\inspectors\AdminUsersWithNoMFAEnforced.ps1: line 72
at , : line 1
at , [REDACTED]\Security\Office365\Test\365Inspect-main\365Inspect.ps1: line 375
at , : line 1

Hello,
In my tenant, roles IDs seem to be different from the official documentation, not sure why yet :
https://learn.microsoft.com/fr-fr/azure/active-directory/roles/permissions-reference
I have these roles and IDs :

$adminRoles = @("9b895d92-2cd3-44c7-9d02-a6ac2d5ea5c3", "58a13ea3-c632-46ae-9ee0-9c0d43cd7f3d", "892c5842-a9a6-463a-8041-72aa08ca3cf6", "158c047a-c907-4556-b7ef-446551a6b5f7", "f28a1f50-f6e7-4571-818b-6a12f2af6b6c", "d37c8bed-0711-4417-ba38-b4abe66ce4c2", "9f06204d-73c1-4d4c-880a-6edb90606fd8", "7be44c8a-adaf-4e2a-84d6-ab2649e08a13", "c430b396-e693-46cc-96f3-db01bf8bb62a", "729827e3-9c14-49f7-bb1b-9608f156bbb8", "62e90394-69f5-4237-9190-012177145e10", "69091246-20e8-4a56-aa4d-066075b2a7a8", "17315797-102d-40b4-93e0-432062caca18", "b0f54661-2d74-4c50-afa3-1ec803f12efe", "29232cdf-9323-42fd-ade2-1d097af3e4de", "194ae4cb-b126-40b2-bd5b-6091b380977d")
$adminRoles doesn't contain any of my tenant roles IDs, hence $RID get empty and Get-MgDirectoryRoleMember throw the error.
As I want to audit all roles, I'm using this for now :

Interesting. Thank you for that explanation.
We will change the way that the RID's are defined to compensate for this.

Ok I got it, I mixed ID and RoleTemplateId attributes :

But in this case if any of $adminRoles RoleTemplateId value doesn't exist in the tenant , the foreach loop stops and throw the error.
To quickly fix it I can either modify $adminRoles values with RoleTemplateId that exist in my tenant or not to use any role filter as previously showed.
Now I can go to sleep :-)

I also had this same error. I fixed it using the above solution by setting the Get-MgDirectoryRole to just return all roles.
After that fix I had another error - "Error message: The term 'Get-MgReportAuthenticationMethodUserRegistrationDetail' is not recognized as the name of a cmdlet"
I was able to fix this error by setting the script to use the beta endpoint - Select-MgProfile -Name "beta"
After that, it ran successfully and returned the correct list of admin  accounts as expected.


torch.cuda("cpu") --> torch.device("cpu")

Minor edit. I think this will prevent CPU users from getting the error 'module' object is not callable :)

Hi, thanks for raising the pull request! You are absolutely correct, that should have been torch.device. Sorry for the bug.


Dashcam Mode Car Unrecognized [2021 Toyota Rav4 XLE Premium]

Describe the bug
unable to use open pilot

The car cannot be recognized by OP.
Tried to follow fingerprint WIKI. BUT carFW section is empty and shown as "carFW = []"
Tried to modified values.py under directory (selfdrive/car/toyota/values.py) as described in https://github.com/commaai/openpilot/commit/d3c7c5f62a4f5e3d04fa492dd80aa8ebc9e2d0dc .
BUT, the devices still cannot recognize the car.

Expected behavior
expecting comma2 recognize the car
Device/Version information (please complete the following information):

Device: [comma two]
Dongle ID: [f7837a0c8f262656]
Route: [f7837a0c8f262656|2021-03-22--16-52-43]
Timestamp: [When in the route the bug occurs (e.g. 4min 30s into the drive)]
Version: [0.8.2]
Car make/model: [2021 Toyota Rav4 XLE Premium]

Have you ensured that your comma power and ethernet cables is properly connected to the OBDII port and the harness box at the camera?

Have you ensured that your comma power and ethernet cables is properly connected to the OBDII port and the harness box at the camera?

Yes. Otherwise, it would show No Panda.

Have you ensured that your comma power and ethernet cables is properly connected to the OBDII port and the harness box at the camera?

No panda will not appear if I disconnect the ethernet cable. The car will popup errors if I disconnect the 10 or 12 pin connector. I guess I don't have a connection problem at the backside of the mirror. But I may have issue with the ethernet to OBDII interface. I just reconnected everything and the issue didn't disappear.

Looks like you opened another issue for this. Will close this one. #20452


get method on Either interface - gets value from a Right, throws for a Left

Do you want to request a feature or report a bug?
Feature request
What is the current behavior?
Either<L, A> has a getOrElse method which can be used to get the right-ish value.
What is the expected behavior?
A get method like in scala's Options, Trys etc. This would return the value if the Either is a Right, and throw if it's a Left.
The current (hacky) workaround for this is doing something like
const num = t.number.decode(123).getOrElseL(val => ThrowReporter.report(left(val)) as any);

with the .get method it'd be more like
const num = t.number.decode(123).get;

Presumably this would require having a statically defined default thrower for Lefts - which I think is a nice solution - fp-ts could provide one, but if the user wants to use something else to make sure errors are parseable by custom logging ingestion tool, the "thrower" could be set on app startup.

This is by design, fp-ts is a (pure) functional library, functions/methods shouldn't throw.
Related: https://github.com/gcanti/fp-ts/issues/429

That's fair, I suspected as much. Most situations I've wanted to use this are either in AWS lambdas in case I want everything to come crashing down, or when I'm dealing with async APIs where I've struggled to be able to chain/map Validation objects around in combination with promises - which is why I opened this, which would provide a more functional-programming solution:  https://github.com/gcanti/fp-ts/issues/526


fishing: prevent false positive session

Adding a space after 'a' prevents chat messages when catching butterflies from passing the check. Butterfly message: "You catch and..."
Fishing message: "You catch a ..."


Make kube-aws node-pools update not to fail

We've been denying all the updates to cfn stacks for node pools in stack policy

Current coverage is 69.90% (diff: 100%)

Merging #140 into master will not change coverage

@@             master       #140   diff @@
==========================================
  Files             5          5
  Lines          1110       1110
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            776        776
  Misses          249        249
  Partials         85         85

Powered by Codecov. Last update 642c504...1fb65f7


onnx: ReduceMin/Max Ops

Adds support for ReduceMin and ReduceMax Ops. Found these in use in a Detectron2 exported model.
Implementation
~Supports Ops version 18+~ Backward compatible as far as I can tell
Open Questions

Backward compatibility: Op version 13 -> 18 has moved the axes from attributes to input. Though it's trivial to handle this with get_attr_opt::<&[i64]>(node, "axes") fallback, not sure if that's the right way of going about this!
The Op definition allows for empty set as input and prescribes -INF/ +INF or Datatype::Min/ Datatype::Max for the ops. Reference. I'm not sure if the return is supposed to be a scalar?

@LaurentMazare, any thoughts around these open questions?

Backward compatibility: Op version 13 -> 18 has moved the axes from attributes to input. Though it's trivial to handle this with get_attr_opt::<&[i64]>(node, "axes") fallback, not sure if that's the right way of going about this!

Falling back to taking the axis from the input if they are not present in the attributes sounds pretty reasonable to me.

The Op definition allows for empty set as input and prescribes -INF/ +INF or Datatype::Min/ Datatype::Max for the ops. Reference. I'm not sure if the return is supposed to be a scalar?

I guess we can just error out in this case for the moment, and if anyone runs into this issue then we can have a look at what the failing onnx model actually expects.

Thanks @LaurentMazare.
I've amended the PR with the following:

Fetching axes defaults to the current standard (via input) and falls back to attribute for backward compatibility.
Errors out on empty Tensors
Tests for backward compatibility and empty set error.

Guess this is good to go!

Thanks!


kafka-manager <IP_ADDRESS>: cannot add cluster

Hello, looks like it's impossible to add new cluster via http://kafka_manager_address/clusters . HTML form target is set to ., after submitting it I get HTTP/404.
However if I change form target to  /clusters and submit form I will successfully add new cluster to configuration.

I'm not sure why, but it works now. I'm closing this issue and will reopen if issue pops up again


support #find_by_attributes in finder

Artist.create(:name => "Mitesh", :title => "Mr.")
Artist.find_by_name("Mitesh")
Artist.find_by_name_and_title("Mitesh", "Mr.")
Artist.find_first_by_name("Mitesh")
Artist.find_first_by_name_and_title("Mitesh", "Mr.")
Artist.find_last_by_name_and_title("Mitesh", "Mr.")
Artist.find_last_by_name("Mitesh")

@mitijain123
I've made this monkey-patch which can be put into config/initializers/
https://gist.github.com/1381824
BTW: I've added support for bang methods (e.g. find_by_name!)
Take it if you like.

@iwinux
I had made this has a gem please add this code in that gem:
https://rubygems.org/gems/mongoid_dynamic_finder
Thanks


Reblog and 401 error

Hi, im trying to reblog some post but i got the 401 error all the time...
my code :
var Tok;
var Stok;
var tumblr = require('tumblr.js');
var OAuth = require('oauth').OAuth;
  var oa = new OAuth(
      'http://www.tumblr.com/oauth/request_token',
      'http://www.tumblr.com/oauth/access_token',
      'MY - consumer_key',
      'MY - consumer_secret',
 "1.0A", null, "HMAC-SHA1");

  oa.getOAuthRequestToken(function(error, oauth_token, oauth_token_secret, results) {
    if (error) return 401;
    console.log(oauth_token,oauth_token_secret,results);
    accessToken=oauth_token;
    accessTokenSecret=oauth_token_secret;
    var Tok = oauth_token;
    var Stok= oauth_token_secret;

      if (Tok != undefined) {
        var client = tumblr.createClient({
          consumer_key: 'MY - consumer_key',
          consumer_secret: 'MY - consumer_secret',
          token: "'"+Tok+"'",
          token_secret: "'"+Stok+"'"
        });
      }
        console.log(Stok);

  function init(){
    console.log("-------- INIT --------");
      if (Stok != undefined) {
        client.reblogPost('my-blog.tumblr.com', { id: postId, reblog_key: key }, function (err, data) {
            if (err) {
              console.log(err);
            } else {
            console.log('Reblog -= [OK] =-');
            }
        });
      }
  }
setTimeout(init,5000);
});

Im very bad with Callback etc... Can you help me to fix my code please ^^. (Im a newbies with nodejs but i can work with that) Please be Kind ^^.

Hi there!
What you're missing is the user token and token_secret. The OAuth tokens you get from oa.getOAuthRequestToken don't have any concept of a Tumblr user account, so you're getting 401 errors basically telling you that you don't have permission to post to that blog.
Depending on what you're trying to do specifically, you can either get your own user tokens from the API console or set up the full three-legged OAuth flow, which will go through the Tumblr authorization page and then redirect to the callback URL you set up on your app with the user token and secret when they click "Allow".
Getting your own user tokens from the API console
If you're just using your own account to make posts, this is the easy way.

 This will take you through the authorization page on the way to the API console.
Click the "Show Keys" button in the top right.
Use the keys in tumblr.createClient

Doing the three-legged OAuth flow
This way is a little more complicated because you'll need to a server to handle redirecting to the Tumblr authorization page and handling the callback URL after the user clicks "Allow", but it's necessary if you're building an app that allows people to post to their Tumblr blogs.

Get the OAuth token and secret the way you're doing with oa.getOAuthRequestToken(). Hang onto them, because you'll need them later.
Redirect to the authorization URL (https://www.tumblr.com/oauth/authorize?oauth_token=OAUTHTOKEN&oauth_callback=http://YOURCALLBACKURL) with the OAuth token and secret, which will redirect to the authorization page.
The callback URL will get the same oauth_token you passed in plus an oauth_verifier parameter. Hang onto these too.
Now you can get the user token and secret by requesting an access token, which is what gives you permission to make API requests as that user. You can get them  oa.getOAuthAccessToken(oauthToken, oauthTokenSecret, oauthVerifier, function(userToken, userTokenSecret) { /* your code here ... */ })
In the callback of oa.getOAuthAccessToken(), you finally have the user token and secret, so that combined with your consumer key and secret allows you to make fully authenticated requests to the Tumblr API.
Make the requests!

HI @kmck Thx for your reply !
I already use my token/secretToken from my API console. But with the reblogPost method J'm getting error 400 bas request.
I use :

client.userInfo - no problem
client.taggedPosts - no problem
client.reblogPost - error 400 bad request but all my function is alright... i'm stuck :/

`function reblogMachine(){
var randKey = Math.floor((Math.random() * reblogKey.length) + 1);
var key = reblogKey[randKey];
var postId = postIdList[randKey];
var blogName = 'stochastique-blog';

if (key != undefined){
  client.reblogPost(blogName,{ id: postId, reblog_key: key}, function (err, data) {
            if (err) {
              throw new Error(err);
            } else {
              console.log('Reblog -= [OK] =-');
            }
      });
    }
  }`

Oh, hmm. A 400 error in that case usually indicates that the post ID and reblog key don't match or are invalid. Could you provide an example pair that you're using?

here a random example :  ID =<PHONE_NUMBER>24Reblog key = jatue9Yf.
Maybe my list are not synchronised 0_o ! I grap random post with tags and create list of key and id. Maybe there are an error with tag...

Odd! That random example just worked for me, no problem. Seems like the ID and reblog key combination are valid, but maybe some of them are mixed up?
If you're not getting the 401, your authentication seems to be fine. You can try this to double-check that the user token/secret are valid:
client.createPost(blogName, {type:'text', body:'test'});

Keep in mind that for client.userInfo and client.taggedPosts it's not actually going to be validating the user token and secret (since they're public read-only routes).

hm...💃
With : client.createPost(blogName, {type:'text', body:'test'});
Uncaught TypeError: requestCallback(...) is not a function
API error: 400 Bad Request

Oh, good call! Didn't think to check that. 😄
You should be in good shape now, so I'm going to close this. Happy API-ing!


[enhancement] Insert "if" stub in site config

Perhaps this could be implemented as --ir (as in if-rules) or something like that.
ee could insert few marked-out examples for easy manual editing, such as by referrer, GeoIP, etc.
# WPSINGLE BASIC NGINX CONFIGURATION
server {
       if ($http_referer ~* "semalt\.com") {
          return 444;
       }

I think #352 will address this.
Please read that issue details, and if you think you need something different, add some details here so we can discuss your use case. :-)