.verify() API is malleable due to DER/non-DER encoding support

Per the comment here:
https://github.com/paulmillr/noble-curves/blob/fb02e93ff66ecd7bc7257d8f76e6cdf88b54bfa9/src/abstract/weierstrass.ts#L1053-L1060
A signature of one type can be trivially transformed into the other (much like low-s vs high-s).
Perhaps this is the intended behaviour - if so, I think it should be documented much more explicitly

This will also fail to verify valid signatures that also happen to be valid DER - I'm not sure what the odds of that are, but it seems plausible

compact signatures that also happen to be valid DER

If we're talking about secp256k1, here how this could look like:

First byte is 0x30
Second byte is 0x3e
Then, there need to be 62 bytes of two valid DER integers. Possible byte lengths of such integers:

30, 30
31, 29
32, 28

DER integers contain integer length as header, which means if there is 30-byte integer, a byte preceding it must be 0x1e (30)

Seems extremely rare, but i'll try to bruteforce it.
The question is, really, how should a better API look like? Your current strategy mitigates this issue.

This condition could be deliberately induced, to produce consensus-breaking side-effects

Induced at which point exactly? Could you describe an attack?

@paulmillr it's precisely the same set of attacks that can be performed if low-s checking is absent

The question is, really, how should a better API look like? Your current strategy mitigates this issue.

IMHO, a safe api would require the user to explicitly declare the format of their pubkey, with no automatic guessing. If a user needs to guess a key format for some reason, they can write their own logic for that, at their own risk.

This is worth considering for the next breaking release, which won't happen soon because of our release cadence (unless there's a probable security issue).
The reason it was made like this is because compact format was added after DER. To simplify verification UX, it was added in the current manner to verify. Not ideal, of course, but at least it worked for all the cases.

It's precisely the same set of attacks that can be performed if low-s checking is absent

High-S malleability allows to produce different signature that would also be valid simply by negating S by its field order. It is trivial.
Here, an attacker needs to find a compact byte sequence which is a valid DER. Seems to happen once per $2^{28.1}$ signatures: $1/256 * 1/256 * 5/256 * 3/256$.
However, even if such a sequence is found, it would be invalid DER signature:
verify_in_compact(sig) // true
is_parseable_der(sig) // true
verify_in_der(sig) // false

I've written the bruteforcer to find such signatures, nothing interesting so far:
import { secp256k1 } from '@noble/curves/secp256k1';
import { bytesToHex } from '@noble/curves/abstract/utils';

const priv = new Uint8Array(32).fill(4);
const msg = new Uint8Array(32);
const view = new DataView(msg.buffer);

const isValidInt = (i) => i >= 0x1c && i <= 0x20
for (let i = 15455938; ; i++) { // 5097214; 8839085; 9720256; 12566853; 13458036; 13831466; 15777036
  view.setUint32(0, i);
  const sig = secp256k1.sign(msg, priv).toCompactRawBytes();
  if (sig[0] === 0x30 && sig[1] === 0x3e) {
    console.log(1, i, bytesToHex(sig));
    const len_1 = sig[2];
    const end_1 = 3 + len_1;
    const len_2 = sig[end_1];

    if (isValidInt(len_1)) {
      console.log(' ', 2, len_1, len_2);
      if (isValidInt(len_2)) {
        console.log(3);
      }
      // console.log(sig.subarray(end_1, 1 + end_1))
    }
  }
}

You can bruteforce many orders of magnitude faster by incrementing k rather than picking a random one each time. I'm working on a bruteforcer now, I expect to have a result in ~hours.

true == verify_in_compact(sig, msg_a, pub) == verify_in_der(sig, msg_b, pub), is the condition we're looking for.

Hm, the brute-force is actually harder than I first thought. You also need an 02 tag byte in front of each integer.
This makes it a roughly 2^45 search- 2^45 curve point additions, that is. Too slow for my single-threaded python script, but if I could be bothered to write some OpenCL, definitely within reach.
303e1fb4a0abbd0145d61b3d17fb005eef60ac89923874bb7c00bc07c3af894673b21d84260d671a646aeb7ba66c7767a6c08732fca7646592e7115e5d6b7a11 is a valid signature for pubkey 0428f81535b7a9eb436367c89a3f4f35bdfee9ec5562498dee23f2c8dafb7433a6b6ab5f2d70da260113a753de793f58deda0d032f58f2cc9880879750ba5ae494 and message sha256("hello"), and fits the constraints you initially described, but unfortunately it doesn't have those 02 tag bytes so it's not parsed as valid DER.

I just thought of an alternative strategy that should have it done in ~2^30 steps, will have a go at writing it tomorrow.

import { secp256k1 } from '@noble/curves/secp256k1';
import { sha256 } from '@noble/hashes/sha256';

const hashed = sha256("hello 172790")
const pubkey = "0428f81535b7a9eb436367c89a3f4f35bdfee9ec5562498dee23f2c8dafb7433a6b6ab5f2d70da260113a753de793f58deda0d032f58f2cc9880879750ba5ae494"

// valid compact sig, not valid DER
const sig0 = "6d30e6d7da48a10230a61f84e86633a672a1c95d3c69e0c6af291dbbe7db96482a02d5521fc5d15e3c65df3b77ea2d32b13c10ae4611e28768cb1c6e1dff37fc"
//const sig0_parsed_as_der = secp256k1.Signature.fromDER(sig0) // would throw
const sig0_parsed_as_compact = secp256k1.Signature.fromCompact(sig0)

// valid compact sig, also valid DER
const sig1 = "303e021f2b50f3e4e236d74c15f4062596ce4ca8c7819f91a1a156187c9072a9675260021b433dcbf8d81115859a8fc3996f3f79192c72a56e6abacd93e72f26"
const sig1_parsed_as_der = secp256k1.Signature.fromDER(sig1)
const sig1_parsed_as_compact = secp256k1.Signature.fromCompact(sig1)

console.log(sig1_parsed_as_der)
console.log(sig1_parsed_as_compact)

console.log(secp256k1.verify(sig0, hashed, pubkey)) // true
console.log(secp256k1.verify(sig0_parsed_as_compact, hashed, pubkey)) // true

console.log(secp256k1.verify(sig1, hashed, pubkey)) // false (!!!)
console.log(secp256k1.verify(sig1_parsed_as_der, hashed, pubkey)) // false
console.log(secp256k1.verify(sig1_parsed_as_compact, hashed, pubkey)) // true

It's really cool that you've spent time on this! Thank you.
For the malleability to happen, an attacker would need to have distinct msg_a verifiable=true under sig (der hex) and msg_b verifiable=true under sig_parsed_as_compact. So, two true outputs under different messages.
Even if we don't find it, it is still a bug, as it can be seen from your provided vector - and it should be fixed. Will need to think what could be done. Perhaps a documentation update and then an API adjustment in v2.

For the malleability to happen, an attacker would need to have distinct msg_a verifiable=true under sig (der hex) and msg_b verifiable=true under sig_parsed_as_compact. So, two true outputs under different messages.

I disagree with this assessment. All normal inputs are malleable. For example:
import { secp256k1 } from '@noble/curves/secp256k1';
import { sha256 } from '@noble/hashes/sha256';

const hashed = sha256("hello 172790")
const pubkey = "0428f81535b7a9eb436367c89a3f4f35bdfee9ec5562498dee23f2c8dafb7433a6b6ab5f2d70da260113a753de793f58deda0d032f58f2cc9880879750ba5ae494"

// valid signature, in compact form
const sig0 = "6d30e6d7da48a10230a61f84e86633a672a1c95d3c69e0c6af291dbbe7db96482a02d5521fc5d15e3c65df3b77ea2d32b13c10ae4611e28768cb1c6e1dff37fc"
console.log(secp256k1.verify(sig0, hashed, pubkey)) // true

// without knowledge of the private key, we create a new byte sequence that is also a valid signature
const sig1 = secp256k1.Signature.fromCompact(sig0).toDERHex();
console.log(sig0 != sig1) // true
console.log(secp256k1.verify(sig1, hashed, pubkey)) // also true

Likewise, if the original signature was provided in DER form, an attacker can convert it to compact form.

Perhaps a documentation update and then an API adjustment in v2.

But yes, I think this is fine. I think the API as-is is a bit of a footgun, but I'm not aware of anyone currently using it to shoot their foot off, as it were, so there's no urgent need to fix it.

Likewise, if the original signature was provided in DER form, an attacker can convert it to compact form.

The attacker could also convert hex to bytes, or to Signature instance. Both bytes and Signatures would also be sigHex != sigBytes; sigHex != sigInstance.

I’ve added format option which prevents this from happening. And older versions can still do Signature.fromCompact() and pass sig instance


Support for HBM on Xilinx FPGAs

Bachelor thesis by Jannis Widmer.

It seems you are accidentally commiting files that don't belong here, .sdfg files for example. Please don't use git add ., preferably add individual files, or at least use git add -u


Add gyro back into mecanum drive

Can we add the gyro back into the mecanumDrive_Cartesian call? It is possible to use a joystick button to enable/disable the gyro input?

No Gyro to integrate!


User Profiles (and Organizations) Spec

Overview
User profiles stores personal information, but only a UserID and Email is required. These profiles can for a person, or for an organization to allow participation as a group.
Details:

User profile: The current version of Codalab v2 does not allow users to edit their profile. This functionality will be implemented (Figure 3). Integration of user accounts on different components (CodaBench, Codalab, ChaGrade) will be supported if needed. User’s profiles will include (* means mandatory; to protect anonymity, only a contact email is required to identify users)
Organizations as User: This will allow users to organize competitions and participate in them as a group. However, being a member of an organization is not mandatory to organize/participate a challenge/benchmark. This will also allow the platform to give credits to organization members for organizing competitions or participating in them (in their visible profile). To claim organizing a challenge or participating in it, an organization member will need to have been a member of that organization at least some time during the active period of the competition. This feature could replace both the notion of teams and of admins. To avoid sharing passwords, a member of an organization, logged in with his own personal ID, could switch to any organization he/she is a member of.
Meta-users (platform administrators) will be granted the right to impersonate users.

Tasks

User ID *: cannot be changed
Display name *: To be shown on leaderboard and profile page; by default, show the user ID; this can be changed
Contact Person Email *: can be changed
Contact Person First Name: optional
Contact Person Last Name: optional
Contact Person Title: optional
Affiliation or Organization Name: optional
Picture: optional (personal photo or organization logo)
Description: optional
Links to websites and social media: optional
is_organization? Check-box

Editable list of current organization members: provide Codalab IDs of organization
members
Non-editable list of former organization members (and dates start-end)
Choice of benchmarks and competitions co-organized to show on profile (check/uncheck) any organization member of a competition can add a competition in this list
Choice of benchmarks and competitions in which participated and rank/number of
participants – any organization member of a competition can add a competition in this list

Opt-in/out to receive emails (to respect GDPR).

Discussion points

Have base Profile object extended by User and Organization objects. Users and Organizations would share a lot of features, but would have more differentiation.

Isabelle:

Create an Organization as a "Group" and the group has an email -- like Google Groups, we should probably stick with Google Groups.

A big concern is email, @codabench.org
Consider "rights" to things:

Competition collaborators
Queue collaborators
Datasets

Maybe like Unix concept of Permissions Groups? "Permission Group" isn't an entity, but a set of people.
Roles in Permissions Groups? Like Admin vs User in group. "Who is allowed to do what?" and "Who is allowed to change permissions?"
Some use cases:

Share data
Run a class and setup multiple teams in your competition

vs Kaggle groups:

not ideal
Groups are formed by competition
Within a competition they are mutually exclusive
First person in group can invite others
"Groups" used to rate people, some credit for participating in a group that wins/gets points
(in Codalab people allowed to have multiple accounts, Kaggle only allows 1 -- "witch hunting" for cheating)

Consider this:

User makes submission to competition, no group, submission just attached to user
User joins a group, does the previous submission become a "submission of that group?"

Google groups permissions matrix

Considering a future point system. It would make sense for a user to change a submission from being from on the user to a group. But we restrict users from moving a group submission to a user. If we don't a user could take a submission from the team for themselves.


compiling error

    crypto.vcxproj -> C:\Users\Administrator\Desktop\Scripts\Discord\Freelance\rust-claimer\target\debug\build\boring-sys-imp-f95bce9fa3443951\out\build\Debug\crypto.lib
  cargo:root=C:\Users\Administrator\Desktop\Scripts\Discord\Freelance\rust-claimer\target\debug\build\boring-sys-imp-f95bce9fa3443951\out
  cargo:rustc-link-search=native=C:\Users\Administrator\Desktop\Scripts\Discord\Freelance\rust-claimer\target\debug\build\boring-sys-imp-f95bce9fa3443951\out/build/Debug
  cargo:rustc-link-lib=static=crypto
  cargo:rustc-link-lib=static=ssl
  cargo:rerun-if-env-changed=BORING_BSSL_INCLUDE_PATH

  --- stderr
  thread 'main' panicked at C:\Users\Administrator\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bindgen-0.69.4\lib.rs:622:31:
  Unable to find libclang: "couldn't find any valid shared libraries matching: ['clang.dll', 'libclang.dll'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Also encounter similar error.
OS: Mac Monterey Ver 12.7.5
CPU: Intel
--- stderr   deps/boringssl/src/include/openssl/base.h:61:10: fatal error: 'sys/types.h' file not found   thread 'main' panicked at /Users/luoqing/.cargo/registry/src/mirrors.ustc.edu.cn-61ef6e0cd06fb9b8/boring-sys-imp-2.1.0/build.rs:558:39:   Unable to generate bindings: ClangDiagnostic("deps/boringssl/src/include/openssl/base.h:61:10: fatal error: 'sys/types.h' file not found\n")   note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
Anyone can help?

You need to install the compilation environment of boringssl related platforms, please refer to: https://github.com/google/boringssl/blob/master/BUILDING.md


Feature request: Allow the excluding of files from producing warnings

In my project I'm using ProtobufJS which produces a whole bunch of warnings.  I'm only really interested in warnings in my own code - not in 3rd party code.  It would be great to be able to exclude node_modues etc from triggering warnings.
Example output at present:
WARNING in ./~/protobufjs/dist/ProtoBuf.js
Critical dependencies:
27:59-66 require function is used in a way in which dependencies cannot be statically extracted
 @ ./~/protobufjs/dist/ProtoBuf.js 27:59-66

WARNING in ./~/protobufjs/dist/ProtoBuf-light.js
Critical dependencies:
27:59-66 require function is used in a way in which dependencies cannot be statically extracted
 @ ./~/protobufjs/dist/ProtoBuf-light.js 27:59-66

WARNING in ./~/protobufjs/dist/ProtoBuf-light.min.js.gz
Module parse failed: E:\Source\MyProj\node_modules\protobufjs\dist\ProtoBuf-light.min.js.gz Line 1: Unexpected token ILLEGAL
You may need an appropriate loader to handle this file type.
(Source code omitted for this binary file)
 @ ./~/protobufjs/dist ^\.\/.*$

WARNING in ./~/protobufjs/dist/ProtoBuf.min.js.gz
Module parse failed: E:\Source\MyProj\node_modules\protobufjs\dist\ProtoBuf.min.js.gz Line 1: Unexpected token ILLEGAL
You may need an appropriate loader to handle this file type.
(Source code omitted for this binary file)
 @ ./~/protobufjs/dist ^\.\/.*$

WARNING in ./~/protobufjs/dist/README.md
Module parse failed: E:\Source\MyProj\node_modules\protobufjs\dist\README.md Line 2: Unexpected token ===
You may need an appropriate loader to handle this file type.
| Distributions
| =============
|
| ProtoBuf.js is available either with or without the .proto parser included.
 @ ./~/protobufjs/dist ^\.\/.*$

WARNING in ./~/protobufjs/dist/ProtoBuf-light.min.map
Module parse failed: E:\Source\MyProj\node_modules\protobufjs\dist\ProtoBuf-light.min.map Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "version":3,
| "file":"",
| "lineCount":88,
 @ ./~/protobufjs/dist ^\.\/.*$

WARNING in ./~/protobufjs/dist/ProtoBuf.min.map
Module parse failed: E:\Source\MyProj\node_modules\protobufjs\dist\ProtoBuf.min.map Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
| "version":3,
| "file":"",
| "lineCount":108,
 @ ./~/protobufjs/dist ^\.\/.*$

WARNING in ./~/protobufjs/~/ByteBuffer/dist/ByteBufferAB.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

WARNING in ./~/protobufjs/~/bytebuffer/dist/ByteBufferAB.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

WARNING in ./~/protobufjs/~/ByteBuffer/~/Long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

WARNING in ./~/protobufjs/~/ByteBuffer/~/long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

WARNING in ./~/protobufjs/~/bytebuffer/~/Long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

WARNING in ./~/protobufjs/~/bytebuffer/~/long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

I'd be happy to see none of the above....

You should be interested in this warnings too. You could be problematic for you.
i. e. ./~/protobufjs/dist/ProtoBuf.js 27:59-66 and ./~/protobufjs/~/ByteBuffer/... vs. ./~/protobufjs/~/bytebuffer/... need to be addressed by you or the package author.

Quite possibly, but I can only maintain my own code. So excluding messages from 3rd party libraries would be really useful. For what it's worth protobufjs is a widely used package and it has caused no problems to me (and presumably others).

I am interested in my own warnings - and my current practice of excluding all warnings because of protobufjs means I don't see them. Which is a shame.

Try these plugins to fix the warnings;
new webpack.NormalModuleReplacementPlugin(/ByteBuffer/, function(data) {
  data.request = data.request.replace("ByteBuffer", "bytebuffer")
}),
new webpack.ContextReplacementPlugin(/protobufjs.dist/, /$^/)

This removed a lot of them.  I'm still left with this:

WARNING in ./~/protobufjs/dist/ProtoBuf.js
Critical dependencies:
27:59-66 require function is used in a way in which dependencies cannot be statically extracted
 @ ./~/protobufjs/dist/ProtoBuf.js 27:59-66

WARNING in ./~/protobufjs/~/bytebuffer/~/Long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

WARNING in ./~/protobufjs/~/bytebuffer/~/long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

Whoa I am working on same issue right now too...
For your last errors try: resolve: {alias: Long: 'long'}

@sokra What is the advantage of using:
    new webpack.NormalModuleReplacementPlugin(/ByteBuffer/, function(data) {
      data.request = data.request.replace("ByteBuffer", "bytebuffer")
    }),

versus
{alias: ByteBuffer: 'bytebuffer'}

@sokra Also,
I get the following message:
27:59-66 require function is used in a way in which dependencies cannot be statically extracted

and have resolved it using:
{alias: {protobufjs: 'protobufjs/dist/ProtoBuf-light'}}

But the warning still remains. How can I disable the warning for this particular file?

Actually - this looks like it broke the build:
Uncaught Error: Cannot find module "."
main.js:2118 Uncaught TypeError: __webpack_require__(...).newBuilder is not a function

What broke it - webpack.NormalModuleReplacementPlugin or alias?

Ah ok I see the same error -- looking into it now.

In my case I am using Electron which allows calling node.js modules. This was causing the UTIL.IS_NODE check to resovle true.
Util.IS_NODE = !!(
            typeof process === 'object' && process+'' === '[object process]' && !process['browser']
        );

I'm running Webpack using Gulp if that helps.

The solution to the particular protobufjs problem appears to be patchable by the following addition to your webpack.config.js:
    "resolve": {
        "alias": {
            "Long": "long",
            "ByteBuffer": "bytebuffer"
        }
    }

See here


Remove no longer extant gulp command from README

I'm totally mostly using this just to look at the perf baseline.

@typescript-bot perf test this


Update 03-run-chroot.sh

Disabled unneeded service and crons. Some commands might be eliminated through the deletion of packages in the future.

I didn't have a chance to merge it yet, i assume i shouldn't because you need more time to figure stuff out?

yeah this still needs work in regards to the journald... So no hurry.

@RespawnDespair This is ready to be pulled now

I think there's a problem with this PR :
Synchronizing state of dbus.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable dbus
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "fr",
        LC_ALL = (unset),
        LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "fr",
        LC_ALL = (unset),
        LANG = "fr_FR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
Unknown host QEMU_IFLA type: 47
Unknown host QEMU_IFLA type: 48
Unknown host QEMU_IFLA type: 43
open: No such file or directory

The problem is the last line which makes the script stop ...

You took the sd card partion name which had been assigned to it and then when you tried to run that within chroot it see no similar partion/name/dir and fails...At least that my understanding.

Ok, but the error occurs before the partition label is changing ...
Executing: /lib/systemd/systemd-sysv-install disable dbus
Never the less I will retry without my mod, and see if I have again this problem.

yes that error is occuring because dbus has been disabled thru mask. That is intentional. along with journald

So, what's the solution to avoid the script stopping ?

I've just run again the script from the beginning (with latest PR) and couldn't reproduce this error.
I don't know what did happend ?
So all looks perfect.
I'm now flashing the image, and will test it ASAP.

That was weird... I thought your code had simply failed to produce the results you wanted. I did not realize the builder script halted.


Automatically set GOMAXPROCS using automaxprocs library

Fixes #1583.
Was only a one-liner and a go get to add, so here's a PR if no-one hates the idea.
/assign @vagababov @markusthoemmes mattmoor

Out of curiosity what happens when the resource limits on our control plane pods changes?

Out of curiosity what happens when the resource limits on our control plane pods changes?

As far as I know you can't change resource limits for pods in a Deployment/StatefulSet without restarting the pods, so any resource change would trigger a restart, and then GOMAXPROCS would be (properly) calculated again.

Yeah, VPA creates pods with new resource blocks.

Yeps, “Whenever VPA updates the pod resources the pod is recreated, which causes all running containers to be restarted.”

I love this idea /lgtm

rebased

hm. Unlikely to be related since these units don't go anywhere near sharedmain. Looks to be flaky in continuous ci too: e.g. https://prow.knative.dev/view/gcs/knative-prow/logs/ci-knative-pkg-continuous/1292718278833082368 :-(.
/retest


profile comes back empty [1.12.2]

21.11 22:55:31 [Server] INFO TPS: 16.163973
21.11 22:55:31 [Server] INFO Single Entity | Time/Tick | %
21.11 22:55:31 [Server] INFO Chunk | Time/Tick | %
21.11 22:55:31 [Server] INFO All Entities of Type | Time/Tick | %
21.11 22:55:31 [Server] INFO Average Entity of Type | Time/tick | Calls
forge: 2531
mc : 1.12.2
spongeforge-1.12.2-2529-7.0.0-BETA-2740.jar
log: https://pastebin.com/wzcrW11y

[22:50:10] [main/WARN] [FML/]: Found FMLCorePluginContainsFMLMod marker in TickProfiler-1.12-0.0.1-sources.jar. This is not recommended, @Mods should be in a separate jar from the coremod.
[22:50:10] [main/DEBUG] [FML/]: Instantiating coremod class CoreMod
[22:50:10] [main/ERROR] [FML/]: Coremod org.minimallycorrect.tickprofiler.minecraft.CoreMod: Unable to class load the plugin java.lang.ClassNotFoundException: org.minimallycorrect.tickprofiler.minecraft.CoreMod
[22:50:10] [main/DEBUG] [FML/]: Examining for coremod candidacy TickProfiler-1.12-0.0.1.jar
[22:50:10] [main/WARN] [FML/]: Found FMLCorePluginContainsFMLMod marker in TickProfiler-1.12-0.0.1.jar. This is not recommended, @Mods should be in a separate jar from the coremod.
[22:50:10] [main/DEBUG] [FML/]: Instantiating coremod class CoreMod
[22:50:10] [main/ERROR] [FML/]: Coremod org.minimallycorrect.tickprofiler.minecraft.CoreMod: Unable to class load the plugin java.lang.ClassNotFoundException: org.minimallycorrect.tickprofiler.minecraft.CoreMod

Looks like you've put TickProfiler's sources jar in the mods folder, and the normal jar.
Don't put the sources jar in the mods folder.

will do  ill let you know how it goes

http://prntscr.com/hdisz1 got it one last question
those cords are wierd  it shows {395.95.20}:6 do you know what the actual cord is cus when i tp to 395 95 22 there is nothing

It's in dimension 6, not the overworld.

ye we figeuerd that out thank you <3 fot the great mod


Search with specific key

I would like to implement search with specific key, and so some tuning like auto complete, or select bar within it.
Sorry but may i ask if there are any hints for the customization on the search bar?

@kavimaluskam So far there is no interface to customize the search bar. I think the simplest way is to hide default header and to put your component instead of it.

Implemented #44


https://lgtm.co appears to be down

Sorry if this is the wrong place to open the issue, but the webpage appears down to me. Also our github webooks aren't working.

back up


Stop Loss Bug

Hi,
Stop Loss don't work as expected. I start a test session with default parameters in config.yml (STOP_LOSS: 3)
The "STOP_LOSS" variable in session_info.json is filled with strange numbre like this:
4433.675640106201
9683876534.500877
4.759015155419072e+16
3.034450382939706e+25
6.132429670462307e+74
6.6884224795857e+193

On first buy of some coins the SL start with huge numbers:
TP:1.28:2.39 or SL:-4484690756570124023740448572080896602373552377797895937321755711979170249294191028339175364001004076712027837302784460278942709748430728250434827881423120834402557335323044126654464.00:-359321429097838669505640101637343327859453470871260005056995556301422959179011934669454782998943547413225179896825135179135730388922616458931735518059590156283531749735606361497010176.00 not yet reached, not selling ADAUSDT for now >> Bought at: 1.24810000 - Now: 1.24460000 : -0.28% Est: -0.06 USDT - CIP: -0.63/-0.51 - TAKE_PROFIT: 2.39
Eventually after a while the SL in both session_info.json and in test_coins_bought.json (or coins_bought.json) stay stuck on NaN
So there's no stop loss at all and only the TRAILING_STOP_LOSS trigger a change from NaN to a normal value.

trade.py Line: 257
coinStopLoss = BUY_PRICE + ((BUY_PRICE * coins_bought[coin]['stop_loss']) / 100)
I may be wrong but shouldn't it be a substraction?

fixed in latest version


Harden containers to CIS Level 1 Benchmarks

For FedRAMP we need to harden container to CIS L1, as we do for VMs.
I think we can use the FIPS container process to apply Ubuntu Advantage CIS hardening

Van and I have decided that on this iteration, all CVEs for the UbuntuAdvantage hardened image are false positives, and then we will inspect all the findings from Grype for the images postinstall.

UA hardening is now working, and then UA (Ubuntu Advantage) token and tools are removed from the image before storing in ECR.

Van, David: Where do matters stand with this? I need to give the JAB an update.

Container hardening has been implemented, and cis audit results are emailed at the beginning of every month. Work that still needs to be done has been separated out into other tickets


[BUG]: Installation(Online) error

Athena OS version
Athena OS live
Which environment or technology are you using?

[X] Baremetal
[ ] VirtualBox Virtual Machine
[ ] VMware Virtual Machine
[ ] Hyper-V Virtual Machine
[ ] QEMU/KVM
[ ] Parallels
[ ] UTM
[ ] Container
[ ] WSL

Kernel version
No response
Involved software
No response
Repro steps
Installation(Online)
[Select Keyboard Layout]
Region China
[Select Timezone]
Region Asia
Zone Shanghai
Expected behavior
install success
Actual behavior
error
Diagnostic logs
install.log

Error related to Chaotic mirror on edex-ui-bin package. I opened an issue to Chaotic AUR team.


Thank you for doing this, I can't believe how lacking the docs are for CC

This is far > than https://docs.computecanada.ca/wiki/AI_and_Machine_Learning

Hi, this is actually a write-up for my class at UVic :-) Thx for the compliment!


Fix clear-containers service unit

I'm not positive this is correct.  The old version (where there were two ExecStarts) errored immediately.  This version works, but causes sudo systemctl docker start to block forever, (though it does actually still work.)

Hi @frioux - thanks for raising, but I don't think this is correct. oneshot won't work for docker as it is a long-running process. Also, the reason we have a seemingly empty ExecStart= is to reset the default docker service's ExecStart= which is required before we then specify a new value (the 2nd ExecStart=).
Can you provide further details of what errors you saw originally?

Thanks for the quick response @jodh-intel.  What made me think this was the following:
Oct 28 09:42:41 zfp systemd[1]: docker.service: Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. Refusing.

I am attaching an (almost) full log of when I was playing with clear containers.  aufs implies clear, and overlay2 implies native cgroups and namespaces.
big.txt.  I can try more if you'd like, but it's a pretty big hassle as I run docker all the time for local tools.  I'm on ubuntu 17.10.

Hi @frioux - it sounds like your systemd config is invalid. You should have:

a /lib/systemd/system/docker.service with a single ExecStart= line.
a /etc/systemd/system/docker.service.d/clear-containers.conf containing two ExecStart= lines, where the first one doesn't have anything after the equals sign (as specified in step (3) in https://github.com/clearcontainers/runtime/blob/master/docs/ubuntu-installation-guide.md).

Hi @frioux - Yes, you are right that the doc doesn't mention /lib/systemd/system/docker.service explicitly. But that will be created when you install the docker-ce package in step (1). Most users won't ever need to touch that file. In fact, our install docs assume (where possible) that all distro-packaged files as pristine/sacred/read-only, so that is why we use the systemd drop-in snippet approach by creating /etc/systemd/system/docker.service.d/clear-containers.conf to override the default (distro-packaged) docker behaviour.
For an alternative we are considering, take a look at #754 and #763.
Thanks - yes, please do confirm that with an unmodified  /lib/systemd/system/docker.service you can now create a Clear Container.

So I just installed docker-ce afresh and here is the default  /lib/systemd/system/docker.service:
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service
Wants=network-online.target
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

[Install]
WantedBy=multi-user.target

But it does seem to be working anyway.  Thanks; I'll close this now.

\o/. Thanks for getting back to use @frioux - glad it's all working now! :smile:


Translated till line 251

Hi @arshadkazmi42,
I had forgotten entirely about my assigned task to translate rendering lists.md. I have translated rendering-lists.md till line 250, waiting for review and feedback to proceed with the further translation.
Kind regards,

Hi @lucifertrj
Thank you for sending PR. I will suggest translating the whole file, then we can start the review process as we have faced issues before when we were working on partial translations and review.
We try to translate the whole file in a go and then we review the translation in parts.
You can let me know once the translation of whole file is completed, post that I can start the review.

to translate the whole file in

Sure

Hi @lucifertrj I have a suggestion . Please keep some keywords in English Itself . Like Array and the function Recipeलिस्ट could have been RecipeList only. and घटक could be कंपोनेंट

Thank you for pointing that out @yogesharma12
Just to add to that for component we use कौम्पोनॅन्ट


Cannot run testsuite in Firefox 38

I cannot run our testsuite in Firefox 38. Opening http://localhost:3000/test/ shows a white page amnd results in the following error in the developer console:
ReferenceError: goog is not defined
    goog.events.listen(this, 'test', function() {});

which is a follow-up error of the error visible on the terminal:
url.js:107
    throw new TypeError("Parameter 'url' must be a string, not " + typeof url)
          ^
TypeError: Parameter 'url' must be a string, not undefined
    at Url.parse (url.js:107:11)
    at Object.urlParse [as parse] (url.js:101:5)
    at Server.closure.Server.getMain (/home/marc/eclipse-kepler-workspace/ol3/tasks/serve.js:52:28)
    at Server._requestListener (/home/marc/eclipse-kepler-workspace/ol3/node_modules/closure-util/lib/server.js:148:23)
    at Manager.handleRequest (/home/marc/eclipse-kepler-workspace/ol3/node_modules/closure-util/node_modules/socket.io/lib/manager.js:565:28)
    at Server.<anonymous> (/home/marc/eclipse-kepler-workspace/ol3/node_modules/closure-util/node_modules/socket.io/lib/manager.js:119:10)
    at Server.emit (events.js:98:17)
    at HTTPParser.parser.onIncoming (http.js:2109:12)
    at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:122:23)
    at Socket.socket.ondata (http.js:1967:22)

Steps to reproduce:
$ make cleanall && make build && make serve
// …
info serve Building examples.
info serve Parsing dependencies.
info serve Listening on http://localhost:3000/ (Ctrl+C to stop)

Then open http://localhost:3000/test/ and see the above error on the terminal.
node -v v0.10.38
npm -v 1.4.28

IE 11 and Chrome 41 are fine.

Line 52 in tasks/serve.js is throwing the error, as var referer = req.headers.referer; evaluates to undefined in Firefox.
      // …
      getMain: function(req) {
        var main;
        var query = url.parse(req.url, true).query;
        var referer = req.headers.referer;
        var pathName = url.parse(referer).pathname;
        if (pathName.indexOf('/test/') === 0) {
          // …
        } else if (pathName.indexOf('/test_rendering/') === 0) {
          // …
        } else {
          // …
        }
        return main;
      }

I don't know why FF doesn't send it, but this is the root of he above error.
@tsauerwein introduced this check in 8e0c21eb580bbd7a659fa5088b9c145030f9b872 and I remember discussions of @tschaub and @elemoine about this different handling of requires.
Can anyone share a light what I/we should do about this?

Note to self, check if referers are disabled, possibly due to a setting in web developer toolbar. Maybe we should switch to an URL parameter instead?

@marcjansen It's working with firefox 43, can we close this issue?

@fredj, sure.


Use https instead of http in urls

This PR changes every instance of an http url to its https version for security reasons.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Reply specificly

We can reply any message specificly in messenger by swiping the message. Thus we can indicate - we are replying of which message. How can it be done in fbchat?

Use the reply_to_id argument when calling send_text on a thread object. See https://github.com/carpedm20/fbchat/blob/master/fbchat/_threads/_abc.py#L102
Example:
import fbchat

session = fbchat.Session.login("<email>", "<password>")
listener = fbchat.Listener(session=session, chat_on=False, foreground=False)

for event in listener.listen():
    if isinstance(event, fbchat.MessageEvent):
        if event.author.id != session.user.id:
            event.thread.send_text(event.message.text, reply_to_id=event.message.id)

@wetmore's answer is correct. This can also be seen in the documentation for send_text. I've added an extra example for this in 65b42e65323de2fedeb424792e9511fdb0e7bf66.
If you're using v1, you would do this using Message.reply_to_id:
client.send(fbchat.Message("text", reply_to_id=message_id), thread_id=thread_id, thread_type=thread_type)

Hello Sir. Can you help me to make a fb chat bot? I am facing some minor
problems but I can't solve these. I need a brief resource also. Please help
me. Thanks in advance.😊
On Sun, Jun 7, 2020, 6:42 PM Mads Marquart<EMAIL_ADDRESS>wrote:

@wetmore https://github.com/wetmore's answer is correct. This can also
be seen in the documentation for send_text
https://fbchat.readthedocs.io/en/v2.0.0a3/api/threads.html?highlight=send_text#fbchat.ThreadABC.send_text.
I've added an extra example for this in 65b42e6
https://github.com/carpedm20/fbchat/commit/65b42e65323de2fedeb424792e9511fdb0e7bf66
.
If you're using v1, you would do this using Message.reply_to_id:
client.send(fbchat.Message("text", reply_to_id=message_id), thread_id=thread_id, thread_type=thread_type)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/carpedm20/fbchat/issues/570#issuecomment-640213260,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APSQSXSELFSYI6GIEYPFRB3RVODLRANCNFSM4NASBYKQ
.


Add autofocus to search input

Because I got tired of that extra click all the time...

Thanks @nikmd23 !

No problem, thank you @maartenba!
Hope this goes live soon.

Changed it a bit so it only happens on / and /packages.


[staging] python3Packages.sphinx: fails to build on staging

Seems to be a recent regression. staging-next is fine, but my naive bisect was stuck on some other failure. Test fails as:
$ nix build -f. python3Packages.sphinx -L
...
python3.9-sphinx> tests/test_intl.py ...........................................FF....     [ 80%]
...
python3.9-sphinx> _________________ test_additional_targets_should_be_translated _________________
python3.9-sphinx> app = <SphinxTestApp buildername='html'>
python3.9-sphinx>     @sphinx_intl
python3.9-sphinx>     @pytest.mark.sphinx(
python3.9-sphinx>         'html',
python3.9-sphinx>         srcdir='test_additional_targets_should_be_translated',
python3.9-sphinx>         confoverrides={
python3.9-sphinx>             'language': 'xx', 'locale_dirs': ['.'],
python3.9-sphinx>             'gettext_compact': False,
python3.9-sphinx>             'gettext_additional_targets': [
python3.9-sphinx>                 'index',
python3.9-sphinx>                 'literal-block',
python3.9-sphinx>                 'doctest-block',
python3.9-sphinx>                 'raw',
python3.9-sphinx>                 'image',
python3.9-sphinx>             ],
python3.9-sphinx>         }
python3.9-sphinx>     )
python3.9-sphinx>     def test_additional_targets_should_be_translated(app):
python3.9-sphinx>         app.build()
python3.9-sphinx>         # [literalblock.txt]
python3.9-sphinx>         result = (app.outdir / 'literalblock.html').read_text()
python3.9-sphinx>         # title should be translated
python3.9-sphinx>         expected_expr = 'CODE-BLOCKS'
python3.9-sphinx>         assert_count(expected_expr, result, 2)
python3.9-sphinx>         # ruby code block should be translated and be highlighted
python3.9-sphinx>         expected_expr = """'RESULT'"""
python3.9-sphinx>         assert_count(expected_expr, result, 1)
python3.9-sphinx>         # C code block without lang should be translated and *ruby* highlighted
python3.9-sphinx>         expected_expr = """#include <STDLIB.H>"""
python3.9-sphinx>         assert_count(expected_expr, result, 1)
python3.9-sphinx>         # C code block with lang should be translated and be *C* highlighted
python3.9-sphinx>         expected_expr = ("""#include """
python3.9-sphinx>                          """<STDIO.H>""")
python3.9-sphinx> >       assert_count(expected_expr, result, 1)
python3.9-sphinx> tests/test_intl.py:1143:
python3.9-sphinx> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
python3.9-sphinx> expected_expr = '#include <STDIO.H>'
python3.9-sphinx> result = '\n<!DOCTYPE html>\n\n\n  \n    <meta charset="utf-8" />\n    <meta name="viewport" content="wid...="_sources/literalblock.txt"\n          rel="nofollow">Page source\n
\n\n    \n\n    \n  \n'
python3.9-sphinx> count = 1
python3.9-sphinx>     def assert_count(expected_expr, result, count):
python3.9-sphinx>         find_pair = (expected_expr, result)
python3.9-sphinx> >       assert len(re.findall(*find_pair)) == count, find_pair
python3.9-sphinx> E       AssertionError: ('#include <STDIO.H>', '
python3.9-sphinx> E         <!DOCTYPE html>
python3.9-sphinx> E
python3.9-sphinx> E
python3.9-sphinx> E         ..."_sources/literalblock.txt"
python3.9-sphinx> E                   rel="nofollow">Page source
python3.9-sphinx> E

python3.9-sphinx> E
python3.9-sphinx> E
python3.9-sphinx> E
python3.9-sphinx> E
python3.9-sphinx> E
python3.9-sphinx> E         ')
python3.9-sphinx> E       assert 0 == 1
python3.9-sphinx> E        +  where 0 = len([])
python3.9-sphinx> E        +    where [] = <function findall at 0x7ffff73593a0>(*('#include <STDIO.H>', '\n<!DOCTYPE html>\n\n\n..."_sources/literalblock.txt"\n          rel="nofollow">Page source\n
\n\n    \n\n    \n  \n'))
python3.9-sphinx> E        +      where <function findall at 0x7ffff73593a0> = re.findall
python3.9-sphinx> tests/test_intl.py:90: AssertionError
python3.9-sphinx> --------------------------- Captured stdout teardown ---------------------------
python3.9-sphinx> # language: xx
python3.9-sphinx> # locale_dirs: ['.']
python3.9-sphinx> # testroot: root
python3.9-sphinx> # builder: html
python3.9-sphinx> # srcdir: /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated
python3.9-sphinx> # outdir: /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/_build/html
python3.9-sphinx> # status:
python3.9-sphinx> Running Sphinx v4.0.2
python3.9-sphinx> loading translations [xx]... done
python3.9-sphinx> building [mo]: targets for 27 po files that are out of date
python3.9-sphinx> writing output... [100%] xx/LC_MESSAGES/versionchange.mo
python3.9-sphinx> building [html]: targets for 28 source files that are out of date
python3.9-sphinx> updating environment: [new config] 28 added, 0 changed, 0 removed
python3.9-sphinx> reading sources... [100%] warnings
python3.9-sphinx> looking for now-outdated files... none found
python3.9-sphinx> pickling environment... done
python3.9-sphinx> checking consistency... done
python3.9-sphinx> preparing documents... done
python3.9-sphinx> writing output... [100%] warnings
python3.9-sphinx> generating indices... genindex done
python3.9-sphinx> writing additional pages... contents search done
python3.9-sphinx> copying images... [100%] i18n.png
python3.9-sphinx> copying static files... done
python3.9-sphinx> copying extra files... done
python3.9-sphinx> dumping search index in English (code: en)... done
python3.9-sphinx> dumping object inventory... done
python3.9-sphinx> build succeeded, 19 warnings.
python3.9-sphinx> The HTML pages are in ../pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/_build/html.
python3.9-sphinx> # warning:
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/glossary_terms_inconsistency.txt:6: WARNING: inconsistent term references in translated message. original: [':term:`Some term`', ':term:`Some other term`'], translated: [':term:`SOME NEW TERM`']
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/label_target.txt:41: WARNING: Duplicate target name, cannot be used as a unique reference: "duplicated sub section".
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/literalblock.txt:13: WARNING: Literal block expected; none found.
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs.txt:16: WARNING: inconsistent references in translated message. original: ['`download Sphinx`_'], translated: ['`A3 DOWNLOAD SPHINX <download Sphinx_>`_', '`A3 DOWNLOAD SPHINX <download Sphinx_>`_']
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs.txt:32: WARNING: inconsistent references in translated message. original: ['`Translation Tips`_'], translated: ['`X TIPS <Translation Tips_>`_', '`X TIPS <Translation Tips_>`_']
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs.txt:34: WARNING: inconsistent references in translated message. original: ['`Translation Tips`_'], translated: ['`TRANS <X TIPS_>`_', '`LATION <X TIPS_>`_']
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs_inconsistency.txt:6: WARNING: inconsistent footnote references in translated message. original: ['[#]_'], translated: []
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs_inconsistency.txt:6: WARNING: inconsistent footnote references in translated message. original: ['[100]_'], translated: []
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs_inconsistency.txt:7: WARNING: inconsistent references in translated message. original: ['reference_'], translated: ['reference_', 'reference_']
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs_inconsistency.txt:8: WARNING: inconsistent references in translated message. original: [], translated: ['`I18N WITH REFS INCONSISTENCY`_']
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/warnings.txt:4:<translated>:1: WARNING: Inline literal start-string without end-string.
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/warnings.txt:4:<translated>:1: WARNING: Inline literal start-string without end-string.
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/admonitions.txt: WARNING: document isn't included in any toctree
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/label_target.txt: WARNING: document isn't included in any toctree
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs_python_domain.txt: WARNING: document isn't included in any toctree
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/rubric.txt: WARNING: document isn't included in any toctree
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/toctree.txt: WARNING: document isn't included in any toctree
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs_inconsistency.txt:11: WARNING: Citation [ref2] is not referenced.
python3.9-sphinx> /build/pytest-of-nixbld/pytest-0/test_additional_targets_should_be_translated/refs_inconsistency.txt:6: WARNING: citation not found: ref3

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: python3Packages.sphinx

Bisected down to
commit 8e1914d43f72ef44cc4d63f662ec4ce589a784ba
Author: toonn<EMAIL_ADDRESS>Date:   Tue Sep 28 14:49:39 2021 +0200

    sphinx: Update FOD source hash

    The hash has changed because of the renamed file but will no longer vary
    depending on whether the filesystem where it is calculated is
    case-sensitive or not.

 pkgs/development/python-modules/sphinx/default.nix | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

@toonn, maybe you know what is happening here? Could it be that test never worked on linux?

Effectively reverting the rename patch seems to fix the test:
--- a/pkgs/development/python-modules/sphinx/default.nix
+++ b/pkgs/development/python-modules/sphinx/default.nix
@@ -36,13 +36,7 @@ buildPythonPackage rec {
     owner = "sphinx-doc";
     repo = pname;
     rev = "v${version}";
-    sha256 = "129x3kljxq7sbg5cc12z3cmjsili6l3qrsxllkx97id1a85f7zmj";
-    extraPostFetch = ''
-      cd $out
-      mv tests/roots/test-images/testimäge.png \
-        tests/roots/test-images/testimæge.png
-      patch -p1 < ${./0001-test-images-Use-normalization-equivalent-character.patch}
-    '';
+    sha256 = "sha256-3YRn+qKMxo8eth8MF+snehY3sji0pR0rl8xH3FexaMQ=";
   };

   propagatedBuildInputs = [

AFAIK the extraPostFetch is needed to prevent sha256 mismatches on darwin hydra builders whose filesystems perform unicode normalization. We should try to make that test work with the renaming if possible, or maybe just skip it.

I'll look into this today. I'm not sure why the rename affects the test. We could go with a different fix for x86_64-darwin if necessary.

I'm looking into this but running into other Python issues, so I figured I'd give some explanation of the diff so someone else could look into this from the Linux side maybe.
The Sphinx source contains a test file named tests/roots/test-images/testimäge.png. The default file system for Darwin used to be HFS+, I think this changed with Big Sur might've been earlier. The x86_64-darwin Hydra builders still run HFS+. The reason the file system matters is that it is case insensitive, not relevant right now but causes similar issues, and it is normalizing, using normal form D. On Linux the custom is to use normal form C even though file systems don't tend to be normalizing. This means the ä in this file name is represented using NFC but when the tarball is unpacked and written to disk it is normalized to NFD. Then when Nix computes the hash to check the FOD it doesn't match. The only way I could think of to solve this was to change the file name so it is unaffected by normalization. The replacement character needs to be unicode because the original intent is for it to have characters outside of the ASCII range. The test using this file can be fixed for Darwin with some calls to Python's unicode normalize() function (but that's a terrible patch because it can hide real issues). However, the problem would remain that the FOD hash does not match when built from a system with a normalizing file system. So the alternative would be to make the FOD hash conditional on the platform (as the closest metric linked to the normalizing properties of the file system) but that would still double the amount of caching Hydra has to do for almost no reason.

Even making the FOD hash conditional on platform isn't sufficient because most modern x86_64-darwin systems use APFS, which behaves like typical Linux filesystems w.r.t. unicode normalization.
I feel like the simplest solution may be to switch the hydra builders over to APFS

That's not really a solution for all the people who're running a macOS install that has been upgraded a couple times from when HFS+ was still the default though.
I think modifying the source with extraPostFetch and patching everything that depends on the file names is the most thorough solution.
Looks like it's not a problem on staging-next anymore though, can't reproduce it locally either.

Let\s close as obsolete.


Cloning of repo

The command to recursively clone the "cFp_HelloKale" repo in section 3.1.3 of the docu fails

there is a second "-" missing in front of the "recursive" statement
ssh-based cloning only works if the person has his/her ssh-key in the account
http-based cloning works without ssh-keys, but then the two repos have to be downloaded independently because the "recursive" option fails

Fixed the missing "-". (Expecting more problems with this MD file which needs to be updated one more time).
ssh-based cloning: correct, you need to associate an SSH key to your GitHub account (https://docs.github.com/en/authentication/connecting-to-github-with-ssh)

Please add a comment to this end (need a github account and ssh-key)

ABOUT CLONING: cFp_helloKale and its README.md are now configured to allow HTTPS-based cloning.


sqlite returner for testing

I've got a whole test setup planned out that will use this to verify (eg) FIM is working as intended. Whether the test setup gets used or not, this might still be useful to someone not using splunk.

Where are we on this? If it needs additional changes, I'm not sure what they would be.

Looks like we got some travis and other testing files in here. Are those ready to be merged or should we move them to another branch?
 I'll clean that up for sure.

OK... I took the "gate" off json (it's got to be there, c'mon) and moved the sqlite3 gate to the top. Then rebased.


[NDTensors] Introduce GradedUnitRangeDual

This PR aims to simplify the use of GradedAxes.dual, especially to be used in a BlockSparseArray. It proposes to remove the UnitRangeDual type:

a generic AbstractUnitRange is now self dual
a labelled GradedUnitRange or GradedOneTo are cast to GradedUnitRangeDual{<:LabelledInteger,<:AbstractGradedUnitRange} <: AbstractGradedUnitRange

This should simplify slicing a BlockSparseArray with dual axes and allows to preserve labels or take their dual.
This branch is similar to #1529 but is based on main.
Currently, this branche solves:

preserving labels in blocklengths(dual(...))

g = gradedrange([U1(1) => 2, U1(2) => 2])
@test GradedAxes.blocklengths(dual(g)) isa Vector{<:LabelledInteger}  #    # broken in main and in non-abelian_fusion

preserving GradedUnitRange behavior when slicing:

blocklengths(dual(gradedrange([U1(0) => 2, U1(1) => 2])[1:3])) isa Vector  # broken in main and in non-abelian_fusion

manipulating arrays with DualGradedUnitRange

r = gradedrange([U1(0) => 2, U1(1) => 2])[1:3]
a = BlockSparseArray{Float64}(r, r)
b = BlockSparseArray{Float64}(dual(r),dual(r))
@test a[:, :] isa BlockSparseArray   # broken in main and in non-abelian_fusion
@test b[:, :] isa BlockSparseArray   # broken in main and in non-abelian_fusion

Meaning one can create and manipulate BlockSparseArray with GradedUnitRange or its dual (not just BlockedOneTo). However axes(a[:,:]) are still Base.IdentityUnitRange(GradedUnitRangeDual{...}), which creates other issues.
It also adds many tests, including broken ones.

[ ] The current plan is to have multiple dual unit range types, such as UnitRangeDual <: UnitRange, GradedUnitRangeDual <: AbstractGradedUnitRange, and maybe BlockedUnitRangeDual <: AbstractBlockedUnitRange. In that case, it would be useful to define an IsDual trait based on the isdual function (say using SimpleTraits.jl). See https://github.com/ITensor/ITensors.jl/pull/1363/files#r1779660676 and https://github.com/ITensor/ITensors.jl/pull/1363/files#r1779708817 for examples of where that would be useful, say for generalizing blocklabels(a::UnitRangeDual) and flip_dual(r::UnitRangeDual).

Overall I still find there are too many kind of UnitRange. I wonder if we could avoid the doublet GradedOneTo and GradedUnitRange. gradedrange creates a GradedOneTo, but when slicing it one gets a GradedUnitRange.

For dispatch it is helpful to have special ranges that start at one since that is used as axes by most arrays, like how Base has UnitRange and Base.OneTo. Handling both shouldn't be too difficult if we write good generic code.

To be more specific, if we write most code in terms of AbstractGradedUnitRange and AbstractUnitRange, that should allow us to have a lot of subtypes (such as GradedOneTo, GradedUnitRage, GradedUnitRangeDual, etc.) without adding too much code complexity. Though perhaps one question is, is there a need for a GradedOneToDual? That does start to sound excessive and it is best to try to design it without that for now.
One thing to consider is that dual only really makes sense for graded unit ranges, so maybe we could consider dropping UnitRangeDual, and just define dual(r::AbstractUnitRange) = r as a generic fallback (i.e. if the unit range isn't graded, the dual is itself).

I have no strong opinion whether there should be a UnitRangeDual type or not. This is mostly a design question for BlockSparseArray, about how much complexity you want there. In the specific case of FusionTensor, I will always use a graded axis. My objective here is to make dual(GradedUnitRange) an AbstractGradedUnitRange to deal with labels in a generic way.
However I believe dual and not dual axes should be handled on equal footing: deciding who is dual and who is not is a pure convention and they should always appear with a 1:1 ratio for contraction to be possible. If there is some gain in having both GradedOneTo and GradedUnitRangeDual, then I think there should be a GradedOneToDual.

I realize that with the current design, there are already GradedUnitRangeDual{<:LabelledInteger, BlockArrays.BlockedOneTo and GradedUnitRangeDual{<:LabelledInteger, BlockArrays.BlockedUnitRange}, so there should be no need for an explicit GradedOneToDual.
I will remove UnitRangeDual and use self-dual axes as the default.

I realize that with the current design, there are already GradedUnitRangeDual{<:LabelledInteger, BlockArrays.BlockedOneTo and GradedUnitRangeDual{<:LabelledInteger, BlockArrays.BlockedUnitRange}, so there should be no need for an explicit GradedOneToDual.
I will remove UnitRangeDual and use self-dual axes as the default.

Sounds good, that's what I was going to suggest (i.e. GradedOneToDual can be a type alias, if needed).

I think this is now ready for GradedAxes. I will focus on BlockSparseArrays in another PR.

@ogauthe I realized that one reason to keep UnitRangeDual is that it may be a reasonable return type when slicing blocks of a GradedUnitRangeDual, i.e. it might make sense for dual(gradedrange([U1(0) => 2, U1(1) => 2]))[Block(1)] to return a UnitRangeDual, to preserve the fact that the block was taken from a dual space.
That could be helpful since then it could make it easier to create a dual graded unit range out of the blocks of another dual graded unit range (say if you have a dual graded unit range and want to remove one of the blocks, it could make it easier to write that kind of code).

Indeed slicing a GradedUnitRangeDual with a Block was an issue I spotted (this is associated with the slicing issues in #1336). Currently it yields a LabelledUnitRange and the dual information is lost.
Now, how generic UnitRangeDual should be?  I guess it should be a subtype of AbstractUnitRange and should be valid axis for a BlockSparseArray. Should we come back to "any AbstractUnitRange can be dualed and become a UnitRangeDual" or should it be more specific, as "obtained from slicing a GradedUnitRangeDual"?
I favor the second as it would make it simpler to use a BlockSparseArray without caring about mathematical details, but I think both options carry the same amount of code complexity (mostly coming from being valid BlockSparseArray axes).

I agree with your assessment that UnitRangeDual should be a subtype of AbstractUnitRange, and also that we should be thoughtful about which operations create UnitRangeDual. I'll try to summarize that view:
I think it should be valid to make a BlockSparseArray with any kind of unit range as long as it satisfies the BlockArrays.jl axis interface, and maybe some additional interface functions we require in BlockSparseArrays, so axes may be dual or non-dual axes of whatever kind.
The question to me more comes down to how and when dual axes get constructed, depending on the unit range type and operation. So to be concrete, if you have a block sparse matrix a, in general a' should have axes: axes(a') == dual.(reverse(axes(a))), so that operations like a' * a make sense (i.e. involve a contraction of a dual axis with a non-dual axis).
However, I think it would be surprising and unnecessarily complicated if a started with non-graded axes, say OneTo or BlockOneTo, and a' had dual axes, say UnitRangeDual{<:Any,<:OneTo} or UnitRangeDual{<:Any,<:BlockOneTo}, since they are self-dual. So that leads me to think that we should define dual(r::OneTo) = r and dual(r::BlockOneTo) = r (since adjoint(::BlockSparseMatrix) will be defined in terms of dual to take the dual of the axes where appropriate), and if a user wants to create a BlockSparseArray with dual axes of type UnitRangeDual{<:Any,<:BlockOneTo}, they'll have to do that manually in another way besides the dual(...) function.
We may want to define that dual(r::LabelledUnitRange) outputs a UnitRangeDual(r), though in that case it may be better to rename LabelledUnitRange to something else to indicate that it is more specifically the block of a GradedUnitRange.

This is copied from this comment: https://github.com/ITensor/ITensors.jl/pull/1363#discussion_r1774410848:
It would be nice to think through how functions like:
flip(g::AbstractGradedUnitRange) = dual(gradedrange(label_dual.(blocklengths(g))))

could be written in a simpler way.
One proposal would be to make this work as an implementation:
flip(g::AbstractGradedUnitRange) = gradedrange(flip.(blocklengths(g)))

In order to make that work, we would need to have flip work on sectors, define the concept of dual sectors, and make it so that gradedrange outputs a dual unit range when constructed from dual sectors.
An implementation could be the following:
flip(l::LabelledInteger) = labelled(unlabel(l), flip(label(l)))

struct CategoryDual{NondualCategory<:AbstractCategory} <: AbstractCategory
  nondual_category::NondualCategory
end
flip(c::AbstractCategory) = CategoryDual(dual(c))
nondual(c::AbstractCategory) = c
nondual(c::CategoryDual) = c.nondual_category
dual(c::CategoryDual) = nondual(c)

In general, dual(flip(x)) can be the canonical/generic way of instantiating a dual sector/space/range, and could supersede label_dual (which I never liked but didn't have a better idea for at the time).
Additionally, gradedrange would have to be able to detect that all of the labels are CategoryDual and in that case construct a UnitRangeDual/GradedUnitRangeDual.

@ogauthe probably at this point it makes more sense to define:
abstract type AbstractGradedUnitRange{T,CS} = AbstractBlockedUnitRange{T,CS}

as opposed to making it a type alias. Also I would drop the T<:LabelledInteger constraint for now, and think of it more as an interface requirement, i.e. the elements of an AbstractGradedUnitRange should act like a LabelledInteger (integer element of the range and associated sector), but it might not be a LabelledInteger.

Indeed we would have more freedom with abstract type AbstractGradedUnitRange{T,CS} . Just to be clear, that would be a significant change in the implementation. It seems to me that implies rewriting several parts of gradedunitrange.jl, as we would need to explicitly write code path that were calling/assuming BlockedUnitRange and BlockedOneTo. I am working on it.

Indeed we would have more freedom with abstract type AbstractGradedUnitRange{T,CS} . Just to be clear, that would be a significant change in the implementation. It seems to me that implies rewriting several parts of gradedunitrange.jl, as we would need to explicitly write code path that were calling/assuming BlockedUnitRange and BlockedOneTo. I am working on it.

I see, I wasn't sure if that would affect other parts of the code, but I think it is the right design so hopefully those changes are not too involved.

Here is a first draft passing test_basics.jl.
Concerning dual, if I remember correctly, the rational behind UnitRangeDual was precisely to have GradedUnitRange = BlockedUnitRange{LabelledInteger}, at a time when BlockArrays did not define AbstractBlockedUnitRange.
Now we have custom GradedOneTo and GradedUnitRange, which are AbstractBlockedUnitRange with extra logic for label manipulation. Therefore I believe the best for dual would be to have a isdual::Bool field inside them, preserved over slicing. That would add a few more lines to the label logic, but would completely avoid to introduce a new type GradedUnitRangeDual. I think that would considerably simplify BlockSparseArrays, where many issues comes from UnitRangeDual being too complex.
We may however still need a UnitRangeDual<:AbstractUnitRange for non-blocked ranges. Or maybe a DualCategory to be used only for that purpose, although I am worried introducing DualCategory would cause other issues.

Concerning dual, if I remember correctly, the rational behind UnitRangeDual was precisely to have GradedUnitRange = BlockedUnitRange{LabelledInteger}, at a time when BlockArrays did not define AbstractBlockedUnitRange. Now we have custom GradedOneTo and GradedUnitRange, which are AbstractBlockedUnitRange with extra logic for label manipulation. Therefore I believe the best for dual would be to have a isdual::Bool field inside them, preserved over slicing. That would add a few more lines to the label logic, but would completely avoid to introduce a new type GradedUnitRangeDual. I think that would considerably simplify BlockSparseArrays, where many issues comes from UnitRangeDual being too complex.

I'm open to considering switching to an isdual flag if there is a convincing argument for it or get convinced that the current design really isn't working, but for the time being why don't we do the best we can improving on the current design and see how it works out. I started designing GradedAxes with an isdual flag but I felt like it got very awkward having to check that flag at runtime within a lot of functions. I thought that putting it at the type level would allow us to separate out the abstraction layers, so one layer of code doesn't have to worry too much about duals, and then duals can be handled in a separate layer of code using dispatch (which I think is working out pretty well as shown in this PR but I agree there seems to be a lot of code duplication). I think it is working out ok and I think that we can probably simplify the code once the dust settles a bit on all the PRs and improvements we need to make to BlockSparseArray, but of course we haven't done that yet so we'll have to see how it works out.
Something I think could help with having multiple dual unit range types is having an IsDual trait and using trait dispatch to define common functionality on the various dual types, but again that is speculation right now and we would have to try it out and see if that helps.

We may however still need a UnitRangeDual<:AbstractUnitRange for non-blocked ranges. Or maybe a DualCategory to be used only for that purpose, although I am worried introducing DualCategory would cause other issues.

One reason we may still want to have UnitRangeDual is because I think that should be the output of slicing operations like r[Block(2)] when r::GradedUnitRangeDual.

GradedAxes and SymmetrySectors  tests are passing. I am now working on issues with BlockSparseArrays. I spotted several cases where the labels are dropped:
d1 = gradedrange([U1(0) => 2, U1(1) => 2])
d2 = gradedrange([U1(0) => 2, U1(1) => 2])
a = BlockSparseArray{Float64}(d1, d2, d1, d2);

@test axes(2 * a, 1) isa AbstractGradedUnitRange
@test_broken axes(a + a, 1) isa AbstractGradedUnitRange
@test axes(a + a, 1) isa BlockedUnitRange{Int64, Vector{Int64}}

b = a[2:3, 2:3, 2:3, 2:3];
@test_broken axes(b, 1) isa AbstractGradedUnitRange
@test axes(b, 1) isa BlockedOneTo{Int64, Vector{Int64}}

bv = BlockVector([Block(1), Block(2)], [1, 1])
b = a[bv, bv, bv, bv];
@test_broken axes(b, 1) isa AbstractGradedUnitRange
@test axes(b, 1) isa BlockedOneTo{Int64, Vector{Int64}}

It is not clear to me where the problem lies as slicing the axes is fine. Also, blockedrange([labelled(2,U1(0)), labelled(2,U1(1))]) still returns a GradedOneTo so we can still use generic code. @mtfishman do you have an idea?

I don't have a good guess without digging into the code... I had to track down a lot of issues like that before, usually it is just a missing overload.

Indeed the problems come from missing overload for BlockArrays.combine_blockaxes. Fixing it in a generic way should solve may related issues in #1336 .

Indeed the problems come from missing overload for BlockArrays.combine_blockaxes. Fixing it in a generic way should solve many related issues in #1336 .

Great! Glad to hear you tracked it down, that makes sense since it is used in block sparse array broadcasting to make the blocks commensurate, and scalar multiplication, addition, and some slicing operations ultimately call broadcasting.

The other problems where due to the change blockedunitrange_getindices/gradedunitrange_getindices. Maybe I should avoid defining gradedunitrange_getindices and just specialize blockedunitrange_getindices for AbstractGradedUnitRange.

The other problems where due to the change blockedunitrange_getindices/gradedunitrange_getindices. Maybe I should avoid defining gradedunitrange_getindices and just specialize blockedunitrange_getindices for AbstractGradedUnitRange.

That seems simpler, for context the primary purpose of blockedunitrange_getindices was to circumvent issues in BlockArrays.jl.

@ogauthe this PR looks good to me, given the current type hierarchy that we are aiming for right now. I definitely get your point about there being a lot of code that needs to be defined to make all of the different unit range types work. I think there are a few potential solutions, some of which are more benign like reassessing what code can be made more generic and can get shared across unit range types or even simplified by making upstream changes to BlockArrays.jl, and some more extreme like making an isdual flag or moving the information about duality into the sectors. As I said let's stick with this design for now and see how far it takes us and reassess once more of the FusionTensors and BlockSparseArrays code is fleshed out.
From what I can tell, at least with the current code design and organization a lot of the complexity for dealing with the different unit range types is living in GradedAxes while BlockSparseArrays and FusionTensors can remain relatively agnostic about those different types, so that should make it easier to try out different designs within GradedAxes in the future without affecting those libraries too much.
Regarding slicing GradedUnitRangeDual, do you plan to fix that in this PR or in a follow-up? I'm fine either way, though this PR has gotten pretty long already so maybe it would be nicer to merge this and work on that in a future PR, for the sake of making reviewing easier.

Thanks for the review. I already started working on a slicing GradedUnitRangeDual, but this is non trivial.
I am pretty happy that many issues appearing in #1336 actually come from GradedAxes and could be fixed here. I added many tests such that test failure would appear clearly in GradedAxes test suit instead of buried deep below a BlockSparseArrays error. So I agree it would be best to merge this PR as it is now.

Oops, it looks like https://github.com/ITensor/ITensors.jl/pull/1562 and https://github.com/ITensor/ITensors.jl/pull/1563 caused test failures, I'll revert those. They passed separately but together they allowed an upgrade to Functors.jl v0.5 which breaks something in NDTensors.jl.

All tests are passing locally. The failures come from unchanged UnallocatedArrays, I suppose they are not related to this PR.

I am pretty happy that many issues appearing in #1336 actually come from GradedAxes and could be fixed here. I added many tests such that test failure would appear clearly in GradedAxes test suit instead of buried deep below a BlockSparseArrays error. So I agree it would be best to merge this PR as it is now.

That's a good sign about the design, my goal with designing BlockSparseArrays was to make it as generic as possible about the axes (and also is the design goal of BlockArrays.jl) so I'm glad to see that seems to be working out so far. Those tests will be helpful, thanks!

@ogauthe looks like the test failures are fixed now, is this ready to merge once the rest of the tests pass?

@ogauthe looks like the test failures are fixed now, is this ready to merge once the rest of the tests pass?

yes it is

Thanks for the PR @ogauthe. Can you check off which issues listed in #1336 that this fixes?


fix dark mode

Compare https://envelop-git-fix-dark-mode-theguild.vercel.app/docs/getting-started
to https://www.envelop.dev/docs/getting-started
Of course you can later change the colors to your preference

@dotansimha any feedback or the reasoning of ignoring this?

@dotansimha any feedback or the reasoning of ignoring this?

Closed automatically because I removed the other branch. I merged website branch to have it available in main ( for vercel deployment...)


(docs): add clarification to the Stack doc details/examples

Subject
Stack
Description
There are a couple of sections in the Stack Component Docs showing arguably redundant, incorrect, and/or confusing information.

Section "Stack Items Horizontally":

The details talk about using the direction prop, but the example uses the HStack component. This might also be a redundant section as it is currently being presented.
Possible consideration would be to remove those contradicting details, and redoing the structure of the page so that this example is shown as part of a group with the previous example under a new header like Use Case Examples

### Use Case Examples
  #### Feature Cards with Stack Component
  #### Feature Cards with HStack Component

Section "Notes on Stack vs Flex":

The details here are not accurate and seem to be unfinished in providing a quality comparison. Combining Stack and Flex (I guess this means <Stack as={Flex}> or vice versa) does not allow you to alter spacing of any one child component -- via margin, say -- as that change will still be overwritten by the styling from Stack. You can use the Spacer component though to add spacing to the value provided by Stack.
justifyContent is a valid prop for both components so it's not clear as to the purpose in including it in the explanation.
Possible considerations are to remove this detail, and either referencing the Flex section "Flex and Spacer vs Grid vs Stack", creating a more clear note in the Stack doc, or provide more clarity in both sections in the Stack and Flex docs to better articulate the differences.

Some other areas of this page may or may not provide enough or clear information, but the points above stuck out the most. I appreciate any thoughts to my observations. 😄

It makes sense, @TylerAPfledderer. I think it's a good idea to refresh the content on the Stack component. I've marked this issue as a "Good first issue" so anyone can participate in fixing it.

@nikolovlazar can I take this? 🙌


Supporting sparklyr in Databricks clusters

What changes are proposed in this pull request?
This patch adds support for Databricks Spark clusters by adding a method type to spark_connect. When method = 'databricks', there is no need for any additional arguments to spark_connect.
Implementation details
The new internal databricks_connection() function, does two main things:

Makes an internal Databricks call to start the sparklyr backend on the currently attached cluster. This call passes the current location of sparklyr JARs to Databricks backend. The backend will perform the work on searching and find the right JAR and will call its main method.
Uses the gateway port returned from step 1 to establish gateway and backend connections

We think sparklyr is going to be an active project with multiple releases in coming months. As a result, we think hard coding the JARs in Databricks spark images will tie users to older versions of the project. We will continue using the JARs that are installed by the R package.
In Databricks notebooks and jobs, the JVM and Spark Context are managed by Databricks. Therefore, applications need to use the existing context and using spark-submit will not work. Also, in Databricks notebooks SparkR connection is already established and provides a mechanism for calling into the JVM. We are using SparkR to bootstrap the sparklyr connection

@falaki LGTM. Thanks for updating the docs, that should fix travis.


Add more PowerShell apps in UnusualGuestActivity.yaml

Fixes #
Unusual activity not detected
Proposed Changes

Add more apps related to PowerShell

Please, could you check more apps, related to PowerShell, that an offensive actor could use?
A red team enumerated our AAD through Azure Active Directory PowerShell with a Guest user. I think it can also be enumerated through Microsoft Azure CLI.
I have seen other apps related to PowerShell commands, like:
Azure Active Directory PowerShell
Microsoft Azure PowerShell
Microsoft Intune PowerShell
Microsoft Exchange REST API Based Powershell
MS Teams Powershell Cmdlets
Microsoft SharePoint Online Management Shell

Maybe the check could be through AppIds, if they are well-known, like:
04b07795-8ddb-461a-bbee-02f9e1bf7b46
1b730954-1685-4b74-9bfd-dac224a7b894
1950a258-227b-4e31-a9cf-717495945fc2

@aprakash13 I have added the AppIds.
I think Office365 Shell WCSS-Client comes from browser activity of Office365 apps.
I wanted to note that, in this detection rule, adding it as condition it does NOT generate noise, for my tenant at least. BUT, for this other detection rule (https://github.com/Azure/Azure-Sentinel/issues/3592) it may generate noise.


ALWAYS SHOWS THE SAME RESULTS

Hi guys
thanks for the wonderful work.
I suggest you review the code, because even if the "likes" increase, it always shows me the same profiles, without updating the new ones.
Thank you!!

I'll look into it, but I don't think there's anything I can do about it.
The script depends on the response Tinder API provides. As the API doesn't return more profiles than what is displayed currently, the script has no data for more profiles to be displayed.

Thank you for fixing this :)

Hi
I think the problem is that the new profile pictures don't appear chronologically. If you have more than 10 "Likes" you might not see a new one, or maybe not in the first position.

Hi
I think the problem is that the new profile pictures don't appear chronologically. If you have more than 10 "Likes" you might not see a new one, or maybe not in the first position.

Exactly

The API returns the same results 10 profiles regardless of the exploit or not. There's unfortunately nothing I can do. What changes are only the returned photos which are aparrently chosen by a different metric and thus do not match to their blurry variants.


Some RSS works, some doesn't, please check

As I loaded a lot of RSS into my feeds (around 10 custom rss) but when deliver to my kindle, only 5 show up.
Here is an example:
RSS that works: http://tramdoc.vn/tu-sach-den-doi-cnE5DW.rss
RSS that doesn't work: http://cafebiz.vn/cau-chuyen-kinh-doanh.rss
I don't know what cause this, can you guys help checking.
Thanks in advance.

The cafebiz site has anti-GAE mechanism, the application of GAE can't get the right RSS xml, instead of a mobile page (html).


context: escape the member value

When passing values with spaces or special characters
the otel/baggage will reject it due to the standard
constraints: https://www.w3.org/TR/baggage/#value
Signed-off-by: crozzy<EMAIL_ADDRESS>

Observed:
WRN failed to create baggage member error="invalid value: \"CVE-2021-33574 glibc\""


Run a baseline simulation of free actin filaments, in readdy and cytosim

In order to have a baseline for variance and magnitude of the metrics
we're going to use the time scale conditions for the fastest velocity

We decided to run cytosim simulations with one end attached, to serve as our baseline

ReaDDy baseline trajectories are listed here
Cytosim are listed here


[BUG] Proposal Creation form validation

Context & versions

The Governance Action Proposal creation form currently supports empty values.
Also it treats spaces as filled characters and allows the addition of random addresses and negative amounts.
Please review the relevant tickets and discussions regarding treasury address validation.
https://github.com/IntersectMBO/govtool/issues/1187
https://github.com/IntersectMBO/govtool/issues/1231

For form validation, please refer to this link.

The form currently supports empty values, but it should not. See reference ticket (#1099).

@vpaunovic, I can see the form validation on the development environment, but there are still some inconsistencies:

An empty title is accepted, and whitespace is taken as a valid title.

Actual Behavior:

Expected Behavior:

Instead of blocking the input that exceeds 80 characters, can you show an error message?

Actual Behavior:

Expected Behavior:

Also, don't forget to add the test ID for those error messages.

@vpaunovic
I can confirm that the form validation and test IDs for the error message are in place.
This issue can be closed now. CC: @bosko-m

Closing as per the comment above.


tests: add test that shows inaccuracy between GitHub diff and AST Diff

Line numbers 497 and 494 are marked as breakpoints in this test case. However, this should not have been the case because they are the diff lines according to GitHub diff view.
If one analyzes their AST diff, it turns out that the nodes corresponding to those lines remain as-is in the AST and the other nodes are operated upon.
A photo to remind us of the entire discussion because I am lazy to draw diagrams on the web.

@khaes-kth documented the discuss and a corresponding "inaccurate" test case here.


CI SBAT use integration tests against alfajoresstaging once it's deployed with BLS

Expected Behavior
Integration tests should work against alfajoresstaging once it's re-deployed.
Current Behavior
Integration are disabled against alfajoresstaging.

@kobigurk can you remind me which tests you mean?

Fixed by https://github.com/celo-org/celo-monorepo/pull/617 already, my bad.


[SPARK-17243] [Web UI] Spark 2.0 History Server won't load with very large application history

What changes were proposed in this pull request?
back port of #14835 addressing merge conflicts
With the new History Server the summary page loads the application list via the the REST API, this makes it very slow to impossible to load with large (10K+) application history. This pr fixes this by adding the spark.history.ui.maxApplications conf to limit the number of applications the History Server displays. This is accomplished using a new optional limit param for the applications api. (Note this only applies to what the summary page displays, all the Application UI's are still accessible if the user knows the App ID and goes to the Application UI directly.)
I've also added a new test for the limit param in HistoryServerSuite.scala
How was this patch tested?
Manual testing and dev/run-tests

@tgravescs here's the 2.0.1 back port

Test build #64675 has started for PR 14886 at commit 22ddf01.

Jenkins, retest this please

Test build #64675 has finished for PR 14886 at commit 22ddf01.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64675/
Test FAILed.

Jenkins, please retest this

Jenkins, retest this please

Test build #64678 has started for PR 14886 at commit 22ddf01.

Test build #64678 has finished for PR 14886 at commit 22ddf01.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64678/
Test PASSed.

Merged build finished. Test PASSed.

+1


python311Packages.fiona: 1.9.5 -> 1.9.6

Description of changes
Changelog: https://github.com/Toblerity/Fiona/blob/refs/tags/1.9.6/CHANGES.txt
cc @imincik @nh2 @nialov @sikmir @willcohen
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Please remove setuptools from nativeBuildInputs.


Platform Capture not enabled after sleep

Hi, thank you for this plugin!
I'm running Hyperion.NG 2.0.0-alpha9 on a Vero 4K+, controlling an WLED instance.
If have ticked the boxes to enable/disable Hyperion, enabled Ambilight for everything except screensaver (because that's what Kodi does on the Vero 4K+, there is no shutdown), and picked "Platform Capture" as the "Component" - so that it's basically always on unless the screensaver is activated (which could be because Kodi is going to sleep).
When sending the Vero 4K+ to sleep, the "Platform Capture" source gets removed from the "Remote Control" -> "Source Selection" table and the LEDs go off, but the the  "Capturing Hardware" -> "Instance Capture" -> "Enable platform capture" checkbox is still ticked.
When waking the Vero 4K+ from sleep, the "Platform Capture" source does not get added to "Remote Control" -> "Source Selection" again, and the LEDs stay off.
I can bring the LEDs back by disabling "Capturing Hardware" -> "Instance Capture" -> "Enable platform capture" , saving, enabling "Capturing Hardware" -> "Instance Capture" -> "Enable platform capture", saving again - but that's a bit annoying ;-)
This is the debug log:
2020-12-19 04:50:38.669 T:3398426848  NOTICE: [script.service.hyperion-control] Settings updated!
2020-12-19 04:50:38.670 T:3398426848  NOTICE: [script.service.hyperion-control] Hyperion ip:           <IP_ADDRESS>
2020-12-19 04:50:38.670 T:3398426848  NOTICE: [script.service.hyperion-control] Hyperion port:         8090
2020-12-19 04:50:38.670 T:3398426848  NOTICE: [script.service.hyperion-control] Enable H on start:     True
2020-12-19 04:50:38.670 T:3398426848  NOTICE: [script.service.hyperion-control] Disable H on stop:     True
2020-12-19 04:50:38.670 T:3398426848  NOTICE: [script.service.hyperion-control] VideoMode enabled:     False
2020-12-19 04:50:38.671 T:3398426848  NOTICE: [script.service.hyperion-control] Hyperion target comp:  GRABBER
2020-12-19 04:50:38.671 T:3398426848  NOTICE: [script.service.hyperion-control] Screensaver enabled:   False
2020-12-19 04:50:38.671 T:3398426848  NOTICE: [script.service.hyperion-control] Video enabled:         True
2020-12-19 04:50:38.671 T:3398426848  NOTICE: [script.service.hyperion-control] Audio enabled:         True
2020-12-19 04:50:38.671 T:3398426848  NOTICE: [script.service.hyperion-control] Pause enabled:         True
2020-12-19 04:50:38.672 T:3398426848  NOTICE: [script.service.hyperion-control] Menu enabled:          True
2020-12-19 04:50:38.672 T:3398426848  NOTICE: [script.service.hyperion-control] Debug enabled:         True
2020-12-19 04:50:38.672 T:3398426848  NOTICE: [script.service.hyperion-control] ChangelogOnUpdate:     True
2020-12-19 04:50:38.672 T:3398426848  NOTICE: [script.service.hyperion-control] tasks:                 0

I also tested with the updated utils.py (COMP_GRABBER), but then it doesn't work at all.

Another thing that may be unintentional: Disabling the plugin itself also disables the instance in Hyperion. I guess that's a bit overreaching? ;-)


RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation:

While training the model, getting following error, can anybody help me here?

I try to change nn.ReLU to nn.ReLU6. I have already solved this problem.


sbtools_DELETE returns session handle

Seems like this isn't the desired output?
Perhaps return the output from the DELETE() call?

Good point. Maybe we could have it check the return value and give TRUE/FALSE for success/failure. Seems to be the approach other delete functions take in various programming languages.

Right, sometimes they return nothing to the user, but that doesn't seem very user friendly

should I do that?

Yeah. TRUE/FALSE is go.
Also, just realized, I think sbtools_PUT and _DELETE should be returning r, the response, not an invisible session object (from session_age_reset). Right?

Yeah, I think so

appears to have been taken care of in 83b85528eed521cb28d49bb17bf0cdb30dd806a9

oops - didn't make that connection. sorry, @sckott!

no problem!


"show full-configuration" and "show" commands return timeout exceptions

Hardware: FortiGate 40C
This code is returning a timeout since it take a long time to retrieve all the configuration :
>>> from pyFG import FortiOS, FortiConfig
>>> d = FortiOS(hostname="<IP_ADDRESS>", username="admin", password="***********")
>>> d.open()
>>> d.execute_command('show full-configuration')
Executing commands:
 show full-configuration
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pyFG/fortios.py", line 152, in execute_command
    for e in error_chan.read():
  File "/usr/local/lib/python2.7/dist-packages/paramiko/file.py", line 192, in read
    new_data = self._read(self._DEFAULT_BUFSIZE)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 1305, in _read
    return self.channel.recv_stderr(size)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 715, in recv_stderr
    raise socket.timeout()
socket.timeout

In my tests (local network) it takes about :

7 minutes to retrieve show full-configuration command result,
10/15 seconds to retrieve show command result.

Hi there!
Just want to acknowledge your reported issue.
Two questions:

How big is the output of show vs show full-configuration ? (amount of lines)
I looked at your PR - just curious how you came to the value of 30 seconds? Was that a sweet spot?

Thanks!
/Stefan

Hi!

I tested with a firewall with about 3400 lines but other clients are between 3000/4000 lines for show full-configuration command. With show command I get about 3200 lines.
I've seen a timeout value of 60 seconds by default when the connection request is made. I think that there is a little chance of connection lost between the connection and the first SSH command but tell me if I'm wrong.

PS: I'm not on local network but inside a VPN. Maybe the 30 seconds make sense when the connection is slow.

Yup, it seems that large configs are truncated.
The full traceback is:   File "/tmp/ansible_fHSN8_/ansible_module_fortios_config.py", line 132, in main     f.load_config(path=module.params['filter'])   File "/backups/scripts/pyfg-0.50/pyFG/fortios.py", line 216, in load_config     self.running_config.parse_config_output(config_text)   File "/backups/scripts/pyfg-0.50/pyFG/forticonfig.py", line 378, in parse_config_output     current_block = current_block.get_parent()
The fact that it seems to be missing a parent for me, implies it truncated/lost lines.
Possibly paramiko's default SSH recv buffer?

I am getting the same thing on firewall with  minimal config running 5.4.5.

from pyFG import FortiOS, FortiConfig
d = FortiOS(hostname="<IP_ADDRESS>", username="ansible", password="************")
d.open()
d.execute_command('show full-configuration')
Traceback (most recent call last):
File "", line 1, in
File "/home/brandon/.local/lib/python2.7/site-packages/pyFG/fortios.py", line 151, in execute_command
for e in error_chan.read():
File "/home/brandon/.local/lib/python2.7/site-packages/paramiko/file.py", line 193, in read
new_data = self._read(self._DEFAULT_BUFSIZE)
File "/home/brandon/.local/lib/python2.7/site-packages/paramiko/channel.py", line 1342, in _read
return self.channel.recv_stderr(size)
File "/home/brandon/.local/lib/python2.7/site-packages/paramiko/channel.py", line 733, in recv_stderr
raise socket.timeout()
socket.timeout

Storing the full configuration does work but took about 8 minutes.
Make sure you have "config system console", "set output standard" set.
a.open()
a.load_config('full')

Increasing the timer from 5s to 30s fixes the same error I hit when I try to run "show" on a firewall from a server on another continent. Apparently network condition (e.g.  network latency) is a contributing factor.