Added Prettier.io code formatter

This PR adds Prettier, changes ESLint config to just the functional rules, and removes stylelint. Prettier supports auto-formatting MD, CSS, SCSS, JS and JSON.
This PR and Prettier come with a defined, opinionated codestyle. Adopting this standard style means ranking the importance of code clarity and community conventions higher than personal style. The whole point of this PR is to save you time by avoiding bikeshedding about code style. There are lots of debates online about tabs vs. spaces, etc. that will never be resolved. These debates just distract from getting stuff done. At the end of the day you have to 'just pick something', and that's what this PR does: its a bunch of sensible 'just pick something' opinions. Hopefully, users see the value in that over defending their own opinions. There are actual real problems that you could spend your time solving!
Two new npm scripts are made available in this PR:

npm run codestyle: This script automatically formats all supported languages with Prettier
npm run codestyle-is-valid: This script checks if the code style passes, and exits with a non-zero code if it does not pass.

Prettier allows for some style rules, located in the prettier property in the package.json, however I advice strongly against changing any rules, because that defeats the purpose.
Some related additional changes:

Moved from json to yaml for the ESLint config file: .eslintrc.yml, so comments are supported.
Updated ESLint version (via babel-eslint and gulp-eslint).
Updated githooks: pre-commit now also runs npm run codestyle-is-valid when a match for *.{md,css,scss,js,json} was found, blocking the commit when the code does not pass the style. It also still blocks the commit when linting fails.
Appended .min to the filenames for minified files, so they are ignored by prettier and linting: bootstrap.min.css and highlight.min.js.

I did not squash the 2 commits because otherwise the actual functional changes will get mingled up together with the huge amount of code formatting changes. See commit 423bc3d for just the new features.

Nice job :) I’ve got some feedback about feedback.
When running npm run codestyle-is-valid in a codebase with some bad formatted code, the following is returned:
[…]
npm ERR! Failed at the<EMAIL_ADDRESS>codestyle-is-valid script 'prettier --list-different --ignore-path .eslintignore "**/*.{md,css,scss,js,json}"'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the frontend-bootstrap package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     prettier --list-different --ignore-path .eslintignore "**/*.{md,css,scss,js,json}"
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs frontend-bootstrap
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls frontend-bootstrap
npm ERR! There is likely additional logging output above.
[…]

As you described in your PR, this is the correct output. I end up with a non-zero code. However, it isn’t clear feedback to the user. It seems like something went wrong. When running gulp lint I end up with something like this:
✖ 2579 problems (2562 errors, 17 warnings)

That’s better feedback. Is it possible to let the script output something similar?
I ran npm run codestyle successfully. If I understood correctly, the files that were OK are grey in the output. Files that are corrected are placed white. It isn’t that clear. Maybe it’s better to end with a summary of what the script has done, like with gulp lint.
When running npm run codestyle-is-valid with correct code, I get no positive feedback :(
I see that you’ve changed the indenting format from 4 to 2 spaces. I’m fine with that, but then you should reformat the HTML (njk-files) as well in this PR.

Thanks for the feedback, will update PR. Not sure how far we should go on getting nicer error messages, but it's a good suggestion.
It also turns out there are non-stylistic rules in Stylelint, namely Possible errors and Limit language features.
I will re-add stylelint to the PR as well with just those rules, in line with ESLint.

@peeke I still think an indentation will improve the readability because it's easier to spot on which line you are. But I'm not sure if it has anything to do with dyslexia. Therefore I tested the line-height, font-size and font-family like @branneman also suggested and that helps a lot!
The plugin system for the Nunjucks part is a nice one, but I could settle with the code reformat shortcut in Webstorm for the time being.
Therefore, if the merge conflicts are fixed and the new commit is working correctly, you have my approval.

I did some more research on the comments about the eslint config file I did earlier. Extending the config is indeed a good way to go. I suggest adding the code below to the .eslintrc file just right after parser: babel-eslint.
extends:
  - eslint:recommended
  - prettier

In order to get this to work the following package needs to be added to the project: https://github.com/prettier/eslint-config-prettier.
Also all the rules within the .eslintrc file need to be checked again since lots of them will be redundant. In this list https://eslint.org/docs/rules/ rules that are defined within eslint:recommended are marked with a check mark.

Options for prettier are defined in the package.json. Wouldn't it be better to use the .prettierrc file, as in their docs?

Options for prettier are defined in the package.json. Wouldn't it be better to use the .prettierrc file, as in their docs?

@peeke: Because I wanted to skip adding another dotfile. If it were up to me, more setting files can go into the package.json since most of the tools we use support it anyway.
Prettier uses the same dependency for settings loading as the rest of them iirc, so it should be in their docs.

Ah, missed that. That's actually nice.
I do have a question though. I've pasted my .prettierrc from another project and expected it to work. I didn't have the prettier field in my package.json yet. I would've expected it to detect the .prettierrc in that case. Do we have codestyle task hardcoded to look for options in the package.json?

@peeke Yes, it's hardcoded within codestyle.js. I also expected it not to be.

I just updated this PR based on the comments I made earlier.

Fixed some linting errors
Fixed wrong config reference to ESlint ignorePath
Added Prettier rulesets for both ESlint as Stylelint and updated existing ruleset (see below)

ESlint
We start with the recommended ruleset from ESlint itself. This ruleset is extended with a ruleset that is optimized for Prettier. Finally we add/override rules with our own config.
Stylelint
We now start with a ruleset optimized for Prettier and then we add/override this set with our own config.

The .babelrc file isn't updated with the new indentation yet. Otherwise approved


ci: Fix Chrome tests on Windows in GitHub Actions

GitHub started installing "Chrome for Testing" builds on Windows and Mac.  Unfortunately, these builds don't have support for H264 and AAC codecs, which we need in our tests.
This fixes the issue by explicitly installing an official build of Chrome on Windows, to overwrite the "Chrome for Testing" build.

I filed https://bugs.chromium.org/p/chromium/issues/detail?id=1476634 to let Chrome know they are missing codecs in these builds.


[WIP] Add a top query source location to the output of each test run.

Perhaps this should be skipped or altered for local vs. CI runs.
For me, it was really useful to see what locations in our application
were hitting the database.  In the case of refactoring and removing a
large shared setup in the miq_report_spec, having visibility helped to
confirm that we were creating too many vm objects among other objects.
Below is an example output for this miq_report_spec before #5448.
The top offenders dealt with tagging vms and confirmed that dropping
the vm count would drop other related queries.
................

Finished in 11.82 seconds
16 examples, 0 failures

Randomized with seed 6225

Top query locations:
2420 .../manageiq/spec/models/miq_report_spec.rb:241:in `block (5 levels) in <top (required)>'
2000 .../manageiq/spec/models/miq_report_spec.rb:238:in `block (5 levels) in <top (required)>'
1015 .../manageiq/app/models/user.rb:66:in `find_by_userid'
991 .../manageiq/app/models/tenant.rb:230:in `root_tenant'
112 .../manageiq/app/models/vm_or_template.rb:1369:in `host_name'
100 .../manageiq/app/models/mixins/relationship_mixin.rb:114:in `parent_rels'
97 .../manageiq/spec/models/miq_report_spec.rb:72:in `block (3 levels) in <top (required)>'
75 .../manageiq/app/models/mixins/tenancy_mixin.rb:24:in `set_tenant'
60 .../manageiq/spec/models/miq_report_spec.rb:71:in `block (3 levels) in <top (required)>'
25 .../manageiq/spec/models/miq_report_spec.rb:248:in `block (5 levels) in <top (required)>'
21 .../manageiq/app/models/mixins/compliance_mixin.rb:18:in `last_compliance'
20 .../manageiq/spec/models/miq_report_spec.rb:223:in `block (4 levels) in <top (required)>'
20 .../manageiq/spec/models/miq_report_spec.rb:220:in `block (4 levels) in <top (required)>'
20 .../manageiq/spec/models/miq_report_spec.rb:222:in `block (4 levels) in <top (required)>'
20 .../manageiq/spec/models/miq_report_spec.rb:221:in `block (4 levels) in <top (required)>'
15 .../manageiq/app/models/rbac.rb:307:in `get_user_info'
15 .../manageiq/app/models/miq_group.rb:72:in `next_sequence'
13 .../manageiq/app/models/rbac.rb:223:in `find_targets_filtered_by_ids'
13 .../manageiq/app/models/mixins/relationship_mixin.rb:244:in `child_rels'
13 .../manageiq/app/models/rbac.rb:184:in `compute_total_count'

I could see reporting application or spec code or just spec code similar to how rails shows the application and framework callstack on errors.  If this is useful, we can decide where to put this and how to enable it locally vs. CI.

Why not just use the query counter that I suggested in https://gist.github.com/Fryguy/ec432fb7ed07767f8491 ?  I ask because this almost duplicates the code we already have in the QueryCounter without having the filtering.


"End-of-central-directory signature not found", when installing sdkman

I install via terminal:
$ curl -s http://get.sdkman.io | bash
... and get the error:
error_install_sdkman.txt

This is a problem with curl not finding your ca certs on your machine. You will probably find that curl will do this for any ssl handshake. This is something that needs to be fixed on your machine.

I'm seeing this too, on a fresh AWS-supported Linux AMI with Tomcat7 and Java7 (ami-796da114).
Any suggestions for where to look?

If you really can't configure curl to pick up CA certs it is now possible
to bypass them altogether, although this is not recommended.
Refer to the configuration section on our site at:
http://sdkman.io/usage.html#config
The config flag you're after is:
sdkman_insecure_ssl=true
On Sun, 3 Jul 2016 at 17:49, trauts2<EMAIL_ADDRESS>wrote:

I'm seeing this too, on a fresh AWS-supported Linux AMI with Tomcat7 and
Java7 (ami-796da114).
Any suggestions for where to look?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/sdkman/sdkman-cli/issues/420#issuecomment-230162847,
or mute the thread
https://github.com/notifications/unsubscribe/AAI0Blh7TZh5o7g3Z1pPz3dI7eh-7ZOCks5qR-gMgaJpZM4ISQBh
.

Note:
sdkman_insecure_ssl=true in the config doesn't work because the install script seemingly doesn't use it.


Create a Screen for the Inspire item

We should create custom Screens to present data based on the project and if there are nested categories
UI is up to you
Cases

[ ] Inspire

[ ] Literature
[ ] Authors

Added to the app


Fixing plot indexing

While working in #2471 I found out that test_plot_incomplete_nodal_selection was not passing, not sure what did I change in that PR (I check I promise).
Hence I'm bringing the fix  I did for that in a separate PR to highlight its importance.
I have also extensively documented the approach so the future developers (me) know what I did.

Needed, so ... LGTM.


calling pointcloud.get_value frequently leaks memory

I am seeing a slow but noticeable increase in memory that I have tracked down to calling get_value on a pointcloud.
I can reproduce this with the depth_sensing tutorial provided as part of the sdk (see below). In this example I call get_value on a point cloud 5000 times to demonstrate the leak.
I am running Windows 10 with CUDA 10, SDK 2.7.1 on Geforce RTX 2080 with the latest pyzed
Please advise if there is a different / better way to retrieve volumetric points from the camera.
In my case I need to retrieve about 500 coordinates for each frame, this is part of an installation that runs 14 hours a day on otherwise memory constricted machine.
import pyzed.camera as zcam
import pyzed.defines as sl
import pyzed.types as tp
import pyzed.core as core
import math
import numpy as np
import sys

def main():
    # Create a PyZEDCamera object
    zed = zcam.PyZEDCamera()

    # Create a PyInitParameters object and set configuration parameters
    init_params = zcam.PyInitParameters()
    init_params.depth_mode = sl.PyDEPTH_MODE.PyDEPTH_MODE_PERFORMANCE  # Use PERFORMANCE depth mode
    init_params.coordinate_units = sl.PyUNIT.PyUNIT_MILLIMETER  # Use milliliter units (for depth measurements)

    # Open the camera
    err = zed.open(init_params)
    if err != tp.PyERROR_CODE.PySUCCESS:
        exit(1)

    # Create and set PyRuntimeParameters after opening the camera
    runtime_parameters = zcam.PyRuntimeParameters()
    runtime_parameters.sensing_mode = sl.PySENSING_MODE.PySENSING_MODE_STANDARD  # Use STANDARD sensing mode

    # Capture 50 images and depth, then stop
    i = 0
    image = core.PyMat()
    depth = core.PyMat()
    point_cloud = core.PyMat()

    #stress to highlight memory leak
    while True:
        # A new image is available if grab() returns PySUCCESS
        if zed.grab(runtime_parameters) == tp.PyERROR_CODE.PySUCCESS:
            # Retrieve left image
            zed.retrieve_image(image, sl.PyVIEW.PyVIEW_LEFT)
            # Retrieve depth map. Depth is aligned on the left image
            zed.retrieve_measure(depth, sl.PyMEASURE.PyMEASURE_DEPTH)
            # Retrieve colored point cloud. Point cloud is aligned on the left image.
            zed.retrieve_measure(point_cloud, sl.PyMEASURE.PyMEASURE_XYZRGBA)

            # Get and print distance value in mm at the center of the image
            # We measure the distance camera - object using Euclidean distance
            x = round(image.get_width() / 2)
            y = round(image.get_height() / 2)

            #stress to highlight memory leak
            for i in range(5000):
                err, point_cloud_value = point_cloud.get_value(x, y)

                distance = math.sqrt(point_cloud_value[0] * point_cloud_value[0] +
                                     point_cloud_value[1] * point_cloud_value[1] +
                                     point_cloud_value[2] * point_cloud_value[2])

                if not np.isnan(distance) and not np.isinf(distance):
                    distance = round(distance)
                    print("Distance to Camera at ({0}, {1}): {2} mm\n".format(x, y, distance))
                    # Increment the loop
                    i = i + 1
                else:
                    print("Can't estimate distance at this position, move the camera\n")
            sys.stdout.flush()

    # Close the camera
    zed.close()

if __name__ == "__main__":
    main()

Hi,
There was indeed a leak in get_value. It has been fixed. Thanks for reporting the issue!
The get_value function is a valid way to retrieve a few values but is quite costly. An alternative would be to read the value using a numpy array for instance. Like this :
np_cloud=point_cloud.get_data()
value=np_cloud[x,y]
distance = math.sqrt(value[0] * value[0] + value[1] * value[1] + value[2] * value[2])

Of course, the numpy array conversion has a cost (but support range operation... and is more flexible)
Here is a sample "benchmarking" it :
import pyzed.camera as zcam
import pyzed.defines as sl
import pyzed.types as tp
import pyzed.core as core
import math
import numpy as np
import sys
import time

def main():

    zed = zcam.PyZEDCamera()
    init_params = zcam.PyInitParameters()
    runtime_parameters = zcam.PyRuntimeParameters()
    point_cloud = core.PyMat()

    err = zed.open(init_params)
    if err != tp.PyERROR_CODE.PySUCCESS:
        exit(1)

    while True:
        if zed.grab(runtime_parameters) == tp.PyERROR_CODE.PySUCCESS:
            zed.retrieve_measure(point_cloud, sl.PyMEASURE.PyMEASURE_XYZRGBA)
            x = round(point_cloud.get_width() / 2)
            y = round(point_cloud.get_height() / 2)

            start = time.time()
            np_cloud=point_cloud.get_data()
            end = time.time()
            print("Numpy conversion %f ms" % ((end - start)*1000))

            start = time.time()
            value=np_cloud[x,y]
            end = time.time()
            print("Numpy get_value %f ms" % ((end - start)*1000))

            start = time.time()
            err, point_cloud_value = point_cloud.get_value(x, y)
            end = time.time()
            print("PyMat get_value %f ms" % ((end - start)*1000))
            print("\n")

    zed.close()

if __name__ == "__main__":
    main()


HTML code preview is not showed correctly

When I input 'Hello' in html code pane without css code, in the preview pane, it just shows Hello, but not h1 font.

Yup, it should do that because there is no CSS. Since Eric Meyer's CSS reset is loaded automatically, it will "normalize" all default browser styles.
It's up to you to define your own typography styles. If you need some quick CSS code to copy/paste, try using https://type-scale.com/. It will generate a heading stack for you.


Support creating C# service applications

Create the FlutterServiceApplication class in Tizen.Flutter.Embedding and refactor for code sharing.
Change the directory structure of templates.

e.g. templates/app/cpp/service-app → templates/service-app/cpp

Add the C# service app template.
Simplify the try-finally block in create.dart.
Remove the "http://tizen.org/metadata/direct-launch" metadata from the C# app template. It was initially added to prevent problems like https://github.com/flutter-tizen/engine/issues/58 but it doesn't seem necessary any more.

Resolved conflicts with https://github.com/flutter-tizen/flutter-tizen/pull/171.


Send event data to command callback, Added integrations tests.

Updated now, to use env variables for each attribute in a product event.


Project Maintenance and Updates

This updates all dependancies except for ora (which can easily at this point). It also updates any packages that has breaking changes in it with updated syntax.

📝 Changed routes:

/404
/_admin

(view full size)
(view full size)

/_error
/about

(view full size)
(view full size)

/faq
/

(view full size)
(view full size)

And 8 other routes:

/api/admin/update
/api/auth/[...nextauth]
/api/stories/[id]
/api/stories
/landing
/new
/story/[id]
/thanks

Commit 882418632ff9197368dcf3e612f2bd67164502fb (https://www-jscsyn8wy-my-covid-story.vercel.app).


Use lib_bagl on nano X and nano S for basic support.

We don't need this anymore, the all-Rust version is working now!


error when using wsinfer outputs -- no matching files

hi @lthealy - i am running the tumor-til analysis pipeline on wsinfer outputs. i'm getting an error that "no predictions had exact pairs".
i have attached a tar file with a small dataset (one slide) to reproduce this error.
data.tar.gz
the dataset has the following folder structure:
data
├── samples.csv
├── tils
│   └── TCGA-3C-AALI-01Z-00-DX1.F6E9A5DF-D8FB-45CF-B4BD-C6B76294C291.csv
└── tumor
    └── TCGA-3C-AALI-01Z-00-DX1.F6E9A5DF-D8FB-45CF-B4BD-C6B76294C291.csv

here is the error:
[1] "TIL Algorithm (Threshold): Frontiers InceptionV4: 0.1"
[1] "=========== Params after R parsing, if any misalignment please check your flags ==========="
$algorithm
[1] "inceptionv4"

$tilDir
[1] "/data/results-tils"

$tilThresh
[1] 0.1

$cancDir

[1] "/data/results-tumor"

$cancThresh
[1] 0.5

$sampFile
[1] ""

$outputFile
[1] "output.csv"

$outputDir
[1] "/data/results-tilalign/"

$writePNG
[1] TRUE

$sampInfo
[1] "/data/sample_info.csv"

 . . . Dropping low_res and color- files . . .
 . . .  All files have pairs . . .
Error: No predictions had exact pairs. Please ensure lymph and cancer pairs have the exact same name.
Execution halted

I think I see it. We have a grep call at line 76 in commandLineAlign.R that only keeps files that start with "prediction" in an effort to drop "color-" files. That returns no entries so everything "matches" because everything is nothing.
WSIinfer never has the prediction/color prefix, so I'll have a flag that if grep(prediction) returns 0 then don't run that trim. Sound good?

Changes made for TIL and Canc sections, shown below for TIL only
Old Code:
tils = tils[grep("^prediction", tils)]
writeLines(" . . . Dropping low_res and color- files . . . ")
if(any(grepl("low_res", tils))){
   tils = tils[-grep("low_res", tils)]
}

New Code:
if(length(grep("^prediction", tils))>0){ ## WSInfer outputs lack prefix, older outputs have prefix.
   tils = tils[grep("^prediction", tils)]
}

writeLines(" . . . Dropping low_res and color- files . . . ")
if(any(grepl("low_res", tils))){
   tils = tils[-grep("low_res", tils)]
}

is there a different path in the code to deal with wsinfer outputs? we would want to take that path if we detect that the files are from wsinfer. there's at least things we can test:

like you say, a lack of prediction- prefix
the presence of .csv suffixes
the presence of a header in the CSV files

there should also be a message printed saying that it has found wsinfer outputs and will use those.
my only worry about assuming that we have wsinfer outputs if there are no files with prediction- prefixes is that if there are no files at all (or maybe the user passed a nested directory), then the error will be confusing.

Yes that'll just require a little shuffling but should be just as straightforward. Currently WSInfer detection is managed after parsing (and really is just a csv suffix check). See lymphFormatCsv object for that detection

Yes, it creates several directories. model-outputs, stitches, patches, and a json file with runtime info.Best,JakubOn Apr 14, 2023, at 12:55, Luke Torre-Healy @.***> wrote:
Question @kaczmarj, does WSInfer spit any log files into the output directory? Something we would have to drop on a glob before running? I dont think so, but wanted to make sure
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

here is a tree of wsinfer outputs. keep in mind that run_metadata_20230225T122426.json includes a timestamp so the actual name will differ across runs.
results-wsinfer
├── masks
│   ├── TCGA-3L-AA1B-01Z-00-DX1.jpg
│   ├── TCGA-4N-A93T-01Z-00-DX1.jpg
│   ├── TCGA-4T-AA8H-01Z-00-DX1.jpg
│   ├── TCGA-5M-AAT4-01Z-00-DX1.jpg
│   ├── TCGA-5M-AAT5-01Z-00-DX1.jpg
│   ├── TCGA-5M-AAT6-01Z-00-DX1.jpg
│   ├── TCGA-5M-AATE-01Z-00-DX1.jpg
│   ├── TCGA-A6-2671-01Z-00-DX1.jpg
│   ├── TCGA-A6-2672-01Z-00-DX1.jpg
│   └── TCGA-A6-2674-01Z-00-DX1.jpg
├── model-outputs
│   ├── TCGA-3L-AA1B-01Z-00-DX1.csv
│   ├── TCGA-4N-A93T-01Z-00-DX1.csv
│   ├── TCGA-4T-AA8H-01Z-00-DX1.csv
│   ├── TCGA-5M-AAT4-01Z-00-DX1.csv
│   ├── TCGA-5M-AAT5-01Z-00-DX1.csv
│   ├── TCGA-5M-AAT6-01Z-00-DX1.csv
│   ├── TCGA-5M-AATE-01Z-00-DX1.csv
│   ├── TCGA-A6-2671-01Z-00-DX1.csv
│   ├── TCGA-A6-2672-01Z-00-DX1.csv
│   └── TCGA-A6-2674-01Z-00-DX1.csv
├── patches
│   ├── TCGA-3L-AA1B-01Z-00-DX1.h5
│   ├── TCGA-4N-A93T-01Z-00-DX1.h5
│   ├── TCGA-4T-AA8H-01Z-00-DX1.h5
│   ├── TCGA-5M-AAT4-01Z-00-DX1.h5
│   ├── TCGA-5M-AAT5-01Z-00-DX1.h5
│   ├── TCGA-5M-AAT6-01Z-00-DX1.h5
│   ├── TCGA-5M-AATE-01Z-00-DX1.h5
│   ├── TCGA-A6-2671-01Z-00-DX1.h5
│   ├── TCGA-A6-2672-01Z-00-DX1.h5
│   └── TCGA-A6-2674-01Z-00-DX1.h5
├── process_list_autogen.csv
├── run_metadata_20230225T122426.json
└── stitches
    ├── TCGA-3L-AA1B-01Z-00-DX1.jpg
    ├── TCGA-4N-A93T-01Z-00-DX1.jpg
    ├── TCGA-4T-AA8H-01Z-00-DX1.jpg
    ├── TCGA-5M-AAT4-01Z-00-DX1.jpg
    ├── TCGA-5M-AAT5-01Z-00-DX1.jpg
    ├── TCGA-5M-AAT6-01Z-00-DX1.jpg
    ├── TCGA-5M-AATE-01Z-00-DX1.jpg
    ├── TCGA-A6-2671-01Z-00-DX1.jpg
    ├── TCGA-A6-2672-01Z-00-DX1.jpg
    └── TCGA-A6-2674-01Z-00-DX1.jpg


Failed to deploy graph node - "Invalid operation type error. at line 59 column 3"

I was able to deploy this subgraph initially, but now I'm unable to deploy new versions. I can run graph codegen and graph build without any issues.
Here's the output of running graph deploy --studio juicebox
The error at the end isn't giving me enough information to debug.
I'm using graph-cli version 0.21.0, node v16.5.0, running on a Mac M1
✔ Version Label (e.g. v0.0.1) · 0.0.2
  Skip migration: Bump mapping apiVersion from 0.0.1 to 0.0.2
  Skip migration: Bump mapping apiVersion from 0.0.2 to 0.0.3
  Skip migration: Bump mapping apiVersion from 0.0.3 to 0.0.4
  Skip migration: Bump mapping specVersion from 0.0.1 to 0.0.2
✔ Apply migrations
✔ Load subgraph from subgraph.yaml
  Compile data source: Projects => build/Projects/Projects.wasm
  Compile data source: TerminalV1 => build/Projects/Projects.wasm (already compiled)
✔ Compile subgraph
  Copy schema file build/schema.graphql
  Write subgraph file build/Projects/abis/Projects.json
  Write subgraph file build/TerminalV1/abis/TerminalV1.json
  Write subgraph manifest build/subgraph.yaml
✔ Write compiled subgraph to build/
  Add file to IPFS build/schema.graphql
                .. Qmd9z2gQh9negCbgJvjcQHEnv12cLUs5PNawWkiLeUxyu4
  Add file to IPFS build/Projects/abis/Projects.json
                .. QmcQ6Jazax2Zs7nS5yiatJfBruLbnDgPh1Z6s8Ymo2R9Bn
  Add file to IPFS build/TerminalV1/abis/TerminalV1.json
                .. Qma83sCmBYhEETWk1FnicgxpVuXFV9ffY6PKzHcVTdDtMX
  Add file to IPFS build/Projects/Projects.wasm
                .. QmbEFthRpcD9A9ZcySbZe9Q2BBZhDSEtzZVUdFbH4QFBhr
  Add file to IPFS build/Projects/Projects.wasm
                .. QmbEFthRpcD9A9ZcySbZe9Q2BBZhDSEtzZVUdFbH4QFBhr (already uploaded)
✔ Upload subgraph to IPFS

Build completed: QmVKSTRM8E693ZVXtBVMmbN32nmTKKx8SkDdwfe1cwSkNV

✖ Failed to deploy to Graph node https://api.studio.thegraph.com/deploy/: subgraph resolve error: resolve error: Serialization error: Invalid operation type error. at line 59 column 3

Are you using solidity 8? I think you have maybe encountered this error https://github.com/graphprotocol/graph-node/issues/2577

I just hit this error, it seems to come from Solidity 0.8.4 errors.
These errors appear in the ABI file like this:
  { "inputs": [], "name": "MustBeOwner", "type": "error" },

It seems that the graph-cli needs to add support for parsing these errors.

@azf20 I am using solidity 8.6, looks like that's exactly what I've been blocked on
@dmihal have you found a workaround yet, like trying to strip customErrors from the abi before using it to compile a subgraph? theoretically that would still be usable no?

Update: I tried deploying this subgraph with the errors removed from the ABI and it failed syncing. I'm guessing this is because of the ABI mismatch https://thegraph.com/studio/subgraph/juicebox/

@dmihal yes - the parsing is actually required on the graph-node side (it is there that the ABI is being parsed, and rejected)
@peripheralist yes that is the correct fix - I don't think there is a reason syncing would fail because of the ABI mismatch, so I expect that something else went wrong. If you can share the deployment hash we could take a look.

Thanks, is this the deployment hash? QmQoZzZCjuAym1Wy31hehErn5HfHwDKAXbEGuRLkv3X78g

Yes it looks like that has failed for another reason:
failed to process trigger: block #12834007 (0x5c38…57b1), transaction c815a8587a49d7379322be6ce69f6c5d7c1995cb3754d01a96ecd549793d6d36: Entity PayEvent[0xc815a8587a49d7379322be6ce69f6c5d7c1995cb3754d01a96ecd549793d6d36-115]: missing value for non-nullable field `memo`\twasm backtrace:\t    0: 0x13ea - <unknown>!generated/schema/PayEvent#save\t    1: 0x1447 - <unknown>!src/mapping/handlePay\t

thanks @azf20 , seems like that's saying an empty value was passed for memo. in that case an error would be thrown, except that i removed the errors from the abi... which is probably causing this to fail? do you have any ideas for a workaround?
also, is there somewhere i could find those logs so i don't have to bug you for them?

@peripheralist the removed errors in the ABI are not related to this subgraph failure (they are thrown on an error in the EVM execution, not the subgraph execution).
it looks like you have a non-nullable field in your schema, and you are trying to save a null value. You can query for these kinds of errors per here

@azf20 yes! didn't understand the non-null GraphQL type before, but i'm clear on it now. syncing succeeded. thanks for the help!
for anyone else coming here using solidity 8, just confirming that you can still deploy your subgraph by manually removing the errors from your ABI 👍

You can query for these kinds of errors per here

@azf20 i'm trying to follow the instructions in that link, but it seems like this is for the legacy explorer—i'm using studio. what should i be using for subgraphName, and where is the deployment hash relevant?

hi @peripheralist apologies yes those docs apply to the legacy explorer, you should be able to run
  indexingStatuses(subgraphs: ["QmQoZzZCjuAym1Wy31hehErn5HfHwDKAXbEGuRLkv3X78g"]) {
    synced
    health
    fatalError {
      message
      block { number }
      handler
    }
    subgraph
    chains {
      chainHeadBlock {
        number
      }
      latestBlock {
        number
      }
    }
  }
}```
on the same endpoint.
Conscious that this discussion has moved off the topic of this issue, please [join our Discord](https://discord.gg/vtvv7FP) and use #subgraph-development for ongoing discussion

Errors are now supported, closing this, thanks!


eslint-plugin-standard is currently deprecated

Since eslint-config-standard v16, eslint-plugin-standard is no longer included in peerDependencies and has been marked as deprecated. Let's remove eslint-plugin-standard from the dependencies!

Let's remove eslint-plugin-standard from the dependencies!
─ https://github.com/sounisi5011/readme-generator/issues/126#issue-751341804

eslint-config-standard-with-typescript still depends on eslint-plugin-standard and is included in peerDependencies. Now we cannot remove eslint-plugin-standard yet.

The time has come. See PR #146 and<EMAIL_ADDRESS>CHANGELOG.


fixes #356 by making examples work also from CLI

Coverage increased (+0.04%) to 99.582% when pulling 5420279e489882d7257726d97e9978f31ca8fb32 on spock:feature/fix-issue-356 into 3e69902d24326f0b8f7180d97a14cc8c54ede737 on timothycrosley:develop.

Looks good, thanks!


basissets and pseudos keys are always present in the cp2k calculation builder.

basissets and pseudos are defined as optional namespaces so we were assuming that they shouldn't be present in the inputs. However, the freshly created builder already contains both of them as empty dictionaries.
This causes the following if statements to be always executed:
https://github.com/aiidateam/aiida-cp2k/blob/0c3c3aad8f1a0151018ce8542fe218c96c37e66c/aiida_cp2k/calculations/__init__.py#L154
and
https://github.com/aiidateam/aiida-cp2k/blob/0c3c3aad8f1a0151018ce8542fe218c96c37e66c/aiida_cp2k/calculations/__init__.py#L159
@dev-zero would you be able to have a look? This is rather urgent as the 'normal' functionality is broken.

proof:

I wonder how other plugins are handling this (e.g. QE seems to have a lot of similar optional ports guarded by conditionals when prepping a calculation). The quickfix is probably doing something like if 'pseudos' in self.inputs and self.inputs.pseudos (test needed), but in the end this seems to be a bug in the ProcessBuilderNamespace.

... and we have to figure out why examples/single_calculations/example_dft.py::example_dft succeeds without the aiida-gaussian-datatypes being installed, since it also uses get_builder()

I talked to @sphuber and he said that there was such a bug in the aiida-core, but it got fixed. So quite probably we have to do nothing except of updating it.
I will try to make a small example to reproduce the problem, and we can then proceed further.

Ok, so I found the origin of the problem. Luckily it has nothing to do with aiida-cp2k or aiida-core. However, it has to do with the way we submit the calculation using SubmitButtonWidget in AiiDAlab.
Essentially, the problem boils down to the way. More specifically to this:
process = submit(process_class, **builder)

So when the builder, that contains empty basis sets and pseudos, gets unpacked - it contains basis sets and pseudos (even though) they are empty. The reason I did things this way is because I wanted users to be able to provide both: input dictionary or input builder.
Now I have a question (more to @sphuber) how to proceed further. Should I modify the SubmitButtonWidget, so that it distinguishes builder and dictionary? Or should we implement some get_dict() method to the input builder which would be able to provide a valid input dictionary?

The submit method is already written to support a ProcessBuilder directly. All you have to do is change that line to:
process = submit(builder)

and it will work as you intended it.

The submit method is already written to support a ProcessBuilder directly.

This I know, see my text above:

The reason I did things this way is because I wanted users to be able to provide both: either input dictionary or input builder.

Essentially, the question here is: are we moving away from using dictionaries as the input to the submit method? Should I drop support of using dictionaries in the  SubmitButtonWidget, or should I changes its API two support both in the following way:
SubmitButtonWidget(process_class, input_dictionary=None, input_builder=None)

I looked at the code that you linked and I am not sure how it is supposed to work, but is it that whatever callable is defined for input_dictionary_function should be able to return either the dictionary of inputs, or a ProcessBuilder instance? If that is the case, then you could just simply use some type checking to make it work, similar to what the submit implementation does in aiida-core:
input_dict = self.input_dictionary_function()
if isinstance(input_dict, ProcessBuilder):
    builder = input_dict
    process_class = builder.process_class
    input_dict.update(**builder._inputs(prune=True))  # pylint: disable=protected-access

Finally, in this case, maybe I would consider renaming input_dictionary_function if it really is also supposed to be able to return a builder.

Essentially, the question here is: are we moving away from using dictionaries as the input to the submit method?

No, both are perfectly fine and will be kept. The builder was added mostly to provide additional functionality in interactive modes, such as a shell or notebook. But we also wanted it to be useful in static code such as scripts, so we added support for it in the launcher methods.

I looked at the code that you linked and I am not sure how it is supposed to work, but is it that whatever callable is defined for input_dictionary_function should be able to return either the dictionary of inputs, or a ProcessBuilder instance?

Yes, that was the intension.

Why don't you just mirror the exact same signature of submit in your SubmitButtonWidget?
You can just do:
SubmitButtonWidget(process, **inputs)

and then you literally just forward this to aiida.engine.submit. Users can then do SubmitButtonWidget(builder) as well as SubmitButtonWidget(PwBaseWorkChain, **{'code': ...}). I don't see why you would need or want a different interface

It is not really possible, as SubmitButtonWidget is a widget in the first place. So it has to be created and shown before any other input generation code is executed. Therefore, we perform the generation of inputs through the input_dictionary_function  function.

Finally, in this case, maybe I would consider renaming input_dictionary_function if it really is also supposed to be able to return a builder.

Yes, I will do that.


Use token based approach for system-agent

What this PR does / why we need it:
Reduce the footprint of the system-agent RBAC.
Per each cluster there will be created:

1 system-agent ServiceAccount

Per each plan there will be temporarily created:

1 Role with access to all plan secrets for each machine
1 Rolebinging for the role and the cluster system-agent ServiceAccount

On plan completion/failure the role and rolebinding will be revoked
Per each machine there will be created:

1 Secret for the system-agent authentication, with unique JWT bound to the secret existence in the API server, and a namespace/name pointer to the plan secret
1 Secret for the plan execution

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #756
Special notes for your reviewer:
Checklist:

[x] squashed commits into logical changes
[ ] includes documentation
[x] adds unit tests
[ ] adds or updates e2e tests

To test issuing token we need a change which is not yet released in CR: https://github.com/kubernetes-sigs/controller-runtime/pull/2969
Getting “ fakeSubResourceWriter does not support create for token”

I’ll try to address the rest of the comments in the followup PRs.


Update Key Vault documentation

Library name
Azure.Security.KeyVault.Keys
Please describe the feature.
We need to update the Key Vault documentation in a few places:

[ ] Authentication in Samples: Our samples should show the use of Managed Identity instead of Service Principals with Client Secrets as the default authentication method. We can still show how to use the former for scenarios where the use of Managed Identity is not possible.
[ ] Cryptographic operations using CBC and CBCPAD: We should make it clear that there are some considerations to have when using these algorithms for local cryptographic operations using our clients:

We should add a notice similar to the following:

Microsoft recommends you not use CBC without first ensuring the integrity of the ciphertext using an HMAC, for example. See https://docs.microsoft.com/dotnet/standard/security/vulnerabilities-cbc-mode for more information.

When it comes to a user provided IV, we should make it clear that if they pass their own IV, they need to make sure they use a cryptographically random, non-repeating IV. Here's an example:

Optional initialization vector (IV). If you pass your own IV, make sure you use a cryptographically random, non-repeating IV. If null, a cryptographically random IV will be choosing using {RandomNumberGenerator | whatever cryptorng your language provides}.

[ ] ExpiresIn in KeyRotationPolicy: Currently, the docstring for this parameter makes it sound like it pertains to the expiration time of the key rotation policy, when it should refer to the expiration of a newly rotated key, as rotation policies don't actually expire. Here's what the swagger documentation says about the expiryTime parameter. We can also refer to the verbiage used in Java, Python or Javascript.

For the last 2 bullets can we just do that as a change to the swagger docs? It's non-breaking. I've already approved and merged a couple of doc updates for @chlowell for Go, for example.

For the last 2 bullets can we just do that as a change to the swagger docs? It's non-breaking. I've already approved and merged a couple of doc updates for @chlowell for Go, for example.

That should be fine, although for the last bullet the swagger already mentions the expiration time refers to the key, as opposed to what .NET does

.NET isn't generated, but I usually took the string from swagger. Interesting. I'll be sure to update it, though.

I actually noticed that all languages have their own flavor for the expiresIn docstring, which makes me think our public layer is not auto-generated for any language.

You can refer to this Java PR for the second item (CBC crypto).


Output not deterministic

I was investigating why sccache was not caching some parts of webrender (which @jrmuizel noticed) and I tracked it down to gleam using gl_generator to generate some source files, and those source files not being generated in a deterministic manner. Here's a diff between two runs.
gleam's build script doesn't seem to be doing anything unusual.
If I had to guess, I would guess it's due to Registry using HashSet / HashMap. Iterating over those items will not use a deterministic ordering. It should be simple enough to change those to BTreeSet / BTreeMap to fix that.

I have a patch that compiles and passes tests: https://github.com/luser/gl-rs/commit/166771f9d79b51219507289ae76529bd093c975f .
I need to test it with a crate that does some generation to ensure that it fixes the issue.

Oooh, thanks for tracking this down!

 I originally thought bindgen was the problem and one of the bindgen authors mentioned that they use BTreeMap / BTreeSet everywhere for this exact reason, so I took a look at the gl_generator code and tried the obvious thing.


Improve MSBuild version parsing

MSBuildDetector was dependent on the current user's culture, and failed to parse for instance 14.0 when it didn't use a decimal point. See issue #244.
Related issue
Closes: #244
Description
Improves version parsing by making it independent of the user's culture settings. Instead of specifying the invariant culture everywhere a version string is parsed, a new helper function attempts to parse the string as a proper version, and falls back to a double in case it fails (e.g., a single integer version).
I also changed the LINQ ordering to ThenByDescending since I thought it made it easier to understand, because then the highest (newest) version will be selected; previously ThenBy was used with negative version numbers to achieve the same effect.
Supplemental testing
The unit tests for MSBuildDetector now runs successfully:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Hi cjberg,
We received your feedback. Thanks for providing your feedback about Porting Assistant for .NET. We will be reviewing your recommendation and contact you if we need further information.
Thanks
AWS Porting Assistant Team

Hi cjberg,
Thank you for providing your feedback about Porting Assistant for .NET! After careful review, your package replacement suggestion was accepted.
Thank you again for your support and contribution towards Porting Assistant for .NET. Please let us know if have any additional questions or concerns.
Thanks,
AWS Porting Assistant Team


Add global --verbose flag to operator-sdk command

Description of the change:
Add global --verbose flag to operator-sdk command. The scorecard command has a verbose option, but this is better defined as a single global flag. The initial use of the flag is to add a log.Debugf call to see the exec calls that are being made.
Motivation for the change:
Experimenting with creating a Bazel build around the generated project from operator-sdk. It was not clear what was actually happening at the runtime exec calls.

/cc @hasbro17 @joelanford @AlexNPavel @LiliC

/ok-to-test

We're able to remove other uses of VerboseOpt in the scorecard code with this change as well, correct?
https://github.com/operator-framework/operator-sdk/blob/1d10eb751d919f2b03becc1f9ff755517babc255/internal/pkg/scorecard/scorecard.go#L64
https://github.com/operator-framework/operator-sdk/blob/1d10eb751d919f2b03becc1f9ff755517babc255/cmd/operator-sdk/scorecard/cmd.go#L54

@joelanford Good eye, I should have done a global search.
Now I'm interested in whether we ought to interact with --v=, which has different dimensionality:
https://github.com/operator-framework/operator-sdk/blob/fc22a333ecaa6ea40391fcc82b9c7050dd2bb3b1/internal/util/projutil/project_util.go#L191-L196

@briantopping it would be nice to rely on --verbose instead of GOFLAGS globally. We could set GOFLAGS="-v" if --verbose is set instead of checking manually.

/retest

/shrug

All requests were made to my understanding of them. Thanks for engaging on this! It's always a pleasure to learn. I'm going to leave the requests in the thread unresolved so the original writers can decide.
(side note: My previous /unshrug was only because I couldn't trigger a retest, just wanted to test what prow access I had.)

/unshrug

/retest

/lgtm

/test sanity

@LiliC No idea why https://github.com/operator-framework/operator-sdk/blob/3c2fee5826c7f691c5879ae47e21174f4f99fba3/hack/check_error_log_msg_format.sh#L10 is failing now. These uppercase strings were there since the start of my PR, now they suddenly emerge after your review!

/retest

/test pr

/test patience

@briantopping Travis tests aren't managed by API CI so sadly we can't trigger retests with comments. If you hit details, there should be a restart test button on the web page. I'll restart the failed test for you now

@briantopping Looks good overall.
One last thing: Can you please document the behavior of the global --verbose flag in the sdk-cli-reference.md doc.
You can add a line right at the top before the first section for the build command.
Additionally we want to point out this change in the Added section of the CHANGELOG.

@hasbro17 all set!

Thanks all! Grateful for your patience while I came up-to-speed with contributing to this project! 😄


GPTQ / ExLlamaV2 (EXL2) quantisation

Feature Description
Please provide a detailed written description of what you were trying to do, and what you expected llama.cpp to do as an enhancement.
Motivation
It sounds like it's a fast/useful quantisation method:

https://towardsdatascience.com/exllamav2-the-fastest-library-to-run-llms-32aeda294d26

https://github.com/mlabonne/llm-course/blob/main/Quantize_models_with_ExLlamaV2.ipynb

https://towardsdatascience.com/4-bit-quantization-with-gptq-36b0f4f02c34
https://huggingface.co/blog/gptq-integration
https://oobabooga.github.io/blog/posts/gptq-awq-exl2-llamacpp/

A detailed comparison between GPTQ, AWQ, EXL2, q4_K_M, q4_K_S, and load_in_4bit: perplexity, VRAM, speed, model size, and loading time.

See Also

https://github.com/ggerganov/llama.cpp/issues/4165

Bump

no support for gptq and awq?

Hi, is there any updates? Thanks!

According to the diagram, it seems that exllama v2 is much better at generating speed.

According to the diagram, it seems that exllama v2 is much better at generating speed.

Also from that link:

Update 2: Gerganov has created a PR on llama.cpp that optimizes the llama.cpp evaluation/processing speeds and should make the values here obsolete. See the numbers and discussion here.

https://github.com/ggerganov/llama.cpp/pull/3776#issuecomment-1781472687

According to the diagram, it seems that exllama v2 is much better at generating speed.

Also from that link:

Update 2: Gerganov has created a PR on llama.cpp that optimizes the llama.cpp evaluation/processing speeds and should make the values here obsolete. See the numbers and discussion here.

cuda : improve text-generation and batched decoding performance ggerganov/llama.cpp#3776 (comment)

Nice, this shows the improvement closer to exllama v2.


go test --json incorrectly reports benchmark output

What version of Go are you using (go version)?
go version go1.11 linux/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/miki/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/miki/Projects/go"
GOPROXY=""
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build566604237=/tmp/go-build -gno-record-gcc-switches"
What did you do?
Running go test -v -bench . --json on a simple test. See https://play.golang.org/p/m2rEGNYaHAL
What did you expect to see?
Benchmark output in it's own test name (e.g. BenchmarkAdd-4)
What did you see instead?
Benchmark output reported under a test function name.
Lines like
{"Time":"2018-09-19T16:16:03.409160347-07:00","Action":"output","Package":"_/tmp/bench","Test":"TestAdd","Output":"2000000000\t         0.29 ns/op\n"}

See full output  below
{"Time":"2018-09-19T16:16:02.805686252-07:00","Action":"run","Package":"_/tmp/bench","Test":"TestAdd"}
{"Time":"2018-09-19T16:16:02.805789664-07:00","Action":"output","Package":"_/tmp/bench","Test":"TestAdd","Output":"=== RUN   TestAdd\n"}
{"Time":"2018-09-19T16:16:02.805805951-07:00","Action":"output","Package":"_/tmp/bench","Test":"TestAdd","Output":"--- PASS: TestAdd (0.00s)\n"}
{"Time":"2018-09-19T16:16:02.806048353-07:00","Action":"output","Package":"_/tmp/bench","Test":"TestAdd","Output":"goos: linux\n"}
{"Time":"2018-09-19T16:16:02.806059884-07:00","Action":"output","Package":"_/tmp/bench","Test":"TestAdd","Output":"goarch: amd64\n"}
{"Time":"2018-09-19T16:16:02.806068435-07:00","Action":"output","Package":"_/tmp/bench","Test":"TestAdd","Output":"BenchmarkAdd-4   \t"}
{"Time":"2018-09-19T16:16:03.409160347-07:00","Action":"output","Package":"_/tmp/bench","Test":"TestAdd","Output":"2000000000\t         0.29 ns/op\n"}
{"Time":"2018-09-19T16:16:03.409188589-07:00","Action":"pass","Package":"_/tmp/bench","Test":"TestAdd","Elapsed":0}
{"Time":"2018-09-19T16:16:03.409195398-07:00","Action":"output","Package":"_/tmp/bench","Output":"PASS\n"}
{"Time":"2018-09-19T16:16:03.40939684-07:00","Action":"output","Package":"_/tmp/bench","Output":"ok  \t_/tmp/bench\t0.604s\n"}
{"Time":"2018-09-19T16:16:03.409405317-07:00","Action":"pass","Package":"_/tmp/bench","Elapsed":0.604}

It is because you are mixing the tests and benchmarks. Try adding -run=xxx.

I got you now. Then this is a different issue. You want the benchmark name to be there while running benchmarks.
According to go doc test2json, this is working as intended.

When a benchmark runs, it typically produces a single line of output giving
timing results. That line is reported in an event with Action == "output"
and no Test field. If a benchmark logs output or reports a failure (for
example, by using b.Log or b.Error), that extra output is reported as a
sequence of events with Test set to the benchmark name, terminated by a
final event with Action == "bench" or "fail". Benchmarks have no events with
Action == "run", "pause", or "cont".

So it seems, if we just run the benchmarks with no logging, the benchmark name is not set. And if both tests and benchmarks are run together, the test name gets set to the last test that was run. Looks like something that should be fixed.
@ianlancetaylor @rsc @bcmills


Chinese translations(Simplified)

This PR has added Simplified Chinese translations
I will strive to ensure the accuracy and fluency of the translations. Any feedback on the translations is welcome to improve their quality.

https://github.com/pizzaboxer/bloxstrap/pull/932#issuecomment-1973847768


Will merge once 🟢.


Authentication error while downlo

Expected behavior
I should be able download and create minecraft image from docker hub.
Actual behavior
I am getting error - unauthorised : authentication required
Information about the Issue
Error screenshot is attached
Steps to reproduce the behavior

Click create button in Kitematics for minecraft official image and I get authentication error after 70% download.

...

Did you login to Docker Hub on the first Kitematic screen? There's a good chance your account needs to be verified:
Login here to do so: https://cloud.docker.com

Hi French Ben,
The issue has been resolved now. Docker icon used to come in my system tray once I log into my machine (mac) and I find myself logged into Kitematic when I open the Kitematic from the system tray docker icon drop down.
I logged out for Kitematic, closed the docker and restarted the machine. I didn’t find the issue in my new login.
Thank you,
Abhay Kumar.

On 18-Jan-2017, at 12:25 AM, French Ben<EMAIL_ADDRESS>wrote:
Did you login to Docker Hub on the first Kitematic screen? There's a good chance your account needs to be verified:
Login here to do so: https://cloud.docker.com https://cloud.docker.com/
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/docker/kitematic/issues/2264#issuecomment-273263214, or mute the thread https://github.com/notifications/unsubscribe-auth/AXGgGl2Q6-Xs7aQzfa2oHb_7Cscg0Bo0ks5rTQ6VgaJpZM4Lj8wE.


Replacing prompt with popup dialog box for creating new circuit

Fixes #1133
Describe the changes you have made in this PR -
I have Removed the prompt and added the script to create a new dialog box for creating new circuit
Screenshots of the changes (If any) -

Coverage remained the same at 84.266% when pulling d19f075c95d272d1ac06f5e9a5cfa0ebc345f0ee on darkshredder:patch-13 into c570b47d12ca5e8aae56bd74e670ebf6c9e34eb0 on CircuitVerse:master.

@tachyons @aayush-05 @nitin10s please review it and tell me how to fix complexity issue too

@Shivansh2407 try it out now it will work i guess

@Shivansh2407 fixed both issues check now
@tachyons @nitin10s @aayush-05 @satu0king please review it.

@Shivansh2407 I m working for all prompts, will create a PR soon

Ok cool, I think you can fix all the prompts in this PR only. Change the PR Title when you do so 👍🏻

@tachyons @satu0king @nitin10s @aayush-05 please review it

@Shivansh2407 I have made X working instead of creating cancel button as it looks weird to make a "X"
and cancel button

@Shivansh2407 I have seen that there are lot of prompt in circuit elements, will do it in another PR, as it will take time to get 7-8 prompt to converted to dialog box

@tachyons @satu0king @nitin10s @aayush-05 please review it

@tachyons @satu0king @nitin10s @aayush-05 please review it

@satu0king movement of code, Done because prompt blocks the line of code but Dialog is Synchronous in nature

@satu0king please review it again.


[2pt] Create login and registration page

Should be built with Devise;
Should follow the layout of the wireframes provided.

Pull request for this task


composer require cwsDump & cwsDebug

And last but not least, this one should auto install dependencies.

Nice :)


dht22 example

The blog goes on to add a DHT22 sensor and additional frames but none of the libraries or sketches seem similar to this.  It would be nice is you included an example sketch for the DHT22 frame to be displayed in the library.

Not an example sketch, but here's the frame code I used with a DHT11.
`void drawInsideConditions(OLEDDisplay display, OLEDDisplayUiState state, int16_t x, int16_t y) {
String t = String(dht.readTemperature(true)) + "°F";
String h = String(dht.readHumidity()) + "%";
int xt = (64 - (t.length() * 8)) / 2;
int xh = ((64 - (h.length() * 8)) / 2) + 64;
display->setTextAlignment(TEXT_ALIGN_LEFT);
display->setFont(ArialMT_Plain_10);
display->drawString(x + 49, y + 14, "Inside");
display->drawString(x + 4, y + 26, "Temperature");
display->drawString(x + 76, y + 26, "Humidity");
display->setFont(ArialMT_Plain_16);
display->drawString(x + 4, y + 38, t);
display->drawString(x + 68, y + 38, h);
}`

Try this, works great for me https://github.com/neptune2/esp8266-weather-station-oled-DST


[feature]Can Jadx expand the expression?

Describe your idea:
Hello,
Can Jadx expand the expression（Similar to three address codes）？This will make it easier to do other analysis.
Code A and code B are semantically the same. Code A is generated by Jadx. For code B, I just expanded the expression in code A.
code A:
public final ZipEntry getNextEntry() {
        String name;
        ZipEntry nextEntry = super.getNextEntry();
        if (nextEntry == null || (name = nextEntry.getName()) == null || (!name.contains("../") && !name.contains("..\\"))) {
            return nextEntry;
        }
        throw new SecurityException(":" + nextEntry.getName());
    }

public final ZipEntry getNextEntry() {
        String name;
        ZipEntry nextEntry = super.getNextEntry();
        if (nextEntry == null)
            return nextEntry;
        ZipEntry c = nextEntry;
        name = c.getName();
        if (name == null)
            return nextEntry;

        boolean b1 = !name.contains("../");
        boolean b2 = !name.contains("..\\");
        if (b1 && b2)
            return nextEntry;

        throw new SecurityException(":" + nextEntry.getName());
    }

Thx! :D

@101142TS yeah, I also don't like long conditions. And here we have two issues actually:

Very aggressive assign inlining, without it code will look like this:

public final ZipEntry test() {
	ZipEntry nextEntry = getNextEntry();
	if (nextEntry != null) {
		String name = nextEntry.getName();
		if (name != null && (name.contains("../") || name.contains("..\\"))) {
			throw new SecurityException(":" + nextEntry.getName());
		}
	}
	return nextEntry;
}

Jadx now trying not to duplicate code, so only one return used. I can force early return here to make code looks like this:

public final ZipEntry test() {
	ZipEntry nextEntry = getNextEntry();
	if (nextEntry == null) {
		return nextEntry;
	}
	String name = nextEntry.getName();
	if (name == null || (!name.contains("../") && !name.contains("..\\"))) {
		return nextEntry;
	}
	throw new SecurityException(":" + nextEntry.getName());
}

And for last condition I don't see a reason to break it further.
Anyway, I will implement both of these fixes, but it is hard and  very error-prone, so it will take some time.

Thanks for the reply！
I'm using semgrep to match vulnerability patterns in Android applications.
However, semgrep cannot match statements in nested expressions. That's why I need to break all the expressions.
Would it be difficult to implement expressions split on my own by modifying Jadx?

Would it be difficult to implement expressions split on my own by modifying Jadx?

I made a quick mod for you: check 6996f94 in mod-1184 branch:
    public final ZipEntry test() {
        ZipEntry nextEntry = getNextEntry();
        if (nextEntry == null) {
            return nextEntry;
        }
        String name = nextEntry.getName();
        if (name == null) {
            return nextEntry;
        }
        boolean contains = name.contains("../");
        if (!contains) {
            boolean contains2 = name.contains("..\\");
            if (!contains2) {
                return nextEntry;
            }
        }
        throw new SecurityException(":" + nextEntry.getName());
    }

Although, these changes may not work for some other cases :(

Thx, I will try it! :D

Although, these changes may not work for some other cases :(

mod-1184 branch worked very well for most expressions, except for a few examples.
// Generated by jadx 1.2.0
ZipEntry nextEntry = zipInputStream.getNextEntry();
if (nextEntry != null) {
    String name = nextEntry.getName();
    if (name == null || !name.contains("../")) {
        if (name == null || !name.equals(aPKInfo.iconPath)) {
            zipInputStream.closeEntry();
        } else {
            ByteArrayOutputStream byteArrayOutputStream2 = new ByteArrayOutputStream();
        }
    }
}

// Generated by jadx mod-1184 branch
ZipEntry nextEntry = zipInputStream.getNextEntry();
if (nextEntry != null) {
    String name = nextEntry.getName();
    if (name != null) {
        boolean contains = name.contains("../");
        if (contains) {
        }
    }    //Here I think it should be an else code block
    if (name != null) {
        boolean equals = name.equals(aPKInfo.iconPath);
        if (equals) {
            ByteArrayOutputStream byteArrayOutputStream2 = new ByteArrayOutputStream();

        }
    }
    zipInputStream.closeEntry();
}

//Here I think it should be an else code block

Well, else is not enough here, it must be a duplicated code or return (if this is last statements).
You can check it yourself using IntelliJ IDEA with only Invert 'if' conditions and Split into 2 if expressions transformations, it will duplicate code to the hell :rofl:
So this is kind of limitations here, jadx don't duplicate code now, and I am not sure if this can be fixed.
Also, jadx can't produce 100% correct code and there are always some errors exists, so be careful and don't rely much on decompiled code.


[9.x] WFLY-4659 Upgrade JGroups to 3.6.3.Final

https://issues.jboss.org/browse/WFLY-4659

Linux with security manager Build 51 is now running using a merge of a7f8f25e5f8f5358449d33eb7a54961a9c29114d

Linux Build 53 is now running using a merge of a7f8f25e5f8f5358449d33eb7a54961a9c29114d

Windows Build 52 is now running using a merge of a7f8f25e5f8f5358449d33eb7a54961a9c29114d

Windows Build 52 outcome was FAILURE using a merge of a7f8f25e5f8f5358449d33eb7a54961a9c29114d
Summary: Exit code 1 (new) Build time: 0:01:10
Build problems:
Process exited with code 1

Linux with security manager Build 51 outcome was SUCCESS using a merge of a7f8f25e5f8f5358449d33eb7a54961a9c29114d
Summary: Tests passed: 825, ignored: 317 Build time: 0:05:54

Linux Build 53 outcome was SUCCESS using a merge of a7f8f25e5f8f5358449d33eb7a54961a9c29114d
Summary: Tests passed: 2867, ignored: 372 Build time: 0:49:32

The upstream one has been merged https://github.com/wildfly/wildfly/pull/7511

Linux Build 67 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Windows Build 66 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Linux with security manager Build 65 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Windows Build 66 outcome was FAILURE using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Exit code 1 Build time: 0:00:54
Build problems:
Process exited with code 1

Linux with security manager Build 65 outcome was SUCCESS using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Tests passed: 825, ignored: 317 Build time: 0:05:38

Linux Build 67 outcome was FAILURE using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Tests failed: 1 (1 new), passed: 2865, ignored: 372, muted: 1 Build time: 0:49:14
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.integration.ejb.stateful.passivation.PassivationTestCase.testPassivationMaxSize: <no details avaliable>

Windows Build 67 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Windows Build 67 outcome was FAILURE using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Exit code 1 Build time: 0:00:47
Build problems:
Process exited with code 1

Windows Build 68 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Linux Build 68 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Windows Build 68 outcome was FAILURE using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Exit code 1 Build time: 0:01:06
Build problems:
Process exited with code 1

Windows Build 70 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Windows Build 70 outcome was SUCCESS using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Tests passed: 2863, ignored: 372, muted: 1 Build time: 0:46:57

Retest this please

Linux Build 72 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Windows Build 75 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Linux with security manager Build 68 is now running using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0

Windows Build 75 outcome was FAILURE using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Exit code 1 (new) Build time: 0:00:51
Build problems:
Process exited with code 1

Linux with security manager Build 68 outcome was SUCCESS using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Tests passed: 825, ignored: 317 Build time: 0:05:39

Linux Build 72 outcome was SUCCESS using a merge of 95a9e4793bf2d581201a199091d0bf1cd03accd0
Summary: Tests passed: 2867, ignored: 372 Build time: 0:47:35


Timestamp verification error if there is an intermediate certificate

I use the TSA http://timestamp.comodoca.com/rfc3161, whose certificate chain includes an intermediate cert besides the root cert and the TSA signing cert.
All certs needed to reproduce this issue can be downloaded here:
certs.zip

comodo-root.pem - The root cert
comodo-intermediate.pem - The intermediate cert
comodo-leaf.pem - The TSA signing cert
comodo-untrusted.pem - Combination of the intermediate cert and the TSA signing cert

Here're the steps to reproduce:
First, create a file sample-data.txt with random content.
Then, get the timestamp response:
openssl ts -query -data sample-data.txt -sha256 | curl -s -S \
-H "Content-Type: application/timestamp-query" --data-binary @- -o sample-data.txt.tsr \
http://timestamp.comodoca.com/rfc3161

Finally, verify it:
openssl ts -verify -in sample-data.txt.tsr -data sample-data.txt \
-CAfile comodo-root.pem -untrusted comodo-untrusted.pem

It shows this error:
time stamp routines:ts_check_signing_certs:ess signing certificate error:crypto/ts/ts_rsp_verify.c:249

I researched ts_rsp_verify.c and found that although there're 2 untrusted certificates, the chain variable has 3 certificates, including the root certificate. But cert_ids only has 2 certificates. When the program searches cert_ids for the first 2 certificates in the chain, it succeeds. But when searching cert_ids for the 3rd certificate (the root certificate), it cannot find it and therefore produces an error.
At first I thought maybe it's the misconfiguration at the TSA side. Then I tested the TSA http://timestamp.entrust.net/TSS/RFC3161sha2TS and it shows the same error. This TSA's chain also contains an intermediate certificate.
When I timestamp a PDF document in Adobe Acrobat Reader DC using these 2 TSAs, they all produce good results. Adobe Acrobat Reader shows the timestamps are valid.
Version: v1.1.1 and v3.0.0

Great timing; I experienced the same error today too.
For me, it is from http://timestamp.sectigo.com, an alias of http://timestamp.comodoca.com/rfc3161, I supposed.
Java validates the response just fine.
I also wanted to add that there are 2 possible certificate paths.

One already provided by the OP
Another one chained to AAACertificateServices

Since the timestamp server can be updated arbitrarily, I will capture the request/response here.

Base64 encoded request

MDkCAQEwMTANBglghkgBZQMEAgEFAAQgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAf8=

Base64 encoded response

[encoded data, 6644 characters]

@zhanzhenzhen
I did not dig into it as deeply as you.
However, from the outsider perspective, I wonder why we need to supply intermediate certificates to OpenSSL at all?
I don't even know what these cert_ids are, but from my understanding:

There are 2 certificates sent along with the timestamp response.
One of them, with the serial number 30:0f:6f:ac:dd:66:98:74:7c:a9:46:36:a7:78:2d:b9, specify X509v3 Authority Key Identifier as keyid:53:79:BF:5A:AA:2B:4A:CF:54:80:E1:D8:9B:C0:9D:F2:B2:03:66:CB, which happened to be X509v3 Subject Key Identifier of the root certificate, with serial number 01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d.
Therefore, all certificates presented in the response are perfectly chained to root already. We should be able to only supply the root certificate 01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d and nothing else.

Could you enlighten me a little? (I am somewhat new to PKI related topics.)

@midnight-wonderer I think with -cert option the response contains one (but only one) intermediate certificates. Without -cert option the response contains no intermediate certificates. So you still have to provide 1 or 2 intermediate certificates when verifying the timestamp. You can find that if you miss some intermediate certificate, the error is different, not the error I mentioned above.

Sorry for misconception. The TSA signing cert isn't an intermediate cert. I changed the title and the comment.

@midnight-wonderer From your pull request, you mean there're 2 "signing certificates" when using Sectigo?
I just see one signing certificate Sectigo RSA Time Stamping Signer #2, with the intermediate certificate Sectigo RSA Time Stamping CA and root certificate USERTrust RSA Certification Authority. What's the other signing certificate?
Yesterday I also found that DigiCert's timestamp works well. They also have an intermediate certificate. So, is the issue not about whether there's an intermediate certificate?
Here's the DigiCert certs:
digicert.zip

I also wanted to add that there are 2 possible certificate paths.
One already provided by the OP
Another one chained to AAACertificateServices

@midnight-wonderer By the other signing certificate, do you mean that?

It seems that DigiCert's response has only one id-smime-aa-signingCertificate, which is the SHA-1 of the leaf cert:
 3030:d=8  hl=2 l=  11 prim: OBJECT            :id-smime-aa-signingCertificate
 3043:d=8  hl=2 l=  28 cons: SET
 3045:d=9  hl=2 l=  26 cons: SEQUENCE
 3047:d=10 hl=2 l=  24 cons: SEQUENCE
 3049:d=11 hl=2 l=  22 cons: SEQUENCE
 3051:d=12 hl=2 l=  20 prim: OCTET STRING      [HEX DUMP]:E1D782A8E191BEEF6BCA1691B5AAB494A6249BF3

While Sectigo's reponse has two, which are the SHA-1 of both the leaf and the intermediate:
  643:d=8  hl=2 l=  11 prim: OBJECT            :id-smime-aa-signingCertificate
  656:d=8  hl=3 l= 221 cons: SET
  659:d=9  hl=3 l= 218 cons: SEQUENCE
  662:d=10 hl=3 l= 215 cons: SEQUENCE
  665:d=11 hl=2 l=  22 cons: SEQUENCE
  667:d=12 hl=2 l=  20 prim: OCTET STRING      [HEX DUMP]:951137101D882F31BD513F949ADA4C68AD8C08F5
  689:d=11 hl=3 l= 188 cons: SEQUENCE
  692:d=12 hl=2 l=  20 prim: OCTET STRING      [HEX DUMP]:02D65B95E28370C1570095FA88F923DD937FAD8F

Is this the reason?

@zhanzhenzhen I just mentioned that the certificate with serial number 30:0f:6f:ac:dd:66:98:74:7c:a9:46:36:a7:78:2d:b9 can be chained to either the one with serial 39:72:44:3a:f9:22:b7:51:d7:d3:6c:10:dd:31:35:95 or 01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d because both certificates contain the same public key.
It is somewhat unrelevant to the main topic, it just might affect the implementation details.
To keep you in the loop, it is because of the number of fingerprints inside id-smime-aa-signingCertificate as you mentioned.
RFC 5035 mentioned that:

If more than one certificate is present, subsequent certificates limit the set of certificates that are used during validation. Certificates can be either attribute certificates (limiting authorizations) or public key certificates (limiting path validation).  The issuerSerial field (in the ESSCertIDv2 structure) SHOULD be present for these certificates, unless the client who is validating the signature is expected to have easy access to all the certificates required for validation. If only the signing certificate is present in the sequence, there are no restrictions on the set of certificates used in validating the signature.

TSAs and OpenSSL seem to interpret the set of certificates used in validating the signature differently.
The details are inside the pull request #14214.
OpenSSL interpretation has more strict security and also more reasonable IMO.
A TSA can just send one fingerprint from the beginning; the only reason they send more fingerprints should be because they want to mention something beyond the certificates included with the response.

There are 2 certificates sent along with the timestamp response.

@midnight-wonderer You're right. I've deleted the old reply. I also don't know why we need to use -untrusted option to specify a cert that's already in the response. Anyone know why? Or, is there an option that can eliminate the use of -untrusted option? It's not about this issue but I'm curious.
I just tested DigiCert. I dumped 1333 bytes starting from position 1470 in the timestamp response, according to what asn1parse shows, and then converted it to PEM. The generated file is exactly the same as my testing DigiCert intermediate cert.

I also don't know why we need to use -untrusted option to specify a cert that's already in the response. Anyone know why? Or, is there an option that can eliminate the use of -untrusted option? It's not about this issue but I'm curious.

I also came across this issue when adding tests to test/recipes/80-test_tsa.t.
I'd say it's an omission, even a bug.
I' providing a fix for that in #14504.

Java validates the response just fine.

Which crypto library are you using there?
If it's BouncyCastle, here is the reason why: they simply ignore any further ESSCertIDs :frowning_face:
https://github.com/bcgit/bc-java/blob/master/pkix/src/main/java/org/bouncycastle/tsp/TimeStampToken.java#L186

A further issue with the OpenSSL implementation is that it silently prefers any given ESSCertID
over any given ESSCertIDv2, while both may be present and RFC 5035 says:

Applications SHOULD recognize both attributes as long as
they consider SHA-1 able to distinguish between two different
certificates, (i.e., the possibility of a collision is sufficiently
low).  If both attributes exist in a single message, they are
independently evaluated.

@DDvO
Confirmed to be BouncyCastle.
Nice catch on

If both attributes exist in a single message, they are independently evaluated.

but in RFC 2634, they also said that:

A SignerInfo MUST NOT include multiple instances of the SigningCertificate attribute.

; not sure if that also extends to v2.
However, it is not like physically impossible for ASN1 to contain both despite what the spec says.
So, probably those branches need rework a little bit.

A SignerInfo MUST NOT include multiple instances of the SigningCertificate attribute.

This only means that there must not be multiple SigningCertificate v1 attribute
(and the same holds for SigningCertificate v2 attributes).

it is not like physically impossible for ASN1 to contain both despite what the spec says.

it is definitely possible, and RFC 5035 explicitly mentions this as references above.
It also says in section 7:

To ensure that legacy implementations are provided protection against
certificate substitution, clients are permitted to include both
ESScertID and ESScertIDv2 in the same message.  Since these
attributes are generated and evaluated independently, the contents
could conceivably be in conflict.  Specifically, where a signer has
multiple certificates containing the same public key, the two
attributes could specify different signing certificates.  The result
of signature processing may vary depending on which certificate is
used to validate the signature.
Recipients that attempt to evaluate both attributes may choose to
reject such a message.


action/upload-artifact v4 breaks Jreleaser

See the issue apparently v4 is a major upgrade that doesn't allow overwrite:true to work anymore.

Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

See: https://github.com/actions/upload-artifact/issues/478
See example JReleaser failure log here: https://github.com/open-sce/fluent-cli/actions/runs/7254116343/job/19762158254
I think the JReleaser script needs to be updated for v4 and to be able to handle this hopefully?

The main action does not rely on actions/upload-artifact and thus should not be affected by it. If your workflow explicitly configures said action then you'll have to handle uploads (and likely downloads as well) according to the rules of v4.
The java builder OTOH (found in the java branch) makes use of old versions and would need an upgrade but should not encounter the error reported in this issue once the uploaded JReleaser logs are renamed to be unique.

I am using the artifact upload based on all the examples provided by Jreleaser?  So I am unclear as to what the action item is to allow for v4?  Even the example on this README.MD uses it?
Can you provide some guidance what I need to do?

There's a migration doc in both upload-artifact explaining the changes you must make of both sides (upload/download): https://github.com/actions/upload-artifact/blob/main/docs/MIGRATION.md

@aalmiray thanks I was able to upgrade to v4 with this commit: https://github.com/open-sce/fluent-cli/commit/900fed54e6680276ffbc62365843f005ceb7e990
If you want to update any JReleaser docs.

Hi just spent half a day on this. Thanks @melloware  for the solution. I was following the wrong trail.
The documentation still points to aalmiray/q-cli, which uses v3 upload/download github actions.


WeightNormalization not working with custom constraints

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):MacOS 10.15.3
TensorFlow version and how it was installed (source or binary):2.2.0 Google Colab
TensorFlow-Addons version and how it was installed (source or binary):
Python version:Python 3 Google Colab
Is GPU used? (yes/no):yes

Describe the bug

WeightNormalization, using Tensorflow Addons, isn't working with custom Keras constraints. Its working with ones previously built(like NonNeg() or MinMaxNorm())
Code to reproduce the issue
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow_addons.layers import WeightNormalization

class Sum2One(Constraint):
  def __call__(self, w):
    w = w*math_ops.cast(math_ops.greater_equal(w, 0.), K.floatx())
    return w/K.sum(w, axis = 1, keepdims = True)

l = K.ones((6,3))
x = WeightNormalization(Dense(3, kernel_constraint = Sum2One()))
x(l)

The constraint works normally without WeightNormalization applied.
Other info / logs
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-52-631b4a6dfe3d> in <module>()
----> 1 x(l)

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    962         # Eager execution on data tensors.
    963         with backend.name_scope(self._name_scope()):
--> 964           self._maybe_build(inputs)
    965           cast_inputs = self._maybe_cast_inputs(inputs)
    966           with base_layer_utils.autocast_context_manager(

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in _maybe_build(self, inputs)
   2414         # operations.
   2415         with tf_utils.maybe_init_scope(self):
-> 2416           self.build(input_shapes)  # pylint:disable=not-callable
   2417       # We must set also ensure that the layer is marked as built, and the build
   2418       # shape is stored since user defined build functions may not be calling

/usr/local/lib/python3.6/dist-packages/tensorflow_addons/layers/wrappers.py in build(self, input_shape)
    113                 layer_config['config']['trainable'] = False
    114                 self._naked_clone_layer = tf.keras.layers.deserialize(
--> 115                     layer_config)
    116                 self._naked_clone_layer.build(input_shape)
    117                 self._naked_clone_layer.set_weights(self.layer.get_weights())

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/serialization.py in deserialize(config, custom_objects)
    107       module_objects=globs,
    108       custom_objects=custom_objects,
--> 109       printable_module_name='layer')

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    373                 list(custom_objects.items())))
    374       with CustomObjectScope(custom_objects):
--> 375         return cls.from_config(cls_config)
    376     else:
    377       # Then `cls` may be a function returning a class.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in from_config(cls, config)
    653         A layer instance.
    654     """
--> 655     return cls(**config)
    656
    657   def compute_output_shape(self, input_shape):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/core.py in __init__(self, units, activation, use_bias, kernel_initializer, bias_initializer, kernel_regularizer, bias_regularizer, activity_regularizer, kernel_constraint, bias_constraint, **kwargs)
   1139     self.kernel_regularizer = regularizers.get(kernel_regularizer)
   1140     self.bias_regularizer = regularizers.get(bias_regularizer)
-> 1141     self.kernel_constraint = constraints.get(kernel_constraint)
   1142     self.bias_constraint = constraints.get(bias_constraint)
   1143

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/constraints.py in get(identifier)
    283     return None
    284   if isinstance(identifier, dict):
--> 285     return deserialize(identifier)
    286   elif isinstance(identifier, six.string_types):
    287     config = {'class_name': str(identifier), 'config': {}}

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/constraints.py in deserialize(config, custom_objects)
    275       module_objects=globals(),
    276       custom_objects=custom_objects,
--> 277       printable_module_name='constraint')
    278
    279

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    360     config = identifier
    361     (cls, cls_config) = class_and_config_for_serialized_keras_object(
--> 362         config, module_objects, custom_objects, printable_module_name)
    363
    364     if hasattr(cls, 'from_config'):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)
    319   cls = get_registered_object(class_name, custom_objects, module_objects)
    320   if cls is None:
--> 321     raise ValueError('Unknown ' + printable_module_name + ': ' + class_name)
    322
    323   cls_config = config['config']

ValueError: Unknown constraint: Sum2One

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Can  you test:
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow_probability.python.layers.weight_norm import WeightNorm
from tensorflow.keras.constraints import Constraint
import tensorflow.keras.backend as K
class Sum2One(Constraint):
  def __call__(self, w):
    w = w*math_ops.cast(math_ops.greater_equal(w, 0.), K.floatx())
    return w/K.sum(w, axis = 1, keepdims = True)

l = K.ones((6,3))
x = WeightNorm(Dense(3, kernel_constraint = Sum2One()))
x(l)

Please subscribe also to https://github.com/tensorflow/addons/pull/1789#issuecomment-629185906 and see how it will evolve.

@bhack, thanks for the help! It works perfectly fine now.


Question about metric

Thanks for your work.
I have a question about the metric used in TransUNet. In the /TransUNet/trainer.py file, I found the metrics defined as follows:
def calculate_metric_percase(pred, gt):
pred[pred > 0] = 1
gt[gt > 0] = 1
if pred.sum() > 0 and gt.sum() > 0:
dice = medpy.metric.binary.dc(pred, gt)
hd95 = medpy.metric.binary.hd95(pred, gt)
return dice, hd95
elif pred.sum() > 0 and gt.sum() == 0:
return 1., 0.
else:
return 0., 0.
I think when pred.sum()==0 and gt.sum()==0, the dice score should be 1.0, not 0.0, and when pred.sum()>0 and gt.sum()==0, the dice score should be calculated by taking the complement of the original pred and gt.

Hi @Lycus99 , thanks for your interest!
For this metric, we directly adopt it from https://github.com/Beckschen/TransUNet, you can refer to this discussion: https://github.com/Beckschen/TransUNet/issues/39


Replace LGTM with Github Actions

Originating Project/Creator

Affected Component

Affected Architectures(s)
GitHub Actions

Related Issue(s)

Has Unit Tests (y/n)

Builds Without Errors (y/n)

Unit Tests Pass (y/n)

Documentation Included (y/n)

Change Description
Adds a Github Actions workflow that runs the security and quality static code analysis in order to phase LGTM out.
Rationale
lgtm.com will be shut down at the end of the year.
Future Work
Remove the LGTM configuration files once we're ok with phasing it out.
Comments
Results should be viewable once the checks are done. It's currently running all queries from the security-and-quality suite, let me know if we want to filter some categories out.

I implemented the recommended way of dealing with the Uncontrolled data used in path expression alerts. Let me know what you think. It's using normpath() to normalize the path (i.e. resolve special instructions like ../ etc...) and check that the requested resource is within logdir.
The two Wrong number of arguments in a call alerts seem to be expected in both cases, so we should probably dismiss those alerts. One has the LGTM suppression flag, the other one is a test case testing the raising of an exception.

@thomas-bc looks good to me.  Can we suppress-by-comment or is there some other way to do it?  Regardless, I agree with the proposed path-forward.


Update Processes section in README

The word list is no longer updated into chunks. The processes just check the next word in the list. The README should be updated to reflect this.

Implemented in #28


replace kafka->storagequeue and redis->blob

Signed-off-by: tanvigour<EMAIL_ADDRESS>Description
Please explain the changes you've made
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation

@halspang, please review when you get a  chance


Improve actor.itemTypes typing

Hello,
I don't know if typescript allow it but could it be possible to automatically types items of actor.itemTypes:
When I'm accessing to types like:
let weapons = itemsByType['weapon'];
// Then I want to access to the data without additional check:
weapon[0].data.data.<property available weapon>

If I create a globa.d.ts with the following
import {ConfiguredDocumentClass} from '@league-of-foundry-developers/foundry-vtt-types/src/types/helperTypes';

declare global {
    class Actor extends ClientDocumentMixin(foundry.documents.BaseActor) {
        get itemTypes(): Record<
            'weapon',
            Array<InstanceType<ConfiguredDocumentClass<typeof foundry.documents.BaseItem>> & {data: {type: 'weapon'}}>
        >;
    }
}

it seems to work but I'm getting an error on class Actor TS2300: Duplicate identifier 'Actor'
1- Why am I getting an error class Actor could you provide an example type merging with more complex case in: https://github.com/League-of-Foundry-Developers/foundry-vtt-types/wiki/A-Quick-Guide-to-Declaration-Merging ?
2- Is there a way to make it available for all types without having to define each overload manually ?

I haven't checked every detail yet but we could potentially make this work by using
    get itemTypes(): {
      [Key in foundry.documents.BaseItem['data']['type']]: Array<
        InstanceType<ConfiguredDocumentClass<typeof foundry.documents.BaseItem>> & {
          data: foundry.data.ItemData & { type: Key; _source: ItemDataSource & { type: Key } };
        }
      >;
    };

The above makes your example work, but I am not sure if there are any negative implications of that. We will need to check.
Regarding your questions:

You can't declare classes multiple times, you need to use an interface to use declaration merging here. However, that still won't work because successive declarations of the same property need to have the same type.
See above.

Tests are still passing, which sounds promising. Any opinions / comments on this @kmoschcau @FloRad?

Unfortunately, this changes breaks other stuff, see #1588, so we are going to revert it for now, until we find a solution.

Just and idea, maybe there could be some kind of config to enable this, like for game with
 let game: 'game' extends keyof LenientGlobalVariableTypes ? Game : Game | {};

Just and idea, maybe there could be some kind of config to enable this, like for game with  let game: 'game' extends keyof LenientGlobalVariableTypes ? Game : Game | {};

I don't think that's a practical solution. I agree that it's good to make stuff configurable for common situations but so far, this seems more like a niche thing, so I don't think it's worth the added complexity.
I'll just leave this issue open for now in case we find a solution, but we won't work actively on this.


Add missing supported regions

Description
The goal of this PR is to add all supported regions to the zappa.core.API_GATEWAY_REGIONS and zappa.core.LAMBDA_REGIONS lists per the relevant AWS documentation.
Relevant documentation:

https://docs.aws.amazon.com/general/latest/gr/rande.html#apigateway_region
https://docs.aws.amazon.com/general/latest/gr/rande.html#lambda_region

Coverage remained the same at 73.222% when pulling f3d403b8313e6f6545b8d22fc2715fd39a2be617 on mbeacom:update-regions into 3ccf7490a8d8b8fa74a61ee39bf44234f3567739 on Miserlou:master.


Check for existing archive folders first

K9 automatically created an IMAP folder called Archiv even though there is already a folder called Archive.
I expect K9 to use that folder instead of creating a new one.

Duplicate of #633


Testing

Have not tested this.

Superseded by #351.


How to calculate accuracy during training?

How to calculate accuracy during training?
I tried the below in the code in ssd7_training.ipynb file and didn't work. Any help would be awesome!
model.compile(optimizer=adam, loss=ssd_loss.compute_loss, metrics=['accuracy'])
    * Operating System - Ubuntu 18.04
    * Keras version - 2.2.5
    * TensorFlow version - TF 1.14

Object detection is not evaluated with accuracy , instead you can use meanAveragePrecision . You can check out my project to see how mAP can be evaluated after every epoch using keras callbacks

This is a regression problem, I think that's why we only can get a loss for the problem.
NOTE: You can add the following line :-
model.compile(optimizer=sgd,loss=ssd_loss.compute_loss, metrics=[tf.keras.metrics.Accuracy()])
But you will get accuracy as "00+e00".(because it's a regression problem.)


use travis homebrew addon

Thank you for your patch.
Because of this repository is a mirror of LuawebDriver of GitLab repository, I pushed your patch into a repository of GitLab.(refs[117d3c71498d56127162d2deeaecf702d0f1b6eb, 9a9ef8e0d5d3d75f78e3312aa3c502d3525d5497])


openbsd support for drogon

Together with Trantor PR104, this gets drogon going on OpenBSD. I haven't done extensive testing, but running webapp and webapp_test succeeds at least.

@an-tao Thanks! Didn't get a chance to sit down at a computer and fix it yesterday 😃


Double input in fifa 16(connect exclusivly doent work)

Is it possible to fix double input if fifa 16 (windows 10) I did it one time and it was working fine but today I cant do it I tried everthing...

anyone?

Have you selected "Hide DS4 Controller" - I'm sure that's what fixed it for me.

The Hide DS4 Controller setting in the settings tab within the DS4Windows program fixes this every time that I have ran into this issue.

Some time this problem i can see in The Witcher III in Windows 10, i don't know if a problem with windows 10 or problem with current DS4 version.


New package: PQPolygonSDK v0.1.0

Registering package: PQPolygonSDK
Repository: https://github.com/Paliquant/PQPolygonSDK.jl
Created by: @Paliquant
Version: v0.1.0
Commit: 49ebb56d3aeaa3ef7880690d5f650764e8de0a0c
Git reference: HEAD
Description: Software Development Kit for Polygon.io

Superseded by #51519, I presume


Add basic thorttler/exponential backoff policy for retry/Defination of throttling exception

Add basic thorttler/exponential backoff policy for retry/Defination of throttling exception
Signed-off-by: Dhwanil Patel<EMAIL_ADDRESS>Description
Reopening PR(https://github.com/opensearch-project/OpenSearch/pull/3527) against feature branch (feature/master-task-throttling).
[NOTE] : For review comments please visit older PR.
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Gradle check fails with:
WorkflowScript: 134: Arguments to "junit" must be explicitly named. @ line 134, column 21.
                       junit allowEmptyResults: true, '**/build/test-results/**/*.xml'

Waiting for fix(https://github.com/opensearch-project/opensearch-build/pull/2328) in build to be merged.

@Bukhtawar want to take on CRing this?

Should we just merge this? Goes on a feature branch. @reta your call

Should we just merge this? Goes on a feature branch. @reta your call

I think so, we reviewed and approved the original one


Tigera-operator helm chart unable to set csiNodeDriverDaemonSet resource memory/cpu requests & limits

I am unable to add resource limits/requests in my K8s cluster for csi-node-driver pods
Please see my config in my values.yaml file:
    csiNodeDriverDaemonSet: #Currently an issue and PR open to allow these values to be supported - https://github.com/projectcalico/calico/issues/8400
        spec:
            template:
                spec:
                    containers:
                    - name: csi-node-driver
                      resources:
                        limits:
                            memory: 100Mi
                        requests:
                            cpu: 75m

I am using projectcalico/tigera-operator  v3.28.0 helm chart.
Expected Behavior
When helm installing/upgrading for csiNodeDriverDaemonSet resource limits & request, the pods and daemonset should be configured afterwards.
Current Behavior
Helm detects the change but the change is not configured, when running helm upgrade:
Comparing release=calico, chart=projectcalico/tigera-operator
tigera-operator, default, Installation (operator.tigera.io) has changed:
...
+   csiNodeDriverDaemonSet:
+     spec:
+       template:
+         spec:
+           containers:
+           - name: csi-node-driver
+             resources:
+               limits:
+                 memory: 100Mi
+               requests:
+                 cpu: 75m
    imagePullSecrets: []
...

Affected releases are:
  calico (projectcalico/tigera-operator) UPDATED

Do you really want to apply?
  Helmfile will apply all your changes, as shown above.

 [y/n]: y
Upgrading release=calico, chart=projectcalico/tigera-operator
Release "calico" has been upgraded. Happy Helming!
NAME: calico
LAST DEPLOYED: Wed May 22 18:08:40 2024
NAMESPACE: tigera-operator
STATUS: deployed
REVISION: 13
TEST SUITE: None

Listing releases matching ^calico$
calico  tigera-operator 13              2024-05-22 18:08:40.510076 -0500 CDT    deployed        tigera-operator-v3.28.0 v3.28.0

UPDATED RELEASES:
NAME     CHART                           VERSION   DURATION
calico   projectcalico/tigera-operator   v3.28.0        50s

Yet after this is applied, the csiNodeDriver daemonSet & pods resource values are still not configured. Unfortunately, there are no errors in the logs indicating where the issue can be occurring.
There has been a couple of similar issues/PRs that claim this issue has been resolved, here are a couple:

https://github.com/tigera/operator/pull/3254
https://github.com/projectcalico/calico/issues/8400
https://github.com/projectcalico/calico/issues/7561

You've not used a correct container name, see the docs for the allowed container names (calico-csi or csi-node-driver-registrar).
I'm not sure why we still allowed csi-node-driver as an allowed value, I'm guessing it was so we didn't break anything for someone that has used that as a container name, it was allowed before and if we removed it from the allowed values then a previously functioning config would have resulted in errors.

You've not used a correct container name, see the docs for the allowed container names (calico-csi or csi-node-driver-registrar).
I'm not sure why we still allowed csi-node-driver as an allowed value, I'm guessing it was so we didn't break anything for someone that has used that as a container name, it was allowed before and if we removed it from the allowed values then a previously functioning config would have resulted in errors.

Perhaps the reason this cannot be set might be due to this line in the chart? https://github.com/projectcalico/calico/blob/master/charts/tigera-operator/crds/operator.tigera.io_installations_crd.yaml#L6869

Yeah it seems the source of the helm chart has not pulled in the proper updates from this repo. You can see the expected allowed values in this repo here https://github.com/tigera/operator/blob/master/api/v1/csi_node_driver.go#L28

I've created https://github.com/projectcalico/calico/issues/8883 in the repo where the helm chart comes from because this is a helm chart issue, not an operator functionality issue. I'm going to close this issue. If you believe there is an operator issue here please comment and I can re-open this.


Change in behavior of mask from 3.3.0 to 3.3.6

This inputmask for a US date
$(".ui-mask-basic-date").inputmask("m/d/y",{
	placeholder: "mm/dd/yyyy",
});

Works correctly  in 3.3.0, putting a placeholder in the input field, but in 3.3.6 it just puts 'm/d/Y' into the field. The mask cannot be typed over.

Hi @mnewnham ,
I should have followed the semantic versioning and bumped to version 4, ... but I didn't.
I removed the d,m,y definitions as they caused many confusion for some in regards with the available aliases for date.  So or you use some of the date aliases or put the definitions back in your code and run the same mask.