Poor HTTP performance vs Node.js

Include the following information (where applicable):

What operating system are you running?

OS X

Are you running any CPU architechute out of the normal?

No

What platform/language are you using?

C++
Node.js

Are you using SSL, if so, how?

SSL is not part of the equation

What version of uws are you using?

Release v0.14.4

What features of µWS are involved in the issue?

Server support (HTTP)

How does the issue reproduce?

All the time

Can you show me some short code that triggers your issue or any other important information?

Yes

I am trying to build a "Hello world" server in uWS and Node.js for comparison purposes.
Here is my uWS server:
#include <uWS/uWS.h>

int main() {

  uWS::Hub h;

  std::string const document = "
Hello, world.
";

  h.onHttpRequest([&document](uWS::HttpResponse *res, uWS::HttpRequest req, char *data, size_t length, size_t remainingBytes) {
    res->end(document.data(), document.length());
  });

  h.listen(3000);
  h.run();

  return 0;
}

I am compiling in LibUV mode with -O3.
Here is my Node.js server:
'use strict';

const express = require('express');

const app = express();

const doc = '
Hello, world.
';

app.get('/', (req, res) => {
  return res.send(doc);
});

app.listen(8080);

Here are the results for uWS:
$ ab -n 10000 -c 100 http://<IP_ADDRESS>:3000/
This is ApacheBench, Version 2.3 <$Revision: 1757674 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking <IP_ADDRESS> (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:
Server Hostname:        <IP_ADDRESS>
Server Port:            3000

Document Path:          /
Document Length:        23 bytes

Concurrency Level:      100
Time taken for tests:   200.560 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      620000 bytes
HTML transferred:       230000 bytes
Requests per second:    49.86 [#/sec] (mean)
Time per request:       2005.601 [ms] (mean)
Time per request:       20.056 [ms] (mean, across all concurrent requests)
Transfer rate:          3.02 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    5   1.7      5      11
Processing:  1989 2000   4.2   2000    2015
Waiting:        1    3   0.7      3       7
Total:       1992 2006   5.6   2005    2020

Percentage of the requests served within a certain time (ms)
  50%   2005
  66%   2008
  75%   2010
  80%   2011
  90%   2013
  95%   2015
  98%   2016
  99%   2017
 100%   2020 (longest request)

Here are the results for Node.js:
$ ab -n 10000 -c 100 http://<IP_ADDRESS>:8080/
This is ApacheBench, Version 2.3 <$Revision: 1757674 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking <IP_ADDRESS> (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:
Server Hostname:        <IP_ADDRESS>
Server Port:            8080

Document Path:          /
Document Length:        21 bytes

Concurrency Level:      100
Time taken for tests:   4.046 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      2210000 bytes
HTML transferred:       210000 bytes
Requests per second:    2471.72 [#/sec] (mean)
Time per request:       40.458 [ms] (mean)
Time per request:       0.405 [ms] (mean, across all concurrent requests)
Transfer rate:          533.45 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        0    0   1.0      0      13
Processing:    14   40  16.1     31     113
Waiting:       12   40  16.1     31     112
Total:         25   40  16.2     32     114

Percentage of the requests served within a certain time (ms)
 50%     32
 66%     39
 75%     46
 80%     50
 90%     64
 95%     76
 98%     87
 99%    102
100%    114 (longest request)

As you can see, Node.js is considerably faster with 2471.72 requests/sec vs 49.86 requests/sec for uWS! This is very surprising given that they use the same underlying library (libuv).
Is this expected? Perhaps there is something wrong with my server implementation?

Don't benchmark http 1.0, benchmark http 1.1 (wrk is a good bench)

Try wrk2

@alexhultman So should I avoid uWS as a solution for HTTP 1.0?

@njlr I can't answer that for you, but Internet Explorer 4 compatibility is maybe a little too anal? Http 1.1 is already 10 years old now.

Correction: 20 years

OK, fair enough. So Node.js is faster for HTTP 1.0, but uWS is only interested in HTTP 1.1.
I will take a look at some HTTP 1.1 benchmarks 👍

@alexhultman
Since HTTP 1.1 is generally speaking a subset of HTTP 1.0 it is a valid performance test.
As far as i can see, a server that only sends 'Hello World' cannot take any advantages of HTTP 1.1 to improve it's performance.
Note that unlike wrk, ApacheBench is not testing throughput but performance of connection processing.
It tests how quickly can an http server write a message, accept and close a connection.
The provided test shows this library is not able to process many connections per second.
After a quick look in the sourcecode it appeared to me that there is no logic for handling the
'Connection: keep-alive' header.
My guess would be that the current implementation assumes 'keep-alive' and timeouts when no further request are performed by the client. This behavior might be acceptable if 99% of clients are upgrading to a Websocket connection but detrimental for a static file server.

The provided test shows this library is not able to process many connections per second.

No, the apache bench (ab) test shows that this lib doesn't care about anal Connection: close behavior since Http 1.1 has keep-alive by default. In actuality it already sent the data and is simply timing out before it closes the connection. Anyways, all of this info is a repeat of another issue report.

#511


Add Compound Parameters

See examples for more.
Fixes #110

@stevendborrelli that TODO codeclimate is complaining about is the one that it'd be really helpful if you'd fill in


Add bin to composer.json and remove extension of the bin

Hi,
This tool looks like something very usefull.
You should add the bin to composer.json and remove the extension of the bin.
{
"bin": ["bin/pagepacker"]
}
Adding the bin makes it usefull as a requirement for other packages and removing the extension makes it platform independent (you could add a .bat file for instance). Also see Composer vendor-binaries.
Line 3 of bin/pagepacker should be changed to work with both ./bin/ and ./vendor/bin/

Hi Giso,
Thanks for your feedback. I've updated the package as you suggested. Could you check if this works for you?

Hi Ron,
It is not working.
I tested the paths using the following code which works for me
// installed via composer symlink?
if (file_exists($autoload = __DIR__.'/../../../autoload.php')) {
    require_once $autoload;
    // installed via composer ?
} elseif (file_exists($autoload = __DIR__.'/../autoload.php')) {
    require_once $autoload;
} else {
    require_once __DIR__.'/../vendor/autoload.php';
}

A test can be performed using the following commands
composer init -n
composer config repositories.ronrademaker vcs https://github.com/RonRademaker/pagepacker
composer require ronrademaker/pagepacker dev-master
php vendor/bin/pagepacker

Hi Giso,
Thanks, that looks like a much better test than the one I applied from the git clone.
I made the change you suggested, if you have any other improvements feel free to make an issue or a pull request.


Type of routing_exception seems too broad

The type of flask.request.routing_exception seems to be too broad. What I can see in your code is, that this field is only se in the case of a werkzeug.exceptions.HTTPException.
Therefore the type of flask.request.routing_exception is Optional[Exception] but could be Optional[HTTPException]
Environment:

Python version: 3.7.1
Flask version: 2.0.1

Other exceptions could be raised. Current annotation seems fine.


Updating to v4.14.2 of bch-js

This uses a new hydrateUtxos() call that uses a waterfall validation method.

:tada: This PR is included in version 2.1.6 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:


Give challenge authors better control test messages

Is your feature request related to a problem? Please describe.
Challenge tests only report success or failure, with no specific information about why a test failed.  If a test is moderately complicated (i.e. has multiple assertions) and can fail at several points, a user may have difficulty understanding what they did wrong.
Describe the solution you'd like
Allow a test writer more control over what a test reports.  For example, if they use an assertion like
assert(expression, "Customised message")

Then, if that fails, "Customised message" should appear in the fcc console.  However, if they write.
assert(expression)

The default message should appear.  Possibly we can achieve this when we handle AssertionErrors, but I haven't confirmed this.
Describe alternatives you've considered
Encourage contributors to write smaller tests.  Generally this is a good idea, but it's not always practical.  Particularly not for backend tests which can often naturally have a few steps.
Additional context
Follows up a  conversation with @SaintPeter in discord

For the projects, there are many steps that require an initial setup in order to get a known id.  For example, with the exercise tracker, you have to create a user every time so you can get an id to add an exercise to it.  For the Personal Library, you need to create a book (or books) before you can add comments.  In order for the tests to be atomic, this needs to happen for each test.
In general, we need to be able to give good feedback about what specifically is failing.
I don't have an issue with breaking the tests out into more/smaller steps, but there is some irreducible complexity.
With regards to the message - The AssertionError has a .message value which is either the default message ("x should be y") or the custom message.  The key would be capturing that it was an AssertionError as opposed to an XHRError (or whatever those are).

Thank you for reporting this issue.
This is a standard message notifying you that the issue has become stale, so we are going to close it.
If you think we're wrong in closing this issue, please request for it to be reopened. Thank you and happy coding.


Spectator instead of Servo (deprecated) ?

As Servo is deprecated and no longer maintained, any plan on using spectator instead of servo library ? I still see there are references to Servo in ThresholdLevelsMetric.java
I am not using Servo anyway. If there is a way to disable Servo instead, that works for me too. I am currently having a blocker due to the usage of Servo
https://github.com/Netflix/servo/issues/443

@ravikancherla thanks for raising this issue. I followed your links and it looks like you have made a PR against servo that resolves this issue. Are you able to pin forward your servo version and achieve your goals?
We will look at updating the eureka dependency versions on servo to the latest soon.
(On the matter of moving to spectator, that is something we wanted to look at but unfortunately is not at the top of our priorities at the moment).


SyntaxError: Unexpected token  in JSON at position 0 on Linux.

A file that actually worked on Windows doesn't work on Linux.
SyntaxError: Unexpected token  in JSON at position 0
at JSON.parse ()
at Y.json (/home/remuring/Desktop/remuring/Music/node_modules/petitio/dist/index.js:30:574)
at ke.json (/home/remuring/Desktop/remuring/Music/node_modules/petitio/dist/index.js:30:2499)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Node.makeRequest (/home/remuring/Desktop/remuring/Music/src/Extenders/Node.js:39:12)
at async /home/remuring/Desktop/remuring/Music/node_modules/erela.js/dist/structures/Manager.js:152:25
When using player.serach() in Linux, the above error is returned
I am using the Build version of erela.js
Is this a problem with my OS? Has anyone got the same error as me on Linux?
OS : Ubuntu 20.04.5 LTS
NodeJS : 16.17.1

its using petitio, the library was removed and replaced with unidici, you have to install erela.js via github


Add a rule to test RDS instance sizes against the engine and licensemodel

Issue #, if available:
Fix #270
Description of changes:

Check RDS instance types against the region, engine, and license model.  Only checks if they are hardcoded values and not REFed or IFed.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

I'm a little torn on this.  Coming up with a really good defined list of valid instance types is a little difficult.  I would appreciate extra validation of my work on this list.


Add workflow_run as eventName

This PR adds workflow_run to the events that can trigger this action.
That way #68 can be solved by using a two staged actions setup like this for example:
First a pytest action that generates pytest-coverage.txt and pytest.xml and uploads these amoung a textfile with the PR number as workflow artifacts.
---
name: Run unit tests
on: # yamllint disable-line rule:truthy
  push:
    branches:
      - main
  pull_request:
jobs:
  pytest:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - name: Install pytest
        run: pip install pytest pytest-cov
      - name: Run pytest
        # yamllint disable rule:line-length
        run: |
          set -o pipefail
          PYTHONPATH=. pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=luxtronik tests/ | tee pytest-coverage.txt
        # yamllint enable rule:line-length
      - name: Save PR number and coverage results
        run: |
          mkdir -p ./pr
          echo ${{ github.event.number }} > ./pr/PR-number.txt
          cp ./pytest-coverage.txt ./pr/pytest-coverage.txt
          cp ./pytest.xml ./pr/pytest.xml
      - uses: actions/upload-artifact@v4
        with:
          name: pr
          path: pr/

And the second stage that actually creates the comment.
---
name: Comment coverage report on the pull request
on: # yamllint disable-line rule:truthy
  workflow_run:
    workflows: ["Run unit tests"]
    types:
      - completed

jobs:
  post-coverage-report:
    runs-on: ubuntu-latest
    if: >
      github.event.workflow_run.event == 'pull_request' &&
      github.event.workflow_run.conclusion == 'success'
    steps:
      - name: 'Download artifact'
        uses: actions/github-script@v7
        # yamllint disable rule:line-length
        with:
          script: |
            var artifacts = await github.rest.actions.listWorkflowRunArtifacts({
               owner: context.repo.owner,
               repo: context.repo.repo,
               run_id: ${{github.event.workflow_run.id }},
            });
            var matchArtifact = artifacts.data.artifacts.filter((artifact) => {
              return artifact.name == "pr"
            })[0];
            var download = await github.rest.actions.downloadArtifact({
               owner: context.repo.owner,
               repo: context.repo.repo,
               artifact_id: matchArtifact.id,
               archive_format: 'zip',
            });
            var fs = require('fs');
            fs.writeFileSync('${{github.workspace}}/pr.zip', Buffer.from(download.data));
        # yamllint enable rule:line-length
      - name: Unzip artifact
        run: unzip pr.zip
      - name: Read the PR number from file
        id: pr_number
        uses: juliangruber/read-file-action@v1
        with:
          path: ./PR-number.txt
      - name: Publish pytest coverage as comment
        uses: bouni/pytest-coverage-comment@workflow_run
        with:
          issue-number: ${{ steps.pr_number.outputs.content }}
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml

Important is that the name of the first worflow name: Run unit tests must be identical to workflows: ["Run unit tests"] in the second workflow.
@asumagic did actually post a solution like the above in #68 but that failed because workflow_run was not an allowd trigger.
The underlayng problem why the comment can not be created directly from a PR that comes from a forked repo is described in GitHubs Blog: https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

😏 The My Pytest Coverage Comment / Live Test (pull_request) action failed exactly because of #68 which will be solved by this PR.

Looks amazing, thank you so much for your contribution! 🚀 Your PR addressing such a complex problem is truly impressive and immensely valuable to this action. Will do a more deep review in the next few days and will release a new version  :)

To be honset, It took me dozens of trail an error attempts on one of my projects and with each iteration I though, now I finally fixed it 😅
I was so happy to see that you already had the workflow_dispatch case in you code, otherwise I would have been completely lost.
An thank you for providing this action by the way 👍🏽

Big thanks again for the effort, I release new version with your code
https://github.com/MishaKav/pytest-coverage-comment/releases/tag/v1.1.51
You can use it:
- name: Pytest Coverage Comment
  uses: MishaKav/pytest-coverage-comment@main

or use a specific version
- name: Pytest Coverage Comment
  uses<EMAIL_ADDRESS>


Remove ipld/hamt source code

Summary of changes
Changes introduced in this pull request:

Remove ipld/hamt source code

Reference issue to close (if applicable)

Closes #1548
Other information and links

2K deleted lines, nothing added. The only joy greater than writing code is deleting code.


Metal3Data and IPAddressClaim objects have unpredictable names

What steps did you take and what happened:
When creating a machine deployment, the Machines, Metal3Machines and Metal3DataClaims use names with a random suffix. When the Metal3DataClaims are fulfilled, those Metal3Data objects aren't using the same randomized names, but incremental ones instead. The IPAddressClaims that get created while creating Metal3Data objects also use those incremental names.
What did you expect to happen:
The Metal3Data and IPAddressClaim objects should use the same names as the the Metal3Machine they belong to. This makes it much easier to determine to which machine they belong to, and is also more in line with the behavior of Kubernetes.
Anything else you would like to add:
We noticed this issue while developing and testing a custom controller based on ip-address-manager to interface with our internal IPAM.
When deleting a Machine (e.g. to redeploy it), everything that belongs to the Machine gets deleted, including the Metal3Data and IPAddressClaim objects. CAPI then immediately creates a new Machine to keep the desired count of Machines. This leads to a new Metal3Data and IPAddressClaim to get created. But, since they use an incremental name, they are almost identical to the previous ones.
Environment:

Cluster-api version: 0.3.20
Cluster-api-provider-metal3 version: 0.4.2

/kind bug
cc @MaxRink

We are using incremental names for the Metal3Data to ensure that the index of the Metal3Data is unique. This way, two metal3Data cannot be created with the same index, Kubernetes would refuse it. The ipclaim then uses the same name as the metal3Data.
Changing this behavior would make it complicated to ensure that the index (or ip address for the ipaddress object) remains unique. We considered alternatives, but did not see one without race conditions.


Annotations

Adds annotations from template to rendered Applications
Makes cluster annotations available for templating (Closes #31)

Would it be feasible to add a small unit test just to ensure we keep it working? It looks relatively straightforward to add in to cluster_test.go. If time is an issue we can merge and add later.


Remove [GT]PREL_[IS] internal binutils relocations

These have never been emitted by GCC or Clang, nor has there ever been
assembly syntax in either toolchain to create them. These are fake
internal relocations inside binutils used as part of its implementation
of linker relaxation, and so should not be part of the psABI. They
should be free to be reallocated for psABI uses (which can include
reintroducing them, if there is a need to expose them to assembly and
code generation).
Closes: #160

Sounds right to me.

LGTM

LGTM

I think that's enough LGTMs!


h36m processed frame number mismatch

https://github.com/Arthur151/ROMP/blob/db7e2a1df1df104909e6b137eaaed653ee433517/romp/lib/dataset/preprocess/h36m_extract_frames.py#L40
Hi there,
Thanks for the great work! I'm trying to train the model on the h36m dataset, and I downloaded the original h36m dataset to process using this script. However, the generated frame number is mismatched with the one in your processed h36m dataset.
Firstly, this dex<20 condition seems not really being applied when you processed the data, otherwise, each video will have at most 20 frames.
Then, even if I removed this condition, the generated frame number for each video is still different from your processed h36m data (on google drive). For example, the video Purchases.60457274.mp4 for S1 generates 231 frames (index from 0-230) using this script (after moving the dex<20 condition). However, in the google drive processed version, there are only 230 frames (index 0-229). For reference, in the original 50 FPS video, the lasting time is 23.2s, and the frame number is 1161.
The similar thing happened for other videos as well, but the delta number is not always 1. For instance, the S1's WalkTogether.60457274.mp4 video has a 2 frame difference (245(your processed) vs 247(actual)).
Since I'm trying to train an event-camera time surface-based model that has a high requirement for time accuracy, I hope you can help clarify where is this missing frame, should I drop some frame at the beginning, or drop the last K frame, or somewhere in the middle. If possible, could you please share the exact processing script you used for generating the current processed h36m dataset? I really appreciate it.

@miracleyoo
Really sorry for the inconvienence this problem caused.
I can't find the exact code for extracting the frame. This code is provided by other developers who use ROMP.
I just remember that I dropped some frames at the beginning of each sequence.
Maybe you can extract all frames of a video and match them with the processed h36m frames to get the precise frame time.

@miracleyoo
in case zou have calculated the smpl models for h36m. could you tell me,  which camera parameters did you utilize for further usage. Are they the original camera parmaters or some other as per ROMP.
I am waiting for your response


Any plans for GSOC 2025?

If yes, where can I submit my ideas?

Consider the program timeline. You'll realize that GSoC was not announced yet, and the Org application phase will start in approx. two months.  Also consider the Contributor Guide, which invites you to join the Matrix channel for further discussion.


Add Tab Support to Formatter

All of the kernel *f functions currently lack support for \t. This should be remedied.

The tab character is not a format character. We only need to support printing it to the TTY.


TestRotations is missing in the Python folder.

@lorensen Please check carefully!
Also print statements now work in Python 3.x

Thanks!


[amazon-corretto] Add 20

Draft, not yet announced on https://aws.amazon.com/corretto/.

Corretto 20 visible on https://docs.aws.amazon.com/corretto/ and was officially announced on 21/03 : https://aws.amazon.com/about-aws/whats-new/2023/03/amazon-corretto-20/.


Couldn't start client Laravel blade lsp

I get the following error:
[Error - 12:51:21 PM] Starting client failed
Launching server using command c:\DevTools\code-insiders\data\extensions\haringsbe-haringsrob.laravel-blade-livewire-lsp-1.2.1\laravel-dev-tools failed.
Am I missing something or am I doing something wrong in the setup?
Thank you very much!
Kind regards,
Florian

Hey @furopi,
I have not really used this with vscode lately. The code may be out of date and need to be build from scratch in order to work.

Most likely this will be an issue with a laravel version mismatch.

To me it looks like Windows doesn't know how to execute the binary file "laravel-dev-tools" in the extension. How can I tell windows to execute this file with the php interpreter or any file beginning with "#!/usr/bin/env php"?
Thank you! 🙃

Oof, good question, I honestly do not know :). Have you considered using docker WSL using vscode remote?

Closing this for now, I feel vscode remote is the way to go until I somehow make an executable for windows (if ever). This is a non goal at this stage of the project.

@haringsrob
You actually don't need to compile a new version, you just have to exec the file with the PHP executable. And maybe add a settings option to specifiy the executable path for systems where the executable is not included in the path variable.
It's kind of sad that there is no Windows support, what should be an easy fix :_(

Hey @furopi I am open to a PR if you have an idea on how to add support.

Well I'm not familiar with you code, @haringsrob , neither with github PR's but what you need to change is executing the devtools binary with "php laravel-dev-tools" instead of "./laravel-dev-tools" when running on Windows. At least from what I have tested the devtools binary runs fine than...
Hope I could help here in a little way!


Dev/1.0.0

@it-hms @oliver-walker-hms None of the previous commits were changed, but I just pushed to add a new commit (https://github.com/hms-networks/flexy-cumulocity-connector/pull/3/commits/11d55f213ef2740ef2fa350ec5703dc874b39b1a) prior to release that addresses MQTTS support.

I only got through update firmware info reporting

@it-hms I restructured the QUICK-START.md document so that registration is performed prior to configuration.
Additionally, I added a new section that describes the download/extraction of the release zip. I did some testing yesterday and found that it was unclear where the files come from without that section.

@it-hms @oliver-walker-hms Pushed a small new commit to add QUICK-START.md to releases. This did not touch/affect any previous commits, and was added to the top of the commit list under the release one.


Microsoft.Azure.Cosmos references many out of support and vulnerable package versions.

Describe the bug
Microsoft.Azure.Cosmos references many packages which are out of support and vulnerable.
To Reproduce
Create a new project with the latest .NET 9.0 preview SDK which includes NuGet audit for security vulnerabilities.  Add a reference to <PackageReference Include="Microsoft.Azure.Cosmos" Version="3.43.0"/> and restore the project.
Expected behavior
No warnings when restoring.
Actual behavior
The following warnings occur:
    C:\scratch\azureCosmos\azureCosmos.csproj : warning NU1903: Package 'Newtonsoft.Json' 10.0.2 has a known high severity vulnerability, https://github.com/advisories/GHSA-5crp-9r3c-p9vr
    C:\scratch\azureCosmos\azureCosmos.csproj : warning NU1903: Package 'System.Net.Http' 4.3.0 has a known high severity vulnerability, https://github.com/advisories/GHSA-7jgj-8wvc-jh57
    C:\scratch\azureCosmos\azureCosmos.csproj : warning NU1903: Package 'System.Text.RegularExpressions' 4.3.0 has a known high severity vulnerability, https://github.com/advisories/GHSA-cmhx-cq75-c4mj

Environment summary
SDK Version: 3.43.0
.NET SDK: 9.0.0-preview.7.24405.7
OS Version: Windows 11 23H2
Additional context
See https://learn.microsoft.com/en-us/nuget/concepts/auditing-packages
https://devblogs.microsoft.com/nuget/nugetaudit-2-0-elevating-security-and-trust-in-package-management/
### Tasks
- [ ] https://github.com/Azure/azure-cosmos-dotnet-v3/pull/4790
- [ ] https://github.com/Azure/azure-cosmos-dotnet-v3/pull/4775
- [ ] https://github.com/Azure/azure-cosmos-dotnet-v3/pull/4756
- [ ] https://github.com/Azure/azure-cosmos-dotnet-v3/pull/4693
- [ ] https://github.com/Azure/azure-cosmos-dotnet-v3/pull/4694
- [ ] https://github.com/Azure/azure-cosmos-dotnet-v3/pull/4676

We cannot update the dependency on Newtonsoft.Json, because there are no patched versions without breaking changes, and we can't force that breaking change on our customers :(
For System.Net.Http and System.Text.RegularExpressions, we should investigate the path they are coming in through and either update an intermediate dependency, or consider pinning to a higher version.
Tagging also @kirankumarkolli and @kundadebdatta.

'System.Text.RegularExpressions' seems like a transitive dependency through 'Newtonsoft.Json'
Unsure of source of System.Net.Http dependency (we do use HttpClient but not explicitly listed in package spec.
'Newtonsoft.Json' 10.0.2 vulnerability is address through a code fix, unfortunately upgrading to the suggested version is a breaking change.
Except 'Newtonsoft.Json' we can at-least fix others as new dependencies directly to override.
@ericstj thoughts on how to way to let analyzer to suppress for Newtonsoft.Json?

You can update the other dependencies, NETStandard.Library and System.Text.RegularExpressions.
FWIW many of your customers are going to be in this same predicament since they'll see this vulnerability warning for Newtonsoft.  Don't you have a major version where you can choose to update Newtonsoft?  Also - have you reached out to @jamesnk to see if he'd be able to produce a 10.0.4 build that has a fix for the CVE without other breaking changes?

Thank you other dependencies are addressed will ship part of next release.
Our next major version will remove dependency on Newstonsoft and just use STJ as default serializer.
Will follow-up with James.

STJ is going to have the same problem if you stay on older versions of it: https://github.com/dotnet/runtime/issues/104619

I believe the STJ in here is inherited via Azure.Core, which currently is a 6.x min (which is not on the official list of versions covered by that cited issue? (Of course your overall point is not necessarily invalidated by that)

Our next major version will remove dependency on Newstonsoft and just use STJ as default serializer. Will follow-up with James.

Is this ever going to actually get fixed?  I've been waiting on this for years.
The 3.46.0 change only makes things worse.  Now I have to have a direct dependency on Newtonsoft and can no longer justify it as an indirect reference through the Cosmos package.

I agree with @AIZ-THerring. IMHO Nuget Microsoft packages have to migrate to STJ if they use Newtonsoft. It may be a long step but it is necessary to avoid depending on third party packages that do the same thing as a proprietary package.


image_to_vm: add initial hetzner cloud support

Along with:

https://github.com/coreos/ignition/pull/667
https://github.com/coreos/coreos-metadata/pull/125
https://github.com/coreos/coreos-overlay/pull/3490
https://github.com/coreos/container-linux-config-transpiler/pull/164

This will greatly enhance working with CoreOS on the Hetzner Cloud.

@LKaemmerling Would you mind taking a look at this from the Hetzner Cloud team? Feel free to suggest changes.

This LGTM.  How does the OS image get uploaded to the cloud?

Okay, so have you tested your changes with Hetzner Cloud?  Does the Hetzner team intend to add Container Linux as an available OS?

I indeed did test these changes in the Hetzner Cloud.
I can't speak for the Hetzner team of course, but I sincerely hope they'll add it as an available OS as soon as there is native support for booting CoreOS in their environment. They are actively developing and supporting Kubernetes related tools (i.e. https://github.com/hetznercloud/csi-driver/) so I can only assume they are welcoming these changes.


ROX-22912: bump google protobuf

Description
This should fix GO-2024-2611, which has been found by our go-scan-binaries CI job. See https://pkg.go.dev/vuln/GO-2024-2611 for more info.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
Here I tell how I validated my change
go scan test passes
Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

/test go-scan-binaries

I think this line needs to be removed so go-scan-binaries runs on PRs: https://github.com/stackrox/stackrox/blob/20e00fb02a5081a3081ab7e8643e2a09cd36cfb3/.github/workflows/build.yaml#L575

/retest

duplicated by https://github.com/stackrox/stackrox/pull/10270


net_class: Split device discovery

This splits the discovery of device/interfaces out from the main
NewNetClass() function.
I want to do this b/c on systems that have a lot of devices in
/sys/class/net/* calling parseNetClassIface in serial takes a lot of
time. On my system with 55 devices it currently takes about 150ms.
By splitting the device discovery I can call parseNetClassIface using
a worker pool and process them much faster. NewNetClass is updated to
use the new NetClassDevices() method so its behaviour does not change.

Seems like a good idea to me. We did a similar split and used golang.org/x/sync/errgroup in sysfs/system_cpu.go.

Follow-up question; with this landing should I update NewNetClass() to use an errgroup or do we want to keep it as-is here and implement that in the node exporter instead?

I think we should implement the errgroup here, to keep the node_exporter's interface the same.

Fixed to use filepath.Join now.


can't use the app on a Samsung device

Samsung Galaxy Note 9 running A14 (OneUI 6.1) and I can't select the app as a voice input, it is not on the list, only Samsung and google voice input are available to select

Do other speech recognition apps show up there? Does it show up if you turn off Arabic?

Okay so apparently I had to set the voice input to none and now the keyboard switching button is visible again and I can select transcribo
Well that's unfortunate because the on device arabic T2S from Samsung is really good, well that's something they need to fix, if I enable voice input, the keyboard switcher button gets replaced by the voice input button

Oh no I was just asking you to try to disable Arabic just in case that's related but it probably isn't needed? So is everything working fine right now?

Oh sorry forgot to mention that, yes I tried disabling Arabic and it unfortunately didn't fix it, anyway yes everything is working fine now, thanks!

Ah I see that's weird on them that they replace the keyboard switcher? Maybe try pressing or holding the globe instead it might have the same function?

Maybe try pressing or holding the globe instead it might have the same function?

Damn you are right

Yayyy lol

And also I figured that these voice input settings are for samsung voice input not global settings for voice input so that's why disabling Arabic didn't do a thing because what it did is just disable Arabic for samsung voice input, when I select google voice input all of these options disappear and now I'm wondering why only samsung and google voice inputs are available, why Transcribe isn't there, is it hardcoded or something need to change in Transcribe for it to be an option🤔
Anyway I can use Transcribe now so yayyy

Yeah that's really weird, seems Samsung might not let you select others?

I don't think its anything to do with Transcribro since it works fine on AOSP based OS' and it follows the Android docs on how to expose itself as a voice input service.


added nightly builds

Signed-off-by: tyvision<EMAIL_ADDRESS>nightly builds added
Signed-off-by: tyvision<EMAIL_ADDRESS>

Description of the Change
Added implementation of nightly builds:
Scheduled at 23:00 UTC every day with the following parameters

build for platforms:

Linux
MacOS
ARMv8

without Doxygen
without bindings

Benefits

Possible Drawbacks

Builds for all branches will be triggered, but then stopped except for 'develop'.
as soon as nightly build is fully configured and managed by Jenkinsfile, if any of the nightly build parameter needs to be changed --> new PR should be opened

Usage Examples or Tests [optional]

Alternate Designs [optional]

Maybe nightly -> night-time
I have an association of unstable packages for nightly builds (e.g firefox)


Uncaught Error: tunneling socket could not be established, statusCode=503

[Enter steps to reproduce:]

...
...

Atom: 1.21.0-beta1 x64
Electron: 1.6.9
OS: Unknown Windows version
Thrown From: Atom Core
Stack Trace
Uncaught Error: tunneling socket could not be established, statusCode=503
At stream.js:74

Error: tunneling socket could not be established, statusCode=503
    at ClientRequest.onConnect (/app.asar/node_modules/tunnel-agent/index.js:166:19)
    at Object.onceWrapper (events.js:290:19)
    at emitThree (events.js:116:13)
    at ClientRequest.emit (events.js:194:7)
    at Socket.socketOnData (_http_client.js:394:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:551:20)

Commands
     -0:34 settings-view:uninstall-packages (div.tool-panel.tree-view)

Non-Core Packages
atom-ide-ui 0.3.3
atom-maven 1.3.0
autocomplete-java 1.2.7
docblockr 0.11.0
highlight-selected 0.13.1
java-generator 2.2.3
java-importer 1.4.1
platformio-ide-terminal 2.6.0
simplified-chinese-menu 5.3.4

Thanks for the report! Can you confirm what were you doing at the time of the error and if it's reproducible?

Happened to me few moments ago, I'm currently using Manjaro Linux 17.0.5 and the error poped while browsing the themes

Was the error temporary @juanicondina?  And are you behind a proxy?

@rsese I got the same error and I am behind a shadowsocks proxy.

This happens whenever I run Atom the first time after a reboot.
If I close and open Atom again the error doesn't pop up.
I am behind a proxy too.

This happens too，when I try to update a package.
I am behind a shadowsocks proxy in windows 10 version 1803.


[opt](inverted index) add inlist condition handling to compound

Proposed changes

Previously, the compound did not support the inlist condition, which could impact performance if an inverted index was created.


Custom dropdown Data Source

Hi,
i am trying to achieve a custom data source for dropdown components, which will help dropdown to be populated from a different data source rather than defining in the editor.
please have a look at the below changes, confirm if this is the right place or will it cause some other problems.
Object.defineProperty(n.prototype, "choices", {
get: function () {
debugger;
//"dataServiceURL" is a new property in dropdown property bag.
if (this.dataServiceURL != undefined) {
this.choicesValues = []; //reinitialize the Array
//loading with a different source.
this.choices = ["1|1st modified Item", "2|2nd Modified Item"];
console.log(this.choicesValues);
return this.choicesValues;
} else {
return this.choicesValues
}
},
set: function (t) {
debugger;
e.ItemValue.setData(this.choicesValues, t)
}

Please refer to this issue: https://github.com/andrewtelnov/surveyjs/issues/67

Hi,
I have implemented the first variant of the feature. Could you check it out?
I am going to work on property editor now. I also thinking about having a parameter support in the url. For example, when you have the employee list and you may show it only after the person select the department. So you do not show the full list, but filtered by a parameter, previous answered question value. Do you need this?
Thank you,
Andrew

Please, take a look at: http://surveyjs.org/examples/questiontype-dropdownrestfull.html

Can i call a function in js instead of URL ?

@PankajKumawat125 you may get a question and set the choices:
var question = survey.getQuestionByName("myQuestionName");
question.choices = [1, 2];
//or
question.choices = [{value: 1, text: "item 1"}, {value: 2, text: "item 2"}];

Thank you,
Andrew

@andrewtelnov  Perfect. work fine.
Can you help me with one more thing ?
If i want to pre-select one of the value and show the dropdown then How do i do that?

@PankajKumawat125 survey.setValue("YourQuestionName", defaultValue); example
or use the defaultValue property in the question, defaultValue: yourDefaultValue.
Please go through our demos. You are asking very basic questions.
Thank you,
Andrew

@andrewtelnov  Well, I have already done that. i was just going to delete my 2nd question.
and
In this case it was fetching dynamically, that is why i asked you this question!
Anyways thank you so much for your help


IGNITE-17915 Implementation of plain osgi and blueprint injection registry

Changes to be applied after apache/ignite#10337 gets merged.

@SammyVimes Can you please have a look on this?


Fix CI with new annotation library

Signed-off-by: Konstantinos Kallas<EMAIL_ADDRESS>

I am leaving this PR open until we resolve issues with the annotation-library dependency. At the moment it doesn't work.


View Menu, Grab & Go Cooler

As a user, I want to be able to order things from the Grab & Go cooler so that I can reserve them for myself.
DONE:
Items display name, price, punch type, and if its in stock. in a text based terminal thingy
Estimate: 3

Duplicate issue


Fix debug_info logic to avoid ValueError on empty _debug_info

In some situations the Template '_debug_info' can be empty. This causes the real exception to be masked by a new ValueError exeception:
  File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 1045, in generate
    yield self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 774, in handle_exception
    traceback = _make_traceback(exc_info, source_hint)
  File "/usr/local/lib/python2.7/site-packages/jinja2/debug.py", line 140, in make_traceback
    return translate_exception(exc_info, initial_skip)
  File "/usr/local/lib/python2.7/site-packages/jinja2/debug.py", line 182, in translate_exception
    lineno = template.get_corresponding_lineno(tb.tb_lineno)
  File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 1112, in get_corresponding_lineno
    for template_line, code_line in reversed(self.debug_info):
  File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 1128, in debug_info
    self._debug_info.split('&')]
ValueError: invalid literal for int() with base 10: ''

This bug fix simply check if '_debug_info' is not empty before working on it.

This needs a test. In what circumstances is _debug_info empty?

This happened when the exception was on python callback in a custom extension. I'll try to build a simple test case.

Duplicate of #674


Cannot get schema if database have hyphen

When trying to develop Schemas node, we get an error : 'sqltools.getChildrenForTreeItem' failed.
Query in snowflake is :
SELECT schema_name as "label", schema_name as "schema", 'connection.schema' as "type", 'group-by-ref-type' as "iconId", catalog_name as "database"
FROM DATA-DEV.information_schema.schemata
WHERE schema_name != 'INFORMATION_SCHEMA' AND catalog_name = UPPER('DATA-DEV')
ORDER BY 2

This error is due to database name : DATA-DEV. In that case database name must be enclosed by double quote.

thanks for reporting this.
New version (0.4.2) with the fix is published to VSCode Marketplace and it should be now ready to upgrade in your VSCode.

Hello,
Not working. It's worth :D
[1616567602620] INFO  (ls): Connection instance created for data-dev.
ns: "conn-manager"
[1616567605287] ERROR (ls): Connecting error: {"code":-1,"data":{"driver":"Snowflake","driverOptions":{}},"name":"DecoratedException"}
ns: "conn-manager"
[1616567605286] ERROR (ext): ERROR: Error opening connection Cannot find data-dev database, {"code":-1,"data":{"driver":"Snowflake","driverOptions":{}}}
ns: "error-handler"


feat: add client for communicating with OCI repository

What type of PR is this?
/kind feature

What this PR does / why we need it:
add client for communicating with OCI registry
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?

Additional documentation e.g., design docs, usage docs, etc.:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

72 of 200 (36.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.7%) to 72.478%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/oci/client/ref.go
0
10
0.0%

pkg/oci/client/client.go
0
20
0.0%

pkg/oci/client/push.go
0
39
0.0%

pkg/oci/client/build.go
72
131
54.96%

Totals

Change from base Build<PHONE_NUMBER>:
-0.7%

Covered Lines:
7279

Relevant Lines:
10043

💛  - Coveralls


SlotRenderEndedEvent type is missing creativeTemplateId property

Both GPT documentation and @types/google-publisher-tags don't mention that SlotRenderEndedEvent has creativeTemplateId, however this is not true. creativeTemplateId is part of the event from the event listener callback, and you can see it in the code example that I'm attaching below.
Code example
I added a slotRenderEnded event listener on lines 67-69 which simply displays creativeTemplateId is present in the event object. If you open the console, you will see that it displays "event object has creativeTemplateId: true" as you scroll past the ads and they are rendered.
https://jsfiddle.net/2h85jzpL/12/

Hi @volodymyr-kl! Thanks for pointing this out, I'll check with the team and see what the status of that property is. There are occasionally API methods/properties that are purposely left undocumented (for example, if they're experimental or being sunset), but I'm not sure if this is one of those.
Do you have a particular use case in mind for this property? If so, could you share that with me so I can relay that to the team as well?

Hey @volodymyr-kl, the creativeTemplateId property along with a few others were added to the type defintions in b6107fa.  The updated version should already be available on npm. GPT documentation will be updated in the next day or so to match.
Thanks again for the report!


About Asynchronous

Is it possible when I code like this:
db.query("use one");
db.query("select * from ...");

and another place:
db.query("use two");
db.query("select * from ...");

there will be some mistakes about asynchronous?

db.commandname is put command to db manager queue. Since next command is dequeued only after previous finished it's completely safe to use code like
 db.query('use db1');
 db.query('select sleep(1)');
 db.query('use db2');
 console.log('ok');

all queries are executed in order here (but 'ok' is printed even before 'use db1' is executed)

But if I code like this:
app.get('/test1',function(){
db.use('two');
db.query('...');
});
app.get('/test2',function(){
db.use('one');
db.query('...');
});
And two users visit test1 and test2 at the same time,what will happen?
2012/3/1 Andrey Sidorov <
<EMAIL_ADDRESS>

db.commandname is put command to db manager queue. Since next command
is dequeued only after previous finished it's completely safe to use code
like
db.query('use db1');
db.query('select sleep(1)');
db.query('use db2');
console.log('ok');

all queries are executed in order here (but 'ok' is printed even before
'use db1' is executed)

Reply to this email directly or view it on GitHub:
https://github.com/sidorares/nodejs-mysql-native/issues/59#issuecomment-4248882

--
QQ:404692150

because 'put command to queue' is non-io, blocking task (it's just sequence of serial JS commands) server won't process second handler until first is processed
app.get('/test1',function(req, res){
    db.query('use one');
    db.query('select sleep(1)');
    db.query('select sleep(1)').on('end', function() {
        res.send('test1');
    });
});

app.get('/test2',function(req, res){
     db.query('select sleep(1.5)');
     db.query('use two');
     db.query('select sleep(1)').on('end', function() {
         res.send('test1');
     });
 });

now, if you have request /test1 and 0.1 sec later '/test2' both get handlers will exit almost immediately and you'll have this db queue:
  use one;
  select sleep(1);
  select sleep(1);  // here /test1 handler
  select sleep(1.5);
  use two;
  select sleep(1); // here /test2  handler

You'll see /test1 response in approx 2 seconds and /test2 in approx 4.5-0.1 seconds from the moment each request issued, respectively.

I see, thanks a lot.
2012/3/1 Andrey Sidorov <
<EMAIL_ADDRESS>

because 'put command to queue' is non-io, blocking task (it's just
sequence of serial JS commands) server won't process second handler until
first is processed
app.get('/test1',function(req, res){
db.query('use one');
db.query('select sleep(1)');
db.query('select sleep(1)').on('end', function() {
res.send('test1');
});
});
app.get('/test2',function(req, res){
db.query('select sleep(1.5)');
db.query('use two');
db.query('select sleep(1)').on('end', function() {
res.send('test1');
});
});
now, if you have request /test1 and 0.1 sec later '/test2' both get
handlers will exit almost immediately and you'll have this db queue:
 use one;
 select sleep(1);
 select sleep(1);  // here /test1 handler
 select sleep(1.5);
 use two;
 select sleep(1); // here /test2  handler

You'll see /test1 response in approx 2 seconds and /test2 in approx
4.5-0.1 seconds from the moment each request issued, respectively.

Reply to this email directly or view it on GitHub:
https://github.com/sidorares/nodejs-mysql-native/issues/59#issuecomment-4252282

--
QQ:404692150


WIP: Locality Weighted Load Balancing

TODO:

set proxy Locality by node annotation.

set Locality LB weight for eds.

@hzxuzhonghu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-presubmit.sh
ffdb1fd34544796393564684ba72efce4885a426
link
/test istio-presubmit

prow/istio-unit-tests.sh
ad717ad4068b9f4fbb0c8dd5d6f1ed8c7bfa56a6
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
ad717ad4068b9f4fbb0c8dd5d6f1ed8c7bfa56a6
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
b7e7e0d187503edc29f24ed1ed32edea35217af3
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
ad717ad4068b9f4fbb0c8dd5d6f1ed8c7bfa56a6
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
b7e7e0d187503edc29f24ed1ed32edea35217af3
link
/test istio-unit-tests

prow/istio-presubmit.sh
b7e7e0d187503edc29f24ed1ed32edea35217af3
link
/test istio-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
ad717ad4068b9f4fbb0c8dd5d6f1ed8c7bfa56a6
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
b7e7e0d187503edc29f24ed1ed32edea35217af3
link
/test istio-unit-tests

prow/istio-presubmit.sh
b7e7e0d187503edc29f24ed1ed32edea35217af3
link
/test istio-presubmit

prow/istio-integ-local-tests.sh
b7e7e0d187503edc29f24ed1ed32edea35217af3
link
/test istio-integ-local-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
ad717ad4068b9f4fbb0c8dd5d6f1ed8c7bfa56a6
link
/test istio-pilot-multicluster-e2e

prow/istio-presubmit.sh
b7e7e0d187503edc29f24ed1ed32edea35217af3
link
/test istio-presubmit

prow/istio-unit-tests.sh
28ad7d2351bb77a71b3941eaf6e59f7897772f78
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
28ad7d2351bb77a71b3941eaf6e59f7897772f78
link
/test istio-unit-tests

prow/istio-pilot-multicluster-e2e.sh
28ad7d2351bb77a71b3941eaf6e59f7897772f78
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


This is my ReallyAnnoyingDirectory Page

Here is my example of a really annoying directory page - thanks for letting me contribute!

modifications to the directory page are allowed EXCEPT adding a link to your page. Maintain the directory structure when adding your link.

Please resubmit the PR and add a link to your page from the directory at the bottom of the current list. Thanks

Thank you for the feedback. I have made the necessary change to the main index.html file and created a pull request.
From: Kevin<EMAIL_ADDRESS>Reply-To: LearnDevelopmentPublic/ReallyAnnoyingDirectory<EMAIL_ADDRESS>Date: Sunday, October 8, 2017 at 6:30 PM
To: LearnDevelopmentPublic/ReallyAnnoyingDirectory<EMAIL_ADDRESS>Cc: William Sampler<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [LearnDevelopmentPublic/ReallyAnnoyingDirectory] This is my ReallyAnnoyingDirectory Page (#9)
modifications to the directory page are allowed EXCEPT adding a link to your page. Maintain the directory structure when adding your link.
Please resubmit the PR and add a link to your page from the directory at the bottom of the current list. Thanks
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Ignore links to GitHub user and organisation profiles

Ignores links to GitHub user and organisation profiles (but not repositories or other pages withing GH users/organizations).
Follows discussion in https://github.com/jupyterlab/jupyterlab/issues/15776

I believe the downstream failures are unrelated. Is it ok to merge and release?

hi, would we merge it to resolve check-link failure?

I opened https://github.com/jupyterlab/jupyterlab/pull/15959 which will allow us to merge this into jupyterlab directly.

It looks it was done in https://github.com/jupyterlab/maintainer-tools/pull/226.


Upgrade and accommodate to GOBL 0.200

Upgrades to GOBL 0.200 and adapts to the changes: mostly, it replaces codes being mapped from scenario summaries with explicit extensions.
Depends on GOBL 0.200 being released but it's not essential to release these changes along with GOBL 0.200: the current version in production is compatible with GOBL 0.200 invoices.

Duplicate of #35


Table Component (Enhancement Request/ Inquiry)

Are there any plans to support a <Table/> Component with 'preact-material-components'?

We can take this as a feature request for post 2.0 release.
But will take some time.


Stopping Streaming for New Chrome

Latest chrome release was not supporting the stream.stop(). Updated it with the Chrome check and new method for stopping the stream. Hope its helpful.

Solution: https://github.com/leemachin/say-cheese/issues/40


⚠ Bofrost.fr has degraded performance

In 86c6761, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 900 ms

Resolved: Bofrost.fr performance has improved in 16692a8 after 313 days, 8 hours, 47 minutes.


Fixed tab formatting in fuzzers

This might need rebase after all the other issues are merged.

This needs a rebase now

Done


RFC: Creates batch_idx on first run

Refactors _get_batch_idx to keep track of the batch index.

I agree that the existing method to count the batch index is flaky since it assumes that all epochs have the same number of batches.
With your proposed change, you would increase the count and return the index. I would prefer such a method not to have a side effect, since it assumes that _get_batch_idx is only called once per batch. I thus have two alternative proposals:

Set a batch_idx_ attribute during initialize, increment it during on_batch_end, get rid of _get_batch_idx.
Count the number of batches (e.g. len(list(flatten(net.history[:, 'batches'])))).

I can think of one way how those two proposals differ, though I'm not sure if it ever happens: When a user de-activates the LR scheduler for some epochs during training (which is possible with set_params), the first proposal would only count the batches during which the scheduler was active, the second would count all batches.

I went with the first approach. This was to avoid the runtime cost of calling len after every batch.

Thanks.


Display supported incremental strategies in a tabular format

Preview
resolves #4634
What are you changing in this pull request and why?
The main aim is to be able to easily answer each of these questions:

which adapters support the delete+insert strategy?
what are the names of the standard incremental strategies that are supported in at least one adapter?
what are the names of the standard incremental strategies that are supported most frequently?
what are the names of the standard incremental strategies that are supported least frequently?

A tabular representation with one column per strategy is the easiest way to answer all of the above.
Design decisions to be made

Style of the icons
Number of tables

Style of the icons
Any of these could work:
✅ - Green check mark
✔ - Heavy check mark
🟢 - Green circle
We probably wouldn't choose any of these:
☑ - Ballot box with check
🆗 - OK button
👍 - Thumbs up
etc.
Number of tables
We have several options:

Keep the status quo
Split into two tables: one for the available strategies, and another for the defaults
Keep a single table, keeping the "default strategy" column, but replacing the "additional supported strategies" with one column per strategy
Defer the default strategy information to each adapter-specific config page instead (see dbt-bigquery or dbt-redshift for examples)

Trade-offs
I strongly prefer not staying with the status quo, or I wouldn't have opened this issue 🤣
Splitting into two separate tables would solve the key problem, but at the cost of taking up more vertical space on the page.
Keeping a single table but refactoring it would work for me.
Completely moving the default strategy information into the adapter-specific pages might be the best of all worlds. It would solve the problem at hand while preserving documentation about the default strategy that is relevant for a particular adapter. dbt-postgres and dbt-redshift in particular have caveats related to the default -- it will change depending if a unique_key is defined or not.
🎩
Preview

Previous iteration
Note: I've since removed the 2nd table that has the defaults, but here's what it looked like with them:

Checklist

[x] Review the Content style guide so my content adheres to these guidelines.

@dbeatty10 Thanks for putting this together. I like the changes you've made here. I'm going to update the tables to use the ✅  icon. It looks good on light mode, but dark mode users may have trouble seeing the gray on black background.


keep y-axis range for overlay and facet

This will resolve https://github.com/VEuPathDB/web-eda/issues/1701

@bobular Thank you for your tests/reviews/suggestions. I addressed your feedbacks 👍


Is there a similar approach in TensorFlow2 when save large model?

Click to expand!
Issue Type
Feature Request
Have you reproduced the bug with TF nightly?
No
Source
source
Tensorflow Version
tf 2.11
Custom Code
Yes
OS Platform and Distribution
No response
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
from transformers import TFAutoModel
model = TFAutoModel.from_pretrained("bert-base-cased")
model.save_pretrained("saved", max_shard_size="200MB")

Is there a similar approach in TensorFlow2 when save large model?
Standalone code to reproduce the issue
from transformers import TFAutoModel
model = TFAutoModel.from_pretrained("bert-base-cased")
model.save_pretrained("saved", max_shard_size="200MB")

Relevant log output
No response

You can always save a model before, during, and after training using the same model.save() API. Once loading, you can always continue training if you so desire. There is no need for a separate API (save_pretrained).
Regarding sharding (max_shard_size), there is no such parameter, TF does automatic sharding.
See https://www.tensorflow.org/api_docs/python/tf/saved_model/save, https://www.tensorflow.org/api_docs/python/tf/saved_model/SaveOptions and https://www.tensorflow.org/api_docs/python/tf/keras/Model#save

Hi @SmileTM ,
Apart from resources mentioned in above, you can also refer to attached tutorial links for end to end examples of creating,training, saving and loading the models.
Please refer serialization_and_saving and save_and_load also for a demo.
Tensorflow has options to save complete model architecture including optimizer state or only weights of the layers also.
If you still looking for a particular use case please let us know.
Thanks!

Thanks for the quick response.


expanding the inline flag to the raws structure as well

stylelint's comment-empty-line-before rule is checking node.raws.inline for inline comments, so this patch will allow inline comments to be ignored properly.

@patsissons, please, see the last updates for Comment Node https://github.com/webschik/postcss-less/commit/057638ff3bd6eb79647da094164b189994108152. According to PostCSS API boolean flags should be properties of Node and raws contain the properties with pure data. For example see how implemented important property.

ah sounds good. I just assumed that this was an oversight, but it sounds more like that stylelint is actually where the bug lies. i'll poke them to see what they are thinking and close this one.

can you point me to the docs that mention that aspect, i can't find it anywhere.

@patsissons, thank you.
https://github.com/postcss/postcss/blob/master/docs/api.md#noderaws
https://github.com/postcss/postcss/blob/master/docs/api.md#declarationimportant
Also it's true according to PostCSS code and result of parsing.

i read the api docs, but didn't see any mention of boolean flags being on the node, just that the raws was for storing parser data.

But they pass to node.important only the state - important or not (true/false) and node.raws.important contains the part of code with !important rule.

ok, i see what you're saying now. important lives in both places, the flag on the node and the actual less text in the raws because it is both a flag and less content. the inline is just a flag because it has no corresponding content in less. That is a reasonable enough explanation for me.

@patsissons Yep, sorry if I clarified it unclear.


nixos/privoxy: add https and settings options

Motivation for this change
This is a major rewrite of the Privoxy module:

As per RFC0042, remove privoxy.extraConfig and replace it
with a privoxy.settings option, which maps a NixOS freeform
submodule to the Privoxy configuration format.

Move all top-level options that mirrored a setting to
the real ones in privoxy.settings. This still keeps the
type-checking, default values and examples in places.

Add two convenience options: userActions and userFilters, which
simplify the operation of creating a file with pkgs.writeText,
converting it to a string and adding it to the actionsfile/
filterfile list.

Add a privoxy.inspectHttps option to automagically setup TLS
decryption support. I don't know how long have been waiting
for this feature: can't believe it has just happened.

Also add a privoxy.certsLifetime to control the periodical
cleanup of the temporary certificates generate by Privoxy.

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via new nixosTests.privoxy
[x] Tested compilation of all pkgs that depend on this change
[x] Tested execution of all binary files
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 115005 at 6b6a20d7 run on x86_64-linux 1

1 package built:

privoxy

Sorry but I don't understand what you're trying to achieve: is the order important?
Can't you just join the two forward definitions?

Upon closer inspection, while the order does matter, in my case I should be able to put the forward-socks5t rule at the end, which leads to this configuration:

  services.privoxy = {
    enable = true;
    settings = {
      toggle = true;
      forward = [ " /                    ."
                  "192.168.*.*/          ."
                  "10.*.*.*/             ."
                  "127.*.*.*/            ."
                  "localhost/            ."
                ];
      forward-socks5t = ".onion/              <IP_ADDRESS>:9050 .";
    };
  };

However, it fails to build with the error:
error: undefined variable 'certificate-directory' at /home/emmanuel/projects/nixpkgs/nixos/modules/services/networking/privoxy.nix:195:12

It's unclear what settings.certificate-directory should be set to, given I'm not using that feature.

This is an actual mistake: I fixed it in 7962df46feb. I tested your configuration in VM and now it's working.
Thank you for reporting it.

Thank you, that worked.


Devtools preview

Just installed new version and I miss dev server preview in Nuxt Devtools.
Will it be included or there is some new undocumented preview solution?

I'm planning to build something like react email preview, but i might also do a nuxt devtools preview that has extra features, im currently busy with work, so it'll take some time.

A possibility to preview would be amazing! :)
Currently I render the mail and display it in an iframe


Cannot compile

When I do 'build grunt', it gives me these error. Installing Java does not help. I am using Ubuntu 14.10
The only change I made away from your tutorial was:
a) When I installed nodejs using sudo apt-get install nodejs. It did not install npm. So I manually installed npm via apt-get install npm
That is the only difference, the rest I followed to the point.
admin@circledisk:~/apprtc-master$ sudo grunt build
Running "shell:buildAppEnginePackage" (shell) task
[Errno 2] No such file or directory
[Errno 2] No such file or directory
Running "shell:genJsEnums" (shell) task
src >>> src/web_app/js
Running "closurecompiler:debug" (closurecompiler) task
Compiling src/web_app/js/analytics.js,src/web_app/js/enums.js,src/web_app/js/adapter.js,src/web_app/js/appcontroller.js,src/web_app/js/call.js,src/web_app/js/constants.js,src/web_app/js/infobox.js,src/web_app/js/peerconnectionclient.js,src/web_app/js/remotewebsocket.js,src/web_app/js/roomselection.js,src/web_app/js/sdputils.js,src/web_app/js/signalingchannel.js,src/web_app/js/stats.js,src/web_app/js/storage.js,src/web_app/js/util.js,src/web_app/js/windowport.js -> out/app_engine/js/apprtc.debug.js
Fatal error: Java is not available, neither the bundled nor a global one.
admin@circledisk:~/apprtc-master$
UPDATE
I managed to fix the Java path etc.
Then it asked me to use force, so I used grunt --force build
Then it said finished but with these errors. Not sure if it is fatal. Please check.
admin@circledisk:~/apprtc-master$ sudo grunt --force build
Running "shell:buildAppEnginePackage" (shell) task
[Errno 2] No such file or directory
[Errno 2] No such file or directory
Running "shell:genJsEnums" (shell) task
src >>> src/web_app/js
Running "closurecompiler:debug" (closurecompiler) task
Compiling src/web_app/js/analytics.js,src/web_app/js/enums.js,src/web_app/js/adapter.js,src/web_app/js/appcontroller.js,src/web_app/js/call.js,src/web_app/js/constants.js,src/web_app/js/infobox.js,src/web_app/js/peerconnectionclient.js,src/web_app/js/remotewebsocket.js,src/web_app/js/roomselection.js,src/web_app/js/sdputils.js,src/web_app/js/signalingchannel.js,src/web_app/js/stats.js,src/web_app/js/storage.js,src/web_app/js/util.js,src/web_app/js/windowport.js -> out/app_engine/js/apprtc.debug.js
Warning: Error: Unable to access jarfile /home/admin/apprtc-master/node_modules/grunt-closurecompiler/node_modules/closurecompiler/compiler/compiler.jar
Used --force, continuing.
Running "grunt-chrome-build:apprtc" (grunt-chrome-build) task
Chrome packaged app build.
Deleting buildDir: out/chrome_app

Cannot delete nonexistent file.
Creating empty buildDir: out/chrome_app
Copying resources to buildDir: out/chrome_app
src/web_app/chrome_app/manifest.json-->out/chrome_app/manifest.json
out/app_engine/js/loopback.js-->out/chrome_app/js/loopback.js
out/app_engine/css/main.css-->out/chrome_app/css/main.css
out/app_engine/images/apprtc-128.png-->out/chrome_app/images/apprtc-128.png
out/app_engine/images/apprtc-16.png-->out/chrome_app/images/apprtc-16.png
out/app_engine/images/apprtc-22.png-->out/chrome_app/images/apprtc-22.png
out/app_engine/images/apprtc-32.png-->out/chrome_app/images/apprtc-32.png
out/app_engine/images/apprtc-48.png-->out/chrome_app/images/apprtc-48.png
src/web_app/js/background.js-->out/chrome_app/js/background.js
src/web_app/js/appwindow.js-->out/chrome_app/js/appwindow.js

Running "grunt-chrome-build-transform" task
Transforming files using jinja.
Running "jinja:chrome-build" (jinja) task
Using context web_app/html/index_template
File "out/chrome_app/appwindow.html" created.
Running "grunt-chrome-build-compress" task
Creating zip file:out/chrome_app/apprtc.zip
Running "compress:chrome-build" (compress) task
Created out/chrome_app/apprtc.zip (28611 bytes)
Running "grunt-chrome-build-package" task
Skipping creation of Chrome package.
Done, but with warnings.
admin@circledisk:~/apprtc-master$
Second Update
Cannot run it. This is what I get.
admin@circledisk:~/google_appengine$ /home/admin/google_appengine/dev_appserver.py ./out/app_engine
Traceback (most recent call last):
File "/home/admin/google_appengine/dev_appserver.py", line 83, in
_run_file(file, globals())
File "/home/admin/google_appengine/dev_appserver.py", line 79, in _run_file
execfile(PATHS.script_file(script_name), globals)
File "/home/admin/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 1033, in
main()
File "/home/admin/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 1026, in main
dev_server.start(options)
File "/home/admin/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 752, in start
options.config_paths, options.app_id)
File "/home/admin/google_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 795, in init
module_configuration = ModuleConfiguration(config_path, app_id)
File "/home/admin/google_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 123, in init
self._config_path)
File "/home/admin/google_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 391, in _parse_configuration
with open(configuration_path) as f:
IOError: [Errno 2] No such file or directory: './out/app_engine'
admin@circledisk:~/google_appengine$

What solved the problem? :)


Bug: Can't await ShutdownSignal receiver

Shutdown signa receiver that is passed to the spawn function needs to be awaited so that users can do tokio::select{} in a loop and in case of that they return Fall::Shutdown() so that cleanup can run
thought, having the signal inside, means that the user could do the cleanup in the select {} closure, without having to write yet another function. Using this construct, the cleanup function doesn't really serve a function.  Can we remove cleanup or remove the need for the signal inside the spawn?

Fixed in #8


Handle failure to launch kernel while connecting

For #10479

Codecov Report

Merging #11454 into master will increase coverage by 0.00%.
The diff coverage is 92.30%.

@@           Coverage Diff            @@
##           master   #11454    +/-   ##
========================================
  Coverage   61.03%   61.04%
========================================
  Files         613      613
  Lines       33630    33647    +17
  Branches     4751     4752     +1
========================================
+ Hits        20527    20540    +13
+ Misses      12657    12076   -581
- Partials      446     1031   +585

Impacted Files
Coverage Δ

...client/datascience/raw-kernel/rawJupyterSession.ts
82.71% <92.30%> (+10.84%)
:arrow_up:

src/datascience-ui/react-common/arePathsSame.ts
75.00% <0.00%> (-12.50%)
:arrow_down:

src/client/common/utils/platform.ts
64.70% <0.00%> (-11.77%)
:arrow_down:

...cience/ipywidgets/cdnWidgetScriptSourceProvider.ts
84.48% <0.00%> (-2.59%)
:arrow_down:

src/client/datascience/debugLocationTracker.ts
76.56% <0.00%> (-1.57%)
:arrow_down:

src/client/common/process/proc.ts
14.49% <0.00%> (-0.73%)
:arrow_down:

src/client/testing/main.ts
14.38% <0.00%> (ø)

src/client/common/editor.ts
8.25% <0.00%> (ø)

src/client/linters/flake8.ts
85.71% <0.00%> (ø)

src/client/linters/pylint.ts
98.27% <0.00%> (ø)

... and 139 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ebfc9e7...790669b. Read the comment docs.


Restart controller pod on config change

LGTM, just check that condition. Also, there is update-templates.yaml that does the same but doesn't redeploy everything so it much faster. Anyway, that's needed in deploy too.


Docs: List possible execution state strings for GET /_api/query/current

Backport of #14223

https://jenkins.arangodb.biz/view/Documentation/job/arangodb-ANY-examples/760/


Extend real-world example data to include Victorian directors

Enhancement
The data used in this package should be extended to include the corporate directorate of Victorian Britain in 1880. This data can be extracted from the O1sims/VictorianBritainDirectors repository.
Specifically, the relevant data could be extracted from here, separated out to form a nodeList and an edgeList and added to the repository with a corresponding example added to the documentation.
This work will be carried out on the branch enhancement_vbd_data.

This has been completed in PR #3, which has now been merged to master.


Add support for HDFS HTTP port (i.e. WebHDFS).

Thanks @kminder - I'll merge this now. I'll work to cut a new release this week or next that includes this change.

The 0.1.3 release has been cut which includes this change. Thanks again.


Move to the voila-dashboards GitHub organization?

We are moving voilà to the voila-dashboards organization, as well as the voilà templates that we work on...
Should voila-gallery also be moved to this organization? It may make sense to only have the examples of the public voila-gallery.org deployment in this organization? Thoughts?
cc @yuvipanda @jtpio

Fine for moving over to voila-dashboards.
If this organization now only contains examples, it could also be renamed to voila-gallery-examples or voila-examples.

The move is done and everyone has been invited to the organization!


implement slice

Pull Request type

Please check the type of change your PR introduces:

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: 180
What is the new behavior?

Tensor slice is added.

Does this introduce a breaking change?

[ ] Yes
[x] No

Other information

@all-contributors please add @dincerguner for code


chore(jellyfish-api-core): WhaleApiClient caching

What this PR does / why we need it:
Creates a WhaleApiClientProvider that lazily initialises WhaleApiClients and caches them by network (i.e. mainnet, regtest, testnet) so they can be reused - improving performance
Which issue(s) does this PR fixes?:

Fixes #1052

The semantic label list is kept to just https://github.com/DeFiCh/jellyfish/blob/f671bb4e166ca3005db94bd2f7633d101113094f/.github/labeler.yml#L4-L32
The label is used to automate the release notes.


[MM-40025] Stop app from redirecting to tabs when not logged in

Summary
In some cases due to the redirect logic, the app think it needs to redirect to another tab immediately after signin. When this happens, the app tries to call a logged-in route and our webapp emits a logged out event when this happens. This creates a race condition potentially logging out the user simply when they try to log in.
This PR stops any redirection to a tab that is not currently logged in, stopping the weird login issues.
Ticket Link
Closes https://github.com/mattermost/desktop/issues/1861
https://mattermost.atlassian.net/browse/MM-40025
Release Note
Fixed an issue where a user might an erroneous "Your session has expired" error and be unable to login.

/cherry-pick release-5.0

Cherry pick is scheduled.

Error trying doing the automated Cherry picking. Please do this manually
+++ Updating remotes...
Fetching upstream
Failed to add the RSA host key for IP address '<IP_ADDRESS>' to the list of known hosts (/app/.ssh/known_hosts).
From github.com:mattermost/desktop
   a0c18bc..fec77d4  release-5.0 -> upstream/release-5.0
Fetching origin
Failed to add the RSA host key for IP address '<IP_ADDRESS>' to the list of known hosts (/app/.ssh/known_hosts).
+++ Updating remotes done...
+++ Creating local branch automated-cherry-pick-of-desktop-#1870-upstream-release-5.0-1641307984
Switched to a new branch 'automated-cherry-pick-of-desktop-#1870-upstream-release-5.0-1641307984'
Branch 'automated-cherry-pick-of-desktop-#1870-upstream-release-5.0-1641307984' set up to track remote branch 'release-5.0' from 'upstream'.

+++ About to attempt cherry pick of PR #1870 with merge commit 3b4437e48061a0cae785e7ce4d0bb9267ddae2f2.

error: could not apply 3b4437e... [MM-40025] Stop app from redirecting to tabs when not logged in (#1870)
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'

+++ Conflicts detected:

UU src/main/preload/mattermost.js
UU src/main/windows/windowManager.ts
Aborting.

+++ Aborting in-progress git cherry-pick.

+++ Returning you to the master branch and cleaning up.


Adding linux/arm64 support and Update Makefile

Adding Arm64 Support.
In addition - gox library is deprecated so change the build making by using  go command   - GOARCH=arm64 GOOS=linux go build.
Also adding option to a specific build by using  make (os)(arch) for example make windows_amd64 will make a exec for windows with amd64 architecture.

Fix for #39


[SPARK-30104][SQL][FOLLOWUP] Remove LookupCatalog.AsTemporaryViewIdentifier

What changes were proposed in this pull request?

As discussed in https://github.com/apache/spark/pull/26741#discussion_r357504518, LookupCatalog.AsTemporaryViewIdentifier is no longer used and can be removed.
Why are the changes needed?

Code clean up
Does this PR introduce any user-facing change?

No
How was this patch tested?

Existing tests

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20150/
Test PASSed.

Test build #115344 has started for PR 26897 at commit e76cf1c.

Test build #115344 has finished for PR 26897 at commit e76cf1c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115344/
Test PASSed.

CC @cloud-fan

If something is only used in tests, it's dead code. I checked the removed tests and they solely test AsTemporaryViewIdentifier.
LGTM, merging to master!


Kip-848: Next Generation Consumer Group

I wanted to know what is the plan on implementing KIP-848. I see a dedicated branch just for this, but it's been inactive for a year. @twmb I would be happy to help move it forward.

I have to evaluate what this entails. The branch is stale because I used to implement protocol support before it was rolled out in Kafka, but stopped once Kafka started changing APIs repeatedly before the API was finally released.
I'm not sure 848 is high priority though because it's released as preview only. I'm not sure how hard it'll be to implement, but likely pretty hard...

Yeah I'm probably going to need help with this, this is a big one.

I definitely missed your reply above^.
I'm starting to think about this (a little bit). One thing I may lean to do first is to improve the guts of the consumer so that moving a partition between brokers does not require a full consumer stop/restart -- and then after, implement KIP-848.
Or it may make sense to just start with 848 logic, I'm really not sure -- I haven't dug into it much.


Errors with returned URIs

Hi,
Thanks for the modules, really working well.
On Get-Cyberduck, the returned URI is missing a "/" in the URI
i.e. it returns **https:/**update.cyberduck.io/......
That seems to be true on all channels for this product.
Thanks
Matthew

Additionally, Get-Filezilla returns a URL which doesn't succeed - wget reports receiving a 301 redirect to https://filezilla-project.org/, and the same experience when pasting into a browser.
Currently Get-Filezilla returns the following for URI: https://dl2.cdn.filezilla-project.org/FileZilla_3.50.0_win64-setup.exe

The Cyberduck update feed is returning an incorrectly formatted URL:

  | <?xml version="1.0" encoding="UTF-8" standalone="no"?>
-- | --
  | <rss xmlns:sparkle="http://www.andymatuschak.org/xml-namespaces/sparkle" version="2.0">
  | <channel>
  | <title>Cyberduck Changelog</title>
  | <link>https://cyberduck.io/changelog</link>
  | <description>Cyberduck Changelog.</description>
  | <language>en-us</language>
  | <copyright>Copyright (c) 2001-2020 David Kocher. Copyright (c) 2011-2020 Yves Langisch</copyright>
  | <item>
  | <title>Cyberduck 7.6.2 (Revision 33520)</title>
  | <pubDate>30/09/2020 10:29 AM</pubDate>
  | <enclosure sparkle:dsaSignature="MC0CFQCPUyJ40sA7OPzlj9RepHNJ2Rr4QAIUKAGnL9D41VbDpJc/b4NjAjUiO9Y=" sparkle:installerArguments="/passive Autostart=1" sparkle:shortVersionString="7.6.2" sparkle:version="33520" type="application/octet-stream" url="https://update.cyberduck.io//Cyberduck-Installer-<IP_ADDRESS>520.exe"/>
  | <sparkle:releaseNotesLink>https://version.cyberduck.io/windows//changelog.html</sparkle:releaseNotesLink>
  |
  | </item>
  | </channel>
  | </rss>
  |


Error with pwa or pwa-typescript installation

First of all, I'm not sure if this is the right repo to put this.
Running npx create-react-app . --template pwa-typescript, installation fails at a statSync error with a missing gitignore file.
Also, after npx create-react-app . --template pwa and running npm start an error occurs with a missing module web-vitals in ./src/reportWebVitals.js. Manually installing web-vitals with npm i --save web-vitals fixed it.
Sorry I don't have screenshots, npm start command cleared my terminal.
Node version: 12.18.3
npm version: 6.14.6

Thanks for pointing that out—we'll get the missing gitignore file added in #4.
In general, though, these templates require create-react-app v4 or higher, which I don't believe has been released yet.
As soon as it's available, https://create-react-app.dev/docs/making-a-progressive-web-app/ will be updated to refer to these templates and you can give them a shot.
(Arguably we could put something in the README saying "requires create-react-app v4 or higher" but that's a little weird since npx create-react-app will just run whatever the latest release is.)

I'm just starting out in PWAs and read the cra docs and some tutorials and expected the default template being pwa ready, but they weren't and it didn't say anything on the docs. I guess I jumped the gun when I read the latest PR on create-react-app, thinking that the pwa-templates were official already.


Page isn't redirecting properly

If I go to this page, Firefox shows the error The page isn’t redirecting properly after many quick  attempts to load the page.
If I can help you somehow, tell me.

I think you copied the wrong link, there is no URL inside the URL and it just works fine. But if you do get this kind of problem, you have to add an entry to your blacklist.

If I turn off Skip Redirect, the site loads just fine. If I turn it on, this is what it's trying to do :

Medium is doing some funny thing there. I've had the same problem and adding "medium.com" to the blacklist fixed it


Pre SVE chip tool testing: Few steps in TC-DL-3.2 are giving general error (0x8b) response

SHA:5d8599d195a93dfe5473f64e4f888c322e5df1d2
Platform: Chip Tool running on Mac, all cluster app running on nRF52840-DK (Thread setup)
As per the steps in TC-DL-3.2, the below commands are giving general error (0x8b) response
Commands used:
./chip-tool doorlock set-week-day-schedule 3 2 2 8 15 9 15 1 1
./chip-tool doorlock clear-week-day-schedule 1 1 1 1
./chip-tool doorlock set-year-day-schedule 3 3 6 21 1 1
./chip-tool doorlock clear-year-day-schedule 1 1 1 1
./chip-tool doorlock set-holiday-schedule 1 20 30 0 1 1
./chip-tool doorlock get-holiday-schedule 1 1 1
./chip-tool doorlock clear-holiday-schedule 1 1 1
[1655342103958] [30412:5015406] CHIP: [DMG] ICR moving to [ResponseRe]
[1655342103958] [30412:5015406] CHIP: [DMG] InvokeResponseMessage =
[1655342103958] [30412:5015406] CHIP: [DMG] {
[1655342103958] [30412:5015406] CHIP: [DMG] 	suppressResponse = false,
[1655342103958] [30412:5015406] CHIP: [DMG] 	InvokeResponseIBs =
[1655342103958] [30412:5015406] CHIP: [DMG] 	[
[1655342103958] [30412:5015406] CHIP: [DMG] 		InvokeResponseIB =
[1655342103958] [30412:5015406] CHIP: [DMG] 		{
[1655342103958] [30412:5015406] CHIP: [DMG] 			CommandStatusIB =
[1655342103958] [30412:5015406] CHIP: [DMG] 			{
[1655342103958] [30412:5015406] CHIP: [DMG] 				CommandPathIB =
[1655342103958] [30412:5015406] CHIP: [DMG] 				{
[1655342103958] [30412:5015406] CHIP: [DMG] 					EndpointId = 0x1,
[1655342103958] [30412:5015406] CHIP: [DMG] 					ClusterId = 0x101,
[1655342103958] [30412:5015406] CHIP: [DMG] 					CommandId = 0xe,
[1655342103958] [30412:5015406] CHIP: [DMG] 				},
[1655342103958] [30412:5015406] CHIP: [DMG]
[1655342103958] [30412:5015406] CHIP: [DMG] 				StatusIB =
[1655342103958] [30412:5015406] CHIP: [DMG] 				{
[1655342103958] [30412:5015406] CHIP: [DMG] 					status = 0x8b (NOT_FOUND),
[1655342103958] [30412:5015406] CHIP: [DMG] 				},
[1655342103958] [30412:5015406] CHIP: [DMG]
[1655342103958] [30412:5015406] CHIP: [DMG] 			},
[1655342103958] [30412:5015406] CHIP: [DMG]
[1655342103958] [30412:5015406] CHIP: [DMG] 		},
[1655342103958] [30412:5015406] CHIP: [DMG]
[1655342103958] [30412:5015406] CHIP: [DMG] 	],
[1655342103958] [30412:5015406] CHIP: [DMG]
[1655342103958] [30412:5015406] CHIP: [DMG] 	InteractionModelRevision = 1
[1655342103958] [30412:5015406] CHIP: [DMG] },
[1655342103958] [30412:5015406] CHIP: [DMG] Received Command Response Status for Endpoint=1 Cluster=0x0000_0101 Command=0x0000_000E Status=0x8b
[1655342103959] [30412:5015406] CHIP: [TOO] Error: IM Error 0x0000058B: General error: 0x8b (NOT_FOUND)
[1655342103959] [30412:5015406] CHIP: [DMG] ICR moving to [AwaitingDe]
[1655342103959] [30412:5015406] CHIP: [EM] Sending Standalone Ack for MessageCounter:22946566 on exchange 26679i
[1655342103959] [30412:5015406] CHIP: [IN] Prepared secure message 0x70000c9cc4f0 to 0x0000000000000001 (1)  of type 0x10 and protocolId (0, 0) on exchange 26679i with MessageCounter:220735223.
[1655342103959] [30412:5015406] CHIP: [IN] Sending encrypted msg 0x70000c9cc4f0 with MessageCounter:220735223 to 0x0000000000000001 (1) at monotonic time: 000000001A37CC26 msec
[1655342103959] [30412:5015406] CHIP: [EM] Flushed pending ack for MessageCounter:22946566 on exchange 26679i
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the commissioner
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the controller
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the commissioner
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the controller
[1655342103959] [30412:5015401] CHIP: [IN] Expiring all connections for fabric 1!!
[1655342103959] [30412:5015401] CHIP: [IN] SecureSession[0x7ff1e1c2bc80]: MarkForRemoval Type:2 LSID:36706
[1655342103959] [30412:5015401] CHIP: [IN] SecureSession[0x7ff1e1c2bc80]: Released - Type:2 LSID:36706
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the commissioner
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the controller
[1655342103959] [30412:5015401] CHIP: [IN] Expiring all connections for fabric 2!!
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the commissioner
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the controller
[1655342103959] [30412:5015401] CHIP: [IN] Expiring all connections for fabric 3!!
[1655342103959] [30412:5015401] CHIP: [CTL] Shutting down the System State, this will teardown the CHIP Stack
[1655342103959] [30412:5015401] CHIP: [DMG] IM WH moving to [Uninitialized]
[1655342103959] [30412:5015401] CHIP: [DMG] IM WH moving to [Uninitialized]
[1655342103959] [30412:5015401] CHIP: [DMG] IM WH moving to [Uninitialized]
[1655342103959] [30412:5015401] CHIP: [DMG] IM WH moving to [Uninitialized]
[1655342103959] [30412:5015401] CHIP: [DMG] All ReadHandler-s are clean, clear GlobalDirtySet
[1655342103959] [30412:5015401] CHIP: [BLE] CancelConnection
[1655342103959] [30412:5015401] CHIP: [DL] Inet Layer shutdown
[1655342103959] [30412:5015401] CHIP: [DL] BLE shutdown
[1655342103959] [30412:5015401] CHIP: [DL] System Layer shutdown
[1655342103960] [30412:5015401] CHIP: [TOO] Run command failure: IM Error 0x0000058B: General error: 0x8b (NOT_FOUND)

Duplicate of #19622


feat(sync): Wipe entries (implement prefix deletion)

Description
This PR implements prefix deletion as specified in Willow
Notes & open questions
There's an important implication of prefix deletion in our willow data model that we'll have to be very explicit about in docs and tutorials:
doc.set_bytes(author, b"foobar", b"important data").await?;
doc.set_bytes(author, b"foo", b"crap").await?;
let x = doc.get(author, b"foobar").await?;
// x == None

In explanation: The willow data model defines that adding an entry removes all older entries that have the new entry's key as their prefix.
What this means is that apps need to model their keyspace carefully. For example, the current file system import/export demo (#1504) will not work as intended, because of the above.
The recommended solution is that for usecases that need nested keys without them being affected by prefix deletion is to append a nul byte to those keys.
Quoting @AljoschaMeyer from Discord:

In defense of prefix-based deletion and its useability: the approach was inspired by regular file-system semantics. If I create /food/pizza.jpg and /food/croissant.jpg, but later created an empty text file /food, it would delete the previously existing /food directory, and hence everything in it.
Prefix-based deletion was the simplemost solution we could find to replicate this behavior. In particular, we do not need to introduce the concepts of nested directories and leaf files (or the formal counterpart to file system paths: strings of strings).
The idea was that anyone who'd create an application layer where users deal with actual filesystems would add a small conversion layer: the user-facing api has (nested) directories, (optionally) containing leaf files. These are converted to willow paths in a way that adds a null-byte at the end of every leaf file (and prefixes every directory with a path separator such as /). Willow then replicates exactly the deletion behavior that the users expect from a file system.

Change checklist

[x] Self-review.
[x] Documentation updates if relevant.
[x] Tests if relevant.

One thing we should make sure to either unify or clarify is the difference (or not) between delete_prefix("foo") and set("foo", "")

One thing we should make sure to either unify or clarify is the difference (or not) between delete_prefix("foo") and set("foo", "")
Yup. They are exactly the same with the current implementation. So should we just note this in the docs?

I think having the same behaviour is the right call, just need to makes sure it is tested & explained in the docs.

I pushed a commit that keeps the behavior, but makes the store return None for entries with the empty hash (and skip them in the iterators).

Changes since @dignifiedquire's review:

Added a test that syncs a prefix wipe
Added validation so that "invalid empty" entries are rejected (empty hash but len not 0, or len 0 but hash not empty hash)
Removed allocations from the ParentIterator in fs impl

So I wrote a test, found a bug in the range calculation, and fixed it.
This should still get more tests. But now we at least have one test that caught edge case.
Should we go ahead and merge, and then likely look into some proptests or similar? Or does someone have ideas what we should test manually?

Last commits have some renames - the bikeshed is over, for now: we settled for del.