HDFS-16348. Mark slownode as badnode to recover pipeline

Description of PR
In HDFS-16320, the DataNode can retrieve the SLOW status from each NameNode.
This ticket is to send this information back to Clients who are writing blocks. If a Clients noticed the pipeline is build on a slownode, he/she can choose to mark the slownode as a badnode to exclude the node or rebuild a pipeline.
In order to avoid the false positives, we added a config of "threshold", only clients continuously receives slownode reply from the same node will the node be marked as SLOW.
Jira ticket: https://issues.apache.org/jira/browse/HDFS-16348
How was this patch tested?
unit test
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@ferhui @tasanuma @Hexiaoqiao Could you help to review this PR?

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 47s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m 21s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
21m 32s

trunk passed

+1 :green_heart:
compile
5m 19s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m  1s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 17s

trunk passed

+1 :green_heart:
mvnsite
2m 28s

trunk passed

+1 :green_heart:
javadoc
1m 45s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 12s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 38s

trunk passed

+1 :green_heart:
shadedclient
22m 24s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 26s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  6s

the patch passed

+1 :green_heart:
compile
5m  9s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m  9s

the patch passed

+1 :green_heart:
compile
4m 52s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
4m 52s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m  8s
/results-checkstyle-hadoop-hdfs-project.txt
hadoop-hdfs-project: The patch generated 3 new + 378 unchanged - 1 fixed = 381 total (was 379)

+1 :green_heart:
mvnsite
2m 10s

the patch passed

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 26s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m  0s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 43s

the patch passed

+1 :green_heart:
shadedclient
21m 58s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 21s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
225m 50s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

356m  7s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux 8b50538bfabd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d7d2dc47ed9dc4e58e0b6fc854f8d9e145e24292

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/1/testReport/

Max. process+thread count
2818 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 44s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 43s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
23m 24s

trunk passed

+1 :green_heart:
compile
6m  8s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m 27s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 16s

trunk passed

+1 :green_heart:
mvnsite
2m 23s

trunk passed

+1 :green_heart:
javadoc
1m 39s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m  3s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 56s

trunk passed

+1 :green_heart:
shadedclient
23m  6s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 12s

the patch passed

+1 :green_heart:
compile
5m 44s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m 44s

the patch passed

+1 :green_heart:
compile
5m 10s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
5m 10s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m  9s
/results-checkstyle-hadoop-hdfs-project.txt
hadoop-hdfs-project: The patch generated 3 new + 381 unchanged - 1 fixed = 384 total (was 382)

+1 :green_heart:
mvnsite
2m 12s

the patch passed

+1 :green_heart:
xml
0m  2s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 27s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 56s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
6m  4s

the patch passed

+1 :green_heart:
shadedclient
23m 15s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 25s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
232m  0s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 47s

The patch does not generate ASF License warnings.

367m 27s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux daf487591f77 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3e323a1a3efe46af94d9068ee66f96ecf56767b5

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/2/testReport/

Max. process+thread count
3869 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@symious Sorry for reviewing it very late. I looked through it, and it mostly looks good to me.
I have one question. This feature is similar to ECN. The configuration of ECN is dfs.pipeline.ecn, which is for the DataNode configuration. dfs.client.mark.slownode.as.badnode seems the client configuration. If it works the same way as ECN, it would be better to make it on the DataNode configuration. What do you think?

@tasanuma Thanks for the review.

I have one question. This feature is similar to ECN. The configuration of ECN is dfs.pipeline.ecn, which is for the DataNode configuration. dfs.client.mark.slownode.as.badnode seems the client configuration

I tried to digest the design of the code again. IMHO, the config of "dfs.pipeline.ecn" is like "dfs.pipeline.ecn.enable", the value of "true" means datanode should add the ECN information in PipelineAck, so this configuration is for DataNode to add or not add ECN. On Client side, there is no config for clients to choose if she should use ECN information, since it will use the information by default.
In this ticket, I think a similar config of "dfs.pipeline.ecn" should be "dfs.pipeline.reply_slownode_in_pipeline_ack.enable", based on which DataNodes will choose to add or skip the slownode information in PipelineAck.
On Client side, "dfs.client.mark.slownode.as.badnode" is more like "dfs.client.handle.ECN_from_pipelineACK".
I think it's more depend on the subject of the operation, that is the subject of "dfs.pipeline.ecn" is DataNode, while the subject of "dfs.client.mark.slownode.as.badnode" is Client.

@symious Thanks for your explanation.
So my question is which do you think better to implement the configuration, on the client side (dfs.client.mark.slownode.as.badnode) or on the datanode side (dfs.pipeline.reply_slownode_in_pipeline_ack.enable)? If this feature is similar to ECN, this ticket would be better to implement dfs.pipeline.reply_slownode_in_pipeline_ack.enable. Or do you think dfs.client.mark.slownode.as.badnode is better?

@tasanuma Sorry for the late reply.

If this feature is similar to ECN, this ticket would be better to implement dfs.pipeline.reply_slownode_in_pipeline_ack.enable. Or do you think dfs.client.mark.slownode.as.badnode is better?

Since it's similar to ECN, I think it's reasonable to set on the server side.
Another problem I was thinking is the configuration of "dfs.client.mark.slownode.as.badnode.threshold", this configuration seems more related to the choice of Clients, it might be a little rigid to set a default value on server side. Should I leave this configuration on client side?

@symious Thanks for your reply.

Since it's similar to ECN, I think it's reasonable to set on the server side.

I agreed. Thanks.

Another problem I was thinking is the configuration of "dfs.client.mark.slownode.as.badnode.threshold", this configuration seems more related to the choice of Clients, it might be a little rigid to set a default value on server side. Should I leave this configuration on client side?

I also agreed that dfs.client.mark.slownode.as.badnode.threshold is more related to clients. Let's leave it on the client side.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 45s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  1s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 47s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m 42s

trunk passed

+1 :green_heart:
compile
5m 41s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m 15s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 19s

trunk passed

+1 :green_heart:
mvnsite
2m 50s

trunk passed

+1 :green_heart:
javadoc
2m  4s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 30s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
7m  9s

trunk passed

+1 :green_heart:
shadedclient
30m  4s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 31s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 37s

the patch passed

+1 :green_heart:
compile
6m 57s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
6m 57s

the patch passed

+1 :green_heart:
compile
6m 21s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
6m 21s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m 26s
/results-checkstyle-hadoop-hdfs-project.txt
hadoop-hdfs-project: The patch generated 1 new + 327 unchanged - 1 fixed = 328 total (was 328)

+1 :green_heart:
mvnsite
2m 29s

the patch passed

+1 :green_heart:
xml
0m  2s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 42s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 23s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
7m 22s

the patch passed

+1 :green_heart:
shadedclient
29m 43s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 32s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
244m 41s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

399m 23s

Reason
Tests

Failed junit tests
hadoop.hdfs.TestLeaseRecoveryStriped

hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux 4428dba61480 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 54045ae141ea1076395cfaa4230878ace3e02c74

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/3/testReport/

Max. process+thread count
3370 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 57s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
21m 33s

trunk passed

+1 :green_heart:
compile
5m 23s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m  5s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 15s

trunk passed

+1 :green_heart:
mvnsite
2m 26s

trunk passed

+1 :green_heart:
javadoc
1m 45s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 12s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 39s

trunk passed

+1 :green_heart:
shadedclient
22m 13s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 27s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  3s

the patch passed

+1 :green_heart:
compile
5m 14s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m 14s

the patch passed

+1 :green_heart:
compile
5m  2s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
5m  2s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m  6s
/results-checkstyle-hadoop-hdfs-project.txt
hadoop-hdfs-project: The patch generated 1 new + 327 unchanged - 1 fixed = 328 total (was 328)

+1 :green_heart:
mvnsite
2m  7s

the patch passed

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 27s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m  2s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 50s

the patch passed

+1 :green_heart:
shadedclient
22m 23s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 24s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
224m 41s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

354m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux f06149a5f0dc 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / df061adfa2fc13ff6f2922dc7c8b1a7cf94aea5a

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/4/testReport/

Max. process+thread count
3223 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 59s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
27m 51s

trunk passed

+1 :green_heart:
compile
5m 22s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m  1s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 18s

trunk passed

+1 :green_heart:
mvnsite
2m 24s

trunk passed

+1 :green_heart:
javadoc
1m 46s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 12s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 37s

trunk passed

+1 :green_heart:
shadedclient
22m 49s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 11s

the patch passed

+1 :green_heart:
compile
5m 37s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m 37s

the patch passed

+1 :green_heart:
compile
5m  8s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
5m  8s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  5s

hadoop-hdfs-project: The patch generated 0 new + 528 unchanged - 1 fixed = 528 total (was 529)

+1 :green_heart:
mvnsite
2m 12s

the patch passed

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 41s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 17s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
7m 35s

the patch passed

+1 :green_heart:
shadedclient
30m 23s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 35s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
237m  7s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 46s

The patch does not generate ASF License warnings.

384m 49s

Reason
Tests

Failed junit tests
hadoop.tools.TestHdfsConfigFields

hadoop.hdfs.TestErasureCodeBenchmarkThroughput

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux b88dbf38ba3a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 020093e041f89907dca85146919fddf9a2ff100d

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/5/testReport/

Max. process+thread count
3410 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 39s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
13m  8s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 31s

trunk passed

+1 :green_heart:
compile
6m 11s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m 52s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 15s

trunk passed

+1 :green_heart:
mvnsite
2m 30s

trunk passed

+1 :green_heart:
javadoc
1m 48s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 12s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
6m  7s

trunk passed

+1 :green_heart:
shadedclient
24m 24s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 27s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  9s

the patch passed

+1 :green_heart:
compile
5m 33s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m 33s

the patch passed

+1 :green_heart:
compile
5m  2s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
5m  2s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  8s

hadoop-hdfs-project: The patch generated 0 new + 528 unchanged - 1 fixed = 528 total (was 529)

+1 :green_heart:
mvnsite
2m 15s

the patch passed

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 29s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 55s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 57s

the patch passed

+1 :green_heart:
shadedclient
22m 46s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 23s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
228m 36s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 46s

The patch does not generate ASF License warnings.

367m 50s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux 858276b9068a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3e8b2807c587aa396909d71dea3426874ef6a3c2

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/6/testReport/

Max. process+thread count
3285 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@tasanuma Updated the patch, could you help to check?

@tasanuma Thanks for the detailed review. Updated as suggested, please have a check.

Thanks for updating PR, @symious. +1 from me, pending Jenkins.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 42s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 52s

trunk passed

+1 :green_heart:
compile
5m 59s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m 39s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 15s

trunk passed

+1 :green_heart:
mvnsite
2m 22s

trunk passed

+1 :green_heart:
javadoc
1m 40s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m  2s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 53s

trunk passed

+1 :green_heart:
shadedclient
25m 26s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 24s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  7s

the patch passed

+1 :green_heart:
compile
5m 53s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m 53s

the patch passed

+1 :green_heart:
compile
5m 27s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
5m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  9s

hadoop-hdfs-project: The patch generated 0 new + 528 unchanged - 1 fixed = 528 total (was 529)

+1 :green_heart:
mvnsite
2m 10s

the patch passed

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 26s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 53s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
6m  9s

the patch passed

+1 :green_heart:
shadedclient
25m 50s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 18s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
320m 50s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 40s

The patch does not generate ASF License warnings.

462m 26s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.balancer.TestBalancer

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux dd4f5befacaf 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 6a6875d588bd3aab072a48409376c29c45b85c80

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/7/testReport/

Max. process+thread count
2257 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 32s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m 29s

trunk passed

+1 :green_heart:
compile
6m  6s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m 32s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 16s

trunk passed

+1 :green_heart:
mvnsite
2m 20s

trunk passed

+1 :green_heart:
javadoc
1m 40s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m  6s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
5m 55s

trunk passed

+1 :green_heart:
shadedclient
25m 27s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 22s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  8s

the patch passed

+1 :green_heart:
compile
5m 54s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m 54s

the patch passed

+1 :green_heart:
compile
5m 26s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
5m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 12s

hadoop-hdfs-project: The patch generated 0 new + 528 unchanged - 1 fixed = 528 total (was 529)

+1 :green_heart:
mvnsite
2m 11s

the patch passed

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
javadoc
1m 27s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 57s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
6m  6s

the patch passed

+1 :green_heart:
shadedclient
25m 52s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 17s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
313m 54s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 39s

The patch does not generate ASF License warnings.

455m  2s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3704

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml

uname
Linux 180c2a1af2dc 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 97485b93f414733d024e567489315b91582e9096

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/8/testReport/

Max. process+thread count
2358 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3704/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Merged it. Thanks for your great work, @symious!


Improve the Assembly interface to support specification of combinatorial designs

Codecov Report

Merging #47 (a5872e0) into master (6048dc4) will increase coverage by 0.98%.
The diff coverage is 82.35%.

@@            Coverage Diff             @@
##           master      #47      +/-   ##
==========================================
+ Coverage   81.52%   82.51%   +0.98%
==========================================
  Files          43       43
  Lines        1532     1590      +58
==========================================
+ Hits         1249     1312      +63
+ Misses        283      278       -5

Flag
Coverage Δ

unittests
82.51% <82.35%> (+0.98%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

supergsl/plugins/builtin/print.py
66.66% <0.00%> (ø)

supergsl/core/eval.py
93.00% <33.33%> (-0.94%)
:arrow_down:

supergsl/core/pipeline.py
94.44% <33.33%> (-5.56%)
:arrow_down:

supergsl/core/types/base.py
60.00% <50.00%> (-6.67%)
:arrow_down:

supergsl/core/types/codon.py
92.85% <50.00%> (-7.15%)
:arrow_down:

supergsl/plugins/sbol/sbol_output.py
28.88% <50.00%> (ø)

supergsl/core/assembly.py
66.66% <66.66%> (ø)

supergsl/core/types/assembly.py
90.78% <89.65%> (+20.78%)
:arrow_up:

supergsl/core/types/part.py
92.30% <100.00%> (+0.30%)
:arrow_up:

supergsl/plugins/builtin/fuse.py
100.00% <100.00%> (+57.14%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6048dc4...a5872e0. Read the comment docs.


pip install doesnt work with python 3.4, windows x64

I tried this with 2 fresh windows installations and got the same error both times.
`c:\Python34\Scripts>pip install quiver_engine
Collecting quiver_engine
Downloading quiver_engine-<IP_ADDRESS>.1.tar.gz (398kB)
100% |################################| 399kB 1.4MB/s
Requirement already satisfied: keras in c:\python34 (from quiver_engine)
Collecting flask (from quiver_engine)
Downloading Flask-0.11.1-py2.py3-none-any.whl (80kB)
100% |################################| 81kB 2.6MB/s
Collecting flask_cors (from quiver_engine)
Downloading Flask_Cors-3.0.2-py2.py3-none-any.whl
Collecting gevent (from quiver_engine)
Downloading gevent-1.1.2-cp34-cp34m-win_amd64.whl (384kB)
100% |################################| 389kB 1.5MB/s
Requirement already satisfied: numpy in c:\python34\lib\site-packages (from quiver_engine)
Requirement already satisfied: pillow in c:\python34\lib\site-packages (from quiver_engine)
Requirement already satisfied: theano in c:\python34\lib\site-packages (from keras->quiver_engine)
Requirement already satisfied: pyyaml in c:\python34\lib\site-packages (from keras->quiver_engine)
Requirement already satisfied: six in c:\python34\lib\site-packages (from keras->quiver_engine)
Collecting itsdangerous>=0.21 (from flask->quiver_engine)
Downloading itsdangerous-0.24.tar.gz (46kB)
100% |################################| 51kB 4.3MB/s
Collecting Werkzeug>=0.7 (from flask->quiver_engine)
Downloading Werkzeug-0.11.11-py2.py3-none-any.whl (306kB)
100% |################################| 307kB 2.1MB/s
Collecting click>=2.0 (from flask->quiver_engine)
Downloading click-6.6.tar.gz (283kB)
100% |################################| 286kB 2.8MB/s
Collecting Jinja2>=2.4 (from flask->quiver_engine)
Downloading Jinja2-2.8-py2.py3-none-any.whl (263kB)
100% |################################| 266kB 3.1MB/s
Collecting greenlet>=0.4.9 (from gevent->quiver_engine)
Downloading greenlet-0.4.10-cp34-cp34m-win_amd64.whl
Requirement already satisfied: scipy>=0.11 in c:\python34\lib\site-packages (from theano->keras->quiver_engine)
Collecting MarkupSafe (from Jinja2>=2.4->flask->quiver_engine)
Downloading MarkupSafe-0.23.tar.gz
Installing collected packages: itsdangerous, Werkzeug, click, MarkupSafe, Jinja2, flask, flask-cors, greenlet, gevent, quiver-engine
Running setup.py install for itsdangerous ... done
Running setup.py install for click ... done
Running setup.py install for MarkupSafe ... done
Running setup.py install for quiver-engine ... error
Complete output from command c:\python34\python.exe -u -c "import setuptools, tokenize;file='C:\Users\idg101\AppData\Local\Temp\pip-buil
d-8ffm_h7y\quiver-engine\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, fi
le, 'exec'))" install --record C:\Users\idg101\AppData\Local\Temp\pip-2amt2zih-record\install-record.txt --single-version-externally-managed --compi
le:
running install
running build
running build_py
creating build
creating build\lib
creating build\lib\quiver_engine
copying quiver_engine\imagenet_utils.py -> build\lib\quiver_engine
copying quiver_engine\layer_result_generators.py -> build\lib\quiver_engine
copying quiver_engine\server.py -> build\lib\quiver_engine
copying quiver_engine\util.py -> build\lib\quiver_engine
copying quiver_engine_init_.py -> build\lib\quiver_engine
running egg_info
writing quiver_engine.egg-info\PKG-INFO
writing dependency_links to quiver_engine.egg-info\dependency_links.txt
writing requirements to quiver_engine.egg-info\requires.txt
writing top-level names to quiver_engine.egg-info\top_level.txt
warning: manifest_maker: standard file '-c' not found
reading manifest file 'quiver_engine.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'quiver_engine.egg-info\SOURCES.txt'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\idg101\AppData\Local\Temp\pip-build-8ffm_h7y\quiver-engine\setup.py", line 20, in <module>
    'pillow'
  File "c:\python34\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "c:\python34\lib\distutils\dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "c:\python34\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "c:\python34\lib\site-packages\setuptools\command\install.py", line 61, in run
    return orig.install.run(self)
  File "c:\python34\lib\distutils\command\install.py", line 539, in run
    self.run_command('build')
  File "c:\python34\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "c:\python34\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "c:\python34\lib\distutils\command\build.py", line 126, in run
    self.run_command(cmd_name)
  File "c:\python34\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "c:\python34\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line 47, in run
    self.build_package_data()
  File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line 103, in build_package_data
    for package, src_dir, build_dir, filenames in self.data_files:
  File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line 59, in __getattr__
    self.data_files = files = self._get_data_files()
  File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line 86, in _get_data_files
    file[plen:] for file in self.find_data_files(package, src_dir)
  File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line 94, in find_data_files
    + self.package_data.get(package, []))
TypeError: can only concatenate list (not "str") to list

----------------------------------------

Command "c:\python34\python.exe -u -c "import setuptools, tokenize;file='C:\Users\idg101\AppData\Local\Temp\pip-build-8ffm_h7y\quiver-engin
e\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install
--record C:\Users\idg101\AppData\Local\Temp\pip-2amt2zih-record\install-record.txt --single-version-externally-managed --compile" failed with error co
de 1 in C:\Users\idg101\AppData\Local\Temp\pip-build-8ffm_h7y\quiver-engine
`

Hmm I do not have easy access to a windows machine to test this at the moment, would love it if someone else can send a PR for this

Can you point me to a tutorial on how files for pip are created?  I'd be happy to debug; I've never deployed with pip before.

I get a similar error with RHEL 6 and Python 2.7.

The same issue for me at windows10 + python3.4

Had the same problem with python 3.5 on Fedora 21.
It worked for me when I replaced 'quiverboard/dist/*' with ['quiverboard/dist/*'] in setup.py, compiled the client and then installed with python setup.py install.

Same for me on Ubuntu 16.04 using Python 3.5.2.
Following @octycs advide and running pip3 install -e . did the trick.

agree with upstair, detail desrcription:

search keyword "quiver“ at pypi webside (https://pypi.python.org/pypi);
download one of quiver package name "quiver_engine <IP_ADDRESS>.4";
unzip it on local disk and "cd ....\quiver_engine <IP_ADDRESS>.4\”;
just run "pip install -e .", it will auto call setup.py, then quiver_engine installed without any problem;
run python under dos "cmd";
just for test, type "from quiver_engine impor server" and no error is reported;
just for use in keras, look detail on function server.lunch, type "help(quiver_engine.server.lanuch", you will find it suggest only 5 parameter like below and NEEDN'T CLASSES MEAN STD TOP these 4 parameters!!!"

help(server.launch)
Help on function launch in module quiver_engine.server:
launch(model, temp_folder='./tmp', input_folder='./', port=5000, html_base_dir=None)

it you add codes below, you MUST first make dir you need first!
server.launch(model = model,temp_folder='./quiver/tmp',input_folder='./quiver/input_imgs',port=5000)
9, run codes with launch function in pycharm or python, quiver_engine will OPEN A WEBPAGE show your images (you FIRST copy to input_folder before run codes) and select it to see feature maps in your diagram show also on web page's left frame.
if it run OK as mentioned in step 9, you trains will BLOCKED by quiver_engine!


feat: dynamic test data generation with faker.js

Purpose
Static test data files are not fully representative of real-world scenarios and can pose problems when using alternate keys. I believe this resolves #40, #16.
Approach
Building upon #41, this will allow us to hook into the popular faker.js library for dynamically generating test data.
TODOs

[x] Automated test coverage for new code
[x] Documentation updated (if required)
[x] Build and tests successful

Happy with the way it looks. Understanding of faker will be required to get this rolling. But it is documented in the readme to be fair.


Differentiate extent and size

Overloading extent to include the size in pages of a book (3 pp vs pp 3-5) is confusing and counterproductive; the former should be medium/size, and medium/size needs to include localities for ranges.

Actually, we cannot use medium/size, that's used for dimensions like 8vo. We need a new field, next to extent.

Agree.

Just discovered that l10n for Chinese is not coping with mixed Chinese–Latin text: spaces should only be stripped between Chinese characters.


Library/package research

Need to use at least one new library, package, or technology that wasn't discussed in class

Possible libraries for sending emails through the application:

https://www.courier.com/docs/getting-started/quickstarts/nodejs/
https://nodemailer.com/about/

Decided to use courier.js for email functionality that sends a welcome email when the user signs up.


Document helm template name and namespace annotations

There's 2 experimental annotations that helps helmTemplate processing to define the name and namespace, but these are not documented:
https://github.com/vmware-tanzu/carvel-kapp-controller/blob/22d3f3a23048f41b9dc7e9c56944ef67c2819758/pkg/packageinstall/app.go#L24-L25
They should be in the docs.

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

Thanks @jorgemoralespou  -
The team that's working in this codebase now didn't write those annotations and we just learned about them through this ticket as well!
Would you describe them something like this:?
These annotations allow the package consumer to override the name and namespace that will be passed to the helm chart.

That seems like it, but you should ask the person that added the annotations, and not me :-D

it looks like there's some descriptions in this issue: https://github.com/vmware-tanzu/carvel-kapp-controller/issues/409

@joe-kimmel-vmw was this ever documented?


Unexpected consent prompts DevEx video incorporation

[Enter feedback here]
Hi, I am a PM in the Developer care team and I have made a DevEx video for the Microsoft Security channel with content from this page I believe we should embed that video on this page to accommodate our customers who prefer video content. The video is here. https://www.youtube.com/watch?v=a1AjdvNDda4 Please add it to the page asap

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 27339f6f-c63d-d2cc-ec02-8727b5587bf0
Version Independent ID: 8edacd13-bbd1-cfc7-e686-7ca7bf4cb2ad
Content: Unexpected consent prompt when signing in to an application
Content Source: articles/active-directory/manage-apps/application-sign-in-unexpected-user-consent-prompt.md
Service: active-directory
Sub-service: app-mgmt
GitHub Login: @kenwith
Microsoft Alias: kenwith

@jeevanbalanmanoj
Thanks for your feedback! I've assigned this issue to the author who will investigate and update as appropriate.

@JamesTran-MSFT Are there any updates on this?
@mmacy @knicholasa Can you two please help expedite this request?

@jeevanbalanmanoj
Thank you for the follow up on this! If this is an urgent doc enhancement request, I'd recommend creating your own PR so one of our doc authors can review/sign-off as soon as possible.
PR process - Quick edits to existing documents
Adding @kenwith for visibility and to provide any updates.

#please-close


cocoapods: 1.15.1 -> 1.15.2

Description of changes
https://github.com/CocoaPods/CocoaPods/releases/tag/1.15.2

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[x] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Would be great to get this merged sooner than later since 1.15.1 has a regression thats causing issues with installing xcframeworks.

I would also appreciate if this could get merged. I currently have to install cocoapods from brew as nixpkgs is still on 1.15.1 which contains a bug.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1524

Result of nixpkgs-review pr 289068 run on aarch64-darwin 1

2 packages built:

cocoapods
cocoapods-beta


API to verify upload ability without actually uploading

What's the problem this feature will solve?
I want to write release scripts which check whether the necessary credentials are in place to upload a package, before it starts making changes as part of the release process.
This could also be integrated with other pre-upload checks, like 'does this release already exist?'
Describe the solution you'd like
An API to which I can make an authenticated request with a package name. The response should indicate whether the credentials given are sufficient to upload a new release.
At the moment, I'm thinking of this as not taking an actual upload. But a possible addition would be to allow uploading a package and check if it could be successfully uploaded, without publishing it.
Additional context

There is now PEP 694: Upload 2.0 API for Python Package Repositories, which has discussions on discuss.python.org which is relevant to this issue.


NLog.Schema not installed when adding NLog.Config

Reproduction steps:
Get netcore 3.0 preview sdk

dotnet new wpf -n SomeProject
open project
add NLog
add NLog.Config

While NLog.Config does reference NLog.Schema it does not end up getting referenced - Manually adding the package does work however.

Thanks for reporting and writing explicit the steps 👍
I think this is #2823, isn't?

@304NotModified thanks for reply - yep. very much does look like it might be related to #2823

OK thanks, closing this as duplicate. Thanks for reporting!


Assistant transferres GMT+01:00 Order timestamps as UTC

Used Shopware Version: <IP_ADDRESS>
Description:
when you migrate orders there seems to happen a timeshift on the timestamps when not both systems are set to utc?
Environment:
PHP 8.2.16,
see SW6 version + Assistance version,
Server time said Fri Feb 23 11:19:05 CET 2024
source and target database-version is
10.6.17-MariaDB-1:10.6.17+maria~deb11
and CURTIME() also always shows CET (currently UTC+01:00) timestamps
Steps to reproduce:
Use a SW6 account with Timezone set to "(UTC +01:00) Europe/Berlin" (not sure if relevant to reproduce) then configure the Migration Assistant - in this case we only migrated Customers + Orders (besides the enforced default Salechannel+Categories Migration setting)
Expected result:
Have the timestamps set and calculated according to the old systems
Actual result:
You happen to even see orders in the new SW6 Admin which are set to happen "in 34 Minutes" then just in the past (due to the one hour time shift)  [Screenshot_20240223_132750-1.jpg|https://sbp-issuetracker-attachments.s3.eu-west-1.amazonaws.com/1708691643_65d890bba313d_1708691596_65d8908c4b4b7_Screenshot_20240223_132750-1.jpg]  [Screenshot_20240223_130234-1.jpg|https://sbp-issuetracker-attachments.s3.eu-west-1.amazonaws.com/1708691644_65d890bc21256_1708691596_65d8908c78f40_Screenshot_20240223_130234-1.jpg]

[public] Automated response: This issue is linked to the internal issue https://shopware.atlassian.net/browse/MIG-946.
[created from MIG-946, comment 484786]

I can confirm this Bug, the data gets migrated with the exact date, but SW5 uses GMT+2, while SW6 uses UTC. Thus all Orders have a timeshift of 2Hours. New Orders are shown to be in the future, as was mentioned in the initial Post.
SW <IP_ADDRESS>
Assistant: 13.2.0
Had the PHP-Timezone default at first, then tried out if setting it to "Europe/Berlin" and UTC would change anything.
Setting the Profile Timezone also did not help, I could change it to UTC to see the actual Times, but i guess that Customers in the Shop-Frontend would still see the wrong time.

[public] Hello,
Within our company roadmap and work capacity, we try to address each bug or improvement request but admit that not each one will be resolved. To continue our culture of honesty and openness, we are closing this issue to focus on our roadmap on behalf of all Shopware users.
If you feel like this issue is still impacting you, please create a new issue and let us know. Thank you again for your valuable feedback.
[created from MIG-946, comment 501322]


Recursive Paths Example Crashes

https://reacttraining.com/react-router/web/example/recursive-paths
Click any link in the example and the console outputs:
The above error occurred in the <Person> component:
    in Person (created by Context.Consumer)
    in Route (at example.js:39)
    in div (at example.js:27)
    in Person (at example.js:18)
    in Router (created by BrowserRouter)
    in BrowserRouter (at example.js:17)
    in RecursiveExample (at index.js:14)

Already fixed in #6957, just not published yet.


current_date syntax in knex

Can anyone help to find a right syntax? I got a complaint on an syntax error near "30 days".
Where(knex.raw('date > (CURRENT_DATE - INTERVAL "30 days")'))
Thanks.

From the MySQL docs on dates/times it looks like quotes are a no go, so INTERVAL 30 DAY should work.


Handle JSON Parser Error

When I call Codebase::TimeSession.delete I get a  JSON Parser Error because the response from codebase is just a blank space.  I added a rescue statement to catch this and print the error, unless there is a better idea?

Great, thanks :+1:


PLAT-81 update ref to terraform-modules

Update ref (v0.13.6) to terraform-modules

I finally tested and verified it works too. (After upping the secgroup quota in osl2/sandbox)


Refine quantizer for supporting larger hidden-dim and group size

Can one of the admins verify this patch?


Update for 0.9.5. Still keep 0.9.4 on index.

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

If this PR is for a new extension or change to an existing extension, use the following to try out the changes in this PR:
docker run -it microsoft/azure-cli:latest
export EXT=<NAME>
pip install --upgrade --target ~/.azure/cliextensions/$EXT "git+https://github.com/digimaun/azure-cli-extensions.git@iotext095#subdirectory=src/$EXT&egg=$EXT"

add to S171


casesfirst commit for array cases

What problem does this PR solve?
this is some special data case for array , and main to use in function and ut for making up the array cases
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[ ] Other reason

Behavior changed:

[ ] No.
[ ] Yes.

Does this need documentation?

[ ] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

run buildall

run buildall

run p0

run buildall


Command History Not Shown In PowerShell 7

Prerequisites

[x] I have read and understand the CONTRIBUTING guide
[x] I looked for duplicate issues before submitting this one

Description
PowerShell 5 can show the last command I typed. But PowerShell 7 can't do that.
Environment

Oh my Posh version: 3.51
Theme: jandedobbeleer (Default Theme)
Operating System: Windows 10 Insiders 2004 Build 20270.1
Shell: pwsh 5 & 7
Terminal: native windows

Steps to Reproduce

Install using Scoop:

mkdir C:\tools
Invoke-Webrequest https://github.com/JanDeDobbeleer/oh-my-posh3/releases/latest/download/posh-windows-amd64.exe -OutFile C:\tools\oh-my-posh.exe

Edit both $PROFILE in respected powershell version folder:
PowerShell 5: C:\Users\User\Documents\WindowsPowerShell
PowerShell 7: C:\Users\User\Documents\PowerShell

[ScriptBlock]$Prompt = {
    $lastCommandSuccess = $?
    $realLASTEXITCODE = $global:LASTEXITCODE
    $errorCode = 0
    if ($lastCommandSuccess -eq $false) {
        #native app exit code
        if ($realLASTEXITCODE -ne 0) {
            $errorCode = $realLASTEXITCODE
        }
        else {
            $errorCode = 1
        }
    }
    $startInfo = New-Object System.Diagnostics.ProcessStartInfo
    $startInfo.FileName = "C:\tools\oh-my-posh.exe"
    $cleanPWD = $PWD.ProviderPath.TrimEnd("\")
    $startInfo.Arguments = "-config=""C:\tools\poshthemes\jandedobbeleer.json"" -error=$errorCode -pwd=""$cleanPWD"""
    $startInfo.Environment["TERM"] = "xterm-256color"
    $startInfo.CreateNoWindow = $true
    $startInfo.StandardOutputEncoding = [System.Text.Encoding]::UTF8
    $startInfo.RedirectStandardOutput = $true
    $startInfo.UseShellExecute = $false
    if ($PWD.Provider.Name -eq 'FileSystem') {
      $startInfo.WorkingDirectory = $PWD.ProviderPath
    }
    $process = New-Object System.Diagnostics.Process
    $process.StartInfo = $startInfo
    $process.Start() | Out-Null
    $standardOut = $process.StandardOutput.ReadToEnd()
    $process.WaitForExit()
    $standardOut
    $global:LASTEXITCODE = $realLASTEXITCODE
    #remove temp variables
    Remove-Variable realLASTEXITCODE -Confirm:$false
    Remove-Variable lastCommandSuccess -Confirm:$false
}
Set-Item -Path Function:prompt -Value $Prompt -Force

Reload profile / reenter powershell.

Expected behavior: It can show autosuggestions (command suggestion based on command history that I typed). oh-my-posh 2 can show that. But it broke, so I deleted everything about oh-my-posh2.
Actual behavior: PowerShell 7 can't show command suggestions.

I tried to reinstall oh-my-posh3 based on: https://ohmyposh.dev/docs/installation#4-replace-your-existing-prompt, it didn't work at all

Are you talking about command history or autocompletion of commands?
I tried here with the latest version and when I type "$PS" and hit tab, the autocompletion takes place:

And for the profile, I use this:
import-module oh-my-posh
import-module posh-git
Import-Module psini
 Set-PoshPrompt jandedobbeleer

it's easier than copy pasting the entier powershell block. It's useful if you want to mess around with it but since everything is in the powershell module, using it is easier.

@rescenic command completion has nothing to do with oh-my-posh (regardless of V2 or V3) as it's a feature enabled by PSReadline. Most likely that one's not set up correctly for your PS7.1 shell, there's more information to be found on that here.
Oh my Posh overwrites the prompt function to display a different styled prompt, and does not touch anything outside of that context (in line with the UNIX philosophy).


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_f7800998-2fa3-4978-b10e-22e388e61b14/pull/1 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-target_f7800998-2fa3-4978-b10e-22e388e61b14.git

Docs Build status updates of commit 858fed0:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error-RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://aka.ms/SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


A few shutdown & re-init bugs

A few issues encountered when trying to get shutdown and re-init to work.
The program simply inits bgfx, shuts it down, then inits it again using the same Init struct.

Single-thread mode, 2nd init() ends up entering multi-thread mode and freezes. There are two conflicting log lines at the end of below:

I0705 02:31:26.442129   71144 app.cc:309] Paused: true, Suspended: true
I0705 02:31:26.445782   71144 bgfx.cpp:3611] BGFX Shutdown...
I0705 02:31:26.525306   71144 bgfx.cpp:3620] BGFX Shutdown complete.
I0705 02:31:30.762280   71144 app.cc:309] Paused: false, Suspended: false
I0705 02:31:30.762474   71144 bgfx.cpp:3559] BGFX Init...
I0705 02:31:30.762566   71144 bgfx.cpp:3566] BGFX Version 1.128.8775 (commit: a476c5b9a42d3779af59a0099d4d222fa8898d36)
I0705 02:31:30.772927   71144 bgfx.cpp:1736] BGFX Reset back-buffer swap chain:
I0705 02:31:30.773095   71144 bgfx.cpp:1745] BGFX       1280x720, format: RGBA8, numBackBuffers: 2, maxFrameLatency: 0
I0705 02:31:30.773172   71144 bgfx.cpp:1747] BGFX       [ ] MSAAx1
I0705 02:31:30.773249   71144 bgfx.cpp:1748] BGFX       [ ] Fullscreen
I0705 02:31:30.773321   71144 bgfx.cpp:1749] BGFX       [x] V-sync
I0705 02:31:30.773387   71144 bgfx.cpp:1750] BGFX       [ ] Max Anisotropy
I0705 02:31:30.773444   71144 bgfx.cpp:1751] BGFX       [ ] Capture
I0705 02:31:30.773504   71144 bgfx.cpp:1752] BGFX       [ ] Flush After Render
I0705 02:31:30.773550   71144 bgfx.cpp:1753] BGFX       [ ] Flip After Render
I0705 02:31:30.773595   71144 bgfx.cpp:1754] BGFX       [ ] sRGB Back Buffer
I0705 02:31:30.773639   71144 bgfx.cpp:1755] BGFX       [ ] Transparent Back Buffer
I0705 02:31:30.773684   71144 bgfx.cpp:1756] BGFX       [ ] HDR10
I0705 02:31:30.773733   71144 bgfx.cpp:1757] BGFX       [ ] Hi-DPI
I0705 02:31:30.773788   71144 bgfx.cpp:1758] BGFX       [ ] Depth Clamp
I0705 02:31:30.773832   71144 bgfx.cpp:1759] BGFX       [ ] Suspend
I0705 02:31:30.776838   71144 bgfx.cpp:1932] BGFX Application called bgfx::renderFrame directly, not creating render thread.
I0705 02:31:30.776922   71144 bgfx.cpp:1948] BGFX Running in multi-threaded mode

Looks like it's relevant to the BGFX_API_THREAD_MAGIC check

Multi-thread mode. Mostly works but after 2nd init clearing stops working, with all backends. (fixed, user error)

In both modes, below warning appears during shutdown:

d3d11 backend:
BGFX WARN<PHONE_NUMBER>776C90 RefCount is 3 (expected 0).

d3d12 backend:
BGFX WARN 000001D96E7DB628 RefCount is 530 (expected 0).

GL and Vulkan seems fine. Metal and GLES not tested.
This issue is not as important as the first 2 since shutdown & re-init is not as useful on Windows.

Check your setup, it looks like something is attaching to your D3D device.

What do you mean by attaching to? Which issue are you responding to? If you mean issue 3, I only mean to report the potential issue and I can live with it. Issue 1 and issue 2 are more blocking.
Reproduced issue 1 on Mac + Metal. Issue 2... never got multi-threading to work with Mac + Metal.

Regarding issue 1, I faced a similar problem. The reason I suspect is that s_renderFrameCalled is not reset on shutdown, causing the second init to not follow the same path. Here is the patch I use to solve it on my end: https://github.com/bkaradzic/bgfx/commit/0a168f901142418a413c1274e4ed83ea09deb634#diff-75ed10dd2a2698c8707c9c9750ec40b3d12bbdf747f0c18514ed89caa3b2c522R3638.
I did not report here nor proposed a PR yet to avoid making a wrong report due to limited BGFX knowledge, and make bkaradzic loose his time on it. We will finish porting our project (Visual Pinball X, OSS pinball simulator) and get back with bug report/fix when we'll have better validations/test cases performed.


Add color's docs

Add docs
Add variable for inverse background
Add inverse variant

Pull Request Test Coverage Report for Build 75

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 73.538%

Totals

Change from base Build 74:
0.0%

Covered Lines:
2176

Relevant Lines:
2796

💛  - Coveralls


Doxygen doc update for heat_transfer.h

Description of the problem

Doxygen documentation incomplete for heat_transfer.h

Description of the solution

Updated it

How Has This Been Tested?

Web verification.

Comments

First PR.
Part of Doxygen documentation hackathon.

Is this ready for a re-review?

Is this ready for a re-review?

Yess it is!


Prevent FocusTrap from being selectable by default.

There is a blue ring when selecting the wrapper element - which is probably not desirable in most cases.

Hey! Thanks for the pull request.
Sadly I'm going to have to reject this one since we should not be opinionated about style. That is much beyond the scope of hotkeys. It's very easy for users of the library to override this behaviour.

Also note that it's not intended that we output a surrounding div at the moment and this is an api flaw for sure which will be resolved in a newer version.
In the future we will simply enhance any element/component (eg. A higher order component) and at that point we cannot provide a solution for merging styles appropriately which covers everyone's needs.


Use kernel.org clang whenever possible

This moves our builds away from Debian's LLVM to the kernel.org LLVM builds I maintain, which are faster and within our control for updates.
I would like to do this for clang-nightly eventually but I have to make sure it is maintainable before doing so, so this just does the stable LLVM builds for now.

oh :(
Why faster? pgo/lto?

oh :(

You have done a great job with apt.llvm.org and I continue to recommend it for people who want continuous updates automatically from their package manager. This pull request is not a reflection of that at all.

Why faster? pgo/lto?

Yes, I apply PGO, ThinLTO, and BOLT whenever possible, which can result in 50%+ speed up at run time.
https://gitlab.com/Linaro/tuxmake/-/issues/215
https://mirrors.edge.kernel.org/pub/tools/llvm/

You have done a great job with apt.llvm.org and I continue to recommend it for people who want continuous updates automatically from their package manager. This pull request is not a reflection of that at all.

thanks

Yes, I apply PGO, ThinLTO, and BOLT whenever possible, which can result in 50%+ speed up at run time.

I could have enabled these for you if you had asked ;)

Yes, I apply PGO, ThinLTO, and BOLT whenever possible, which can result in 50%+ speed up at run time.

I could have enabled these for you if you had asked ;)

Where is the best venue to formally ask? llvm's issue tracker, another bug tracker, email?

Yes, I apply PGO, ThinLTO, and BOLT whenever possible, which can result in 50%+ speed up at run time.

I could have enabled these for you if you had asked ;)

I think it is still worth doing even in spite of this pull request (I maintain those toolchains for other developers, not just our CI), especially for the main Debian packages because I know many kernel developers who use Debian and Ubuntu clang versions and I am sure they would enjoy those speed ups :)

Where is the best venue to formally ask? llvm's issue tracker, another bug tracker, email?

https://github.com/llvm/llvm-project/issues/new for example :)


Error with random.py when not specifying UA file

Awesome tool!
Quick thing, it errors out if you do not specify a file with user agents:

Is this file a necessity for the tool to function? Seems like the intent is to use a random one if this is not specified.
Thanks!

Hey, thanks for the bug report! Fixed in the most recent commit, just a cursory error on my part. Happy hunting!


Require Python 3.7+, fix test failures, test against Py 3.10 and 3.11, README/RELEASE update

Based on #18 to avoid merge conflicts. Excluding that #18, this PR involves a flurry of minor fixes.

ci: fix failure to test old python versions
Transition to hatchling/pyproject.toml from setuptools/setup.py
Let module include __version__
Fix typo in comment
Add a basic description to the README
Closes #15
Use relative module imports within the module
Closes #12
ci: reduce verbosity of pytest
Update RELEASE.md and rely on tbump
Drop support for Python 3.6
Adopt python 3.7 text=True for subprocess.Popen
docs: increase development status beyond beta
ci: resolve intermittent test failures by sleeping longer
Properly closes #12

DRAFT
Currently working to fix test failing intermittently

@consideRatio do we use hatchling in other Jupyter projects already?  Also I merged #18, but github doesn't seem to acknowledge that here in showiong me diffs :(

Thanks @yuvipanda!!!
I saw @minrk start to use it in https://github.com/minrk/jupyter-keepalive/blob/main/pyproject.toml so that is my inspiration towards this.

I'm not sure, do you need that defined at all? Would it work to remove it entirely and have a pyproject.toml without hatchling defined as a build system?

I'm quite lost as well, but I think this is fine! I'm happy to avoid having setup.py, setup.cfg, pyproject.toml all side by side. If adding hatchling to pyproject.toml makes us able to remove the setup.py thats nice.
I think can't use Python 3.5 and remove setup.py going for only pyproject.toml + a declared build-system, but you can if you have python 3.6+! So, this pattern is something we can adopt wider as well I think.

Thanks, @consideRatio! If pip install -e . works I'm happy to try :)

Wiee thanks for your quick review of this @yuvipanda, greatly appreciated!!! :heart: :tada:

I've been exploring hatchling, which is kinda the PyPA successor to flit, but with hooks for optional build steps. It's being adopted in the JupyterLab/jupyter-releaser projects, so there's a plugin to build jupyterlab extensions, so we don't have to cargo-cult our npm build steps around everywhere in setup.py. That's why I picked it for jupyter-keepalive. I probably would have stuck with setuptools in keepalive were it not for the lab extension.
If you have a simple Python-only package like this one, there isn't really a difference between using setuptools and hatchling with pyproject.toml. You don't need setup.py with either one, but if you have setup.py with setuptools, it's the two-line from setup.py import setup; setup() since you can put all your metadata in pyproject.toml's [project] section instead of args to setup.
pip install -e does work for pyproject.toml installs, thanks to pep 660.
As I understand hatchling more, I think we can make the jupyterhub build/install much simpler than the complicated steps we have now.


Convert dice_coefficient_x86 glue code to Cython

Cython seems to be nicer as a glue layer than using CFFI.
Aha! Link: https://csiro.aha.io/features/ANONLINK-56

@nbgl this can be closed right?


PSR-4 autoloading

#37

I haven't investigate why but after running composer update today (Ran it on daily as part of deploy), twilio no longer work for my site and this is kinda urgent due to a sch presentation is 2-3 hours time.
I previously did not use the stable versions because they did not work at all.

You can lock the version to a previous commit has. Google it. It's
something like "dev-master#{hash}"
On 26 Feb 2015 06:22, "Ziming Lan"<EMAIL_ADDRESS>wrote:

I haven't investigate why but after running composer update today (Ran it
on daily as part of deploy), twilio no longer work for my site and this is
kinda urgent due to a sch presentation is 2-3 hours time.
I previously did not use the stable versions because they did not work at
all.
—
Reply to this email directly or view it on GitHub
https://github.com/aloha/laravel4-twilio/pull/45#issuecomment-76124281.


add meta generator tag

Intended primarily to assist in debugging so that we can tell the version of Framework used to build the site. Also standard practice for site generators.

Thanks for the fixes!


Remove benchmarking comment bot; Run benches weekly and save results as an artifact

The benchmark comments have generally been pretty noisy and I think unpopular, especially when iterating on one open PR.
This PR removes the current benchmark commenting functionality and instead runs a once-weekly (Monday morning at midnight) benchmark on main and saves the results as JSON. These results can then be easily interpreted by a separate tool that I'm putting together. This will help us catch any performance regressions or trends, but in a once-weekly retrospective report of all merges into main, instead of every single commit.
The key thing here is that after this PR goes in, we will have a mechanism to retrospect over performance data without having it commented on every commit of every PR.

I agree with removing the current benchmarks from the pipeline, and having a trend of noisy benchmarks over time is more useful than having no data.


Rewrite admin.site.register() calls using model sequence and admin kwarg

A little extra for #190.

FYI @UnknownPlatypus 😊


Lines starting with "all" missing in coverage output

Hi,
I am using bedtools v2.26.0 and used following command to find coverage of my sample (paired-end exome). Acoording to http://tagteam.harvard.edu/hub_feeds/1981/feed_items/382070 and https://groups.google.com/forum/#!msg/bedtools-discuss/n2rNbRGRTvI/gg8TcbnT698J;context-place=searchin/bedtools-discuss/coverage$20histogram|sort:relevance there are lines starting with all, but they are missing from my output. Kindly guide how I should go further to plot.
bedtools coverage -hist -a sample.bed -b cds.bed > out.coveragehist

############### sample.bed has been generated as follows: ########################

extracting properly-paired reads

samtools view -bf 0x2 sample.bam | bedtools bamtobed -i stdin -bedpe > sample.bedpe

bedpe to bed conversion

cut -f 1,2,6 sample.bedpe > sample.bed

already sorted by chromosome and location, so did not use: sort -k1,1 -k2,2n sample.bed
############################################################################

Thanks !

Any hints please?

@arq5x  I am still waiting for the answer. :P

I cannot reproduce this problem. Can you send minimal data files that create this issue?


Add patch to fix ERR_CERTIFICATE_TRANSPARENCY_REQUIRED timebomb

patches from now on will use full-index to eliminate prefix difference among git versions
Auditors: @bridiver, @bbondy

++


[IDEA] Unified approach for ASP and AlertDialogService properties in DataExchangeViewModel

closes #186

update documentation article stating the need to provide auth provider