HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients.

Adds a semaphore to throttle the #of parallel clients which can be
created simultaneously, set in "fs.creation.parallel.count".
added some durationtracker counting @ debug level to measure time to acquire a lock and time to instantiate an FS. There's still some opentrace code near that ... didn't touch it.

For interest of @snvijaya @rajeshbalamohan @mukund-thakur @mehakmeet
Not thought of a good test yet, at least not one overly destructive. Probably

private Cache instance in test suite (in o.a.h.fs. package) and small semaphore set (2 ?)
Slow to create FS with sleep(30s) in initialize
executor pool to submit a set of workers, where time to instantiate should be << time to instantiate the set sequentially.

May help to have Cache have an atomic long to count #of FS instances discarded. Actually publishing that as an FS stat would be interesting but out of scope.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
28m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
30m 12s

trunk passed

+1 :green_heart:
compile
20m 57s

trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
19m 18s

trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
0m 56s

trunk passed

+1 :green_heart:
mvnsite
1m 31s

trunk passed

+1 :green_heart:
shadedclient
17m 31s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 41s

trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
1m 39s

trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
2m 19s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
2m 17s

trunk passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 54s

the patch passed

+1 :green_heart:
compile
19m 17s

the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javac
19m 17s

the patch passed

+1 :green_heart:
compile
17m 13s

the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
javac
17m 13s

the patch passed

-0 :warning:
checkstyle
0m 56s
/diff-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 2 new + 189 unchanged - 0 fixed = 191 total (was 189)

+1 :green_heart:
mvnsite
1m 29s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
14m 44s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 35s

the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
1m 35s

the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
findbugs
2m 21s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
10m  1s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 53s

The patch does not generate ASF License warnings.

196m  0s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2396

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 46d1b1749654 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d60d5fe43d9

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/1/testReport/

Max. process+thread count
1379 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/1/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.1.3

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Looks good. Just the pending test.

yeah, let me get that out the way before I forget about that cache. I'll use a counter of discarded instances and test off that. I'll plan some tricks to avoid the test being fussy about timing...have a semaphore inside the fake FS I'll be instantiating to block its construction, as sleep() is brittle as well as slowing down test runs

Also, I plan to move the close in a discard out of the locked area. It's not needed and an FS is doing anything there, we don't want to block the other threads

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
31m 23s

trunk passed

+1 :green_heart:
compile
21m 58s

trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
18m 18s

trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
0m 56s

trunk passed

+1 :green_heart:
mvnsite
1m 30s

trunk passed

+1 :green_heart:
shadedclient
18m 17s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 39s

trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
1m 33s

trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
2m 28s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
2m 26s

trunk passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 59s

the patch passed

+1 :green_heart:
compile
21m  4s

the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javac
21m  4s

the patch passed

+1 :green_heart:
compile
18m 40s

the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
javac
18m 40s

the patch passed

-0 :warning:
checkstyle
0m 56s
/diff-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 3 new + 197 unchanged - 0 fixed = 200 total (was 197)

+1 :green_heart:
mvnsite
1m 28s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 36s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 34s

the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
findbugs
2m 26s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
10m  8s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 52s

The patch does not generate ASF License warnings.

174m 10s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2396

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux ecc70d53f09d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7f8ef76c483

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/2/testReport/

Max. process+thread count
1878 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/2/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.1.3

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:3512:      checkArgument(permits > 0 , "Invalid value of %s: %s",:33: ',' is preceded with whitespace. [NoWhitespaceBefore]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:3553:              new DurationInfo(LOGGER, false, "Acquiring creator semaphore for %s",: Line is longer than 80 characters (found 83). [LineLength]
./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java:465:    private static final Semaphore sem = new Semaphore(1);:36: Name 'sem' must match pattern '^[A-Z][A-Z0-9]*(_[A-Z0-9]+)*$'. [ConstantName]

Is there particular reasoning for "64" as the default value?

No. I wondered whether to make it smaller or just leave large.
Large: no visible impact of this change anywhere, so lowest risk
Small: should always speed up applications which always use it.
I think I should add a mention of this in the s3a performance doc, so it's not forgotten about

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 31s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
11m 52s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 26s

trunk passed

+1 :green_heart:
compile
20m  1s

trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
17m 13s

trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
2m 41s

trunk passed

+1 :green_heart:
mvnsite
2m 26s

trunk passed

+1 :green_heart:
shadedclient
20m 30s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 40s

trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
2m 17s

trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
1m 17s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m 30s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 26s

the patch passed

+1 :green_heart:
compile
19m 22s

the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javac
19m 22s

the patch passed

+1 :green_heart:
compile
17m 16s

the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
javac
17m 16s

the patch passed

+1 :green_heart:
checkstyle
2m 41s

the patch passed

+1 :green_heart:
mvnsite
2m 24s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m  5s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 39s

the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
2m 23s

the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
findbugs
3m 46s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
9m 55s

hadoop-common in the patch passed.

+1 :green_heart:
unit
1m 41s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 56s

The patch does not generate ASF License warnings.

182m 32s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2396

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 4f592c5ad05f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f17e067d527

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/3/testReport/

Max. process+thread count
1946 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/3/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.1.3

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  1s
test4tests
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 51s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m  2s

trunk passed

+1 :green_heart:
compile
20m 38s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
compile
17m 12s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
checkstyle
2m 39s

trunk passed

+1 :green_heart:
mvnsite
2m 26s

trunk passed

+1 :green_heart:
shadedclient
21m  0s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 41s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
2m 22s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
1m 16s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m 29s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 26s

the patch passed

+1 :green_heart:
compile
19m 17s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javac
19m 17s

the patch passed

+1 :green_heart:
compile
17m 12s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
javac
17m 12s

the patch passed

+1 :green_heart:
checkstyle
2m 43s

the patch passed

+1 :green_heart:
mvnsite
2m 25s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 43s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 39s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
2m 23s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
findbugs
3m 46s

the patch passed

_ Other Tests _

-1 :x:
unit
9m 52s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch passed.

+1 :green_heart:
unit
1m 40s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 56s

The patch does not generate ASF License warnings.

188m 51s

Reason
Tests

Failed junit tests
hadoop.security.ssl.TestSSLFactory

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2396

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux eb56005e63db 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a7b923c80c6

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/4/testReport/

Max. process+thread count
2481 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/4/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Don't get the test failure. Will rebase
[ERROR] testServerKeyPasswordDefaultsToPassword(org.apache.hadoop.security.ssl.TestSSLFactory)  Time elapsed: 0.103 s  <<< ERROR!
java.security.GeneralSecurityException: The property 'ssl.server.keystore.location' has not been set in the ssl configuration file.
	at org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory.init(FileBasedKeyStoresFactory.java:153)
	at org.apache.hadoop.security.ssl.SSLFactory.init(SSLFactory.java:187)
	at org.apache.hadoop.security.ssl.TestSSLFactory.checkSSLFactoryInitWithPasswords(TestSSLFactory.java:520)
	at org.apache.hadoop.security.ssl.TestSSLFactory.checkSSLFactoryInitWithPasswords(TestSSLFactory.java:440)
	at org.apache.hadoop.security.ssl.TestSSLFactory.testServerKeyPasswordDefaultsToPassword(TestSSLFactory.java:397)

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 44s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m  8s

trunk passed

+1 :green_heart:
compile
21m 46s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
compile
17m 55s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
checkstyle
2m 51s

trunk passed

+1 :green_heart:
mvnsite
2m 31s

trunk passed

+1 :green_heart:
shadedclient
21m 20s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 31s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
2m 15s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
1m 24s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m 42s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 27s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 36s

the patch passed

+1 :green_heart:
compile
20m 55s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javac
20m 55s

the patch passed

+1 :green_heart:
compile
18m  9s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
javac
18m  9s

the patch passed

+1 :green_heart:
checkstyle
2m 46s

the patch passed

+1 :green_heart:
mvnsite
2m 29s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 44s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
2m 21s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
findbugs
3m 55s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
10m  0s

hadoop-common in the patch passed.

+1 :green_heart:
unit
1m 40s

hadoop-aws in the patch passed.

-1 :x:
asflicense
0m 56s
/patch-asflicense-problems.txt
The patch generated 2 ASF License warnings.

194m 37s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2396

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux 1c1e123c2239 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 34aa6137bd8

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/5/testReport/

Max. process+thread count
2427 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/5/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
27m 18s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:

0m  0s
test4tests
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 23s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m 29s

trunk passed

+1 :green_heart:
compile
21m 38s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
compile
18m 23s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
checkstyle
2m 49s

trunk passed

+1 :green_heart:
mvnsite
2m 18s

trunk passed

+1 :green_heart:
shadedclient
21m 24s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 41s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
2m 12s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+0 :ok:
spotbugs
1m 11s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m 33s

trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 26s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 27s

the patch passed

+1 :green_heart:
compile
19m 40s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javac
19m 40s

the patch passed

+1 :green_heart:
compile
18m 28s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
javac
18m 28s

the patch passed

+1 :green_heart:
checkstyle
2m 50s

the patch passed

+1 :green_heart:
mvnsite
2m 26s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
16m  8s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 42s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04

+1 :green_heart:
javadoc
2m 25s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

+1 :green_heart:
findbugs
3m 57s

the patch passed

_ Other Tests _

+1 :green_heart:
unit
9m 58s

hadoop-common in the patch passed.

+1 :green_heart:
unit
1m 39s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 51s

The patch does not generate ASF License warnings.

221m 29s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2396

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint

uname
Linux e02f8c2973a5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f13c7b1b026

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/6/testReport/

Max. process+thread count
1377 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2396/6/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@surendralilhore -could you look @ this again? Think I've tried to address your comments

@surendralilhore -could you look @ this again? Think I've tried to address your

+1

thanks -merging to 3.3+!


Sub-team support

Does this module support sub-team?
If not, how can we customize this to support sub-team?

Hi,
Teamwork does not support sub teams.
If you want, you are welcome to create a PR.


Inappropriate exception from BlockBlobClient.Upload()

Occurs When
Using Azure.Storage.Blobs 12.6.0
Calling BlockBlobClient.Upload() with a 250MB MemoryStream with a slow-ish network that cannot upload that much data before timeout.
Behaviour
Upload() threw an AggregateException containing 6 TaskCancelledException instances (presumably because of retry). Caller had not cancelled any CancellationToken. None of the exceptions had a message giving any insight into the root cause.
Expected
Public API should not throw an OperationCancelledException (of which TaskCancelledException is a subclass) unless it is because a CancellationToken passed in by the caller has been cancelled - not the case here.
Public API should avoid throwing AggregateException unless genuinely unavoidable, as it is hard for the caller to handle.
The only exception documented for BlockBlobClient.Upload() is RequestFailedException. Ideally, such an exception should be thrown. If not, something like a TimeoutException would at least make the cause clear and be easy to handle separately from other exceptions that might need different treatment.
Exception message should clearly indicate what happened.
Note
No StorageTransferOptions were passed in when this happened, so the BlockBlobClient attempted to send the 250MB as a single PUT.
We are now passing appropriate StorageTransferOptions to avoid this particular instance of the inappropriate exception, but the poor exception could still be encountered, e.g. even slower network.

//cc: @tg-msft

Thank you for your feedback.  Tagging and routing to the team best able to assist.

I would have expected the AggregateException to contain 6 OperationCancelledException instead of 6 TaskCancelledException, because TaskCancelledException would indicate that a CancellationToken was passed (which is not the case).
If you are experiencing network issues and the requests that are being sent are reaching the timeout, then OperationCancelledException is the correct exception to see. Seeing that it's 6 exceptions within AggregateException, it's safe to assume it is due to the timeout.
A RequestFailedException should indicate a failed HTTP error code (400+), so it might be best not to throw this type of exception. I like the idea of throwing a TimeOutException instead though, it's more fitting since the request failed due to network issues, and not storage service issues.
I did some digging and looks like this is a common HttpClient issue https://github.com/dotnet/runtime/issues/21965 with dotnet  2.1 and looks like they were looking to change it to TimeOutException in dotnet 5.0. We actually recently upgraded to dotnet 5.0 so I will to look to writing a specific test for dotnet 5.0 and see if we're throwing the correct TimeOutException, instead of the TaskCancelledException.
Thanks for submitting this issue.

@amnguye was this done? marking it a feature request as this is specific to .net 5


How to get the inner text of elements from the return values of page.$$

Steps to reproduce
Tell us about your environment:

Puppeteer version: 0.12.0
Platform / OS version: 10.12.6 Mac OS
URLs (if applicable):

What steps will reproduce the problem?
I have a grid, with several rows and columns. My action creates a new entry into the grid and I would like to validate the same.
In order to do that, I am trying to use page.$$ and get all elements, loop through each and validate whether my required text is present and there by get the row info and eventually perform other actions.
(I would do this in a simpler way if xpath was available.)

				5f3c557

				043c6c0-

				5f3c557043c6c0

				f3c43

I am getting all elements using the below. elements.length gives proper value of element count.
const elements = await page.$$("div.grid div.rows div.row div.cell div.label");
However, I tried so many ways to get the properties of each element but could not succeed.
const data1 = await elements[i].getProperty('value');

const data2 = await elements[i].getProperty('innerText');

const data3 = await elements[i].getProperty('innerHTML');

Currently any of the properties are returning same info - the entire node info.

Is the above, right way to get info of each node?
Could you suggest what is the best way to get text?
Also, how can I get the class name?

What is the expected result?
I was expecting the getProperty returns the property/attribute values of elements.
What happens instead?
Entire JSHandle node info is getting displayed when I console.log data1 or data2 or data3.

getProperty resolves to a JSHandle (imagine if you wanted to get 'parentElement')
Does this work for you? I'll admit it is a bit verbose:
const data1 = await (await elements[i].getProperty('value')).jsonValue();

I did try this as well. It says data1 is "undefined".

@JoelEinbinder thanks for your suggestion.
I could get it working, by using getProperty('innerHTML')).jsonValue() instead of 'value', as its a div element.

innerHTML only get it'self tag  ?

const options = await this.page.$$('li');
for (const option of options) {
const label = await this.page.evaluate(el => el.innerText, option);

answer by @ukrexpo worked for me. Thanks.

Answer by @ukrexpo
More understandable, natural, more Front-End :)

Thanks, @ukrexpo ! Works here! Helped me a lot!!

you could also use the $$eval directly without additional calls to the page.evaluate for each item.
It would be smth like this:
const labels =  await this.page.$$eval('li', option => options.map(item => item.innerText));


Echo when enabling grid mode

When someone in the call is switching to grid mode, audio is echoing

Fixed


🚀 Feature Request: specify worker env for getPlatformProxy()

Describe the solution
Hi,
It would be very useful if the options object of getPlatformProxy() would accept an env property which would be used to get bindings from the wrangler.toml file of that specific environment. Currently, only top-level bindings are used (at least for workers, not sure about pages).

Hi @hrueger 👋 🙂
Thanks for the issue
It should be pretty simple to add support for specifying an env
It should be just a matter of extending the API and pass the value here:
https://github.com/cloudflare/workers-sdk/blob/fbe1c9c816f2b5774060d721ff830e70d9b7d29f/packages/wrangler/src/api/integrations/platform/index.ts#L89-L90
I'll have a look into it soon 🙂👍

@hrueger PR opened 🙂👍
(If you want please give the PR's prerelease a shot and let me know if it works for you 😄)

Wow, that was fast. Works perfectly for me 🎉👍


Searchbar allows to search for non existing repos, leads to 404

Description
The search box "Search for a repo" on the main page allows for every input. If I type in "codekernai" and hit enter, this leads to a new page with a 404 error. The problem occurs for every search string that does not match an existing repository.
Suggested Fix
Only allow submitting the search if the input matches one of the suggested repositories, and stating that no matching repository was found otherwise.

@garrrikkotua we should add a tooltip that explain the search

please assign this work to me


Sync commands are undiscoverable

These are the commands that we show when clicking on sync:

And these are the commands that are in the command palette:

I think it would be worthwhile to add a command in the sync menu that's labeled "Advanced" where we surface the synced data, local backups, logs, etc. otherwise they become hard to discover. I was specifically looking for the Synced Data view and couldn't find it until I remembered it's not in this list.

Added following entries to the sync menu

Preferences Sync: Show Settings
Preferences Sync: Show Synced Data

An action to show logs is available in Synced Data view. Other commands related to local backup are way too advanced and showing them might not be worth.


Update to ember-cli 2.13 family

I'm trying to decouple ember-exam of jQuery, and the first step is to update to the latest ember-cli & family.
The only addon I haven't updated (deliberately) is ember-qunit, since it not requires some manual initialization that I wasn't sure how to perform conditionally, since the test suite also runs in mocha.

@trentmwillis This is a weird kind of error actually. Any idea why it's tying to replace a file all the time?

@cibernox thanks for working on this.

This is a weird kind of error actually. Any idea why it's tying to replace a file all the time?

Unsure what error you're referring to. Looking at Travis I only see errors related to the test-loader not being present.

Hi @cibernox @trentmwillis! May I help you with this PR in some way? There is already ember-cli 2.15 wheres this addon uses 2.7.
This would allow to eliminate some duplications within node_modules for projects that use ember-exam.

@SergeAstapov feel free to open another PR.
I'm not sure what you mean by:

This would allow to eliminate some duplications within node_modules for projects that use ember-exam.

As

@trentmwillis created #111
Sorry, wasn't clear - I meant ember-cli-babel which is listed as dependency.

Ah okay, that makes sense.
I'm going to close this in favor of #111.


Creating RasterDataStoreItem not idempotent (enough)

When rerunning a configuration, where the Role "RasterDataStoreItem" with its corresponding parameters
"DataStoreItems": {
            "RasterStore": {
                "FileShareLocalPath": "D:\\RasterFileShare",
                "FileShareName": "RasterFileShare"
            }

is specified, it fails with the error-message:
Die PowerShell DSC-Ressource "ArcGIS_FileShare" konnte die Funktion "Test-TargetResource" nicht ausführen. Fehlermeldung: "FileShareLocalPath already exist. Please Choose Another One!"
Die Funktion 'SendConfigurationApply' war nicht erfolgreich.
As far as I understood, the reason can be found in "ArcGIS_FileShare.psm1", row 168. In there it "throws" an error, if the file share-path already exists. But why not only check, if a file share is also present on that folder and continue?
This behavior could be reproduced with arcgis-powershell-dsc in version 2.0, ArcGIS Enterprise 10.6.1 on Windows Server 2016 Datacenter (German).

Introduced a new variable "ExternalFileShare" for Raster Datastore Items. Fixed with v2.1.1!


Create questions for another user

Hello everyone. This is my first issue in this project.
Is your feature request related to a problem? Please describe
We use a cloud-hosted Answer instance. It's common that some possible users do not even know the app and use another channel to ask something, like an issue in a project, for example, or just think something is a bug while it's not. We need a quick way to do a soft move, without forcing the user to rewrite everything.
Describe the solution you'd like
I think best experience for user would be having an admin endpoint to create the question as another user. So we can use it in workflows. It's already possible to have a user to create the question using the API, but it should be the asking user because he'll choose the right answer.
Describe alternatives you've considered
A simpler feature may be fill the new question form with query parameters, like "http://my-instance/questions/ask?title=Test Title&description=My %test�scription&tags=mytag1,mytag2. This is a good feature in GitLab and GitHub while opening issues.
I can work in the solution if needed.


Reduce CPU use in callbacks demo

Reduce number of iterations
Use FGP_TV instead of TotalVariation
Use astra FBP (maybe don't need to change this?)

When changing from TotalVariation to FGP_TV, I see some difference in the image metrics for the final example - image metric callbacks with region of interest.
With TotalVariation after 500 iterations

With FGP_TV after 500 iterations

I don't think it's over-fitting to the noise in the data so we don't see the PSNR reduce.
If we want to show this, it may be worth sticking to TotalVariation but just using fewer iterations


properly handle the load and store cache control operand types

Without handling these operand types, disassembling a SPIR-V module that uses the cache control extension produces an invalid operand type error: "error: 33: Internal error: Unhandled operand type: 81".

Please add a test, and rebase the PR. There are conflicts now.

I fixed the merge conflict and I added a test, which fails before my change and passes after my change.  @s-perron please let me know if this is what you had in mind - thanks!
I think the CI-macos-clang-release failure is unrelated to my changes.

The CI-windows-VS2019-debug  failure is not because of this PR. It should pass when we rerun it but I cannot get it to rebuild. I rebased and did a force push. Hopefully the test will now pass.


Logs written by ILogger does not shown in Monitor tab.

I create an azure function(v2) from visual studio, it's blob trigger. And I use the ILogger to write logs to application insights.
The code in function.cs like below:
public static class Function1
{
[FunctionName("Function1")]
public static void Run([BlobTrigger("f22/{name}", Connection = "AzureWebJobsStorage")]Stream myBlob, string name, ILogger log)
{
log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes !!!");
log.LogTrace("this is a trace info from ilogger !!!");
log.LogInformation("this is a information from ilogger!!!");
}
}
And after I publish it to azure, I add application insights ikey to the application settings and save it.
Then upload some files to azure blob, the function works and i can see the logs are shown on the azure function log console.

But in the azure function Monitor tab, I cannot see the logs generated.

but I can find the logs in application insights portal via search.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 13672b67-22ea-3dda-8b6b-3332c24e6030
Version Independent ID: fdc60aeb-9971-1d9f-a6ba-b3ad818cf0d5
Content: Monitor Azure Functions
Content Source: articles/azure-functions/functions-monitoring.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@imeya On clicking a row for one of the executions, you should see a new blade called Invocation Details slide in from the right with the logs.

@PramodValavala-MSFT , actually, I cannot see any rows of executions, it's empty. I'm not sure if I'm missing something.

@imeya The second image that you've shared does show executions though right? Try clicking one of those rows.

yes, the second image show executions. as you can see, these are all failed. because it is a blob triggered function. when I click run button on the first image, it will cause an error which said "blob not found etc". So I just upload a file to my blob storage, then no error messages, but still cannot see the logs in the monitor tab.

@imeya Could share a screenshot of what you see when clicking any of the executions? Even if it has failed.

see screenshot below:

@imeya Aren't those the logs that you are looking for?

@PramodValavala-MSFT , not only this.
In my function, I used ILogger to log other messages, like log.LogTrace("this is a trace info from ilogger !!!");, I want to see this information there.

@imeya Ohh... Seems like I missed that before.
I believe the default log level is Information, that is why you do not see any debug or trace logs.
To enable them too, you would have to set your host.json file to something like this
{
  "version": "2.0",
  "logging": {
    "logLevel": {
      "Function": "Trace"
    }
  }
}

Refer the Configure categories and log levels section of the doc for more information about this.

@PramodValavala-MSFT , thank you very much. And I will give it a try 👍

@imeya Just following up here... Hope my previous comment clears things up.

@PramodValavala-MSFT , I'm trying it now, and will let you know asap.

@PramodValavala-MSFT , it works now. thank you 👍

@imeya Glad we could help!
We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.

In case anyone else stumbles upon this issue, I tried using the above host.json configuration and it did not work. I was not only not seeing any trace logs in the monitor tab at all, but I was also missing function invocation rows entirely from the monitor tab.
{
  "version": "2.0",
  "logging": {
    "logLevel": {
      "Function": "Trace"
    }
  }
}

I found the configuration documentation for the host.json file here which for conservation purposes is the following:
{
  "logging": {
    "fileLoggingMode": "always",
    "logLevel": {
      "default": "Information",
      "Host.Results": "Error",
      "Function": "Error",
      "Host.Aggregator": "Trace"
    }
  }
}

I dropped the log levels down to Trace for each category and finally was able to see rows of invocations in the monitor tab. This pertains to Azure Functions v2. The final configuration I am using is as follows:
{
  "logging": {
    "fileLoggingMode": "always",
    "logLevel": {
      "default": "Trace",
      "Host.Results": "Trace",
      "Function": "Trace",
      "Host.Aggregator": "Trace"
    }
  }
}


[language] Detect and error on circular dependencies

Make the loader check for circular dependencies while loading.
That seems important given modules are only pushed through WriteSets
and thus never going through the verifier. If a circular dependency
was introduced, we would enter an infinite loop.
So it's a protection against an unlikely but horrible scenario.
It also fixes a problem with publishing that was loading and caching
all dependencies except the published module, though it was intended
to load nothing. Now publishing a module only verifies the dependencies
of the published module and nothing else, it also does not load any module.
That hopefully simplifies the code a bit

/land

:clipboard: Looks like this PR is still in progress, unable to queue for landing

I'm not very familiar with the loader code, so I'm not going to approve this. Anyway, we should get more people to look at this, since it is a non-trivial change coming late in the schedule. I left a few comments. Overall it seems like this is probably worth doing, but if anyone has concerns about risk from the code change, we could get by without this for now because we don't have open publishing yet.

to be clear, the concern for me is more related to the fact we mostly publish code via WriteSet and do not run any verification over that. But having said so, I am with you overall....

@tnowacki you'd be a good candidate to look at this. I'd appreciate if you could

I am going to abandon this PR given the feedback and the speed is moving at.
Let me know if anybody thinks otherwise, though I am not sure you'd be able to convince me to put it in.
Thanks and sorry for the churn!

Close it?


resolve the type of lambda auto parameters

template <typename F>
void foo(F&& callback) { callback(5); }
...
foo([](auto p){});

It would involve looking into foo and finding the calls to the lambda.

Ah, ok. This is not what I thought you meant in this comment; I thought you meant cases where the lambda's instantiated type can be deduced from the declaration of the function it calls.
If we're talking about a case like this, where we have to look into the body of the function, this becomes a much more involved feature to implement.
Just to make sure I understand correctly: the editor feature you're intending to use here is hover, i.e. you'd like to hover over the auto in the lambda signature and see int?

Yes this is about complex callback based code using lots of autos where you are trying to figure out types.

What would you expect the behaviour to be if foo() calls the lambda with arguments of multiple different types?

I guess it's ok to bail out in that case.

In my mind, this is a special case of a more general feature: given a template, search the project for instantiations of that template to see what sorts of arguments it gets instantiated with, and use that information to provide better analysis of the template. I've thought of this before in the context of code completion (where we may want to provide completion proposals that would be valid in all the known instantiations), but it could be useful for things like this too in cases where there is only one instantiation.
The reason this is a special case is that in the case of a generic lambda being passed directly to a function, we know that all instantiations will be inside that function or things it (transitively) calls, and should be available in the current translation unit (so no need to search the whole project).

That would indeed be useful. In Visual Studio you have to manually specify type arguments to get Intellisense.

You'd need to compute something like Typescript's Union Types for the template parameters. But then there's constexpr if which breaks this.

You'd need to compute something like Typescript's Union Types for the template parameters. But then there's constexpr if which breaks this.

A glaring issue with this is templates are often defined in header files. When building the preamble, the bodies of these template are typically skipped so there wouldn't be a way to determine what arguments the lambda would get invoked with.

A glaring issue with this is templates are often defined in header files. When building the preamble, the bodies of these template are typically skipped so there wouldn't be a way to determine what arguments the lambda would get invoked with.

There's a simpler case here which almost works perfectly:
auto f(std::function<void(std::string)>) -> void {}

auto testy() -> void {
    f([](auto && thing) {
        thing.sized();
    });
}

sized gives me a red squiggle saying

No member named 'sized' in 'std::basic_string'clang(no_member)

and indeed, size has no red squiggle.
So that's great! But sadly:

autocompletions are not available when I type thing.
the auto&& in the parameter list does not tell me that it's a string when I hover over it.

Is it possible that just a few small changes would make this use case work?
Thanks all

sized gives me a red squiggle saying

No member named 'sized' in 'std::basic_string'clang(no_member)

and indeed, changing it to size has no red squiggle.

This matches the diagnostics that clang itself would (or wouldn't) give when compiling this code.
As part of resolving the call to f, the generic lambda's templated call operator gets instantiated (with the type of the auto parameter being mapped to std::string), and then the body of the instantiated function does or does not compile.

So that's great! But sadly:

autocompletions are not available when I type thing.

the auto&& in the parameter list does not tell me that it's a string when I hover over it.

Is it possible that just a few small changes would make this use case work?

Those are editor features that you'd like to work when editing the uninstantiated code.
It's possible to support these, but unfortunately it requires more than "a few small changes". It requires implementing the mechanism described in this comment, to search for instantiations and use information from the instantiations to inform queries (such as completion) about the uninstantiated code.

search the project for instantiations of that template to see what sorts of arguments it gets instantiated with, and use that information to provide better analysis of the template

Is the infrastructure for that in place already?

search the project for instantiations of that template to see what sorts of arguments it gets instantiated with, and use that information to provide better analysis of the template

Is the infrastructure for that in place already?

There's two parts here:

Instantiations in the current translation unit. These are already available in the AST (with the notable caveat pointed out in this comment, that instantiations triggered from the body of a template defined in a header will not be present, because clangd skips parsing such bodies for performance reasons), so it's just a matter of searching for them (which is just an AST traversal, or not even that) and making use of them. The "making use of" part is still a considerable amount of work. I think what we'll need in general, is some mechanism to take a descendant AST node inside an uninstantiated template body (say, an Expr) and match it to the corresponding AST node inside an instantiation. I haven't thought through how that would work, but perhaps a specialized version of SelectionTree could be used for it. Then the remaining work is specific to each editor feature, how it wants to make use of this additional information.

Instantiations throughout the project. This is considerably more difficult, and in fact I'm not sure how we would make it work with clangd's index model. (When I originally envisioned this, I was working on Eclipse CDT, whose index is more like the serialized ASTs of all translation units, so you could just look up instantiations from the index and work with them more or less as though they were instantiations in the current translation unit.)

Time to revive an old thread. I have a patch out for the original request. https://reviews.llvm.org/D119537
@njames93

A glaring issue with this is templates are often defined in header files. When building the preamble, the bodies of these template are typically skipped so there wouldn't be a way to determine what arguments the lambda would get invoked with.

Yes. This is a problem when function types are template parameters, like in the STL.
When functions are type-erased instead, by passing as std::function, llvm::function_ref and what-have-you, the lambda->wrapper conversion occurs in the scope of the current file, so we get to see the instantiations. So the technique does work widely, but not universally.
@HighCommander4

In my mind, this is a special case of a more general feature: given a template, search the project for instantiations of that template to see what sorts of arguments it gets instantiated with, and use that information to provide better analysis of the template. I've thought of this before in the context of code completion (where we may want to provide completion proposals that would be valid in all the known instantiations), but it could be useful for things like this too in cases where there is only one instantiation.

The general idea of combining/intersecting results across several instantiations seems like it will require a lot of effort per-feature, and we should only do it where it really makes sense.
(I'm guilty of this in the patch I linked above: hover will show the type if multiple instantiations agree on the type).
Where there's exactly one instantiation, it seems we could get at this in a cross-cutting way. e.g. a flag on RecursiveASTVisitor to traverse specialization bodies instead of template bodies in this case.
Sadly, I think it's going to be very hard to apply this technique to code completion/signature help, because it runs after AST building and yields an AST, and code completion disrupts AST building and can't really operate on just an AST.

Works surprisingly well so far. It at least gives me an idea of the type now.
Some observations:

Goto Definition works but does not help in my case since it's often a shared_ptr<T> and thus jumps to shared_ptr instead of the T I'm interested in, but that's by design ofc. Though I guess that's a more general problem and could maybe be "solved" by special-casing smart pointers or offering multiple goto destinations for templated types.
The type resolution only works on the auto part, not the parameter itself which you usually hover on (esp. where it's actually used inside the function). I guess that's due the mentioned AST limitation:

param payload
Type: const auto &

// In subscribe::(anonymous class)::operator()
const auto& payload

type-alias auto
struct Foo

With the current implementation this is a prime candidate for a type inlay hint so you don't have to manually check the auto every time. Might be a low-hanging fruit to improve the experience.
If the calling code is itself inside a template it doesn't work:

template <typename F>
void foo(F&& callback)
{
    callback(5);
}

template <typename T>
void bar()
{
    foo([](auto) {});
}

int main()
{
    bar<int>();
}

The issue got auto-closed but there are still plenty of open aspects as previously mentioned.

Indeed, closed too early.

Goto Definition works but does not help in my case since it's often a shared_ptr

Yes. This is independent of this issue as you noted. Opened #1026.

The type resolution only works on the auto part, not the parameter itself which you usually hover on

OK, let's use this bug to track the more general idea of using the non-template AST to analyze the whole function.
(We need a scope for this bug that allows it to eventually be closed, so I'm going to call non-AST features like code completion out of scope here).

With the current implementation this is a prime candidate for a type inlay hint

https://reviews.llvm.org/D120258 is out for this.

If the calling code is itself inside a template it doesn't work:

Strictly speaking "inside a template" is fine, but:

if the callee's instantiation depends on a template parameter of the caller, we won't perform it until the caller is instantiated. So dependent calls inside a template won't trigger deduction.
function bodies in header files aren't visible, so if the instantiation is via a function in a header (possibly templated) we won't trigger deduction

I don't see a realistic way to fix either of these.

Regarding https://reviews.llvm.org/D120258#3336114,
I just noticed that at least in one case the inlay hint becomes visible when I change auto&& p to auto p.
In a simple testcase both work though.

@sam-mccall I found a testbed.
https://vittorioromeo.info/index/blog/zeroalloc_continuations_p2.html
https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L489
https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L499-L501
Hover shows the correct type but the inline hint isn't shown.
https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L551
And here if you add && to the auto t the type inference stops working altogether.

https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp

I'm seeing a lot of <dependent type> hints in this file, presumably we just want to omit those?

/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L489
...
Hover shows the correct type but the inline hint isn't shown.

This is a length issue. The hint is tuple<basic_string<char>, basic_string<char>>, which is longer than the limit of 32 characters we added in #824. There are some other ideas in that bug for how we could try harder to shorten hints instead of omitting them; contributions welcome.

https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L499-L501

But here the hints are below the length limit, so there must be something else going on and further investigation is needed.

https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L551
And here if you add && to the auto t the type inference stops working altogether.

Confirmed, this will need some further investigation as well.

This is a length issue. The hint is tuple<basic_string<char>, basic_string<char>>, which is longer than the limit of 32 characters we added in #824.

Ah ok good to know. Showing something (like tuple<...> or just tuple) would indeed be better than nothing, and avoid the confusion about hint vs hover discrepancies.

Confirmed, this will need some further investigation as well.

Any way to help? I guess reducing the code, but would need a commandline way to get and check the inlay hints.

I often reduce such code examples manually.

https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L499-L501

But here the hints are below the length limit, so there must be something else going on and further investigation is needed.

Taking another look, this is simply a case of the parameter being unnamed. If you add a name, the hint appears.
This case was discussed during the review.

https://github.com/vittorioromeo/vittorioromeo.info/blob/master/extra/zeroalloc_continuations/p2_parallel.cpp#L551
And here if you add && to the auto t the type inference stops working altogether.

Confirmed, this will need some further investigation as well.

This one seems to be related to the assert a couple of lines below: if the assert is removed, the type hint appears even with &&.
What must be happening is that the assert triggers a second instantiation of the lambda's call operator template. Recall that the logic we implemented only shows a hint if the call operator template has a single instantiation.
My guess would be that then() triggers a call to the lambda with an lvalue tuple<int, int> argument, and the assert with an rvalue (or vice versa).
When the parameter type is auto, both calls deduce to the same signature (auto = tuple<int, int>).
But when the parameter type is auto&&, the lvalue call deduces auto = tuple<int, int>& while the rvalue call deduces auto = tuple<int, int>, so we get two instantiations.

Taking another look, this is simply a case of the parameter being unnamed. If you add a name, the hint appears.
This case was discussed during the review.

Ah I remember, it was getLocation pointing at the wrong token, which sounded like a general clang bug.
It got removed and then forgotten. An (auto : type) hint would be fine with me and could be useful to understand code.
But in general I think (also regarding the other case of type length limit) consistency is important. If it silently doesn't show anything in just a few cases you always think something is broken. Maybe a placeholder or ellipsis should be displayed then with a tooltip explaining what went wrong. Or a log entry at least.


Create Contributing.md

I'm submitting this pull request more for feedback than expecting this to be done.  I need to get the formatting a little cleaned up, and have a few TBA sections.  Feel free to bounce back with feedback, or fill in the blanks, or remove the TBA's and post as is (i.e. good enough and we'll iterate)

Thanks for getting this started!  Let's keep this here in pull-request form for the time being.  We can continue to discuss and you can simply push additional commits to your branch for them to show up here.
Perhaps start with adding URLs where appropriate?  That would help. 😄


[BEAM-7533] Fix CoderRegistry for Float

There's a typo in CoderRegistry.CommonTypes causing Float type to be resolved with SerializableCoder.
builder.put(
    FloatCoder.class, CoderProviders.fromStaticMethods(Float.class, FloatCoder.class));

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

R: @reuvenlax

Run Java PreCommit

Run Java_Examples_Dataflow PreCommit

Run Direct ValidatesRunner

Run Java_Examples_Dataflow PreCommit


Remove RATA Traverse Data

As an ECMPS user, I want the ability to remove RATA Traverse Data so that I can update Relative Accuracy Data as needed
Acceptance Criteria:
Given I am in the RATA Traverse data section
When I click on the 'Remove' option for RATA Traverse Data
Then I see that the relevant RATA Traverse data record is deleted
Given I chose to remove RATA Traverse Data
When I click on the 'Remove' option
Then I see a message 'Are you sure you want to remove the selected data?'
And when I click Yes
Then the data is removed
And the update shall be reflected in the RATA Traverse data section
When I select Cancel
Then I am returned to the landing screen for RATA Traverse Data
Invision Board:
https://projects.invisionapp.com/freehand/document/SrHrFhGQ3?zoomToItems=Y2w2YXc0OThtMDBqaDNiNnlnanNjaWF4Mg%3D%3D
Testing
Test Case: TC933
Tester: @mosesdeeCVP

Test Passed


Make setting Atom as Default Editor for filetypes easier

Currently it is not so easy to assign atom as default editor on windows eg for all .css files
the path to atom.exe is changed with every update.
no exe in C:\Users\hebbet\AppData\Local\atom

Chrome has eg an chrome.exe in:
C:\Users\username\AppData\Local\Google\Chrome SxS\Application which starts chrome with the currect folder.

Have you tried using C:\Users\hebbet\AppData\Local\atom\bin\atom?
That should be a consistent location that always points to the latest installed release

With atom it doesn't work. Tested with Windows 7 and 8.
With atom.cmd it works but with an pretty annoying cmd Window for a very
short time.
On Jan 7, 2015 10:35 PM, "Kevin Sawicki"<EMAIL_ADDRESS>wrote:

Have you tried using C:\Users\hebbet\AppData\Local\atom\bin\atom?
That should be a consistent location that always points to the latest
installed release
—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom/issues/4893#issuecomment-69096304.

I tried to use .lnk file in the start menu instead of the .cmd file and it seems to work. The annoying cmd window is no longer appearing.

Thanks @sick51, it works for me!

Doesn't work for me - I tried opening a C file in Windows 8 using the Start Menu shortcut, and it showed a Command Prompt window (which itself is alright), but then didn't proceed after that.

@samirotiv The shortcut target should be C:\Users\[you]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\GitHub, Inc\atom.exe

I think there's something wrong with that since the executable surely
wouldn't be in the Start Menu (and it indeed isn't), and the shortcut in
the Start Menu won't point to an executable in the same folder.
The location I wrote above is the target of the shortcut in the Start Menu.
Regards,
Samir.
On Tuesday, March 24, 2015, Joseph Dykstra<EMAIL_ADDRESS>wrote:

@samirotiv https://github.com/samirotiv The shortcut target should be C:\Users[you]\AppData\Roaming\Microsoft\Windows\Start
Menu\Programs\GitHub, Inc\atom.exe
—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom/issues/4893#issuecomment-85591426.

It's easy to set Atom as default editor.

Go to Windows Explorer
Select a .txt file, right click, go to Open with, then Choose default program
In dialog, copy and paste this: %LOCALAPPDATA%\atom\bin\atom.cmd and click OK

Here's a GIF
atom.cmd is a shortcut to the latest version of Atom.

I used the atom installer for win 7.  There is no bin directory in %LOCALAPPDATA%\atom

@Amleto
Create a new batch file and save it as atom.cmd and put it in %LOCALAPPDATA%\atom\bin\ then make sure that directory is in PATH.
@echo off
FOR /F "delims=" %%i IN ('dir %LOCALAPPDATA%\atom\app-* /b /ad-h /t:c /o:d') DO SET foldername=%%i
%LOCALAPPDATA%\atom\%foldername%\resources\cli\atom.cmd

This batch file I created will get the latest version of Atom and start it.

:+1: on this issue. While targeting atom.cmd works, it's not really a great workaround, since

you have to know and manually paste the path to that cmd file
you get a blank rectangle as the file icon, not the Atom logo, so the file association is not clear.
you get an annoying short flicker of a terminal when opening files, which you don't get when opening Atom itself via the start menu.

Touché.
On Mon, May 11, 2015 at 6:24 PM, Christoph Buchner<EMAIL_ADDRESS>
wrote:

[image: :+1:] on this issue. While targeting atom.cmd works, it's not
really a great workaround, since

you have to know and manually paste the path to that cmd file
you get a blank rectangle as the file icon, not the Atom logo, so
the file association is not clear.
you get an annoying short flicker of a terminal when opening files,
which you don't get when opening Atom itself via the start menu.

—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom/issues/4893#issuecomment-100897507.

:+1:

:+1:

:+1:

As @bilderbuchi said, that terminal dialog before Atom is really annoying and takes a lot of time.

:+1:

:+1:

:+1:

:+1:

:+1::+1:

:+1:

:+1:

:+1:

A solution could be: A C program (compiled with MinGW) that finds latest folder, starts Atom
This program would find the latest Atom folder (AppData\Local\atom\app-1.0.0\), and behave like the batch file resources\cli\atom.cmd

:+1: Love the editor but this is an annoying issue.

Hello Windows 95:

Please add this! :+1:

The solutions here work, but the Icon wrong (it is the ugly system default) and it isn't clear what program you are opening the file with.

There's a lot going on in here, so I'll just summarize the feedback as I see it:

it'd be nice to have Atom appear in the "Default Programs" list - how this will work with the potentially unknown number of file extensions available, I'm not sure.
when manually setting Atom to be associated with a specific extension using atom.cmd, the icon is not used. That should be addressed with the previous step.
the current workaround for pointing to the atom.cmd script means a command prompt is launched - perhaps an alternative approach is necessary here.

Did I miss anything?

:+1:

Please, the added +1s are just adding noise at this point. People are aware of the gravity of the request, as Windows is a big user base for Atom (and Electron incidentally), and the issue is being worked on.

+1 to your comment. ;)
On Saturday, July 18, 2015, Patrick Metzdorf<EMAIL_ADDRESS>wrote:

Please, the added +1s are just adding noise at this point. People are
aware of the gravity of the request, as Windows is a big user base for Atom
(and Electron incidentally), and the issue is being worked on.
—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom/issues/4893#issuecomment-122426071.

and the issue is being worked on.

@batjko is that so? There is no label on this issue that would indicate that (like in-progress, work-in-progress, under-review)

@bilderbuchi

is that so? There is no label on this issue that would indicate that

You're right, the issue is still unassigned. Sorry for not making it clearer what I've been up to.

@batjko is that so? There is no label on this issue that would indicate that (like in-progress, work-in-progress, under-review)

Well, it's just something one adds to a comment like that ;)
Nonetheless, awareness has been raised to the point where any further +1's don't add any value.

Mfw my email has been spammed with these DaMN +1's chill please. BTW you can easily just make a simple wrapper exe that calls the function  for opening files in atom and have that automatically set to be default on install.

+1

All these +1s and :+1:s are just adding noise, so to minimize that I'm going to lock this issue for now – we may unlock it once someone starts working on this enhancement to get further feedback.

I made a workaround using AutoHotKey.
atom.ahk:
Run, %LocalAppData%\atom\bin\atom.cmd %1% %2% %3% %4% %5% %6% %7% %8% %9% %10%,,Hide

Then using Ahk2Exe for AutoHotkey:

Now when I set filetypes to "Open With..."

They have this nice icon:

That autohotkey script doesn't work for files with spaces in their file name. With quotes, it does.
Run, %LocalAppData%\atom\bin\atom.cmd "%1%",,Hide

I tried your solution @wmhilton but the Open With... dialog wont "find" the generated exe (tested to be working) even though I'm pointing right at it...

Definitely need a fix for this. It would be good to have an Atom setting with all the main file extensions to choose from.

:+1:

:+1:

Stop the thumbs up madness. If you are here to add one, please stop and think. GitHub is not a feature voting system. All these thumbs up are giving Windows users a bad name and not helping Atom folks or the community get this issue fixed.

They do help to make the Atom team be aware of the scope of this issue / the big number of affected users, so they may prioritize issues accordingly.
This is not giving Windows users a bad name, this is giving the Github issue tracking system a bad name for not having something like google code "stars" or Launchpad "affects me too!" -equivalent systems implemented. ;-)

They do help to make the Atom team be aware of the scope of this issue / the big number of affected users, so they may prioritize issues accordingly.

Unfortunately, not really. :+1:s are only helpful up to a certain threshold - beyond that they're just noise.
I unlocked this issue to let someone post a useful workaround for subscribers to this issue, but now that it's devolved into +1s again I'm going to re-lock it to keep the noise levels down.

Fix checked in to master. Will now appear in the list of apps from "Open With..." and the update will take care of updating/correcting the exe path.

To clarify, Atom will only appear in the "Open With" list starting in Atom 1.7.0.  After upgrading to 1.7.0 (when it's released), if Atom is still missing in the Open With menu, please file a new issue.  Thanks!

Screenshots showing this fix in action are at https://github.com/atom/atom/pull/10818


select2 ajax $request.then(success) can't run on android

Prerequisites

[ ] I have searched for similar issues in both open and closed tickets and cannot find a duplicate
[ ] The issue still exists against the latest master branch of Select2
[ ] This is not a usage question (Those should be directed to the community)
[ ] I have attempted to find the simplest possible steps to reproduce the issue
[ ] I have included a failing test as a pull request (Optional)

Steps to reproduce the issue

Expected behavior and actual behavior
When I follow those steps, I see...
I was expecting...
Environment
Browsers

[ ] Google Chrome
[ ] Mozilla Firefox
[ ] Internet Explorer

Operating System

[ ] Windows
[ ] Mac OS X
[ ] Linux
[ ] Mobile

Libraries

jQuery version:
Select2 version:

Isolating the problem

[ ] This bug happens on the examples page
[ ] The bug happens consistently across all tested browsers
[ ] This bug happens when using Select2 without other plugins
[ ] I can reproduce this bug in a jsbin

select2 ajax $request.then(success) can't run on android,straight do $request.fail(failure)
function request () {
var $request = options.transport(options, function (data) { ####todo,even don't access here。
var results = self.processResults(data, params);
    if (self.options.get('debug') && window.console && console.error) {
      // Check to make sure that the response included a `results` key.
      if (!results || !results.results || !$.isArray(results.results)) {
        console.error(
          'Select2: The AJAX results did not return an array in the ' +
          '`results` key of the response.'
        );
      }
    }

    callback(results);
  },function () {           **#### straight access here**
    // Attempt to detect if a request was aborted
    // Only works if the transport exposes a status property
    if ($request.status && $request.status === '0') {
      return;
    }

    self.trigger('results:message', {
      message: 'errorLoading'
    });
  });

  self._request = $request;
}

It is unclear what this issue is asking.  OP appears to be quoting part of the codebase in /src/js/select2/data/ajax.js, but there is no explanation of OP's actual issue or how to replicate it in Android.
Closing as a low-quality issue.


A \n was added when pasting a string, and not removable.

There's a \n attached to the end of translated string when I copied that from my browser and pasted to the translation input box. I tried to remove such \n but it came back when I focused on another segment and focus back.
Procedures:

Go to an empty segment.
Copy a string from browser.
Paste string on the input box.
Remove the \n from the end of the pasted string.
Focus another segment and focus back this segment.
Check results.

Expected: no \n at all or won't be there after removed.
Observed: the \n is not removable.

\n highlights newlines. It is correct and expected behavior that it is shown after every newline you entered.
If you don't want to have a trailing newline, don't enter it, or delete it (the newline preceding \n, not only \n).

On second thought, reopening because you're not the first person confused by this and not realizing that newlines are editable whitespace characters.

Thanks for your quick response.
I know I can edit the \n, so I removed that immediately. However, the problem is when I switched to another segment and come back, the \n appears again.

As I already wrote above: You need to delete THE ACTUAL NEWLINE, not the \n marker. You didn't. You STILL have a newline in your translation. Poedit merely highlights it. You need to delete it. The newline, not (only) the visual indicator of its presence. If the cursor is under the text you wrote, there's evidently a newline involved. If you don't want to have a newline at the end of the text, you must end editing it with a cursor to the right of the text (as in any other text editor), not under it.

Thanks for your patience. I got what you mean now. Is that possible to have that marker "not editable" to avoid misunderstandings from users?

Yes, it's "possible". It's also insanely huge amount of work (low level changes to three layout engines, of which only one has direct-yet-complicated-and-poorly-documented support for it). Would be worth doing if 10% of Poedit users were confused by this, but not when it's 0.001% or so.

No problem.


authentication with Wallabag failed

I add the Client id, client secret , username and password but I still can't authenticate with Wallabag. The url is https://app.wallabag.it/

See this: https://github.com/huseyz/obsidian-wallabag/issues/38#issuecomment-2450510295
Remove the slash at the end and retry.

As this seems to be a common problem, additional wording around this field would help.

Thank you! IS working now.


Issue related to 'MODEL_KWARGS' while testing custom model.

I am getting the below error while testing my model, which I built by finetuning the icdar19 model with one custom class(Table). i used 'https://layoutlm.blob.core.windows.net/dit/dit-fts/icdar19modern_dit-b_cascade.pth' as the base model while training. and I used the config file generated while training the model.
Error
Traceback (most recent call last):
File "/content/unilm/./dit/object_detection/inference.py", line 81, in
main()
File "/content/unilm/./dit/object_detection/inference.py", line 44, in main
cfg.merge_from_file(args.config_file)
File "/usr/local/lib/python3.9/dist-packages/detectron2/config/config.py", line 69, in merge_from_file
self.merge_from_other_cfg(loaded_cfg)
File "/usr/local/lib/python3.9/dist-packages/fvcore/common/config.py", line 132, in merge_from_other_cfg
return super().merge_from_other_cfg(cfg_other)
File "/usr/local/lib/python3.9/dist-packages/yacs/config.py", line 217, in merge_from_other_cfg
_merge_a_into_b(cfg_other, self, self, [])
File "/usr/local/lib/python3.9/dist-packages/yacs/config.py", line 478, in _merge_a_into_b
_merge_a_into_b(v, b[k], root, key_list + [k])
File "/usr/local/lib/python3.9/dist-packages/yacs/config.py", line 478, in _merge_a_into_b
_merge_a_into_b(v, b[k], root, key_list + [k])
File "/usr/local/lib/python3
.9/dist-packages/yacs/config.py", line 474, in _merge_a_into_b
v = _check_and_coerce_cfg_value_type(v, b[k], k, full_key)
File "/usr/local/lib/python3.9/dist-packages/yacs/config.py", line 534, in _check_and_coerce_cfg_value_type
raise ValueError(
ValueError: Type mismatch (<class 'str'> vs. <class 'dict'>) with values ({} vs. {}) for config key: MODEL.VIT.MODEL_KWARGS
real	0m2.466s
user	0m1.980s
sys	0m0.538s
Generated config file is -
AUG:
DETR: true
CUDNN_BENCHMARK: false
DATALOADER:
ASPECT_RATIO_GROUPING: true
FILTER_EMPTY_ANNOTATIONS: false
NUM_WORKERS: 4
REPEAT_THRESHOLD: 0.0
SAMPLER_TRAIN: TrainingSampler
DATASETS:
PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000
PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000
PROPOSAL_FILES_TEST: []
PROPOSAL_FILES_TRAIN: []
TEST:

table_detection_test
TRAIN:
table_detection_train
GLOBAL:
HACK: 1.0
INPUT:
CROP:
ENABLED: true
SIZE:

384
600
TYPE: absolute_range
FORMAT: RGB
MASK_FORMAT: polygon
MAX_SIZE_TEST: 1333
MAX_SIZE_TRAIN: 1333
MIN_SIZE_TEST: 800
MIN_SIZE_TRAIN:

480
512
544
576
608
640
672
704
736
768
800
MIN_SIZE_TRAIN_SAMPLING: choice
RANDOM_FLIP: horizontal
MODEL:
ANCHOR_GENERATOR:
ANGLES:

-90
0
90
ASPECT_RATIOS:

0.5
1.0
2.0
NAME: DefaultAnchorGenerator
OFFSET: 0.0
SIZES:

32

64

128

256

512
BACKBONE:
FREEZE_AT: 2
NAME: build_vit_fpn_backbone
DEVICE: cuda
FPN:
FUSE_TYPE: sum
IN_FEATURES:

layer3
layer5
layer7
layer11
NORM: ''
OUT_CHANNELS: 256
KEYPOINT_ON: false
LOAD_PROPOSALS: false
MASK_ON: true
META_ARCHITECTURE: GeneralizedRCNN
PANOPTIC_FPN:
COMBINE:
ENABLED: true
INSTANCES_CONFIDENCE_THRESH: 0.5
OVERLAP_THRESH: 0.5
STUFF_AREA_LIMIT: 4096
INSTANCE_LOSS_WEIGHT: 1.0
PIXEL_MEAN:

127.5
127.5
127.5
PIXEL_STD:
127.5
127.5
127.5
PROPOSAL_GENERATOR:
MIN_SIZE: 0
NAME: RPN
RESNETS:
DEFORM_MODULATED: false
DEFORM_NUM_GROUPS: 1
DEFORM_ON_PER_STAGE:

false
false
false
false
DEPTH: 50
NORM: FrozenBN
NUM_GROUPS: 1
OUT_FEATURES:
res4
RES2_OUT_CHANNELS: 256
RES5_DILATION: 1
STEM_OUT_CHANNELS: 64
STRIDE_IN_1X1: true
WIDTH_PER_GROUP: 64
RETINANET:
BBOX_REG_LOSS_TYPE: smooth_l1
BBOX_REG_WEIGHTS: &id002
1.0
1.0
1.0
1.0
FOCAL_LOSS_ALPHA: 0.25
FOCAL_LOSS_GAMMA: 2.0
IN_FEATURES:
p3
p4
p5
p6
p7
IOU_LABELS:
0
-1
1
IOU_THRESHOLDS:
0.4
0.5
NMS_THRESH_TEST: 0.5
NORM: ''
NUM_CLASSES: 80
NUM_CONVS: 4
PRIOR_PROB: 0.01
SCORE_THRESH_TEST: 0.05
SMOOTH_L1_LOSS_BETA: 0.1
TOPK_CANDIDATES_TEST: 1000
ROI_BOX_CASCADE_HEAD:
BBOX_REG_WEIGHTS:
&id001

10.0
10.0
5.0
5.0

20.0
20.0
10.0
10.0

30.0
30.0
15.0
15.0
IOUS:

0.5
0.6
0.7
ROI_BOX_HEAD:
BBOX_REG_LOSS_TYPE: smooth_l1
BBOX_REG_LOSS_WEIGHT: 1.0
BBOX_REG_WEIGHTS: *id001
CLS_AGNOSTIC_BBOX_REG: true
CONV_DIM: 256
FC_DIM: 1024
NAME: FastRCNNConvFCHead
NORM: ''
NUM_CONV: 0
NUM_FC: 2
POOLER_RESOLUTION: 7
POOLER_SAMPLING_RATIO: 0
POOLER_TYPE: ROIAlignV2
SMOOTH_L1_BETA: 0.0
TRAIN_ON_PRED_BOXES: false
ROI_HEADS:
BATCH_SIZE_PER_IMAGE: 512
IN_FEATURES:
p2
p3
p4
p5
IOU_LABELS:
0
1
IOU_THRESHOLDS:
0.5
NAME: CascadeROIHeads
NMS_THRESH_TEST: 0.5
NUM_CLASSES: 1
POSITIVE_FRACTION: 0.25
PROPOSAL_APPEND_GT: true
SCORE_THRESH_TEST: 0.05
ROI_KEYPOINT_HEAD:
CONV_DIMS:
512
512
512
512
512
512
512
512
LOSS_WEIGHT: 1.0
MIN_KEYPOINTS_PER_IMAGE: 1
NAME: KRCNNConvDeconvUpsampleHead
NORMALIZE_LOSS_BY_VISIBLE_KEYPOINTS: true
NUM_KEYPOINTS: 17
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 0
POOLER_TYPE: ROIAlignV2
ROI_MASK_HEAD:
CLS_AGNOSTIC_MASK: false
CONV_DIM: 256
NAME: MaskRCNNConvUpsampleHead
NORM: ''
NUM_CONV: 4
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 0
POOLER_TYPE: ROIAlignV2
RPN:
BATCH_SIZE_PER_IMAGE: 256
BBOX_REG_LOSS_TYPE: smooth_l1
BBOX_REG_LOSS_WEIGHT: 1.0
BBOX_REG_WEIGHTS: *id002
BOUNDARY_THRESH: -1
CONV_DIMS:
-1
HEAD_NAME: StandardRPNHead
IN_FEATURES:
p2
p3
p4
p5
p6
IOU_LABELS:
0
-1
1
IOU_THRESHOLDS:
0.3
0.7
LOSS_WEIGHT: 1.0
NMS_THRESH: 0.7
POSITIVE_FRACTION: 0.5
POST_NMS_TOPK_TEST: 1000
POST_NMS_TOPK_TRAIN: 2000
PRE_NMS_TOPK_TEST: 1000
PRE_NMS_TOPK_TRAIN: 2000
SMOOTH_L1_BETA: 0.0
SEM_SEG_HEAD:
COMMON_STRIDE: 4
CONVS_DIM: 128
IGNORE_VALUE: 255
IN_FEATURES:
p2
p3
p4
p5
LOSS_WEIGHT: 1.0
NAME: SemSegFPNHead
NORM: GN
NUM_CLASSES: 54
VIT:
DROP_PATH: 0.1
IMG_SIZE:
224
224
MODEL_KWARGS: '{}'
NAME: dit_base_patch16
OUT_FEATURES:
layer3
layer5
layer7
layer11
POS_TYPE: abs
WEIGHTS: https://layoutlm.blob.core.windows.net/dit/dit-fts/icdar19modern_dit-b_cascade.pth
OUTPUT_DIR: ./dit/object_detection/Training_Output_base
SEED: 42
SOLVER:
AMP:
ENABLED: true
BACKBONE_MULTIPLIER: 1.0
BASE_LR: 5.0e-05
BIAS_LR_FACTOR: 1.0
CHECKPOINT_PERIOD: 1000
CLIP_GRADIENTS:
CLIP_TYPE: full_model
CLIP_VALUE: 1.0
ENABLED: true
NORM_TYPE: 2.0
GAMMA: 0.1
IMS_PER_BATCH: 1
LR_SCHEDULER_NAME: WarmupCosineLR
MAX_ITER: 4000
MOMENTUM: 0.9
NESTEROV: false
OPTIMIZER: ADAMW
REFERENCE_WORLD_SIZE: 0
STEPS:

30000
WARMUP_FACTOR: 0.01
WARMUP_ITERS: 1000
WARMUP_METHOD: linear
WEIGHT_DECAY: 0.05
WEIGHT_DECAY_BIAS: null
WEIGHT_DECAY_NORM: 0.0
TEST:
AUG:
ENABLED: false
FLIP: true
MAX_SIZE: 4000
MIN_SIZES:

400
500
600
700
800
900
1000
1100
1200
DETECTIONS_PER_IMAGE: 100
EVAL_PERIOD: 1000
EXPECTED_RESULTS: []
KEYPOINT_OKS_SIGMAS: []
PRECISE_BN:
ENABLED: false
NUM_ITER: 200
VERSION: 2
VIS_PERIOD: 0

I added some custom code to load the config, remove MODEL.VIT.MODEL_KWARGS if it exists, and then write it back to disk before starting inference/training which fixed the issue for me. Can be done pretty quickly with the native yaml library.

@ alexjiao-chrono it's quite late, but do you still have the code snippet for it?

Sure, this is basically the code I used:
import yaml

def remove_vit_kwargs(config):
    with open(config) as f:
        cfg_dict = yaml.safe_load(f)
        try:
            del cfg_dict['MODEL']['VIT']['MODEL_KWARGS']
        except Exception as e:
            print(
                f'Failed deleting MODEL.VIT.MODEL_KWARGS. This could be because the config key does not exist.')
        with open(config, 'w') as f:
            yaml.safe_dump(cfg_dict, f)


One in ten peer TMGetLedger requests do not return a response

Hi,
I'm pulling the ledger and associated transactions via the peer network. I'm issuing TMGetLedger requests with a specified LedgerSeq field and the TMLedgerInfoType set to liBase. These commands are sent to, on average 18 peers with qualifying ledger ranges, at a combined rate of 20 requests/second. That is, roughly one ledger request per client per second.
I'm finding that approximately 1 in 10 of these requests never receives a response. It's not particular to any one other node. Is this by design? Dropping requests that the node does not want to handle, or is it a bug that I need to trace? I'm currently patching the gaps by reissuing the requests later, which works fine.
Cheers!

If the server is not going to reply to you, it simply doesn't reply. The query logic rippled uses handles this case correctly. The peer ledger exchange logic is built on the assumption that servers are peers and are sharing work equally. It's not designed to tolerate a peer that makes other peers do all the work. You should issue such queries to your own rippled that's participating in the network, not to other people's rippleds that have no obligation to do extra work for you.
There are really only two alternatives. A server could never ever be allowed to refuse a request. That's an obvious non-starter. Alternatively, a server could be required to always send an active rejection of a request if it wasn't going to reply to it. That would add overhead in particularly the case where the server is trying to minimize overhead. And the intention of delaying the response is to force the queries to back off (to time out waiting for it). It doesn't seem to make sense to shift this work to the server that receives the query when the whole intention of the design is to put as much work on the querier as possible.

The peer network protocol is undocumented, so I gave up on this some years ago.


[QT] Failed to create OpenGL context for format QSurfaceFormat

Environment

OS Version: Ubuntu ionic
Source or binary build? Source Installation on Docker

Description

Expected behavior: I have successfully build the packages (although I have to admit it requires huge RAM, otherwise it doesn't work even if use --parallel-works 1 flag). I sourced my environment and run "gz sim shapes.sdf" to start gazebo.
Actual behavior: I get: [QT] Failed to create OpenGL context for format QSurfaceFormat

Steps to reproduce

1.Build the Pakcage.
2.Source the environment
3.gz sim shapes.sdf
Output

@Roboticistprogrammer any updates?

Might be worth looking at https://github.com/gazebosim/gz-sim/issues/2595 as well.


outline  parameter in json

Is the outline.png required for the manifest or can we omit this?
Do i need to create an outline file with my custom logo.png file?
Can I delete the outline parameter in the json file if it was the case?

Icons (color and outline) are required for this app. Check this link for more details.