HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep"  -everything but the switch

This change has all of PR #5689 except for changing the default value of marker retention from keep to delete.

leaves the default value of fs.s3a.directory.marker.retention at "delete"
no longer prints a message when an S3A FS instances is instantiated with any option other than delete.
Updates the directory marker documentation

Switching to marker retention improves performance on any S3 bucket as there are no needless marker DELETE requests -leading to a reduction in write IOPS and and any delays waiting for the DELETE call to finish.
There are very significant improvements on versioned buckets, where tombstone markers slow down LIST operations: the more tombstones there are, the worse query planning gets.
Having versioning enabled on production stores is the foundation of any data protection strategy, so this has tangible benefits in production.
Marker deletion is not compatible with older hadoop releases; specifically

Hadoop branch 2 < 2.10.2
Any release of Hadoop 3.0.x and Hadoop 3.1.x
Hadoop 3.2.0 and 3.2.1
Hadoop 3.3.0 Incompatible releases have no problems reading data in stores where markers are retained, but can get confused when deleting or renaming directories.

Contributed by Steve Loughran
Change-Id: Ic9a05357a4b1b1ff6dfecf8b0f30e1eeedb2fe75

Description of PR
How was this patch tested?
For code changes:

[ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

testing: s3 london. this is a backport and as it doesn't include the contentious issue "actually changing the switch" then I'm happy to cherrypick as is. lets see what the tests say now I've rolled back the default

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 55s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ branch-3.3 Compile Tests _

+1 :green_heart:
mvninstall
49m 15s

branch-3.3 passed

+1 :green_heart:
compile
0m 42s

branch-3.3 passed

+1 :green_heart:
checkstyle
0m 37s

branch-3.3 passed

+1 :green_heart:
mvnsite
0m 50s

branch-3.3 passed

+1 :green_heart:
javadoc
0m 42s

branch-3.3 passed

+1 :green_heart:
spotbugs
1m 16s

branch-3.3 passed

+1 :green_heart:
shadedclient
37m 32s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 43s

the patch passed

+1 :green_heart:
compile
0m 32s

the patch passed

+1 :green_heart:
javac
0m 32s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 22s

the patch passed

+1 :green_heart:
mvnsite
0m 38s

the patch passed

+1 :green_heart:
javadoc
0m 27s

the patch passed

+1 :green_heart:
spotbugs
1m 11s

the patch passed

+1 :green_heart:
shadedclient
37m 19s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 42s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 41s

The patch does not generate ASF License warnings.

139m 44s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5859

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux d6e4a71d159d 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.3 / 6a2b198dd70e08a1e93fddb8af44c3b752dd5f73

Default Java
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/1/testReport/

Max. process+thread count
585 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/1/console

versions
git=2.17.1 maven=3.6.0 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m  4s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ branch-3.3 Compile Tests _

+1 :green_heart:
mvninstall
53m 22s

branch-3.3 passed

+1 :green_heart:
compile
0m 38s

branch-3.3 passed

+1 :green_heart:
checkstyle
0m 30s

branch-3.3 passed

+1 :green_heart:
mvnsite
0m 45s

branch-3.3 passed

+1 :green_heart:
javadoc
0m 34s

branch-3.3 passed

+1 :green_heart:
spotbugs
1m 12s

branch-3.3 passed

+1 :green_heart:
shadedclient
42m 50s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 39s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed

+1 :green_heart:
javac
0m 31s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 19s

the patch passed

+1 :green_heart:
mvnsite
0m 36s

the patch passed

+1 :green_heart:
javadoc
0m 25s

the patch passed

+1 :green_heart:
spotbugs
1m 14s

the patch passed

+1 :green_heart:
shadedclient
40m 14s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 31s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

160m 52s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5859

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 4b8cb5861852 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.3 / 6a2b198dd70e08a1e93fddb8af44c3b752dd5f73

Default Java
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/2/testReport/

Max. process+thread count
527 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/2/console

versions
git=2.17.1 maven=3.6.0 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
7m 18s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ branch-3.3 Compile Tests _

+1 :green_heart:
mvninstall
52m 43s

branch-3.3 passed

+1 :green_heart:
compile
0m 43s

branch-3.3 passed

+1 :green_heart:
checkstyle
0m 36s

branch-3.3 passed

+1 :green_heart:
mvnsite
0m 50s

branch-3.3 passed

+1 :green_heart:
javadoc
0m 42s

branch-3.3 passed

+1 :green_heart:
spotbugs
1m 17s

branch-3.3 passed

+1 :green_heart:
shadedclient
37m 48s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 40s

the patch passed

+1 :green_heart:
compile
0m 33s

the patch passed

+1 :green_heart:
javac
0m 33s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 22s

the patch passed

+1 :green_heart:
mvnsite
0m 38s

the patch passed

+1 :green_heart:
javadoc
0m 27s

the patch passed

+1 :green_heart:
spotbugs
1m 10s

the patch passed

+1 :green_heart:
shadedclient
37m  9s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 47s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 40s

The patch does not generate ASF License warnings.

149m 39s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5859

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 7e5380e1a46b 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.3 / b986515adc2ece7f099809305ed4fb83e030bfa2

Default Java
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/3/testReport/

Max. process+thread count
552 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/3/console

versions
git=2.17.1 maven=3.6.0 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
5m 39s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ branch-3.3 Compile Tests _

+1 :green_heart:
mvninstall
36m  5s

branch-3.3 passed

+1 :green_heart:
compile
0m 30s

branch-3.3 passed

+1 :green_heart:
checkstyle
0m 24s

branch-3.3 passed

+1 :green_heart:
mvnsite
0m 37s

branch-3.3 passed

+1 :green_heart:
javadoc
0m 30s

branch-3.3 passed

+1 :green_heart:
spotbugs
0m 54s

branch-3.3 passed

+1 :green_heart:
shadedclient
23m 48s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 24s

the patch passed

+1 :green_heart:
javac
0m 24s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 15s

the patch passed

+1 :green_heart:
mvnsite
0m 27s

the patch passed

+1 :green_heart:
javadoc
0m 19s

the patch passed

+1 :green_heart:
spotbugs
0m 50s

the patch passed

+1 :green_heart:
shadedclient
23m 14s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 20s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 28s

The patch does not generate ASF License warnings.

99m 11s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5859

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux ca5c4acd6d1b 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.3 / 25e932285ac5713fcc1033ffb6bcf729a0e88c47

Default Java
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/4/testReport/

Max. process+thread count
681 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5859/4/console

versions
git=2.17.1 maven=3.6.0 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

OK, backport has gone in cleanly. This does not change the default value, as discussed, even though I'd like to.


Replace println! with log functions (info!, warn!, etc)

Requested feature
Replace println! with log functions (info!, warn!, etc).
Use debug! for dumping debugging material, e.g. in https://github.com/veracruz-project/veracruz/blob/main/runtime-manager/src/platforms/linux.rs#L261
Motivation
println! is still used extensively, especially in the Runtime Manager. This degrades performance and is potentially dangerous

@gbryant-arm how's the status on this? I think it's complete, but can't be sure

When running VOD with log level set to info, I'm getting a log over 15K lines.
I'd suggest we replace info! with debug! or trace! for most debug messages that keep showing up. Cf. tls_data() and read_tls_data() in veracruz-server/src/platforms/linux.rs


Profile in context of RAM usage

Try remove MqttDebugComponent and see how ram usage changes
... same with other components and especialy strings
Do it on both platforms 32 and 8266

Seems like done as a lot have been moved to progmem.
ESP32 doesn't support progmem, this does not matter there.


Issues with heading estimation

@priseborough
There seems to be serious issues present regarding heading estimation at places with high magnetic declination.
The problem is easy to reproduce in SITL. Just change the location to e.g. Seattle with a magnetic declination of ~15 degrees, make clean, run make posix_sitl_default gazebo and fly a mission.
Here is a logfile from such a case. I also set EKF2_MAG_DECL to -5 degrees for illustration purpose but it is not needed to expose the bug.
The following happens:
t = t0
The system boots up. The true heading is 25 degrees and the true declination is 15 degrees.
The EKF starts to initialize the filter by collecting and low pass filtering the magnetometer and barometer sensor data.
t = t1
When the EKF has collected 2 times the observation buffer length it proceeds with the initialization by running the resetMagHeading function. Here it initialize the heading by taking the last magnetometer measurement, rotate it into the inertial frame using the current attitude estimate ignoring the yaw, calculate the angle between the N and E component and add the magnetic declination. However, the declination is taken from the class variable _mag_declination and since the function calcMagDeclination() has not yet run a single time it will use the default value of zero.
Now the function initialiseFilter() returns true and the function controlFusionModes() is called. First thing that happens here is that we call calcMagDeclination(). In here since _NED_origin_initialised is false we set the _mag_declination variable to the EKF2_MAG_DECL parameter, in our case -5.
After that we start to wait for the tilt alignment and when that is done we again run the function resetMagHeading.
This time the _mag_declination is set to EKF2_MAG_DECL and the heading estimate is still zero so the earth magnetic field states are set to correspond to a declination of -5 degrees.
Now both tilt and yaw alignment is finished and the EKF is up and running. Since the heading is estimated to be zero degrees and the _mag_declination is using the parameter value the measured heading from the magnetometer will correspond to 10 degrees (true heading is 25 deg, true declination is 15 deg). The innovations will thus start at 10 degrees and slowly decrease simultaneously to the estimated heading rotating to 10 degrees. The estimated heading will even overshoot since the gyro z bias will grow.
t = t2
Now the EKF gets GPS lock and sets _NED_origin_initialised to true. In the next iteration of calcMagDeclination() it then updates the _mag_declination variable to the value returned by the look up table on the GPS position. _mag_declination is hence set to the true value of 15 degrees. Now the heading innovation starts at 15 degrees, the estimated heading will start to rotate against the true value of 25 but will overshoot massively because of the now very large gyro Z bias.
t = t3
The quad takes off and starts flying the mission with a heading estimate error of 10 degrees.
t = t4
The quad finished the mission. The heading estimate has now converged to the true value

Heading innovations

blue is the magnetic declination calculated in postprocessing from the estimator states, red is the ecl class variable _mag_declination added to logging

Flying almost the same mission but with EKF2_MAG_TYPE = 0 (automatic switching from 3D axes and simple heading fusion) is even worse.
This is the true heading, estimated heading, the magnetic declination calculated from the estimated earth magnetic field states and the EKF2_MAG_DECL parameter added to logging

Note that the second time the exact same mission is flown everything is fine since the EKF2_MAG_DECL automatically is updated to the correct value in the first flight

As a first step what about grabbing the declination in resetMagHeading? At least then it would be restored if previously saved to the parameter.
This still doesn't on a completely fresh system or the first use in a new area. What about forcing realignYawGPS() as soon as the GPS checks pass and NED origin is initialized?

https://github.com/PX4/ecl/tree/pr-ekf_mag_decl_cleanup
https://github.com/PX4/Firmware/tree/pr-ekf_mag_decl_cleanup

@CarlOlsson If you're using heading fusion you are directly extracting the measured heading from the magnetometer measurement. By definition there are no magnetic earth field states involved.

The estimated earth magnetic field states could still be used to calculate a more accurate value of the magnetic declination than the one obtained from the GEO lookup table. Which, actually is already done, just missed it before

I like the idea of a running realignYawGPS() as soon as the WGS-84 origin has been set. Will submit a patch for that.

The estimated earth magnetic field states could still be used to calculate a more accurate value of the magnetic declination than the one obtained from the GEO lookup table. Which, actually is already done, just missed it before
@CarlOlsson That's true as long as your not using heading fusion only. In that case the mag alignment never takes place.

The resetMagHeading() function currently does not reset the states and covariances unless the change in magnetic yaw is greater than 15 degrees. This was done originally to avoid unnecessary disturbance of the covariance matrix when doing the in-air yaw alignment. As a compromise we could reset the yaw angle when the change is <15 degrees, but leave the covariance matrix alone.

See https://github.com/PX4/ecl/pull/520


feat(tls): Allow the underlying TLS implementation to be selected by feature flags.

Content
The mithril-client library uses the reqwest crate.  By default, reqwest uses OpenSSL for TLS support in HTTPS requests.  This causes any consumer of the library to have a dynamic dependency on a system OpenSSL library.
reqwest has a number of options for fulfilling tls functionality.  This patch just exposes that, using the same feature flags that reqwest uses.
Pre-submit checklist

Branch

[x] Tests are provided (if possible)
[x] Crates versions are updated (if relevant)
[x] CHANGELOG file is updated (if relevant)
[x] Commit sequence broadly makes sense
[x] Key commits have useful messages

PR

[x] No clippy warnings in the CI
[x] Self-reviewed the diff
[x] Useful pull request description
[x] Reviewer requested

Documentation

[x] Update README file (if relevant)
[ ] Update documentation website (if relevant)
[ ] Add dev blog post (if relevant)

Comments
This PR came about because our project requires static linking of dependent libraries, and this was not possible when using the mithril-client crate.
Issue(s)
Closes #1737

@stevenj could you bump the patch version of the mithril-client crate before we can merge the PR?


WIP: [Release] Verify release-13.0.0-rc1

PR to verify Release Candidate

@github-actions crossbow submit  --group verify-rc-source --param release=13.0.0 --param rc=1

Revision: cbf8e418212de9a40e35375fa7b97afd2b0b2cf0
Submitted crossbow builds: ursacomputing/crossbow @ release-13.0.0-rc1-0

Task
Status

almalinux-8-amd64

almalinux-8-arm64

almalinux-9-amd64

almalinux-9-arm64

amazon-linux-2023-amd64

amazon-linux-2023-arm64

centos-7-amd64

centos-8-stream-amd64

centos-8-stream-arm64

centos-9-stream-amd64

centos-9-stream-arm64

conan-maximum

conan-minimum

debian-bookworm-amd64

debian-bookworm-arm64

debian-bullseye-amd64

debian-bullseye-arm64

debian-trixie-amd64

debian-trixie-arm64

java-jars

nuget

python-sdist

r-binary-packages

test-ubuntu-default-docs

ubuntu-focal-amd64

ubuntu-focal-arm64

ubuntu-jammy-amd64

ubuntu-jammy-arm64

ubuntu-lunar-amd64

ubuntu-lunar-arm64

wheel-clean

wheel-macos-big-sur-cp310-arm64

wheel-macos-big-sur-cp311-arm64

wheel-macos-big-sur-cp38-arm64

wheel-macos-big-sur-cp39-arm64

wheel-macos-mojave-cp310-amd64

wheel-macos-mojave-cp311-amd64

wheel-macos-mojave-cp38-amd64

wheel-macos-mojave-cp39-amd64

wheel-manylinux-2-28-cp310-amd64

wheel-manylinux-2-28-cp310-arm64

wheel-manylinux-2-28-cp311-amd64

wheel-manylinux-2-28-cp311-arm64

wheel-manylinux-2-28-cp38-amd64

wheel-manylinux-2-28-cp38-arm64

wheel-manylinux-2-28-cp39-amd64

wheel-manylinux-2-28-cp39-arm64

wheel-manylinux-2014-cp310-amd64

wheel-manylinux-2014-cp310-arm64

wheel-manylinux-2014-cp311-amd64

wheel-manylinux-2014-cp311-arm64

wheel-manylinux-2014-cp38-amd64

wheel-manylinux-2014-cp38-arm64

wheel-manylinux-2014-cp39-amd64

wheel-manylinux-2014-cp39-arm64

wheel-windows-cp310-amd64

wheel-windows-cp311-amd64

wheel-windows-cp38-amd64

wheel-windows-cp39-amd64

Revision: cbf8e418212de9a40e35375fa7b97afd2b0b2cf0
Submitted crossbow builds: ursacomputing/crossbow @ release-13.0.0-rc1-1

Task
Status

test-ubuntu-default-docs

Revision: cbf8e418212de9a40e35375fa7b97afd2b0b2cf0
Submitted crossbow builds: ursacomputing/crossbow @ release-13.0.0-rc1-2

Task
Status

test-ubuntu-default-docs

Closing this because we will be creating a new RC

Closing this because we will be creating a new RC

Any timeline for this? :)
(no pressure, just so that I know when I need to allocate a bit of time; talking about 13.0.0 GA)

Closing this because we will be creating a new RC

Any timeline for this? :)
(no pressure, just so that I know when I need to allocate a bit of time; talking about 13.0.0 GA)

I believe that the blocker for the creation of the next RC (and discussion about it) can be be found here: https://github.com/apache/arrow/issues/36892#issuecomment-1655445724

Thanks @thisisnic !
@h-vetinari, as shared by @thisisnic I am waiting for some feedback on the issue above. I plan to start creating the RC tomorrow if I don't get any more info. Once the RC is created running all the verification tasks and creating binaries take a couple days + the vote itself which takes 2-3 more days, etc.


Can you first expose the training scripts, PT scripts and sft scripts, and synthesis scripts?

Can you first expose the training scripts, PT scripts and sft scripts, and synthesis scripts?

You can refer to our first-gen script first, as Llama-2 7B/13B do not use GQA, which makes them compatible with their firsr-gen. We will indeed release our code in a future time, but we are now occupied.

@ymcui
Thank you for your reply
I tried the first generation of scripts, it seems that it can be trained, but I don't know why the trained LoRa merges with the llama2-HF model and makes an error, but it can be merged with the ALPACA model, that is, the final reasoning will be garbled

@ymcui  你好在你所知道的范围内，有没有一款 微型模型的训练，并且可以进行组合的。多个模型组合成一个专家模型的


Update azure-api-management-certs.md

Hi Team,
Can you please update the change. This feature in Azure Portal works only if the user is a Co-Administrator on the Subscription.
Best
Santhosh
Senior Program Manager, Azure Portal

@santhoshsomayajula,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@santhoshsomayajula : Thanks for your contribution to the Azure documentation! The author, @Thraka, has been notified to review your proposed change.

@cwatson-cat

#sign-off


Restoring mergify_linter in CI

This PR is for the issue: https://github.com/python-wheel-build/fromager/issues/172
I am new to writing workflows, CI and GitHub actions so not sure whether what I have done in this PR is correct. Please let me know if any changes are required

Looks like the yaml module is not installed: https://github.com/python-wheel-build/fromager/actions/runs/9996065530/job/27629569454?pr=198#step:5:15.
You will have to install it in the install dependency step. I also don't think you need tox unless you are adding a new target to run the mergify linter

Oh okay! Let me add that step

Okay, I am a bit confused...
tox.ini already has this (https://github.com/python-wheel-build/fromager/pull/62/files ) included in main branch.
Is there any other way to restore or am I missing something?

It looks like maybe a merge conflict was resolved poorly.
I see the linter invoked here, in the pkglint section of tox.ini https://github.com/python-wheel-build/fromager/blob/main/tox.ini#L67
I expected that to show up in the linter section.

okay, I will move it there


Patch for heading bug fix

Patches problem in fix PR #309.
Another solution is to revert #309 and to delete line 301 (if it is not needed).

This PR can be closed.


Example of using Dataframe row

I was wondering if there is an example of how to extract a row of a dataframe into a struct in rust; such as after the dataframe was returned from r.  Below is an example:
Thanks.
use extendr_api::prelude::*;

fn main() {
    #[derive(Debug)]
    struct Iris {
        sepal_length: f64,
        sepal_width: f64,
        petal_length: f64,
        petal_width: f64,
        species: String,
    }

    extendr_engine::start_r();

    let df: Dataframe<Iris> = eval_string("head(iris)").unwrap().try_into().unwrap();

    // how to convert Dataframe to Vec of struct, or extract an individual row as an Iris struct?
    // let x: Vec<Iris> = df.try_into().unwrap();

    extendr_engine::end_r();
}

Thank you for the response.  I was able to do something like this but since I already set the struct type of the Dataframe when I created it, I figured there might be an easier way without having to rewrite the column type in the as_ calls.
  df.as_list().unwrap().dollar("sepal_length").unwrap().as_real_iter() ....

There is a more 'Rust' way, I will try to come up with something more elegant later today / tomorrow.

Ok, now I am convinced we completely lack proper support for DataFrames access in extendr. It can be improved of course, especially to support scenarios similar to yours, but I cannot promise when this can be added and in which form.
In the meantime, you can as_list() on your dataframe and then iterate over name/value pairs, and then zip iterators to get iterator over a row.

Ok. I think we can close this for now.


Hugging face models direct compatibility with doc.embed() method

At present, docarray is supporting PyTorch, Keras, Paddle and ONNX format in doc.embed() method for automatic filling of .embeddings object. It would be great if docarray supports adding Hugging face models (transformers and sentence-transformers) directly.

To be tackled in docarray v2


update android gradle<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

To target SDK 26 in future releases of React Native, we started to use build tools version 26.0.3 and compile SDK version 26 (commit https://github.com/facebook/react-native/commit/a3931e95317acaff5ceb1d2fb0f26ae67744c8a3)
Now we can update android gradle plugin 2.3.3, which requires build tools versions 25 and above, to make builds faster and support Instant Run in Android Studio.
Android gradle plugin 2.3.3 requires gradle 3.3, thus gradle-download-task supports gradle 3.3 from version 3.2 and above.
Next goal is gradle plugin 3.0.0, which introduced many new features and requires migration work.
https://developer.android.com/studio/releases/gradle-plugin
Test Plan
React Native and React Native apps must build and run as usual

Warnings

:warning:

:clipboard: Release Notes - This PR appears to be missing Release Notes.

  Generated by :no_entry_sign: dangerJS

read through gradle 3.x release notes and found that version 3.5 improves performance, but don't introduce breaking changes.
Please merge, I think that it would help to target sdk 26 in the future.


DELIA-64243:  Add Conditional WAI Compile Code for Amlogic and RTK

Reason for change: Added Conditional Compile Code for
WhoAmI in Amlogic and RealTek Platforms
Test Procedure: Check if the Maintenance Manager gets WAI data from SecMgr
Risks: Medium
Priority: P1

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Failed Test Cases after enabling fulfillment

I have started using action-on-google-testing package for our enterprise bot. During testing, I found some issue regarding failed scenarios -:
1.sometime intent doesn't match with my utterance.
2. When I enable fulfillment for webhook, I not getting any response but when I disable the fulfillment and added static response that time it was working fine.
error:
2 exceptions
Uncaught Exception
TypeError: Cannot read property 'url' of undefined
ClientDuplexStream.conversation.on (node_modules/actions-on-google-testing/src/actions-on-google.ts:559:62)
Object.onReceiveMessage (node_modules/grpc/src/client_interceptors.js:1292:19)
InterceptingListener.recvMessageWithContext (node_modules/grpc/src/client_interceptors.js:607:19)

Do these things work if you were to test using the simulator, or a physical device?

Yes Fleker !
We tested so many times manually from physical devices as well as simulator.It was giving the correct response.


[BUG] The default genesis.json doesn't work on the last dev branch.

OS and Environment
MacOS, Docker Hub
GIT commit hash
fbc7d346
Minimum working example / Steps to reproduce
docker compose up

Actual result
genesis.json did not apply. Peers did not rise.
Expected result
genesis.json applied.
Logs

Log contents
iroha-iroha0-1  | Error: Failed to deserialize raw genesis block from "/config/genesis.json"
iroha-iroha0-1  |
iroha-iroha0-1  | Caused by:
iroha-iroha0-1  |     data did not match any variant of untagged enum ExpressionDeserializeWrapper at line 246 column 9
iroha-iroha0-1  |
iroha-iroha0-1  | Location:
iroha-iroha0-1  |     /iroha/core/src/genesis.rs:254:41

Who can help to reproduce?
@astrokov7
Notes
Problem is reproducible. This genesis doesn’t work.

Hi, @astrokov7. Do you have a solution for this?

Hi, @astrokov7. Do you have a solution for this?

yes, you can re-generate the default genesis with
cargo run --bin kagami -- genesis > configs/peer/genesis.json

Hi, @astrokov7. Do you have a solution for this?

Just update docker image docker pull hyperledger/iroha2:dev


kpt v0.36.x preventing gcloud.core.project auto setter from working

When following this guide: https://www.kubeflow.org/docs/gke/deploy/management-setup/
At the step to fetch the management blueprint with:
kpt pkg get<EMAIL_ADDRESS>"${MGMT_DIR}"

I was receiving the following error (using kpt v0.37.1):
failed to set "gcloud.core.project" automatically in package "management" with error: setter "gcloud.core.project" is not found

reverting to kpt version v0.35.0 fixes this issue and the changelog for kpt (https://github.com/GoogleContainerTools/kpt/releases) shows some updates in v0.36.0 to how setter inheritance is handled which is likely responsible for this.

Added an issue to the docs page as well: https://github.com/kubeflow/website/issues/2437

Added an issue to the docs page as well: https://github.com/kubeflow/website/issues/2437

Hi @sidpalas! This is a behavior change on kpt side, can you report an issue to kpt if it doesn't exist right now.
For Kubeflow deployment purpose, I think you can ignore the error,
because kpt pkg get<EMAIL_ADDRESS>"${MGMT_DIR}" would still succeed in fetching the package to local dir, we will set values by ourselves after that.

Hi @sidpalas! This is a behavior change on kpt side, can you report an issue to kpt if it doesn't exist right now.
For Kubeflow deployment purpose, I think you can ignore the error,
because kpt pkg get<EMAIL_ADDRESS>"${MGMT_DIR}" would still succeed in fetching the package to local dir, we will set values by ourselves after that.


Flask-RESTPlus is unmaintained, development has moved to Flask-RestX.

We are using FlaskPlus for the API, but the project is dead.
It had to be forked by the old maintainers ( drama here ), the new repo is at https://github.com/python-restx/flask-restx
Since the code is backwards compatible, we have to do an import name change and change the dependency from the dnf package of restplus to restx's pip package. (not in dnf yet)

uhhhh...
@IceWreck thanks for raising this
[django? :smiling_imp:]

oh, thanks, will take care of the fedora RPMs

https://bugzilla.redhat.com/show_bug.cgi?id=1817535
needs a Fedora Reviewer
(https://fedoraproject.org/wiki/Package_Renaming_Process#Re-review_required)


Quick and dirty fixes for 3.5.1

SPT team changed blacklist config in 3.5.1, so I made quick and dirty fix - feel free to correct and suggest fixes.
Tested it with my config from 3.5.0 - works fine.

Well, looks like it's not needed anymore.


evn file is default but error

2024-08-06 07:52:43 | INFO     | 285 - bae | Balance: 125970
2024-08-06 07:52:58 | ERROR    | 181 - bae | Unknown error while getting boosts | Error: 503, message='Service Unavailable', url=URL('https://www.okx.com/priapi/v1/affiliate/game/racer/boosts?extUserId=7255409818&t=1722905577999')
2024-08-06 07:52:58 | ERROR    | 342 - bae | Unknown error: 'NoneType' object is not iterable

This error means that the problem was on the server side


Automatic Differentiation

No longer defaults to PolyesterForwardDiff but can be opted in if installed. This is to prevent some confusing behavior regarding before and after loading PolyesterForwardDiff.
Entry Point for ReverseDiff added.
Entry Point for Tracker added.

the macos failure is that same broyden one which I haven't yet been able to reproduce anywhere else.


[Ubuntu] Set flags for curl

Description
Set -fsS flags for curl to catch errors early. Also add -L flag to handle redirects and improve build stability.
Related issue: -
Check list

[ ] Related issue / work item is attached
[ ] Tests are written (if applicable)
[ ] Documentation is updated (if applicable)
[x] Changes are tested and related VM images are successfully generated

/azp run ubuntu20,ubuntu2004

/azp run ubuntu2204,ubuntu2004


FastAPI + SQLAlchemy + REST + GraphQL

Codecov Report

Merging #23 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #23   +/-   ##
=======================================
  Coverage   94.73%   94.73%
=======================================
  Files           7        7
  Lines         228      228
=======================================
  Hits          216      216
  Misses         12       12

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update befd916...7daa793. Read the comment docs.


CSV fields not quoted properly on copy-to-clipboard

(Issue) CSV output: when the data has a comma in it, the CSV isn't enclosing it in quotes or anything. So, when copying to clipboard and then importing to a spreadsheet, the data of that cell gets split into columns due to the rogue comma.

Looks like I improperly implemented converting the HTML table to CSV here: https://github.com/tabulapdf/tabula/blob/feature/newUI/webapp/static/js/pdf_view.js#L216-L223
from https://github.com/tabulapdf/tabula/issues/303

fixed in 5448d84


fix Poetry dependency

fix Poetry dependency
upgrade Gym to Gymnasium
support for Python 3.10
fix minor bugs in Stock_NeurIPS2018_3_Backtest.ipynb and plot the backtesting results of all models (A2C, DDPG, PPO, TD3, SAC).

The backtesting results of all models did not outperform the benchmark. Did I do something wrong?

thanks for your effort
have you tested it?
in software engineering, we hope that a PR only solves a problem for review. could you pls split this PR to several PRs?

Hi @zhumingpassional,
I upgraded Python to 3.10 and Gym to Gymnasium in order to fix the dependency issue, so I don't think we should split them into multiple PRs.
I was running Stock_NeurIPS2018_3_Backtest.ipynb to test the new dependency. I can but I don't think it's a good idea to split the test into another PR.
After the fix of this PR, you should be able to install dependencies simply with poetry install and run Stock_NeurIPS2018_3_Backtest.ipynb with zero error.

Hi @dancju
Thanks for your commits. I just tested the updates, and will do few more. If no errors are found, I will merge the PR soon.

it raises errors in the notebook Stock_NeurIPS2018_SB3.ipynb
the PR will be merged after multiple tests
this PR will be reverted considering of error
could you pls revise the PR and submit it again?

pyfolio is no longer maintained and its last commit was in 2020, which is why I removed it from pyproject.toml. I have put it back and will create a new PR.

Does tat mean we need to upgrade python to 3.10 to be able to use Gymnasium instead of Gym?

@lcavalie You can try yourself.
Edit pyproject.toml. Change from python = "~3.10" to python = "~3.9".
poetry env use 3.9
poetry lock
poetry install


Create package list when packages are installed or removed

Another shot at #18; fixes #12. Registers one callback that toggles a boolean when all packages are activated on startup, and another callback that on package load, if initial package activation has completed and the config value is true, writes to the package list.

Thanks for your perseverance :grinning:


Missing possible edge cases in Sorted Union challenge

Affected page
https://www.freecodecamp.org/learn/javascript-algorithms-and-data-structures/intermediate-algorithm-scripting/sorted-union
Your code

function uniteUnique(arr)
{
  //new array, made to make a copy of arguments to follow functional programming practices
  let narr = [];

  //make a copy of all argments and push into narr
  for(let i = 0;i<arguments.length;i++)
  {
    narr.push(arguments[i]);
  }

  //filters out duplicates contained in
  for(let i=1;i<narr.length;i++)
  {
    narr[i] = narr[i].filter(a => !narr[0].includes(a));
  }

  //combines all arrays together
  for(let i = 1;i<narr.length;i++)
  {
    narr[0] = narr[0].concat(narr[i]);
  }

  return narr[0];
}

console.log(uniteUnique([1, 3, 2], [5, 2, 1, 4], [2, 1]));

Expected behavior
The code above should not pass the challenge however it does. For example uniteUnique([1, 3, 2, 3], [5, 2, 1, 4], [2, 1]) returns [ 1, 3, 2, 3, 5, 4 ]. [ 1, 3, 2, 3, 5, 4 ] has two 3s in it but it still pass the checks for the challenges despite not meeting the requirement of only having unique values in the final array. This edge case has duplicate elements in the first array of arguments.
Another example where my code should fail the challenge is when other arrays have duplicates with each other in arguments but not the first array. uniteUnique([1, 3, 2], [5, 4], [5, 6]) returns [ 1, 3, 2, 5, 4, 5, 6 ], which again has duplicate elements yet it still passes the challenge.
To conclude their should 2 edge cases added to make sure the challenge is passed

An edge case that has the first array in arguments have duplicate elements such as uniteUnique([1, 3, 2, 3], [5, 2, 1, 4], [2, 1])
An edge case where duplicates are found between arrays but not with the first array in arguments such as uniteUnique([1, 3, 2], [5, 4], [5, 6])

Screenshots

System

Device: Desktop
OS: Windows 11
Browser: Firefox
Version: 95.0.2

Additional context
I see this an issue because I missed it myself originally and almost went to the next problem thinking I learned how to actually solve the issue. It wasn't until I took a look at my code to clean it up that I noticed the two edge cases that would make my code not meet the requirements of the challenge yet still pass the challenge.

I would like to help with this issue.

Note: for anyone willing to take on this issue, you may need to adjust the solution function

@marsonya
An issue with the help wanted or first timers only label is open for contribution. The first comprehensive PR created will be reviewed and merged. We typically do not assign issues to anyone other than long-time contributors.
If you would like to contribute, and have not read the contributors docs, please do so here: https://contribute.freecodecamp.org/#/
If you have any issues with contributing, be sure to join us on the contributors channel, or on the contributors sub-forum


Invalid API Key Google STT

I've got Jasper up and running on my Raspberry Pi 3 (running Raspbian Jessie) and I've managed to get it working with the Pocketsphinx and Wit.ai STT engines, but I haven't been happy with the performance of either of those. I'd really like to use the Google STT but whenever I run it, I get this error in the console:
CRITICAL:client.stt:Request failed with http status 403
WARNING:client.stt:Status 403 is probably caused by an invalid Google API key.

I've enabled the Speech API in Google Dev and generated an API key. I've triple checked the key multiple times, tried formatting it in the profile.yml multiple ways, and I even tried inserting the key directly into the stt.py source code, but no luck. Any suggestions?

I have the exact same problem. Have you found a way to fix it since December

same here


helm install --wait doesn't work for Deployment pods

helm version
Client: &version.Version{SemVer:"v2.4.2", GitCommit:"82d8e9498d96535cc6787a6a9194a76161d29b4c", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.4.2", GitCommit:"82d8e9498d96535cc6787a6a9194a76161d29b4c", GitTreeState:"clean"}

I used the following command:
helm install stable/etcd-operator --name etcd-operator --wait --timeout 300
I thought that helm will wait for all deployment pods to be ready, unfortunately it returns almost immediately and pod is in pending state.
I tested it with kubernetes versions 1.5, 1.6 and master branch

At first glance it seems like podsReady function is checking only pod status, not its phase (which is where pending lives)? Is it expected behaviour? @thomastaylor312

In scenario where Deployment has replicas set to 1 and maxUnavailable is not set to 0 as part of rolling update strategy, --wait will return as ready as it has satisfied the minimum Pod in ready condition.
https://github.com/kubernetes/helm/blob/master/docs/using_helm.md#helpful-options-for-installupgraderollback

Oh, I see. Is there a way to set maxUnavailable to 0 when running helm install?

@lukaszo there's .Release.IsInstall and .Release.IsUpgrade  you can use in your templates.

Ok, I understand how it works. Thank you for your help.


Refactor to target .NET standard 1.1; unified naming conventions.

Suggestion given in #5 implemented. Additionally, some types and files were renamed to better follow the .NET naming conventions: MIPS* -> Mips*, x86* -> X86*.

I've been trying to make my mind up about the naming conventions but I think this should be the naming convention to use.

I don't think I missed anything, besides some samples or file names.

Looks good to me 🎉


Modifier.clickable and .pointerMoveFilter don't seem to respect z-order

This seems unexpected - that is the mouse is still being detected on elements that are under others. This happens for both hover and click events.

Hmm seems I can only repro this with stickyHeader on the LazyColumn. This used to be consistently happening when I took this recording. Closing issue for now.


Added Tictactoe Game.

Please describe the changes this PR makes and why it should be merged:
Added a TicTacToe Game built using million.js and React. It allows two users to play the game simultaneously.
Status
closes: kitchenSink/TicTacToe#648

[x] Code changes have been tested against prettier, or there are no code changes
[x] I know how to update typings and have done so, or typings don't need updating

Semantic versioning classification:

[ ] This PR changes the codebase

[ ] This PR includes breaking changes (methods removed or renamed, parameters moved or removed)
[ ] This PR changes the internal workings with no modifications to the external API (bug fixes, performance improvements)

[ ] This PR only includes non-code changes, like changes to documentation, README, etc.

@tobySolutions Please review my PR.

Awesome! Sure thing!

Resolves #648


latest version of yt-uploader is not working   unable to extract uploader id;

Checklist

[x] I'm asking a question
[x] I've looked through the README and FAQ for similar questions
[x] I've searched the bugtracker for similar questions including closed ones

Question

It appears latest version of yt-uploader is not working or not able to pull from youtube?
youtube-dl --get-filename -o '%(title)s.%(ext)s' BaW_jenozKc --restrict-filenames
WARNING: unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

[x] I've searched the bugtracker for similar questions including closed ones

Review #30839 as asked.


Current Top/Records EHP/EHB links

On the /top or /records page, if i click someone's name while viewing ehp/ehb, it takes me to the player's activities gains/records.
EHP should link to skilling, EHB should link to bossing

The fix would be in the getPlayerURL functions on both of those pages.


Nf core template merge 2.12

Hi, this is the first time I'm trying to merge an automated PR, so I was unsure about some things. I tried following the nf-core docs on "Merging automated PRs".
Can some more experienced people double-check to see what I missed?
As I'm new to this, feel free to delete this PR if it's more trouble trying to fix it, than just starting from scratch, no problem!
These are the changes I made:

whenever it was a 'version update', I approved the change

for example: .github/workflows/awsfulltest.yml

some more elaborate updates of 'standard' files I also approved the change

.github/workflows/fix-linting.yml
.github/workflows/linting.yml
modules/nf-core/fastqc/tests/main.nf.test
assets/nf-core-circrna_logo_light.png

some files I kept the original nf-core/circrna dev code, as the proposed PR changes seemed to remove specific pipeline stuff (and often replace it with nothing), so it made more sense to me to keep the original

CITATIONS.md
conf/modules.config
main.nf
nextflow_schema.json

sometimes I made a combination of both. I guess these are the files that need the most proof-reading

README.md
docs/output.md
docs/usage.md
modules.json
nextflow.config

I'm not sure about these two files where args is added. For now, I did not follow the update. Could someone check if this is correct?

lib/WorkflowMain.groovy
main.nf

I'm also not sure about this one. It suggested to remove the params.species check, but I assume it's useful to keep it? So I kept it

lib/WorkflowMain.groovy

and about this one, which tried to change the INPUT_CHECK process
line 116 in workflows/circrna.nf

Lastly, I noticed linting fails, but I think this is due to this previous commit #83

I changed the target to dev

So there are three things that should be adressed here I think:

[ ] Linting: There are some inconsistencies between nextflow.schema (the file that is relevant for the documentation) and nextflow.config (the file that is used for the pipeline). They should be pretty easy to fix
[ ] Pipeline execution: There is some problem in the pipeline, which makes it not executable. Could you try running it with the test configuration, to make sure everything works properly?
[ ] Documentation: Sometimes it is a bit hard to keep track of all the changes that are made, so once the other problems are sorted out, we can check if something got messed up in documentation


Start/end frames, BT.2020 HEVC output, adjusted label/flash location, all Python 3

Added the following functionality:

Added optional start (dark green) and/or end (dark red) frames to signal the start/end of the test stream using the --start-end-indicators argument. Would it be useful to use a test pattern for the first frame instead or is a single color best for test automation reasons?
Added support for BT.2020 HDR output. If the source content is BT.2020, the script encodes the output using HEVC. (Note: ffmpeg ProRes output colorspace, primaries and transfer function signalling does not seem to be working as expected.)

Change:

Location of label and AV sync flash adjusted to provide enough space for a JS app to also display a timecode (top-centre) and QR code (right side of the screen).

Changes to address issues from Zack's review:

Added requirements.txt and updated the README with some more details (once we agree on the final version we can clean up/merge the older parts).
All scripts now require Python 3 instead of a mix of Python 2 and 3. Used 2to3 to translate the test sequence generator scripts to Python 3 and resolved a new related issue.
Now uses pathlib to avoid using OS-specific directory paths.

Remaining issues to look into:

Avoid generating QR codes, flashes and beeps into files and instead directly synthesize them in memory.
Adapt annotations when source content is HDR (to avoid overly bright graphics).
From Eurofins comments in issue #1 : "To make automated capture easy it may be necessary to silence spoken audio either side of the beeps". - This isn't done at present. If this is still a concern, perhaps the weighting of the original source audio vs the beeps can be adjusted to avoid any detection issues.
From Eurofins comments in issue #1 : "We would suggest having an audio tone at the start (for audio start up delay) and end, and at a consistent jump to point (e.g. 20s) to enable automated checking." - Can add a tone to accompany the start/end frames. Considering the irregular AV sync pattern repeats after a configurable duration, do we need to additionally define a specific jump point?
Comment in issue #1 : "With the audio need to make sure that every audio sample has a defined start and end tone (apply a different frequency than the beep - could also be a short jingle (identifying) and it is longer - like 1 seconds)." - The configurable irregular AV sync pattern should already address this, but to be confirmed.

Resolves #10 , resolves #9.

Updated mezzanine.py:

Inverted QR codes (now white on black), to minimize amount of peak white graphics shown.
Added 1kHz beep to accompany start/end indicator frames.
Made configuration of start/end frame colour and duration easier.
Fixed outdated parameters for start only and end only indicator frame options.

Updated mezzanine.py:

Fix for fractional frame rate handling (confirmation from a reviewer would be welcome).
Increased precision of timecode written to QR codes to match with timecode displayed by ffmpeg, as there was a mismatch with longer content durations.

Please add the non-tracked issues to the issue tracker


Mondo's names use x-linked instead of X-linked

Mondo term (ID Label)
mondo_id | name
MONDO_0010275 | spondyloepimetaphyseal dysplasia x-linked with mental deterioration
MONDO_0010278 | Angelman-like syndrome x-linked
MONDO_0010278 | intellectual disability x-linked syndromic Christianson type
MONDO_0010337 | intellectual disability x-linked 60 (formerly)
MONDO_0010337 | intellectual disability x-linked with cerebellar hypoplasia and   distinctive facial appearance
MONDO_0010337 | mental retardation x-linked 60 (formerly)
MONDO_0010337 | mental retardation x-linked with cerebellar hypoplasia and distinctive   facial appearance
MONDO_0010519 | Alpha-thalassemia x-linked intellectual disability syndrome
MONDO_0010580 | autoimmunity-immunodeficiency syndrome x-linked
MONDO_0010580 | polyendocrinopathy, immune dysfunction and diarrhea x-linked
MONDO_0010626 | hyper IgM immunodeficiency, x-linked
MONDO_0024772 | intellectual developmental disorder, x-linked, syndromic, Pilorge type
MONDO_0024773 | spermatogenic failure, x-linked, 4
MONDO_0024773 | spermatogenic failure, x-linked, 4
MONDO_0025351 | multiple congenital anomalies-neurodevelopmental syndrome, x-linked
MONDO_0035826 | symptomatic form of x-linked centronuclear myopathy in female carriers
MONDO_0800250 | graves disease, susceptibility to, x-linked 2
(new terms from new release)
MONDO:0800318 | Emery-dDeifuss muscular dystrophy 6, x-linked
MONDO:0800319 | cone dystrophy 5, x-linked
MONDO:0800320 | cone dystrophy 1, x-linked
MONDO:0800321 | congenital heart defects, multiple types, 1, x-linked
Bug/Typo/Error description
These names use a lower case x instead of upper-case X when referring to X chromosome linked disorders. (Most are synonyms, but some are the preferred terms.)
Would Mondo consider using the capitalized form X instead?  (NOTE in onecases the preferred and alternate names both have the lower case x, see MONDO_0024773)
Your nano-attribution (ORCID)
If you don't have an ORCID, you can sign up for one here

related to Issue #5771


Removing subtitle formatting

Is it possible for the automator to remove any font formatting on external subtitles?

Not that I'm aware of, I'm sure there are outside programs that can that might be able to be integrated into the 'extra scripts' functionality to scrub the formatting

I'm using Sonarr/Radarr with Deluge and SABnzbd. How do I get sickbeard_mp4_automatator to execute a script after converting but before passing back to sonarr/radarr?


on_create_set() with set() generates invalid cypher

I'm trying to merge a node, set a property if a new node was created, and set another property regardless of create or match. The cypher for this works perfectly:
MERGE (t:Thing { name: 'something' })
ON CREATE SET t.created = 'yes'
SET t.foo = 'bar'

However, the neo4j-core Query:
q.merge(t: { Thing: { name: 'something' } })
  .on_create_set(t: { created: 'yes' })
  .set(t: { foo: 'bar' })

generates the cypher:
MERGE (t:Thing { name: 'something' })
SET t.foo = 'bar'
ON CREATE SET t.created = 'yes'

which is invalid because ON CREATE must immediately follow MERGE.
Runtime information:
Neo4j database version: 3.5.6 (Community)
neo4j-core gem version: 9.0.0

I will try to create a pull request for this one


Fix php8 deprecation warning

When using this module with PHP8, following deprecated warning is logged:
PHP Deprecated:  Required parameter $dataTable follows optional parameter $options in /var/www/html/vendor/omines/datatables-bundle/src/Column/AbstractColumn.php on line 41
Since the $options parameter always needs to be set (because $dataTable is required), we can safely remove the default value for $options.

Cheers, thanks!

@curry684 You're welcome 😄  Do you think it would be possible tag a new release with this change?

Done.


Research why (and ideally stop) Renovate suddenly started opening PRs for GO patch releases

Example: https://github.com/terraform-google-modules/terraform-google-org-policy/pull/112

Perhaps related to https://github.com/renovatebot/renovate/pull/25479


user guide, misleading/incomplete update instructions 1.7.2 -> 2.0

Hi,
Upgrade instructions:  Upgrading from 1.7.2 to 2.0.0
http://codeigniter.com/user_guide/installation/upgrade_200.html
I recommend to put a bright notice 'Read the changelog, a lot of stuff has changed'.
There ara a lot of changes which are not mentioned in the upgrade info and I guess it's easier to just read that one.
I did not read the change log and it did bite me.

Tom, can you itemize what the changes were that were not in the upgrade notes but were in the Change Log that affected you?

I got bitten by getwhere, somewhere forgotten in a very old, long forgotten  (super stable 'til now) routine.
I've compared the change log 1.72. to 2.0.0 with the upgrade instructions,here's the list :
General
PHP 4 support is removed. CodeIgniter now requires PHP 5.1.6.
Scaffolding, having been deprecated for a number of versions, has been removed.
The CAPTCHA plugin has been converted to a Helper and documented.
The JavaScript calendar plugin was removed.
The system/cache and system/logs directories are now in the application directory.
The deprecated Validation Class has been emoved.
The config folder
the follwing files have been changed:

config.php
database.php
mimes.php, see Other Changes further down
routes.php
user_agents.php

the follwing files have been added:

foreign_chars.php
profiler.php

Libraries
The xss_clean() function has moved to the Security Class.
do_xss_clean() now returns FALSE if the uploaded file fails XSS checks.
The Session Class requires now the use of an encryption key set in the config file.
The default encryption mode in the Encryption library is now CBC.
Input library mehods post() and get() will return all POST and GET items (respectively) if there are no parameters passed in.
Database
The following deprecated functions have been removed: orwhere, orlike, groupby, orhaving, orderby, getwhere.
The deprecated _drop_database() and _create_database() functions have been removed from the db utility drivers.
Helpers
The dohash() function is now deprecated in favour of do_hash() for naming consistency.
The second parameter of directory_map() in the Directory Helper is modified to accept an integer to specify recursion depth.
The File Helper delete_files() returns no FALSE on failure.
The img() function of the HTML helper will now generate an empty string as an alt attribute if one is not provided.
If CSRF is enabled in the application config file, form_open() will automatically insert it as a hidden field.
Other Changes
Some DIR_WRITE_MODE constants use no FILE_WRITE_MODE where files and not directories are being operated on.
"default" is now a reserved name.
Added 'application/x-msdownload' for .exe files and ''application/x-gzip-compressed' for .tgz files to config/mimes.php.
Added audio/mpeg3 as a valid mime type for MP3.

Awesome thank you.


Allow to unresolve nodes by clicking on a resolved node

If you rebase to master, you'll be able to work on this PR using Gitpod.

@spoenemann ok did it


commit to test codecov comparison

:bulb: Description

:hammer_and_wrench: Type of change

[ ] :bug: Bug fix (non-breaking change which fixes an issue)
[ ] :gift: New feature (non-breaking change which adds functionality)
[ ] :boom: Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] :book: This change requires a documentation update

:microscope: How Has This Been Tested?

[ ] Test A
[ ] Test B

Codecov Report

Merging #10 (76bc3ad) into dev (43f9554) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                dev      #10   +/-   ##
=========================================
  Coverage     70.17%   70.17%
  Complexity       20       20
=========================================
  Files            12       12
  Lines            57       57
=========================================
  Hits             40       40
  Misses           17       17

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Color of Chord settings

Hey,
What setting does the color of the chords depend on? In different designs no color is assigned to the chords so that you have no highlighting!

That setting depends on the following shcemes:

To be honest I only tested a couple of existing vscode color themes and it was working fine.
I'll also do a revision on this setting next time I'll edit the project.
If you can/want to contribute with any thing, feel free to submit a PR! :)


Use AppendableCharSequence.charAtUnsafe(int) in HttpObjectDecoder

Motivation:
HttpObjectDecoder pre-checks that it doesn't request characters
outside of the AppendableCharSequence's length. 0 is always allowed
because the minimal length of AppendableCharSequence is 1. We can
legally skip index check by using
AppendableCharSequence.charAtUnsafe(int) in all existing cases in
HttpObjectDecoder.
Modifications:

Use AppendableCharSequence.charAtUnsafe(int) instead of
AppendableCharSequence.charAt(int) in HttpObjectDecoder.

Result:
No unnecessary index checks in HttpObjectDecoder.

@netty-bot test this please

@netty-bot test this please


Get benchmarks runner working correctly again

Waiting on this PR: https://github.com/TranslatorSRI/Benchmarks/pull/36


Do we have any Command line for Gradle as well? like we have it for maven (like using with -Drewrite)

Do we have any Command line for Gradle as well? like we have it for maven (like using with -Drewrite)

firstly just for trying added the dependency in  build.gradle just to orderImports if that works or not . that works fine when i do gradle rewriteRun.

But i want to write custom recipes  which i'm checking the link and trying to replicate is not working. i created a init.gradle file and pasted whatsoever mentioned and created a rewrite.yml file just added one recipe in it which idk is not working i'll share all those to you could you please help me in that :) . And i'm new to Gradle

rewrite.yaml
type: specs.openrewrite.org/v1beta/recipe
name: org.openrewrite.FindSpringUses
displayName: Find all Spring uses
description: This is an example of a custom recipe.
recipeList:

org.openrewrite.java.spring.boot2.UpgradeSpringBoot_2_7

init.gradle
initscript {
repositories {
maven { url "https://plugins.gradle.org/m2" }
}
dependencies {
classpath("org.openrewrite:plugin:latest.release")
}
}
rootProject {
plugins.apply(org.openrewrite.gradle.RewritePlugin)
dependencies {
rewrite("org.openrewrite:rewrite-java")
}
rewrite {
activeRecipe("org.openrewrite.FindSpringUses")
}
afterEvaluate {
if (repositories.isEmpty()) {
repositories {
mavenCentral()
}
}
}
}

and one more thing please how i can run the same in shell script ?
for maven what we're doing is like this we have one migration.sh file
mvn $build_options -f "$project_dir" -U "$rewrite_plugin_artifact:run"
-Drewrite.recipeArtifactCoordinates="$rewrite_spring_artifact"
-Drewrite.activeRecipes=UpgradeSpringBoot2_7AndThirdPartyLibraries
-Drewrite.configLocation="$PWD/rewrite.yml"
please help :)

In your init gradle it seems you're missing a dependency on rewrite-spring, since UpgradeSpringBoot_2_7 is in there:
https://docs.openrewrite.org/recipes/java/spring/boot2/upgradespringboot_2_7
On that documentation page we also provide a sample Gradle init script for you to use or customize

Hope that helps!

Cannot add extension with name 'rewrite', as there is an extension already registered with that name.
still getting this error

Build file 'path\build.gradle' line: 5
An exception occurred applying plugin request [id: 'org.openrewrite.rewrite', version: '6.7.0']

Failed to apply plugin 'org.openrewrite.rewrite'.
Cannot add extension with name 'rewrite', as there is an extension already registered with that name.

If you use init.gradle you should remove any rewrite plugin or configuration from build.gradle; it's either/or, not both. :)

From your thumbs up I'm guessing that's resolved your issues there @Sanskarbajaj ?
Perhaps we can update the docs to

use rewrite-spring as a dependency: that fits in with the example recipe, and includes a lot of the other recipe modules
emphasize not to add the plugin to build.gradle when using init.gradle

cc @mike-solomon

yeah all those issues are gone could you please help me how i can run as a shell script.  i already sent i'm able to run shellscript for maven ones could ypu please help :)
what we're using for maven
mvn $build_options -f "$project_dir" -U "$rewrite_plugin_artifact:run"
-Drewrite.recipeArtifactCoordinates="$rewrite_spring_artifact"
-Drewrite.activeRecipes=UpgradeSpringBoot2_7AndThirdPartyLibraries
-Drewrite.configLocation="$PWD/rewrite.yml"

On the Gradle side the similar invocation for what you have above would be:
# Assuming PWD is the root of the project directory
gradlew rewriteRun -I init.gradle -Drewrite.activeRecipes=org.openrewrite.java.spring.boot2.UpgradeSpringBoot2_7

The default config location is the ${rootProject.projectDir}/rewrite.yml, but it can also be changed by using the initscript. The active recipe can be defined either with the JVM argument above OR it can be defined within the initscript itself as shown in the documentation, but you'd want to pick one or the other to use. Lastly, the recipe id must be it's fully qualified name. So as an example, for a recipe defined in rewrite.yml this must be the name of that recipe as found in that YAML file.

From a docs standpoint, everything that Tim and I have shared is described here:
https://docs.openrewrite.org/running-recipes/running-rewrite-on-a-gradle-project-without-modifying-the-build
https://docs.openrewrite.org/reference/gradle-plugin-configuration
@mike-solomon, unless I missed something it seems like we do miss noting on the Gradle side that you can also control the active recipe and style using JVM arguments. (activeRecipe, activeRecipes, rewrite.activeRecipe, or rewrite.activeRecipes; then the same for activeStyle, activeStyles, rewrite.activeStyle, and rewrite.activeStyles)

I'm going to summarize what I think are the open issues here so it's clear for anyone picking this up (potentially future me). If any of you could confirm/deny/assist with this I'd appreciate it:

[ ] In the running rewrite on a gradle project without modifying the build doc, we have a dependency on rewrite-java. That doesn't fit in with the Spring recipe example. Instead, we should add a dependency on rewrite-spring.
[ ] In the same file, we need to clarify that if you use an init script, you can not have a rewrite section in your build.gradle file.
[ ] In both the above file, and in the gradle plugin configuration file, we should add that as part of running Gradle from the command line that you can add specific JVM args to it. For instance, if you were in the same directory as the init.gradle file, you could run the following command to execute the UpgradeSpringBoot2.7 recipe: gradlew rewriteRun -I init.gradle -Drewrite.activeRecipes=org.openrewrite.java.spring.boot2.UpgradeSpringBoot2_7. These arguments include: (activeRecipe, activeRecipes, rewrite.activeRecipe,  rewrite.activeRecipes, activeStyle, activeStyles, rewrite.activeStyle, and rewrite.activeStyles)

Yes I agree with those points; for that last one it might even make the most sense to move as much configuration from the init.gradle script to command line arguments. That way folks are better able to use a single file and modify just the arguments. A familiar pattern, also as compared to the Maven approach of having mostly commandline arguments.

I tried following along with the existing guide and noticed I ran into IntelliJ warnings when making an init.gradle script:

Have any of you ran into that before? Maybe not a big deal as I can still run the command in the command line -- but it feels a bit bleh to have other people run into this if they're following along.
Things I've tried include:

Deleting the .idea folder and reopening
Deleting the same folder and opening from the build.gradle file as a project
Invalidating the cache and restarting

Was looking at: https://stackoverflow.com/questions/32905270/intellij-idea-and-gradle-cannot-be-applied-to-groovy-lang-closure -- but none of those seemed to fix it for me.

So this is something that I see relatively regularly... Intellij's support for Project and Settings scripts is great, but Init just isn't there. This is probably due to it being a fairly uncommon one to work with for the vast majority.

So this is something that I see relatively regularly... Intellij's support for Project and Settings scripts is great, but Init just isn't there. This is probably due to it being a fairly uncommon one to work with for the vast majority.

Fair enough! I just won't worry about it then - thanks!

They are all the same, just various permutations. I believe what happened was that the Gradle plugin and the Maven plugin has diverged, so the @sambsnyd had added the permutations as a mechanism to keep backwards compatibility, but add alignment.

PR that fixes all of the above issues: https://github.com/openrewrite/rewrite-docs/pull/260


open failed: ENOENT (No such file or directory)

Hi, I try save result uri to file, that upload to server, but i get this error to upload    "open failed: ENOENT (No such file or directory)"

Can you add your code snippet?
Also, when you use the URI to display the image locally, does it work correctly?

even for me issue is the same
var file = File(userProfileUrl.toString())
val reqFile = file.asRequestBody("application/image".toMediaTypeOrNull())
val imageBody = MultipartBody.Part.createFormData("image", file.name,reqFile)

Please try I this new version: 2.3.21 and let me know if it fixed the issue or not

I am facing the same issue.
ImagePicker.with(this)
            .galleryOnly()
            .galleryMimeTypes(
                arrayOf(
                    Constants.MIMETypes.IMAGE_PNG,
                    Constants.MIMETypes.IMAGE_JPEG,
                    Constants.MIMETypes.IMAGE_JPG,
                ),
            )
            .createIntent()
            .also {
                selectImageCropResultLauncher?.launch(it)
            }

Uri: content://media/picker_get_content/0/com.android.providers.media.photopicker/media/1000154898
File path Data -> /sdcard/.transforms/synthetic/picker_get_content/0/com.android.providers.media.photopicker/media/1000154898.jpg
use this method to upload file with retrofit
val multipartBody = MultipartBody.Part.createFormData(
            name = "file",
            filename = imageFile.getName(),
            body = imageFile.asRequestBody(contentType = "image/*".toMediaType()),
        )

Please help to fix this issue.


[TEST] flaky test case test_volume_wait_for_backing_image_condition on ubuntu arm64

What's the test to develop?


Please invite me to the community

Name
Naman
Discord Username (if applicable)
n_m_n#2797
Additional Context
Hi, i got to know about Web 3 community through twitter and hacktoberfest. I like the idea of decentralized technology and open source, web 3 meets both these preferences of mine. Moreover, I have just started open source and It would be a great oppurtunity to network and grow with people here, and contribute to something that has so much to offer.

Welcome to Web3Community ✨


Correspondence to PHP-CS-Fixer v2

What does this implement/fix? Explain your changes.
Correspondence to PHP-CS-Fixer v2.
Does this close any currently open issues?
#1465
Any other comments?

This is breaking change for PHP. 💥
This PR is subject to #1510 merged.

Checklist
Check all those that are applicable and complete.

[x] Merged with latest master branch
[ ] Added examples for testing to examples/ directory
[ ] Travis CI passes (Mac support)
[ ] AppVeyor passes (Windows support)

why is this still not implemented?

I just finished my final exams yesterday. I should now have time to revisit working on Atom-Beautify and review and merge these Pull Requests. Thank you for your patience!

Is it reasonable to expect all PHP-CS-Fixer users to upgrade to 2.x? Since this PR is a breaking change for PHP-CS-Fixer 1.x users, maybe we should add two separate beautifiers? PHP-CS-Fixer (original) and PHP-CS-Fixer2 (new, from this PR). Then make PHP-CS-Fixer (1) the default and users can opt-in to PHP-CS-Fixer2 when they are ready. Thoughts? I personally do not use PHP-CS-Fixer, so this would be for your fellow PHP developers advantage so I will leave the decision up to you 😃 .

backward compatibility for older PHP-CS-Fixers sounds like a good idea

Thanks for contributing!

Published to v0.29.19


aider-chat: fix build

Description of changes
Hey first of all sorry for merging the litellm PR, it seems it broke aider.
When trying to build the latest version I got two missing dependencies error
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[X] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 341760 run on x86_64-darwin 1

2 packages built:

aider-chat
aider-chat.dist

Result of nixpkgs-review pr 341760 run on x86_64-linux 1

2 packages built:

aider-chat
aider-chat.dist


Unexpected text formatting with colon and formatoptions+=m

Steps to reproduce
test.md content
abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc:::

abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: https://google.com

example.vimrc content
set formatoptions+=m
set textwidth=80
set colorcolumn=81

To Reproduce

Edit with vi -u example.vimrc test.md
Use V2j to select everything.
Reformat with gq

The first line doesn't get split despite it being 81 characters wide.
The second line's https://google.com gets broken but it shouldn't.

Expected behaviour
Without set formatoptions+=m, both lines get formatted correctly.
Colon \x3a shouldn't be affected set formatoptions+=m since that should only change the formatting behavior for characters above 255.
Version of Vim
8.2.2434
Environment
Debian
Logs and stack traces
No response

cannot reproduce using: vim --clean --cmd ':set textwidth=80 colorcolumn=81 formatoptions+=m' -c ':norm! ggV2jgq' testfile

With --cmd, it seems like formatoptions+=m gets ignored. If I do set formatoptions? once the file is open, I get formatoptions=tcqln.
However if I replace --cmd with -c, I am able to reproduce the issue:
vim --clean -c ':set textwidth=80 colorcolumn=81 formatoptions+=m' -c ':norm! ggV2jgq' test.md

This is the resulting file output:
abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc:::

abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: abc: https:
//google.com

In the output, the colon is in column 81 for both lines, and https://google.com has been broken apart.


[Docs] SAP UI 5 Docs setup for mobile apps missing script tag

Hey,
just wanted to inform you that on the official developer Guide there is the bootstrap script tag on this page missing (step 5): https://openui5.hana.ondemand.com/#docs/guide/058c7fd461d94223a11ef4c89b2259ea.html
I know this is not the correct repo. But this is the easiest way to get in contact with SAP for now.

Hi,
perfectly fine to report this here, thanks!
I just checked the state in the upcoming 1.30 and... this short tutorial has been replaced by a more comprehensive one. So maybe this problem automatically resolves itself, but maybe we can still fix it for the remaining 1.28 time. And maybe it would be good to also keep this brief tutorial in parallel...
I'll discuss this first and then let you know how this continues.
Thanks
Andreas
P.S.:
the missing part should read:
<script  src= "http://<server>:<port>/sapui5/resources/sap-ui-core.js"
       id= "sap-ui-bootstrap"
       data-sap-ui-libs= "sap.m"
       data-sap-ui-theme= "sap_bluecrystal">
   </script>

Replace <server> and <port> with your local UI5 installation.
Results
At this point, SAPUI5 including the mobile controls is loaded and ready to use.

Ok, I found this also in the 1.30 version.
But the issue does not appear in all generated flavors of this page in 1.28, so I opened an internal ticket to analyze and fix this, in whatever version:<PHONE_NUMBER>

Hi Felix,
the reason for the issue (wrongly marked docu content - the code sample is actually in the HTML, but with display:none!) has been identified and fixed for the next 1.28 patch (should be 1.28.10 because 1.28.9 has already been built yesterday and is about to be published).
Thanks again
Andreas


fix: if inputs are provided do not use data

Discarded as finally we are using the official AKS module: https://github.com/Azure/terraform-azurerm-aks


timezone are not correctly used in the pandas client

the index timezone is always used even if a timezone column is provided. Moreover, if the index is not tz-aware, the local timezone is used by default.

Fixed in #97


Resend emails work locally but not on Cloudflare

I followed the guide to set up email with Resend, added the API key to the .env file, and configured the DNS records in Cloudflare. Everything works fine locally, and emails are sent successfully when I test them. However, it doesn't work on Cloudflare.
I've checked the dev console, and there are no errors, and the DNS records in Cloudflare aren't proxied, as far as I know. Any ideas on what might be causing this issue?

99% of the time this is because you don't await the call to an async API like the mailer APIs. Cloudflare shuts down as soon as main thread exits, so you need to wait for it to complete.
Can you check that and confirm/close if that's the case.

@scosman Thanks for the answer. The function is almost identical to sendAdminEmail() and it uses await

Are you calling it with await wherever you call it? What you are describing is almost definitely a missing await somewhere. Classic issue with serverless hosting like pages.
The stock version works; I can see it's sending welcome emails from cloud flare.
I'm going to close since it looks like I think this issue lives in your adaption, not the template. But let us know if you find any details/issues.

Yes, I did use await when calling the function. I even tested it with your sendAdminEmail() function, but got the same result. The email sends successfully locally, but not when running on Cloudflare.
I'd appreciate any further assistance you could provide.

Can't help much from here.
See the examples in code which do work:
// Send welcome email
await sendUserEmail({
If you are sure you are awaiting properly, check your environment var setup matches in both envs.

@scosman Does the primary CNAME in Cloudflare need to be unproxied for sending emails? If so, this would disable many security features. I know that MX records shouldn't be proxied, but I'd like to understand if the CNAME proxy setting impacts email functionality.


A question about  N_cells_per_location

Dear developers,
Thank you for creating the amazing tool, cell2location. It has been incredibly helpful in my research. According to the documentation, the N_cells_per_location parameter can be estimated from paired histology images, and the value should be adjusted based on the actual number of cells observed in the tissue. However, after reading the explanation, I am still unsure how to accurately estimate this parameter from histology images.
Could you please provide more detailed guidance or steps to help me determine the appropriate N_cells_per_location value based on the images?
Thank you once again for your help and support. I look forward to your response!

Hi @beilouer
The easiest way is to use 10x Loupe browser and count cells under 20-30 random spots and compute the average. You can also segment histology images. If you cannot see cells in the histology images you can provide estimates of N cells in round areas with diameter 50um from similar tissues.


Show more specific error message when importing unsupported types of ArcGIS layers

As noticed by @julianamori in #9013, unsupported layers types (e.g. Raster) are simply ignored when importing a multi-layer resource, but cause an import error when importing a single layer.
In this case we log this error message internally:

2016-07-25 16:25:44 UTC: Additional error info: Invalid layer type: 'Raster Layer'

But the error the user gets is a generic one, which might be misleading:

Error retrieving data from datasource (1011)
There was an error retrieving data from the datasource. Check that the file/data is still present.

We can map the error to mor appropriate error code or add a new one for this case.
The error shown when a multilayer resource consisting only of unsupported layers should also be reviewed.

@jgoizueta is this still happening?

Yes, still happening. Can be reproduced with this layer:
https://raster.nationalmap.gov/arcgis/rest/services/LandCover/USGS_EROS_LandCover_NLCD/MapServer/21
Internally we log this:
2016-11-22 14:33:47 UTC: Additional error info: Invalid layer type: 'Raster Layer'

But the users sees only this:

@saleiva I'd move this straight to the next sprint, unless this takes precedence on what was planned (I don't think so, the rest affects way more builder stability)


Store version number in a separate file (fixes #1183)

The version number displayed in the console currently does not match the actual version number since the v0.9.0 release. To fix that, I simply stored the version number in a separate file (as briefly suggested in #1183) so that  it would be easier to update at each release.

:+1:

@lorenzo Makes me wonder: Do we even need to maintain a version number in composer lock file days?...

It seems directly reading the composer.lock file would remove the need to have a separate “version file” whether YAML/PHP/text or whatever, but would also tighten the coupling between Phinx and composer by adding the composer.lock file specification/format as a dependency of Phinx and this feature. I’m not sure if that’s a “net” improvement – trading one dependency for another.
Also, wouldn’t reading composer.lock require logic to detect a global versus project-level installation of Phinx so that the correct .lock file is read?

I meant not displaying at all. It is a dev tool. devs can look that up in easier qaxs otherwise.

Phinx is self-described as “ridiculously easy...migrations for everyone.” I think providing the version from the CLI is more consistent with that description than not.
I agree most users of Phinx are technically-oriented. However, most won’t – and shouldn’t need – crack open the source. That’s probably true for most packages.  It’s common for commands to provide a mechanism for reporting their version to users through, at least, an option. Often when there’s an issue of any kind, the first step is to check the version of the command, and it’s very useful, if not considerate of users, if identifying the version is NOT an exercise source code review.

Maybe a quick/easy/adequate solution is to provide a Phinx command/option like this:
phinx -i
phinx --info

Or adjust the existing:
phinx -V
phinx --version

So these are just a wrapper for:
composer show robmorgan/phinx

Or maybe even take it a step further and parse the version from this command’s output and display it with other minimal information.

Why not just use the version config in composer.json?  Then we can have a release script which updates composer.json when tagging a new release like we have for CakePHP (which updates VERSION.txt).

Closing in favor of #1275


Professional store assets

They should at least not look like sh***... like I made them
https://overwolf.github.io/start/submit-your-app#prepare-your-store-assets

A better tile (258X198 at 72PPI JPG)

A better logo (55X55)

Screenshot is okay (1200x750)

HomeAssistant logo repository https://github.com/home-assistant/assets/tree/master/logo

New assets have been added thanks to an awesomely talented designer.
Logo

Tile


Remove reliance on IRCompiler in unrelated objects

In the previous PR we separated the loaded function pointers (strictly: "just the library") from a created IRCompiler instance, because those instances need to be mutated and are not thread-safe.  We also borrowed the IRCompiler inside IRObject (#14) to pass into some other fn new() constructors, making sure that they would use the same compiler instead of taking that as a separate argument.
This is super inconvenient, because IRCompiler cannot be mutated (mutably borrowed) while an IRObject is live.  And for no good reason: when splitting IRCompiler we forgot to make sure that most constructor functions only take the function pointers as they don't need the full IRCompiler in the first place.  By replacing it with the function pointers in every constructor we no longer need to borrow IRCompiler inside IRObject.
Also move some public constructor functions directly on the MetalIrConverter struct instead of having to pass it in as borrowed argument.  For constructor functions that don't need to be used publicly (i.e. only via special getter functions that create an object before filling it with data, specifically IRShaderReflection and IRMetalLibBinary), the constructor is made private and takes the function pointer structure directly.

Tested, and self-reviewed, this is now good to go!


Preserve 'signed'-ness of a verilog wire through RTLIL

As per suggestion made in https://github.com/YosysHQ/yosys/pull/1987, now:
RTLIL::Wire now holds an is_signed field.
signed is exported via JSON backend, and read in via JSON frontend
Also exported via dump_rtlil command, and read back via ilang_parser
I am sure I missed a few other places, and would love to add a test somewhere. Please feel free to advise!

It would be nice to have this used in the Verilog backend, but without extensive testing it is likely that this change would backfire.

@whitequark Have a look. I tested it with some verilog files at hand. However, I tried to look for something like dump_module or *Backend::execute in /tests, but failed to see anything that I can append a test to.

@jersey99 I'm not comfortable signing off on this without some VLogHammer testing. (@cliffordwolf could greenlight this, but I suspect her view would be the same as mine here...)

This implementation LGTM. I do share @whitequark's concerns regarding the verilog backend bit. A simple way of "playing it save", if we really want this in the verilog back-end, would be to always use signed variables with $unsigned() in the generated verilog code.
However, as previously pointed out in https://github.com/YosysHQ/yosys/pull/1987#issuecomment-618293763 I'd like to see a use-case for this.

@cliffordwolf My use case for this is basically reflected in my comment in #1987. We have been using attributes heavily for bus independent code-generation and register map automation. I find that using Yosys parser and rtlil will provide more benefits to us into the future. However, if we want to get hold of the output of write_json or dump_rtlil, the signedness is lost and this PR will help preserve it.
@whitequark I spent some time with VlogHammer after your suggestion. syn_yosys and check_yosys run fine. However, there is no exercise of verilog_backend inside VlogHammer. From what I can see read_verilog, write_ilang and read_ilang are the ones used. The report.html generated between this PR and master is practically identical. Any more guidance is appreciated!

@cliffordwolf I'm in favor of having this functionality. It seems entirely in line with having upto and start_offset: all three are not directly useful to Yosys but could be used by downstream tooling; by PNR and codegen tools in case of upto and start_offset, and codegen tools alone in case of signed. If we have upto and start_offset, I think there is a good case to have signed.
It is true that Yosys seems to be biased towards synthesis tools, but I see no reason it should remain biased. nMigen is pushing Yosys towards HDL conversion and simulation, I think it's good that @jersey99 is pushing Yosys to be more useful for code generators.

@cliffordwolf  @whitequark To give you both a little more context (maybe a bit too much :) ), although this isn't quite at the level of (n)-migen and/or Litex's AutoCSR (where python-DSL within python makes life easier), this is at the moment more verilog-centric and could use some clean-up.
Code here is taken from https://github.com/BerkeleyLab/Bedrock/tree/newad_yosys_take1/rtsim
Take a verilog module (prng) defined in file (prng.v):

`timescale 1ns / 1ns

`include "prng_auto.vh"
// Almost useless shell around two tt880v modules
module prng(
	input clk,
	output [31:0] rnda,
	output [31:0] rndb,
	(* external *)
	input [0:0] random_run,  // external
	(* external, signal_type="plus-we" *)
	input [31:0] iva,  // external plus-we
	(* external, signal_type="plus-we" *)
	input [31:0] ivb,  // external plus-we
	// Everything else above is robust and pretty good.
	// Minor loss in efficiency from having two 32-bit registers when zero would do.
	// The following get hacked in.
	input iva_we,  // special for above, keep a trailing _we
	input ivb_we  // special for above, keep a trailing _we

);

reg iva_strobe=0; always @(posedge clk) iva_strobe <= iva_we; wire tt800_ena = random_run | iva_strobe;
reg ivb_strobe=0; always @(posedge clk) ivb_strobe <= ivb_we; wire tt800_enb = random_run | ivb_strobe;

tt800 tt800a(.clk(clk), .en(tt800_ena), .init(iva_strobe), .initv(iva), .y(rnda));
tt800 tt800b(.clk(clk), .en(tt800_enb), .init(ivb_strobe), .initv(ivb), .y(rndb));

endmodule

Being instantiated in another file (cav_mech.v):
`timescale 1ns / 1ns
`define AUTOMATIC_self
`define AUTOMATIC_decode
`define AUTOMATIC_noise_couple
`define AUTOMATIC_resonator
`define AUTOMATIC_prng
`include "cav_mech_auto.vh"

module cav_mech(
  input clk,
  input start_eig,
  input start_outer,
  output signed [17:0] mech_x,
  output signed [17:0] noise_eig_drive,
  input signed [17:0] eig_drive,
  // Output status
  output res_clip,
  `AUTOMATIC_self
);

`AUTOMATIC_decode

parameter n_mech_modes = 7;
parameter n_cycles = n_mech_modes * 2;

// Couple randomness to mechanical drive
wire signed [17:0] environment;  // filled in later
(*lb_automatic*)
outer_prod noise_couple  // auto
        (.clk(clk), .start(start_outer), .x(environment), .result(noise_eig_drive),
         `AUTOMATIC_noise_couple
);

// Instantiate the mechanical resonance computer
(*lb_automatic*)
resonator resonator // auto
  (.clk(clk), .start(start_eig),
         .drive(eig_drive),
         .position(mech_x), .clip(res_clip),
         `AUTOMATIC_resonator
);
// Pseudorandom number subsystem
wire [31:0] rnda, rndb;

(*lb_automatic*)
prng prng  // auto
  (.clk(clk), .rnda(rnda), .rndb(rndb),
   `AUTOMATIC_prng);

// Create a white noise term for environment
// This is a strangely-scaled CIC filter
// Each iteration adds a variance of 12, consuming 6 bits of PRNG
// Result has variance of n_cycles*12, mean of 0, possible peak n_cycles*7
// e.g. if n_cycles=14, std.dev.=12.96, peak=98, peak/rms=7.56
reg signed [11:0] noise_accum=0, noise_1=0, noise_out=0;
always @(posedge clk) begin
         noise_accum <= noise_accum + rndb[2:0] - rndb[5:3];
         if (start_eig) begin
                  noise_1 <= noise_accum;
                  noise_out <= noise_1-noise_accum;
         end
end
assign environment = noise_out;

endmodule // cav_mech

User settable registers are marked (* external *)
Those signals marked (* external *) are automatically decoded (bus independent)
externally (higher up the hierarchy) and routed through the module instantiation sites.
Handling generation of write strobes, and arrays etc.
Instantiation sites that need routing these signals through are marked with (* lb_automatic *)
So far, we haven't ventured into modifying verilog files directly, and all the code generation
is done within *_auto.vh files. This generated code is for macros that are sprinkled
throughout the verilog. We are thinking of directly modifying verilog and avoid using macros and
header files with RTLIL/JSON.
With this, writing error-prone boiler plate code can be avoided, by automatically generating:
a) Register decoder at `AUTOMATIC_decode in cav_mech.v
b) Instantiation site wiring with macros like `AUTOMATIC_prng and `AUTOMATIC_resonator
c) As the decoder is generated, addresses for these registers is automated as well.
Things not seen here:
a) automated clock domain crossing for some of these registers
b) generate loop handling
c) comments exported as notes inside the register map etc.
d) Register readback

There are several things to improve/iron out, to list a few:

The ugly `define 's at the top of the file (there to ensure the code is valid verilog
before code generation step). These are in place as we decided to not modify the source
verilog directly.
Code with sprayed attributes and missing signals may not be readable for users unfamiliar
with the automation. But readable code can be re-emitted.
Compatibility with VHDL.

Yosys makes this easier to deal with than iverilog or verilog-perl. Also promising to integrate
with ghdl with ghdl-yosys-plugin.
If you are interested in talking more about this, perhaps we can take the discussion off-line. Also, notifying @ldoolitt in case he wants to add his thoughts here.

Just to be clear--I was always in favor of the functionality, I was only concerned about testing the Verilog backend.

I, too, am concerned about testing of the Verilog back-end changes. Could we maybe factor the Verilog back-end changes into a separate pull request and merge the rest now?

Could we maybe factor the Verilog back-end changes into a separate pull request and merge the rest now?
@clairexen Let me do that. Also do note that I ran VlogHammer against changes in the verilog_backend. We can pick this up at the other PR.

@clairexen @whitequark verilog_backend changes were in a single commit. So I undid that commit and force pushed. Let me know if there is anything else you need from me. Once we clear this I will open the other PR.


Update protobuf to v22.0

Closing, because there was no response, PR is incomplete and there's another PR with upgrade.