logging: add way to configure logging level via cilium-agent option

The new option can be provided as follows to cilium-agent:
--log-opt level=error

If no option is set, or the provided option does not match a level for logrus,
use the default logging level (info level). If debug mode is not enabled,
endpoint loggers will use the logging level provided via the command-line
option, if there was one provided. If not, the endpoitn logger will default
to info level as well.
I added this because I thought I would be able to inject the logging level for
unit testing via a -X parameter when running go test, but given how we
set up our loggers (in their own package), that didn't work very well. But,
I didn't want my work to go to waste, so I posted this PR.
Signed-off by: Ian Vernon<EMAIL_ADDRESS>

This change is

test-me-please

test-me-please

test-me-please

test-me-please

test-me-please

Coverage increased (+0.01%) to 44.238% when pulling 7b11f625613dd8485cc6b7f048f234f650a6fdd6 on pr/ianvernon/log-level-argument into d9c3256a4af69a372313d14accc203f4a355aca6 on master.

Where is the option flag being set in the cilium-agent?

@aanm it is set as a --log-opt, which is a mapping of key-value pairings for configuration related to logging.

test-me-please

test-me-please

@joestringer will add some unit tests.
Re: why options are cached in a var - outside packages consume the configuration (specifically the endpoint log). The value has to be inspected as well when we are changing from user-specified log level --> debug --> user-specified log level as well.

test-me-please

@joestringer added the unit tests - created some types wrapping around the logging options to make unit testing easier.

test-me-please

test-me-please


Binding to a Pin's Tag property

SUMMARY
Not exactly a feature request, but:
The Tag property of Pin is typed as an object. So, when using the newly added option to bind the pins directly in XAML, the binding for Tag is the actual bind to it.
As in, the binding isn't the int object I'm trying to bind as an id, but the Xamarin.Forms.Binding object referring to it.

DETAILS

Changing Tag to int would fix this for me (but, as Pin is sealed, I can't implement it). However, as far as I understand, Tag's supposed to be any kind of identification for the pin (any type), so this may not be the most ideal solution.
PLATFORMS

[x] Android
[x] iOS
[ ] UWP

Hi @gabrielbunselmeyer did you ever find a solution here? thanks!

Hey, sorry for the delay @aherrick. I didn't, no. Just had to do whatever it was I wanted without using a binding, if I remember correctly. Has been a while since I left that project.
This pull request would help, though: #701.

This never was resolved was it? Would be helpful to have an additional space to store values to the pin without showing a label or address...


Problems with threads

Hi, I wanted to extend your project so I created a Dockerfile to run tests in container. I did some modifications and when I ran tests they failed from time to time. So I reverted my changes and ran it again against current HEAD.
See results here http://pastebin.centos.org/34206/
Simply put, tests for threading randomly fail - in this case in 6th run. I am not very familiar with multi threaded applications, but will be happy to assist with debugging.

Yikes! I have some questions:

Are you saying cached_property fails on 1 out of every 6 runs on Fedora?
How many attempts have you made?
My docker-fu is weak. How exactly is cached_property being installed. Is it these lines: https://github.com/vpavlin/cached-property/blob/disk-stored-caching/Dockerfile#L6-L8

Tomorrow I'll try and replicate this on Fedora and Ubuntu.

No, in this case it was 1 of 6, in other run it was one of 10..it seems to be completely random. I'll let it run for longer time today and see the ratio.
Not many 5, maybe
Yes, ADD line copies repo content to the image, WORKDIR changes working directory so that I don't have to use absolute path for requirements.txt

@ncoghlan_dev, do you have any idea why this is happening? I can't reproduce it on Mac OS X or on the CI server.

I was able to reproduce it without Docker on Fedora 22 - same symptoms as @vpavlin, with only 4 entries in the cache rather than the expected 5.
Dependencies were installed in the virtualenv via "pip -r requirements.txt".
Looking at the test case, I'm not seeing anything wrong except that I'm not sure the block comment is right about there being a race condition: run_threads() joins all the created threads before returning, so "cached_total" should always have been updated correctly before the assertions are checked.

Running py.test --pdb rather than plain py.test so it drops into the debugger when the test fails, I can see that check.cached_total is 10, while check.add_cached is only 9.

Oh, now I understand that block comment in test_threads - I didn't realise what cached_property actually did :)
In that case, I suppose it's possible that for some reason one of the threads is dragging its feet and not starting running until after one of the other threads has already shadowed the property with a normal value.

I'm heading out, but I've kicked off a test run using this modified implementation of run_threads in the tests:
    def run_threads(self, num_threads):
        launch = Event()
        def _replace_in_thread(started):
            return lambda: (started.set(), launch.wait(), self.add_cached)
        threads = []
        start_events = []
        for _ in range(num_threads):
            started = Event()
            thread = Thread(target=_replace_in_thread(started))
            thread.start()
            threads.append(thread)
            start_events.append(started)
        for started in start_events:
            started.wait()
        launch.set()
        for thread in threads:
            thread.join()

That means the sleep in the cached property test is only covering the threads reacting to the event being signalled, rather than actually starting them. (There's still be a problem to report to the Fedora python and/or kernel teams though, so it would be weird for 5 threads to be taking an entire second to all start)

Even with that modified implementation, that still failed within 28 runs of the test suite. I'd suggest pinging the Fedora python-devel list at this point.

@ncoghlan Do you want me to do it, or will you? (Anyway thanks for looking at this)

@vpavlin I was suggesting you do that. I'm now assuming there's something odd in the Fedora Python patches, and I'm the wrong person to ask about that.

@rkuska asked me to file a bug against python in Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1261445

We just moved cached-property's Travis builds to use the container-based infrastructure. There, we ran into what appears to be the same issue: https://travis-ci.org/pydanny/cached-property/jobs/93074100

I got a similar bug report for the Gentoo Ebuild: https://bugs.gentoo.org/638250

Is this still an issue?

With 1.5.1:
==================== FAILURES ======================
____ TestCachedPropertyWithTTL.test_threads_ttl_expiry _________

self = <tests.test_cached_property.TestCachedPropertyWithTTL testMethod=test_threads_ttl_expiry>

    def test_threads_ttl_expiry(self):
        Check = CheckFactory(self.cached_property_factory(ttl=100000), threadsafe=True)
        check = Check()
        num_threads = 5

        # Same as in test_threads
        check.run_threads(num_threads)
        self.assert_cached(check, num_threads)
        self.assert_cached(check, num_threads)

        # The cache expires in the future
        with freeze_time("9999-01-01"):
            check.run_threads(num_threads)
>           self.assert_cached(check, 2 * num_threads)

tests/test_cached_property.py:207:
_ _ _ _ _ _ _ _
tests/test_cached_property.py:69: in assert_cached
    self.assertEqual(check.add_cached, expected)
E   AssertionError: 6 != 10
======= 1 failed, 29 passed in 8.89 seconds ==========

All current tests are passing

Perhaps Travis CPU isn't loaded enough for the race to trigger?


Is there a way to reset the round in Descent?

A command to reset the round (mark all heroes as not activated) would be very handy, especially when designing multi encounter quests.
In this way the heroes can start encounter 2 as a new round instead of being in the activation state they were when ending encounter 1.

My current thinking is that this is perhaps best as two quests, with an automatic progression to the next quest.  With the campaign system you will be able to carry over important vars and we could mark encounter 2 so that it doesn't show up in quest selection.  Thoughts?

This sounds like a good solution.


mk.math.maxD4() doesn't work

val tensor = mk.ndarray(output, 272, 480, 7, 4)
val result = mk.math.maxD4(tensor, axis = 1)
If I execute the above action, Logcat as below will appear and the app will be forced to shut down. I don't know the cause.

I've done all of these things, including allowing Deprecated NDK and  setting Hardware Acceleration to false.

Are you using the default module?
The default module uses jvm and native implementations. The native implementation does not currently support Android. Although the max4D() function is implemented in jvm, the library still tries to load the native library ibmultik_jni.so, as a result of which this error appears.
For now, please use multik-jvm for android.


Feature/get baladyiats for wilaya

This pull request holds changing of :

Adding get baladyia by wilaya code (matricule).
Test of getBaladyiaByWilaya.
lbledSdk = require('../src/index'); to => lbledSdk = require('../src');

Related to #36

Codecov Report

Merging #41 into develop will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##           develop       #41   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           14        15    +1
  Lines          101       108    +7
  Branches        16        16
=========================================
+ Hits           101       108    +7

Impacted Files
Coverage Δ

src/api/getBaladyiatsForWilaya.js
100.00% <100.00%> (ø)

src/index.js
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 029d9f6...b2b0ee8. Read the comment docs.

@all-contributors add @Redaloukil for code


feat: improve seo by adding important meta elements 🚀✨

Closes #185
Description

PR aims to improve overall SEO by adding every essential META elements.

Hi there @c4coderandcreator 👋🏻, Your review is appreciated.

Hey @0xabdulkhalid  Thanks for contributing !! If you liked contributing here , Leave a star to the Repo for us !!


README.md: Added Codecov badge for dev branch

Changes to README.md

Added Codecov badge for dev branch

This Pull Request (PR) fixes the following issues:
n/a

[ ] Change details added to Unreleased section of CHANGELOG.md?
[ ] Added/updated documentation, comment-based help and descriptions in .schema.mof files where appropriate?
[ ] Examples appropriately updated?
[ ] New/changed code adheres to Style Guidelines?
[ ] Unit and (optional) Integration tests created/updated where possible?

This change is

Codecov Report

Merging #422 into dev will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff         @@
##            dev   #422   +/-   ##
===================================
  Coverage    69%    69%
===================================
  Files        31     31
  Lines      3178   3178
===================================
  Hits       2206   2206
  Misses      972    972

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f54fbdf...e5cddab. Read the comment docs.

If someone could review this PR and come with suggestions, or approve it if it looks good, I would appreciate it. 😄
Here is the markdown in more readable form: https://github.com/johlju/xSQLServer/blob/af3056f431c5d87ef1bd4f5dcf86b7a520662c11/README.md

Reviewed 1 of 1 files at r3.
Review status: all files reviewed at latest revision, 2 unresolved discussions.

README.md, line 15 at r3 (raw file):

[![codecov](https://codecov.io/gh/PowerShell/xSQLServer/branch/master/graph/badge.svg)](https://codecov.io/gh/PowerShell/xSQLServer>)

This is the branch containing the latest release - no contributions should be made to this branch.

'...should be made directly to this branch.'

README.md, line 22 at r3 (raw file):

[![codecov](https://codecov.io/gh/PowerShell/xSQLServer/branch/dev/graph/badge.svg)](https://codecov.io/gh/PowerShell/xSQLServer)

This is the development branch to which pull request (changes) should be proposed by contributors. This development branch will periodically be merged to the master branch, and be released to [PowerShell Gallery](https://www.powershellgallery.com/).

Use 'contributions' here as well

Comments from Reviewable

Merging this as soon as the tests passes.

Reviewed 1 of 1 files at r5.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Feature request: API Swagger documentation generator

Use case
I'd like to generate swagger JSON files describing my service out of my lambda handlers functions.
I've seen utilities such as FasTAPI do that but I want a more simplified and native lambda experience.
Solution/User Experience
A simple decorator is added to the function handler. In runtime it will not do anything and will not impact performance.
However, the utility will supply a CLI command that will generate the JSON output file so it can be committed into the repository.
Similar to FastAPI it can supports the following arguments:

general description
path parameters description
Pydantic body schema support
response codes
query parameters support

We can create a wrapper around FastAPI or write it from scratch.
Alternative solutions
No response
Acknowledgment

[X] This feature request meets Lambda Powertools Tenets
[X] Should this be considered in other Lambda Powertools languages? i.e. Java, TypeScript

Hi @ran-isenberg! Thank you for opening this issue. I'm moving this topic to "On Hold" because I know @heitorlessa wants to get some ideas on this and he is out this week.
Thank you.

@ran-isenberg this issue already describes it #1236

hey Ran, I'm closing this in favour of generating it at runtime with a new route (Swagger UI + OpenAPI generated for you): https://github.com/awslabs/aws-lambda-powertools-python/issues/2421
It'd be great to get your feedback from a Pydantic customer perspective - I suspect we might need a new Event Handler implementation altogether to make Pydantic a first class citizen like FastAPI.
https://github.com/awslabs/aws-lambda-powertools-python/issues/2421


ParJoin is not implemented for AntiStorage

If you try to convert example in the book to use par_join:
(
    &*entities,
    &mut pos_storage,
    &vel_storage,
    !&freezed_storage,
).par_join().for_each(|(ent, pos, vel, ())| {
    println!("Processing entity: {:?}", ent);
    *pos += *vel;
});

It fails to compile because ParJoin isn't implemented for AntiStorage.
This seems to be oversight as DistinctStorage is implemented for it.

Nevermind, it's the Deref to MaskedStorage that will make it miss.


[Question]

Would like to know the dependent variable (y) used in the ML model. Is it the Cycle_Index  for all the cycler data types ?

In general, no. There are several different targets for an ML model, depending on the application. In our 2019 publication, the target was the cycle_index to 80% of initial capacity, but this was because the SOC window was the same for all cells.


Update dependency from MSBuild12 to MSBuild14

Hi,
I made a PR that changes lots of thing:

change dependency from MSBuild12 to MSBuild14
Add appveyor support & publish package as BuildTasks.TextTemplating.MSBuild14
Drop nuget project (appveyor does that)
add a bit more information to the readme (dependency, license, requirements)

If you wish to merge that, then you can change back the ID to BuildTasks.TextTemplating if you consider acceptable to loose compatibility with MSBuild12 or I can delete my package and let you use the ID BuildTasks.TextTemplating.MSBuild14
Regards,
Thomas

This change is

Note that I think it might be possible to avoid linking to MSBuild14 (and in the future MSBuild 15) by adding some binding redirects to the project:
Here is a part I copied from MSBuild15:

See in C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\MSBuild.exe.config if you have VS 2017 RC installed

Hi, with VS 2017 & MSBuild 15, you don't need this nuget anymore:
https://docs.microsoft.com/en-us/visualstudio/modeling/code-generation-and-t4-text-templates


"UIViewRepresentables must be value types: MarkdownUI" Exception in iOS 16 beta

To make it clear, in the stable iOS 15.5 release, this issue does not occur. App built using Xcode 14 beta.
At the @main line in My_Awesome_AppApp.swift, an exception is thrown, saying UIViewRepresentables must be value types: MarkdownUI. I don't even know what that's supposed to mean, can anyone help?

Here's a demo project to demonstrate the problem:
TestAppDemo.zip

Hi @Terabyte1385,
Thanks for reporting this. I will look into it before the official iOS 16 is released.

Hi @Terabyte1385,
The demo project you have provided is not using this library (MarkdownUI). Instead, it is using https://github.com/keitaoouchi/MarkdownView.
In any case, I ran the sample project from this library, MarkdownUIDemo on Xcode 14.0 Beta 3 and I couldn't reproduce the issue you mention.

Oh my days, I'm truly sorry, I haven't realised that. Thanks for investigating anyway!


refactor: use logic voltages for mount detect

In EVT and previous, we had a scheme (or set of frequently-changing schemes) to have a passives network establish an appropriate voltage for a given (pipette type, attach point) pair that could be read by both the attached pipette and the head board. This kept having problems with rise time, consistency between assembled units, the quality of the onboard ADCs, and so on.
Instead, what we're going to do  in DVT is tie the lines to logic levels at the head board. This removes the ability for the head to sense a pipette's presence, but it lets the pipettes use GPIO inputs to distinguish between the left and right mount much more reliably.
This lets us remove all the ADC handling, tool detection, voltage conversions, etc.
We also remove the messages to do with head-side tool detection, since they'll no longer be used and we'll rely on pipette announce messages.
Test cases:

[x] Pipette plugged into left at system boot
[x] Pipette plugged into right at system boot
[x] Pipette plugged into left after system boot
[x] Pipette plugged into right after system boot
[x] Pipette sent to bootloader on left
[x] Pipette sent to bootloader on right

Closes RET-1297

Codecov Report

Merging #544 (e008445) into main (53075ad) will decrease coverage by 0.47%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main     #544      +/-   ##
==========================================
- Coverage   83.94%   83.46%   -0.48%
==========================================
  Files          88       78      -10
  Lines        2790     2710      -80
==========================================
- Hits         2342     2262      -80
  Misses        448      448

Impacted Files
Coverage Δ

bootloader/core/pipette_node_id.c
0.00% <0.00%> (ø)

include/pipettes/core/mount_sense.hpp
100.00% <100.00%> (ø)

...rol/core/stepper_motor/motor_interrupt_handler.hpp
96.51% <0.00%> (-0.93%)
:arrow_down:

include/motor-control/core/motor_messages.hpp
100.00% <0.00%> (ø)

.../brushed_motor/brushed_motor_interrupt_handler.hpp
89.51% <0.00%> (ø)

include/head/core/adc.hpp

include/common/core/adc_channel.hpp

include/common/tests/mock_adc_channel.hpp

Yeah this PR is a little time constrained but the followup that re-adds how tool detection should work should maybe also just delete the proto code.

In EVT and previous, we had a scheme (or set of frequently-changing schemes) to have a passives network establish an appropriate voltage for a given (pipette type, attach point) pair that could be read by both the attached pipette and the head board. This kept having problems with rise time, consistency between assembled units, the quality of the onboard ADCs, and so on.
Instead, we'll eventually have a system where the head sets its mount id pins to input with internal pulls. This establishes a voltage the pipettes can see; they'll read it, and then they'll drive the pin to the opposite voltage, which the head can detect to indicate pipette presence.
However, this scheme won't work with EVT pipettes or head boards because of the presence of external pulls which are stronger than the internal pulls. That means the head board has to drive its detection pins directly, and that means the pipettes have to keep their mount ID pins on input so we're not driving current between two GPIOs. To get the above scheme to work, either both sides need to be DVTs or they need to be modified to remove the external pulls.
For now, and what's in this PR, what we're going to do is drive the pins from the head and keep the pipette pins inputs. This removes the ability for the head to detect the presence of a pipette, so we're going to keep this change in a branch sent to the factory to do QC of DVT pipettes and not distribute it further. We'll add, in a separate branch, the new behavior that allows detection again; and eventually, when we get more DVT hardware we'll either move away from the remaining EVT hardware or modify it to work with the new scheme.
What we have now is:
The head drives its left/z mount id high
The head drives its right/a mount id low
The pipettes sense this and assign their ids from it
Note: The above is switched from the first push of this PR because driving the voltage high on the left/z mount of EVT systems results in an unusable intermediate voltage because of the values of the resistor divider for the old analog system.
Test cases:
Pipette plugged into left at system boot
Pipette plugged into right at system boot
Pipette plugged into left after system boot
Pipette plugged into right after system boot
Pipette sent to bootloader on left
Pipette sent to bootloader on right

Closes RET-1297


Merge prTesting_201902120852 branch to master branch

OPS Build status updates of commit 920eaa5:
:clock10: Preparing: average preparing time is 48 sec(s)

OPS Build status updates of commit 920eaa5:
:clock10: Incremental building: average incremental building time is 15 sec(s)

OPS Build status updates of commit 920eaa5:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details
Dependent Files

split/updateTestRef.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


chore(main): release 1.1.1

:robot: I have created a release beep boop
1.1.1 (2023-03-12)
Bug Fixes

new version detection message (c9dd3c4)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/joke/zim-github-cli/releases/tag/v1.1.1 :sunflower:


Custom Markers disappear in Snapshots

PLEASE DON'T DELETE THIS TEMPLATE UNTIL YOU HAVE READ THE FIRST SECTION.
What happens if you skip this step?
Someone will read your bug report, and maybe will be able to help you,
but it’s unlikely that it will get much attention from the team. Eventually,
the issue will likely get closed in favor of issues that have reproducible demos.
Please remember that:
* Issues without reproducible demos have a very low priority.
* The person fixing the bug would have to do that anyway. Please be respectful of their time.
* You might figure out the issues yourself as you work on extracting it.

PLEASE NOTE THAT ISSUES WITHOUT THE TEMPLATE IN PLACE WILL BE CLOSED RIGHT FROM THE START.
Thanks for helping us help you!
-->
Is this a bug report?
Yes.
Have you read the Installation Instructions?
Yes.
Environment
"react": "16.4.1",
"react-native": "0.55.0",
"react-native-maps": "^0.22.1"

 iOS - 11.4 Simulator/iPhone 8 Plus Using Apple Map.

Steps to Reproduce
(Write your steps here:)
1.  {this.state.markers.map((marker,key) => (
          <Marker
            key={key}
            coordinate={marker.latlng}
            title={marker.title }
            description={marker.description}
            onCalloutPress={()=> this.onMarkerCalloutPress(key)}>
           {marker.type == "label" ?
          <View style ={{flex:1,backgroundColor:"rgba(0,0,0,1.0)",borderRadius:0,padding:2,justifyContent: 'center',
    alignItems: 'center'}}>
          <Text style={{flex:1,backgroundColor:"rgba(0,0,0,1.0)",color:"white",fontSize:12,justifyContent: 'center',
    alignItems: 'center'}}> {marker.title}</Text>
          </View>
   : null } </Marker>

2 const snapshot = this.map.takeSnapshot({

          format: 'png',   // image formats: 'png', 'jpg' (default: 'png')
          quality: 1.0,    // image quality: 0..1 (only relevant for jpg, default: 1)
          result: 'file'   // result types: 'file', 'base64' (default: 'file')
        });
        snapshot.then((uri) => {

          console.log(uri);
        });

Expected Behavior
get an image with all markers.
(Write what you thought would happen.)
Actual Behavior
only default marker on the image!
Screenshot
On simulator

Generated image

Any solution?

Also facing this issue please fix, no custom markers and user location marker are being shown

Especially user location marker - seems to be something people missed out on


Add support to get the bios serial boot logs

We need to provide an option for users to download the bios serial logs.
ssh into an openbmc @port 2200 and reboot server

work on this kicked off today. Basic pathway is clear re manual steps to retrieve these logs.

A request has come in to also grab the openbmc console output as well.

thanks @gregg-shick - we will add.

based on internal discussions, we will tackle the openbmc console requirement seperately.

status: tackling some debug issues on collection sequence that is beyond the scope of this ticket.

status: issue is facing a blocker due to collection reliability. Will resolve once we clear that out.

My thoughts

As soon as connect power is done, begin logging the 2200 session.
Present the user with an interface option to download the bios serial log once session is established.
User can do whatever he chooses in the session, including powering on the unit from the openbmc web interface.
It is then up to the user to download bios serial log before sessions times out or session is ended by user.
Clean up logs at session closure.

yep, agreed @gregg-shick - we now no longer worry about anything beyond logging the 2200 session.


[question] Example for Cusom Policy for SAC with combined image and box type observation

I am trying to use SAC for a custom robot env which has both image and box type data as observations. The plan is to use a custom network which looks something like this.

Does anyone have an example of how to implement this? Any help would be appreciated.

Hello,
Duplicate of https://github.com/hill-a/stable-baselines/issues/947 and https://github.com/hill-a/stable-baselines/issues/133#issuecomment-533806931
Anyway, I would recommend you to use Stable-Baselines3 as such modifications should be easier to implement.

@araffin Thanks a lot

Update for SB3: the fix will be merged with master soon ;)
https://github.com/DLR-RM/stable-baselines3/pull/198


Add Round Spark function

std::round will get 0.57 for round(0.575,2).
But in  spark, we should get 0.58 in default mode.
The main reason is that double type "0.575 * 100" is actually "57.499999999999996". So we need use nextafter function to get real value.

Hi, @Yuhta  would you review this PR?

Yes, yuhta is backing out presto's change. We already discussed yesterday. PR is here https://github.com/facebookincubator/velox/pull/5991

Hi @zhejiangxiaomai , can you rebase with latest main and push to this PR, thanks ?

Hi @zhejiangxiaomai , can you rebase with latest main and push to this PR, thanks ?

Just rebased.

After discussed with yuhta offline.  we think we can use decimal_round in gluten. And I found postgres also use "Numeric"( close to decimal) type to implement double/float round. https://github.com/postgres/postgres/blob/REL9_0_STABLE/src/backend/utils/adt/numeric.c#L827-L865
And spark also use decimal type to handle it. https://github.com/apache/spark/blob/branch-3.2/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1341-L1370
And https://github.com/facebookincubator/velox/pull/5597 already implement decimal_round.

Just Remainder:
I found nextafter can handle 0.574999999999996. because double is 15 decimal digits precision. 0.49999999999999994 can not get correct result because it is 17 digits.

To use decimal_round, I assume we need to cast double as decimal first. But for number like 0.57499999999999996 which has 17 igits, if precision drift happens on this double value during casting process, possibly, unmatched decimal could be got and the round result is affected.

@rui-mo The change to decimal should be something at SQL parser level, and Velox should be called with decimal already.  If Velox received double, there is no way for us to recover the original value.

Discussed with @zhejiangxiaomai offline, this is not solvable inside Velox so close this for now


All missing distros plus hardcoded maint

closes #46

this ones are not currently building (blame QEMU):
make erlang_latest_rockylinux_8_linux-arm64
make erlang_latest_ubuntu_bionic_linux-arm64
make erlang_latest_ubuntu_focal_linux-arm64
make erlang_latest_ubuntu_jammy_linux-arm64
make erlang_latest_ubuntu_bullseye_linux-arm64
make erlang_latest_debian_buster_linux-arm64
make erlang_latest_centos_7_linux-amd64
make erlang_latest_centos_7_linux-arm64
centos_8 => needs to be clarified...

Will tackle this with a dedicated instance if there is time:
thinkg that need to be done:

create EC2 instance
install docker multi arch
s3 sync
import gpg
maybe runner

I am leaving ARM for the future 😉. After this run I will comment the failed builds in the github workflow.


Nuitka doesn't work with latest sentence_transformers

Hi @kayhayen ,
I am trying to compile a basic sentence transformer code using Nuitka. The code is given in the sentence-transformer tutorial at :
https://sbert.net/docs/quickstart.html
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium.",
]

embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities)

The nuitka command is very basic with no modules and no packges:
python -m nuitka --main=main.py --standalone --onefile --static-libpython=yes --noinclude-unittest-mode=allow --no-pyi-file
I am getting the following error while running:
(sentence_transformer_env) pingaksh@us01odc-sc4-1-gpu064:/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test % ./main.bin
Traceback (most recent call last):
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1778, in _get_module
File "importlib.py", line 90, in import_module
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1310, in _find_and_load_unlocked
File "", line 488, in _call_with_frames_removed
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/init.py", line 15, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/albert/init.py", line 31, in
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 2197, in define_import_structure
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1946, in create_import_structure_from_path
FileNotFoundError: [Errno 2] No such file or directory: '/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/albert'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/main.py", line 1, in
from sentence_transformers import SentenceTransformer
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/sentence_transformers/init.py", line 10, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/sentence_transformers/cross_encoder/init.py", line 3, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/sentence_transformers/cross_encoder/CrossEncoder.py", line 14, in
File "", line 1412, in _handle_fromlist
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1766, in getattr
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1783, in _get_module
RuntimeError: Failed to import transformers.models.auto because of the following error (look up to see its traceback):
[Errno 2] No such file or directory: '/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/albert'
"/remote/vg_llm/pingaksh/chroma_cache' is my temp dir which is clean and nothin to do with chromadb.
Following are my pip installs in my env:
certifi==2024.8.30
charset-normalizer==3.4.0
filelock==3.16.1
fsspec==2024.10.0
huggingface-hub==0.26.2
idna==3.10
Jinja2==3.1.4
joblib==1.4.2
MarkupSafe==3.0.2
mpmath==1.3.0
networkx==3.4.2
Nuitka==2.4.11
numpy==2.1.2
nvidia-cublas-cu12==<IP_ADDRESS>
nvidia-cuda-cupti-cu12==12.4.127
nvidia-cuda-nvrtc-cu12==12.4.127
nvidia-cuda-runtime-cu12==12.4.127
nvidia-cudnn-cu12==<IP_ADDRESS>
nvidia-cufft-cu12==<IP_ADDRESS>
nvidia-curand-cu12==<IP_ADDRESS>
nvidia-cusolver-cu12==<IP_ADDRESS>
nvidia-cusparse-cu12==<IP_ADDRESS>
nvidia-nccl-cu12==2.21.5
nvidia-nvjitlink-cu12==12.4.127
nvidia-nvtx-cu12==12.4.127
ordered-set==4.1.0
packaging==24.1
pillow==11.0.0
PyYAML==6.0.2
regex==2024.9.11
requests==2.32.3
safetensors==0.4.5
scikit-learn==1.5.2
scipy==1.14.1
sentence-transformers==3.2.1
setuptools==75.2.0
sympy==1.13.1
threadpoolctl==3.5.0
tokenizers==0.20.1
torch==2.5.0
tqdm==4.66.6
transformers==4.46.0
triton==3.1.0
typing_extensions==4.12.2
urllib3==2.2.3
zstandard==0.23.0
Any help will be appreciated.

Hi @kayhayen ,
I am trying to compile a basic sentence transformer code using Nuitka. The code is given in the sentence-transformer tutorial at :
https://sbert.net/docs/quickstart.html
`from sentence_transformers import SentenceTransformer
1. Load a pretrained Sentence Transformer model
model = SentenceTransformer("all-MiniLM-L6-v2")
The sentences to encode
sentences = [
"The weather is lovely today.",
"It's so sunny outside!",
"He drove to the stadium.",
]
2. Calculate embeddings by calling model.encode()
embeddings = model.encode(sentences)
print(embeddings.shape)
[3, 384]
3. Calculate the embedding similarities
similarities = model.similarity(embeddings, embeddings)
print(similarities)
tensor([[1.0000, 0.6660, 0.1046],
[0.6660, 1.0000, 0.1411],`
The nuitka command is very basic with no modules and no packges:
python -m nuitka --main=main.py --standalone --onefile --static-libpython=yes --noinclude-unittest-mode=allow --no-pyi-file
I am getting the following error while running:
(sentence_transformer_env) pingaksh@us01odc-sc4-1-gpu064:/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test % ./main.bin
Traceback (most recent call last):
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1778, in _get_module
File "importlib.py", line 90, in import_module
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1310, in _find_and_load_unlocked
File "", line 488, in _call_with_frames_removed
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/init.py", line 15, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/albert/init.py", line 31, in
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 2197, in define_import_structure
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1946, in create_import_structure_from_path
FileNotFoundError: [Errno 2] No such file or directory: '/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/albert'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/main.py", line 1, in
from sentence_transformers import SentenceTransformer
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/sentence_transformers/init.py", line 10, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/sentence_transformers/cross_encoder/init.py", line 3, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/sentence_transformers/cross_encoder/CrossEncoder.py", line 14, in
File "", line 1412, in _handle_fromlist
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1766, in getattr
File "/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/utils/import_utils.py", line 1783, in _get_module
RuntimeError: Failed to import transformers.models.auto because of the following error (look up to see its traceback):
[Errno 2] No such file or directory: '/remote/vg_llm/pingaksh/chroma_cache/onefile_1497776_1730145894_164938/transformers/models/albert'
"/remote/vg_llm/pingaksh/chroma_cache' is my temp dir which is clean and nothin to do with chromadb.
Following are my pip installs in my env:
certifi==2024.8.30
charset-normalizer==3.4.0
filelock==3.16.1
fsspec==2024.10.0
huggingface-hub==0.26.2
idna==3.10
Jinja2==3.1.4
joblib==1.4.2
MarkupSafe==3.0.2
mpmath==1.3.0
networkx==3.4.2
Nuitka==2.4.11
numpy==2.1.2
nvidia-cublas-cu12==<IP_ADDRESS>
nvidia-cuda-cupti-cu12==12.4.127
nvidia-cuda-nvrtc-cu12==12.4.127
nvidia-cuda-runtime-cu12==12.4.127
nvidia-cudnn-cu12==<IP_ADDRESS>
nvidia-cufft-cu12==<IP_ADDRESS>
nvidia-curand-cu12==<IP_ADDRESS>
nvidia-cusolver-cu12==<IP_ADDRESS>
nvidia-cusparse-cu12==<IP_ADDRESS>
nvidia-nccl-cu12==2.21.5
nvidia-nvjitlink-cu12==12.4.127
nvidia-nvtx-cu12==12.4.127
ordered-set==4.1.0
packaging==24.1
pillow==11.0.0
PyYAML==6.0.2
regex==2024.9.11
requests==2.32.3
safetensors==0.4.5
scikit-learn==1.5.2
scipy==1.14.1
sentence-transformers==3.2.1
setuptools==75.2.0
sympy==1.13.1
threadpoolctl==3.5.0
tokenizers==0.20.1
torch==2.5.0
tqdm==4.66.6
transformers==4.46.0
triton==3.1.0
typing_extensions==4.12.2
urllib3==2.2.3
zstandard==0.23.0
Any help will be appreciated.

If I do not use the --onefile option, I am getting the following error:
main.dist/main.bin
Traceback (most recent call last):
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/utils/import_utils.py", line 1778, in _get_module
File "importlib.py", line 90, in import_module
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1310, in _find_and_load_unlocked
File "", line 488, in _call_with_frames_removed
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/models/init.py", line 15, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/models/albert/init.py", line 31, in
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/utils/import_utils.py", line 2197, in define_import_structure
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/utils/import_utils.py", line 1946, in create_import_structure_from_path
FileNotFoundError: [Errno 2] No such file or directory: '/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/models/albert'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/main.py", line 1, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/sentence_transformers/init.py", line 10, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/sentence_transformers/cross_encoder/init.py", line 3, in
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/sentence_transformers/cross_encoder/CrossEncoder.py", line 14, in
File "", line 1412, in _handle_fromlist
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/utils/import_utils.py", line 1766, in getattr
File "/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/utils/import_utils.py", line 1783, in _get_module
RuntimeError: Failed to import transformers.models.auto because of the following error (look up to see its traceback):
[Errno 2] No such file or directory: '/remote/vg_llm/pingaksh/pycharm/chatsv_code2doc/sentence_transformet_test/main.dist/transformers/models/albert'

@huyz-git seems very relevant information, finding files to know what to import is a frequent thing we need to workaround for Nuitka, will look at that now.

Hi,
I am trying to do RAG using langchain. I used the following libraries:
from langchain.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import Chroma
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import TextLoader
from langchain.prompts import ChatPromptTemplate
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains import create_retrieval_chain
The environment is same as above. But when I run the executable, it shows the following error:
main.dist/main.bin --gen_code --file_name ./data/query.txt
Error: Could not import sentence_transformers python package. Please install it with pip install sentence-transformers.
However, sentence-transformer is already present in the environment. Is there an way to overcome this error?
~Pingu

@pingakshya2008utd bottom posting other issues is a sure fire way to get ignored, please open a new issue with full set of information

I got newer transformers to work somewhat, but now the various transformers are having requirements for metadata, which they tell me about one by one, so I am in a compiling loop right now.

It's supposed to work with current factory version, I am checking now if transformers before 4.45 still works, since I made the detection of what needs to be done dynamic. Previously we had a list of models to cover, but that is not tenable.

Thanks for your report, this is fixed on the factory branch, which is a development version under rapid development. You can try it out by going here: https://nuitka.net/doc/factory.html
Feedback on whether this is working is very welcome. Please do not share plans to do it; only confirm or deny that it is working.

Using older version worked for me at the mean time. Dont want to use the factory version as there may be some other issues as mentioned in the website. I am creating a new thread for the other issue.

This is now part of the current pre-release and will be part of 2.5 release.

I met the exactly the same issue when packing transformer 4.45.0, any pre-release version for verification?

Part of the 2.5 release I just made.


Fix identifying volume for Windows 10 Anniversary Edition

ObjectId is more verbose than DeviceId in Windows 10 Anniversary
Edition. In one instance, mounting a disk image and checking the
ObjectId returns
{1}\\WIN-56OJJQTNIEP\root/Microsoft/Windows/Storage/Providers_v2\WSP_Volume.ObjectId="{4c19525a-f384-11e4-8044-806e6f6e6963}:VO:\\?\Volume{b153fdb9-87e2-11e5-80b3-005056fce5a0}\"

The associated volume has a DeviceId of
\\?\Volume{b153fdb9-87e2-11e5-80b3-005056fce5a0}\

look for DeviceId in the ObjectId string instead of an exact match to
identify volumes. Fixes #48.

This change is

Hi @MikaelSmith, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

@MikaelSmith, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, MSBOT;

Other than the alias and positional parameter issue, it looks good to me. However, this resource doesn't have any unit or integration tests on it. But this change should probably go through regardless as it seems like it is broken in AE. @KwirkyKat?

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

DSCResources/MSFT_xMountImage/MSFT_xMountImage.psm1, line 95 [r1] (raw file):

        #Verify drive letter. ObjectId is more verbose than DeviceId in Windows 10 Anniversary Edition, look for
        #DeviceId in the ObjectId string to match volumes.
        $CimVolume = Get-CimInstance -ClassName Win32_Volume | where {$Image.ObjectId.IndexOf($_.DeviceId) -ne -1}

I know this code was already there, but are you able to change the where to be:
Where-Object -FilterScript {$Image.ObjectId.IndexOf($_.DeviceId) -ne -1}
We want to get rid of positional parameters and aliases.

Comments from Reviewable

Done.

 Thanks @MikaelSmith

Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


support serverless-next.js ?

hi.
Our project uses https://github.com/serverless-nextjs/serverless-next.js to deploy next.js application to lambda edge.
We are having trouble installing the apm agent because the newrelic documentation does not provide instructions on how to set it up when using serverless-next.js.
Does serverless-newrelic-lambda-layers support serverless-next.js? If so, how do I use it?

@bonobozzzz Thank you for bringing this to our attention! I think that this New Relic plugin is unlikely to work for your use case. This plugin's primary purpose is to install and configure an instrumentation agent and extension via Lambda Layers, but your application requires Lambda@Edge, which does not support Lambda Layers. Among those restrictions are also restrictions on the use of environment variables.
Despite those restrictions, you could instrument your application with New Relic, with a little manual work:

Install newrelic in your application as a dependency, so that the NR Node agent is included in your deployment package.
Follow the instructions for wrapping a function manually...although you'll want to depart from those instructions this way:
Since you can't use environment variables, do not set NEW_RELIC_NO_CONFIG_FILE to true. Instead, define the app name, account id, and trusted account key in your project's newrelic.js configuration file (included with the agent), along with any other required environment variables.
Without the use of Lambda Layers, you won't have the option of ingesting telemetry via the New Relic Lambda Extension, so you could set up a CloudWatch Log Subscription using the Lambda CLI


floor return type is not integer

see http://php.net/manual/en/function.floor.php

@steros nice find. Can we also add a test so we can ensure that a valid value is returned?

Closing this one since it is stale


wallet: extract and document legacy (bdb) derivation (refactor only)

Follow-up for #2977, with the intent of cleaning up and clarifying legacy wallet key derivation.

Proposes a new structure for Dogecoin-specific code to prevent circular dependencies - having just a dogecoin.h is nice if we only touch validation structures but this is already no longer the case in current 1.14. Instead of prefixing a bunch of files with dogecoin- in src/, I think it's more useful to make a directory for it and put everything under that. Makes it easier to find too. This is up for discussion of course - please let me know if that concept is agreeable.
Extracts BIP32_HARDENED_KEY_LIMIT from wallet/scriptpubkeyman.cpp. Because we don't have a specific bip32 header I decided to simply copy this into the dogecoin/wallet.h file to use there, because we only need it there and in the cpp file it was formerly defined in and I don't think we should create a header with just 1 constant in it.
Extract values for the first 2 elements of the legacy wallet derivation path. account and chain were magic numbers inside wallet/scriptpubkeyman.cpp but are now declared and documented inside dogecoin/wallet.h

Note 1: I found one remaining legacy derivation value in wallet/scriptpubkeyman.cpp on line 1078 - I think I caused that myself in an early review. To not mix refactoring with bug fixing, I left it as-is, but marked it with a TODO note - I will propose a separate PR that fixes it as it will change the test. I do think that this makes the benefit of not using magic numbers visible.
Note 2: I have not changed the BIP44-based descriptor wallet yet, because there the derivation path is much cleaner and can be documented inside chainparams instead. I will open a separate PR for that too.

Built and tested x86_64-pc-linux-gnu before and after a rebase with upstream/1.21-dev and on jammy. 🔥

Forgot about that, sorry! Rebased.

utACK; the changes look straightforward and workable, but I'm still hitting the secp256k1 test errors. I'm inclined to give a passing review based on @xanimo and CI however. Any concerns?

did you do the autoreconf?

I did both make clean and autoreconf yes. Still a little baffled. Even with a rebase in 1.21-dev, I still get these failures.

@chromatic I just ran another successful build on focal with the following:
make clean && make -C depends -j$(nproc) NO_QT=1 && ./autogen.sh && ./configure --prefix=`pwd`/depends/x86_64-pc-linux-gnu --enable-zmq --enable-glibc-back-compat LDFLAGS=-static-libstdc++ && make check -j$(nproc) && ./test/functional/test_runner.py

Let me know if this helps and/or if it would be possible to see build logs if you still run into errors.

I'm trying this again on Jammy, @xanimo. Let's see how far I get.

Hmm hitting this Boost issue for an undefined PTHREAD_STACK_MIN.


Why was perspective(augmentation) removed?

Hey, can I get some perspective on why it was removed, I'm using it in my model so it would be helpful to know!

Perspective is not removed, actually.  Only normalization is removed.


PartObjaverse-Tiny segmentation ground truth tooling

Hi, Which tool was used to generate semantic and instance level ground truths for PartObjaverse-Tiny?
I would like to use the same for re-training and testing.

I am also interested in the annotation tools. Look forward for the answer.

Hi, Thanks for your interest in our work!
we manually annotate the PartObjaverse-Tiny dataset.
Maybe you can try to use blender.


Keep compatibility with 1.18.2

1.18.2 is still a go to version for lots of mods. Perhaps it should be supported for a bit longer?

keep 1.18 support !

I suspect 1.18.2 is another major modding version right now (like 1.7.10 and 1.12.2) due to 1. the lackluster content of 1.19 in the view of many players, 2. the apparent performance detriment of 1.19, and 3. rejection of the chat report system. 1.18.2 is a must, at least in current times.

I suspect 1.18.2 is another major modding version right now (like 1.7.10 and 1.12.2) due to 1. the lackluster content of 1.19 in the view of many players, 2. the alleged performance detriment of 1.19, and 3. rejection of the chat report system. 1.18.2 is a must, at least in current times for now.

Chat Reporting isn't included in 1.19.0 tho

1.18.2 compatibility is done, just unreleased since it hasn't been reviewed yet. Until then, you can grab a test jar from the 1.18.2 pull request, or try the one in my fork's releases.


How to control the speed of the Fetch's arm?

Hi all,
I have some code set up that can move the robot's arm (well, end-effector) to a pose. When I do this to the physical Fetch robot, the arm moves very quickly. Thus, I am hoping to control the speed to slow it down.
My code is similar to this code base, but I'm thinking of now using stuff from fetch_pbd and merging it into my code, which is why I'm reading this code repository. I noticed a few observations:

The arm.py code here uses a velocities argument in the move_to_joints_plan method.  However, it's commented out. Is this supposed to be something that could control the speed of the robot?

move_to_pose has a self._move_group.set_planning_time(1.0) call. I assume this is for the planning time but does not affect how fast the arm moves once the plan is set. Is that correct?

fetch_pbd_interaction/arm_control.py has some module-level constants: MOVE_TO_JOINTS_SLEEP_INTERVAL  and TRAJECTORY_COMPLETE_SLEEP_INTERVAL . But the code comments there imply that these are invoked after the arm has finished moving, and does not control the speed while the arm is moving.

I wonder if there is an example script somewhere that could show an example of controlling the velocity of the robot? There seems to be some code re-use between several arm-related scripts so I also appreciate any code recommendations or best practices. Or maybe the best practice is to try and put a rospy.sleep(...) somewhere during the move methods?
Thanks all! I'm happy to clarify my questions as needed.

Update: turns out it's probably best to adjust MoveIt's max_velocity_scaling_factor. I tried that on my physical Fetch robot and it works.


Added link text to cloud storage bulk operations

Added the link to the end of this paragraph in its own sentence. Avoiding a long link out in the middle of the current sentence and making it clearer to the reader what page they're heading to re cloud storage.
Closes #10573

This change is


[Bug]: No tests found with JUnit 5.9.0

Module
Core
Testcontainers version
1.17.3
Using the latest Testcontainers version?
Yes
Host OS
Any OS
Host Arch
Any arch
Docker version
Client: Docker Engine - Community
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:02:57 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:03 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.7
  GitCommit:        0197261a30bf81f1ee8e6a4dd2dea0ef95d67ccb
 runc:
  Version:          1.1.3
  GitCommit:        v1.1.3-0-g6724737
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

What happened?
Any tests annotated with @TestContainers not found by Maven Surefire plugin 2.22.0 with JUnit 5.9.0.
All works fine with JUnit 5.8.2.
Relevant log output
No response
Additional Information
No response

Please share a reproducer. You still need to annotate the tests with the actual JUnit5 annotations. Also, it is @Testcontainers.

yup @kiview sorry for typo about @Testcontainers
What I was meaning is that I do have some tests annotated by @Testcontainers which are running fine when
      <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
      </dependency>

If I simply upgrade to 5.9.0 then such tests are not detected.
I'll try to come up shortly with a reproducer.

Alright, I think this is actually an issue with Surefire and JUnit5 and has nothing to do with Testcontainers. At least this is my assumption. Which version of Surefire are you using? Try to use the latest version, potentially a milestone release, such as 3.0.0-M7.

Here you go: https://github.com/ilgrosso/testcontainers5680
As you can read from pom.xml, the default versions are
    <junit.version>5.8.2</junit.version>
    <surefire.version>2.22.2</surefire.version>

So, when you run
mvn clean test
you obtain
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running net.tirasa.test.testcontainers5680.ReproducerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.809 s - in net.tirasa.test.testcontainers5680.ReproducerTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

but when you go with
mvn clean test -Djunit.version=5.9.0

then you get
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

with
mvn clean test -Djunit.version=5.9.0 -Dsurefire.version=3.0.0-M7

you receive
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (default-test) on project testcontainers5680:
[ERROR]
[ERROR] Please refer to /home/ilgrosso/work/testcontainers5680/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] There was an error in the forked process
[ERROR] TestEngine with ID 'junit-jupiter' failed to discover tests
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR] TestEngine with ID 'junit-jupiter' failed to discover tests
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:701)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:311)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:268)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1334)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1167)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:931)
[ERROR]         at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:301)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:211)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:165)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:157)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:121)
[ERROR]         at org.mvndaemon.mvnd.builder.SmartBuilderImpl.buildProject(SmartBuilderImpl.java:178)
[ERROR]         at org.mvndaemon.mvnd.builder.SmartBuilderImpl$ProjectBuildTask.run(SmartBuilderImpl.java:198)
[ERROR]         at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[ERROR]         at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[ERROR]         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[ERROR]         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:829)

which is basically the same.

My wild guess is that there is some dependency between Testcontainers 1.17.3 and JUnit 5.8.2 as shown by
mvn dependency:tree -Djunit.version=5.9.0

[INFO] net.tirasa.test:testcontainers5680:jar:1.0-SNAPSHOT
[INFO] +- org.testcontainers:postgresql:jar:1.17.3:test
[INFO] |  \- org.testcontainers:jdbc:jar:1.17.3:test
[INFO] |     \- org.testcontainers:database-commons:jar:1.17.3:test
[INFO] +- org.testcontainers:junit-jupiter:jar:1.17.3:test
[INFO] |  +- org.testcontainers:testcontainers:jar:1.17.3:test
[INFO] |  |  +- junit:junit:jar:4.13.2:test
[INFO] |  |  |  \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] |  |  +- org.slf4j:slf4j-api:jar:1.7.36:test
[INFO] |  |  +- org.apache.commons:commons-compress:jar:1.21:test
[INFO] |  |  +- org.rnorth.duct-tape:duct-tape:jar:1.0.8:test
[INFO] |  |  |  \- org.jetbrains:annotations:jar:17.0.0:test
[INFO] |  |  +- com.github.docker-java:docker-java-api:jar:3.2.13:test
[INFO] |  |  |  \- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.3:test
[INFO] |  |  \- com.github.docker-java:docker-java-transport-zerodep:jar:3.2.13:test
[INFO] |  |     +- com.github.docker-java:docker-java-transport:jar:3.2.13:test
[INFO] |  |     \- net.java.dev.jna:jna:jar:5.8.0:test
[INFO] |  \- org.junit.jupiter:junit-jupiter-api:jar:5.8.2:test
[INFO] |     +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] |     +- org.junit.platform:junit-platform-commons:jar:1.8.2:test
[INFO] |     \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] \- org.junit.jupiter:junit-jupiter:jar:5.9.0:test
[INFO]    +- org.junit.jupiter:junit-jupiter-params:jar:5.9.0:test
[INFO]    \- org.junit.jupiter:junit-jupiter-engine:jar:5.9.0:test
[INFO]       \- org.junit.platform:junit-platform-engine:jar:1.9.0:test

Additional finding: if adding the following exclude:
    <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>junit-jupiter</artifactId>
      <version>${testcontainers.version}</version>
      <exclusions>
        <exclusion>
          <groupId>org.junit.jupiter</groupId>
          <artifactId>junit-jupiter-api</artifactId>
        </exclusion>
      </exclusions>
      <scope>test</scope>
    </dependency>

then the test is run even with -Djunit.version=5.9.0

Hey @ilgrosso, thanks for your detailed triaging. You are completely right, this is an issue with org.testcontainers.junit-jupiter bringing in the transitive org.junit.jupiter:junit-jupiter-api:jar:5.8.2 dependency.
So either there might be a better way to build our junit-jupiter extension here (we would need to check no best practices around extensions in the community), or we just update the dependency itself (although this does not seem like the clean solution to me).

IntelliJ IDEA 2022.2.2 (Community Edition) has no problems to find the check test when tasked to run all test in the  ReproducerTest Testclass.
Class name > Test Marker left of Text Pane > Run 'ReproducerTest'

You are completely right, this is an issue with org.testcontainers.junit-jupiter bringing in the transitive org.junit.jupiter:junit-jupiter-api:jar:5.8.2 dependency.

Would switching the dependency declaration api 'org.junit.jupiter:junit-jupiter-api:5.9.1' to implementation help? Gradle's Java Library plugin explains the subtle difference between api and implementation.

@edysli yes. However, the concern is doing that change could break existing projects due to the absence of junit-jupiter in the classpath.

@eddumelendez it will indeed break projects relying on this transitive dependency. However, they shouldn't do this and instead declare a dependency on org.junit.jupiter:junit-jupiter -- just like @ilgrosso did -- since they are using it directly. For me, this is OK to break, but I'll leave this up to the maintainers, of course. :blush:

junit-pioneer, which I consider a reference implementation of JUnit5 extensions, is using implementation as well:
https://github.com/junit-pioneer/junit-pioneer/blob/main/build.gradle.kts#L59

This is an issue I ran into at work and exposing the conflicting dependency as api(platform( was the solution i settled on.
A couple of things I found was that actually, in some cases (e.g. not through multiple transitive deps' api references) you can use "compileOnly platform" and it will completely hide your junit version from the consumer. This seems to violate some api contract because it fails unpredictably and causes frustration. When it does, you can revert it, and ask the consumer to use enforcedPlatform instead of platform. both of these are not the gradle way of doing things. ergo, the solution being presented is what i went with.
The dependency in question was spring-boot-dependencies (and friends) - using the solution here, the consumer still prints the right spring banner, so gradle seems to be smart about using the consumers implementation(platform( before the libraries' api(platform('s.


Make Alt-F behaviour native on Windows

Make Alt-F work natively - to open file menu - as per standard behaviour on Windows platform.
Refer issue #474

Is this still an issue? Can you elaborate on the issue? Issue #474 doesnt look like it is in reference to this change.

What about keybindings such as Alt-H or Alt-E?  This doesn't seem like it'll fix all of them.

Seems to be no more traction on this so going to close. If somebody wants to answer the questions and push it through please reopen.


Stone Paper Scissors with GUI

I do checked the repo and  found that there is stone paper scissors   pygame included but its implemented using Command Line .

I will be using pygame and tkinter library to make same with Graphical User Interface
It can be more interactive and playful

I would like to work

@Lakhankumawat assigned to you


Header/Type not working correctly

Hello, following the README.md Create Request section I found that there is an issue setting the content-type for a post request. Tried both with type method and the header method. I made a gist for it.
Steps to reproduce it

git clone https://gist.github.com/6917b308ecaf1a64fd5f.git
npm install
DEBUG=* npm test

You will see that the body parser says that the request has the content type set to "application/x-www-form-urlencoded"

This is already reported in #53

@Nijikokun it is not. They are different issues. One refers to the header is not sent and the other that the received header is not used correctly.

You're right, my apologies, I read this quickly, I'll run the tests specified now and see if I can debug this.

@nicosommi can you try master and let me know if that solves your issues? There are tests but I want to confirm before releasing the version.

@Nijikokun yep both gists working

Perfect, publishing :+1:


rocALUTION build following ROCm versions

Thanks for all the updates of rocALUTION library. We would be glad to use the new features and take advantage of bug fixes. But the fact that rocALUTION releases follow closely ROCm versions is really annoying. Our HPC cluster is still on ROCm 5.2.x... Any hope rocALUTION main branch is rebased on older ROCm version? Thanks

Currently, there are no plans in changing our release cycles independent of rocm. You can, however, compile latest rocalution with out-dated rocm stack on your cluster to get access to latest features.

The last time we tried to build the develop branch with ROCm 5.3.0, it failed with some errors:
1/ First some AMD GPU architectures were not recognised : gfx1100;gfx1101;gfx1102. I removed them in CMakelists.txt (DEFAULT_AMDGPU_TARGETS).

2/ Then, when compiling I hit the following error :

make[4]: Entering directory '/lus/home/NAT/cpa/dechardj/TRUST/TRUST.dev/ThirdPart/src/LIBROCALUTION/rocALUTION/build'
[  0%] Building HIPCC object src/CMakeFiles/rocalution_hip.dir/base/hip/rocalution_hip_generated_hip_sparse.cpp.o
/lus/home/NAT/cpa/dechardj/TRUST/TRUST.dev/ThirdPart/src/LIBROCALUTION/rocALUTION/src/base/hip/hip_sparse.cpp:1164:16: error: no matching function for call to 'rocsparse_sbsrmv'
        return rocsparse_sbsrmv(handle,
               ^~~~~~~~~~~~~~~~
/opt/rocm-5.3.0/include/rocsparse/rocsparse-functions.h:2073:18: note: candidate function not viable: requires 15 arguments, but 16 were provided
rocsparse_status rocsparse_sbsrmv(rocsparse_handle          handle,

So probably that we need also to build rocSPARSE too...

You are right, it will not work out of the box with older rocsparse. Alternatively to compiling rocsparse yourself, you can also modify rocALUTION/src/base/hip/hip_sparse.cpp
You will need to adjust all calls to rocsparse_sbsrmv and remove the NULL pointer argument, e.g. line line 1164 onwards:
1164         return rocsparse_sbsrmv(handle,
1165                                 dir,
1166                                 trans,
1167                                 mb,
1168                                 nb,
1169                                 nnzb,
1170                                 alpha,
1171                                 descr,
1172                                 bsr_val,
1173                                 bsr_row_ptr,
1174                                 bsr_col_ind,
1175                                 bsr_dim,
1176                                 nullptr,
1177                                 x,
1178                                 beta,
1179                                 y);

has to be changed to
1164         return rocsparse_sbsrmv(handle,
1165                                 dir,
1166                                 trans,
1167                                 mb,
1168                                 nb,
1169                                 nnzb,
1170                                 alpha,
1171                                 descr,
1172                                 bsr_val,
1173                                 bsr_row_ptr,
1174                                 bsr_col_ind,
1175                                 bsr_dim,
1176                                 x,
1177                                 beta,
1178                                 y);

Once done, it should compile successfully.

Closing this issue. Please re-open, if you face additional issues.

Thanks, it works. I build last rocALUTION version with rocm 5.2.3.