add redis storage integration

This PR adds support for using Redis as the storage service of Apollon Standalone (instead of file-based storage). After merging this PR, Apollon Standalone will look for APOLLON_REDIS_URL environment variable, and if present, switch to Redis as its storage backend. See these docs for further info on the format of the Redis URL. If the environment variable is not present, Apollon Standalone will fallback into using the filesystem based storage it was already using.
The main benefit of using Redis as the storage backend is stability of high-frequency writes. Specifically in context of realtime collaboration, it can happen that diagrams are constantly updated, which, using a file based storage, means potentially colliding write operations resulting in corrupted files. The DiagramFileStorageService was already equipped with rate-limiting logic to mitigate this issue, but this means occasionally clients would get the wrong diagram if they requested it at the wrong time (for example, while a change is being debounced).
This PR refactors the rate limiting logic into a separate DiagramStorageRateLimiter class, which is in turn used by both the old DiagramFileStorageService and the new DiagramRedisStorageService. However, the Redis storage service can use a much more relaxed rate limiting configuration as Redis is much faster than a file-based storage scheme. Additionally, Redis, through its JSON module, allows for atomic patching of diagrams, which was again not possible in a file-based storage, further increasing the stability and performance of the storage.

Looks good to me! We should eventually introduce unit tests for functionalities like "redis-patch.ts" but not as part of this PR.


Add support for different activation functions

Hello team, this PR closes #812,
This PR adds support for activation dictionaries in the create_and_compile_model function, allowing to accept either a string or a dictionary, enabling different activation functions for recurrent and dense layers.
I followed the example from the issue:
model = create_and_compile(
    ...
    recurrent_units=[32, 64],
    dense_units=[64, 32],
    ...
    activation={
        "recurrent_units": ["relu", "relu"],
        "dense_units": ["relu", "tanh"],
    },
    ...
)

Acceptance Criteria:

[x] Modify the function to accept a list of activation functions for both recurrent and dense layers.

If a single activation function is provided, it should be applied to all layers (as in the current behavior).
If a list is provided, each layer should use the corresponding activation function from the list.

[x] Validate that the number of activation functions matches the number of layers in the network.
[x] Update the documentation and docstrings to explain the new functionality.
[x] Add unit tests to verify the behavior for both single and multiple activation functions.
[x] Ensure backward compatibility: If a single activation function is passed, it should be applied to all layers (recurrent or dense), preserving the current behavior.
[x] If a list of activation functions is passed, ensure that it matches the number of layers for proper assignment.
[x] Provide clear error messages if the list of activation functions does not match the number of layers.

Hi @pablorodriper,
Thanks alot for your contribution and the great quality of your code! We will review th PR in the following days.
@FernandoCarazoMelo could you help on this?

@pablorodriper Thanks for the suggestion! Using a dictionary could indeed add more flexibility, especially for cases where granular control over each layer’s activation is needed. On the other hand, a list might keep things simpler and more explicit in terms of the order of activations, aligning directly with recurrent_units and dense_units.
Do you have any other specific reasons for preferring a dictionary? I’m open to exploring this further if you think it would better serve a broader range of use cases.
Thanks again for your input and collaboration on this!

@pablorodriper
Thanks a lot for the PR! It is part of skforecast 0.14.0 which was released yesterday 😄
Your contribution has been included in the release notes and your profile in the Authors section.
Best


Exposing viewport mouse events with a Qt signal

See: https://github.com/ros-visualization/rviz/issues/1206#issuecomment-371438135
But, I will not have time to properly test this out right now, so unfortunately you'll have to wait for us to have time to spend on rviz in the coming months.

Hi!
I think this PR and the issue #1206 can be closed. I made this PR to have a more comprehensive question. This solution tackles the problem by changing the principles on how Rviz is made. Thus I would not merge it and I would close it.
I have seen that the new Rviz 2 is being planned and I will put my inputs if needed in there.
Thanks for the replays :)


Index tags used in yaml wildcard files

Parsing YAML takes about 0.4s on an M1 Mac and the script runs twice. Not sure why and how to fix it.
It will now create an additional file wcet.txt in temp with a list of tag,count pairs. Tell me if there's anything to fix :).

Alright, thanks. I'll probably implement it tomorrow if nothing gets in the way.


Add setters and getters for imageLoadFunction

We have the same for tileLoadFunction so adding here as well makes sense to me

I just added a doc-related comment. Otherwise looks good to me.

thanks @elemoine but not merge as yet until you have released right?

thanks @elemoine but not merge as yet until you have released right?

Others did not care :-) Please merge.


can not show emoji on linux

Does any one knows why?

Even my terminal can printout emoji:

flutter: get projects: 2
flutter: {🚖 工作 2: 2.0, 🍓 生活 0: 0.0}

Most likely the font is missing emojii's.
Make sure you have the font you want to use in the assets folder and linked in pubspec.yaml.
If the font is not in your app, then Flutter will try to use the fonts from the operating system, but you can never be sure your users have the font you want to use. So always bundle fonts with your app.
Please re-open this issue if the problem did not go away after adding the font to your app.

Tested with https://github.com/samuelngs/apple-emoji-linux and  https://github.com/MorbZ/OpenSansEmoji and the emoji where properly displayed.
Still, you have to bundle the font in the app. Like in this example: https://github.com/go-flutter-desktop/examples/blob/768b3c12b46778d78c7bc1edf29b957777be87e6/text_demo/pubspec.yaml#L43-L57

All other font related information can be found here: https://github.com/flutter/flutter/issues/30700#issuecomment-541269268

@pchampio  thanks for your reply. I actually buddled the fonts to my app, and the normal text ca be shown (charactors), only emoji can not shown.
Did u guys means, my custom fonts doesn't contain emoji? I suppose emoji is not related with fonts but system encoded pattern to show. My system is linux, and my linux can show emoji in terminal (as you can see ) at least the terminal shows like a normal emoji rather than boxes.
So, I have already added fonts in my pubspec.yml, what else need I do to make it show? By using default fonts?

Did u guys means, my custom fonts doesn't contain emoji? I suppose emoji is not related with fonts but system encoded pattern to show. My system is linux, and my linux can show emoji in terminal (as you can see ) at least the terminal shows like a normal emoji rather than boxes.

Please read the above comment https://github.com/flutter/flutter/issues/30700#issuecomment-541269268
Here is an example:
https://file.io/2xJLLY7q

https://file.io/EBfmqaQ7 this link shows 404.
So, is because linux can not found the last fallback font which is Robo font on linux?
how to set a fallback font on linux since every font maybe missing and different paltform may have different fallback font.
What's the standard solution for this? I think this is a simple and foundamental using scenarios of apps on desktop.

https://we.tl/t-J5V6fZcKtl

So, is because linux can not found the last fallback font which is Robo font on linux?
how to set a fallback font on linux since every font maybe missing and different paltform may have different fallback font.
What's the standard solution for this? I think this is a simple and foundamental using scenario of apps on desktop.

I don't have answer to your questions, the level of which go-flutter operates isn't on that level.
What's I can say is that bundling a font that has support for emoji and using this font in your applications works.

@pchampio  thanks! weird thing I test again on windows, it can show emoji, if it related about fonts, this shouldn't happen.
Anyway, I will test add fallback fonts on ubuntu


Proposal: New rule for attribution of active/ inactive holidays

A holiday name may change over time and get replaced or discontinued.
The same applies if a holiday is effective from a certain timestamp on.
Currently this is attributed by the active array using {to, from}.
For renaming a name this then will case a duplication of the rule name which then is resolved with a numbering scheme using #1.
This is not ideal.
It is therefore proposed to introduce a set of new rules:
<rule> since YYYY(-MM-DD)?

means that the rule is effective since year
<rule> prior to YYYY(-MM-DD)?

means that the rule is effective prior to year
<rule> since YYYY(-MM-DD)? and prior to YYYY(-MM-DD)?

means that the rule is effective since year and prior to another year

I like this proposal. I also agree that making the month and day portions optional is reasonable as these things commonly change on calendar year boundaries.

Great will work on the change.


pipenv does not work in a pipenv shell

First, thanks for pipenv -- it's a great step forward in Python package management.  I'd like to roll it out for my team, but a few snags are preventing me.
I'd like to be able to run pipenv shell and have it work like a virtualenv; specifically, when I run pipenv xxx in that shell, from any directory, even outside the project directory, I expected it to operate on that virtualenv.  But alas.
Create a new empty pipenv:
$ mkdir proj
$ cd proj
$ pipenv install
Creating a virtualenv for this project…
Using /usr/bin/python (2.7.14) to create virtualenv…
⠋Already using interpreter /usr/bin/python
New python executable in /home/me/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/me/.venv/proj-9zRXrcHj
Creating a Pipfile for this project…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (b44f36)!                                                                                Installing dependencies from Pipfile.lock (b44f36)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run the following:
 $ pipenv shell

Activate that environment:
$ pipenv shell
Spawning environment shell (/bin/bash). Use 'exit' to leave.
$ . /home/me/.venv/proj-9zRXrcHj/bin/activate

Leave the project directory:
(proj-9zRXrcHj) $ cd ..

...And things fall apart:
(proj-9zRXrcHj) $ pipenv graph
Warning: Unable to display currently–installed dependency graph information here. Please run within a Pipenv project.

I expected that to work.
(proj-9zRXrcHj) $ pipenv install requests
Creating a virtualenv for this project…
^C

I expected that to install to the existing virtual environment (/home/me/.venv/proj-9zRXrcHj) instead of create a new one.
And I can't even provide debugging info, because pipenv does not install itself in the virtualenv:
(proj-9zRXrcHj) $ python -m pipenv.help
/home/me/.venv/proj-9zRXrcHj/bin/python: No module named pipenv

It's pipenv 11.9.0 on Redhat 7 with Python 2.7.14.

Pipenv is a project-oriented dependency manager. Other people like to be able to safely move into a new project and create it while still being in a pipenv shell, which has been a requirement in the past -- we've had a few other issues on this in the past and this is a design decision

It will work in all subdirectories of the project however.


[FEAT]: Possibility to "white label" meta information

Pull Request Type

[x] ✨ feat
[ ] 🐛 fix
[ ] ♻ refactor
[ ] 💄 style
[ ] 🔨 chore
[ ] 📝 docs

Relevant Issues

resolves #1373 (except the related closed #1256 that should be another issue)
What is in this change?

Enables possibility to while label metadatas.
Additional Information

Everything works just fine but I'd catch your attention on the way the image that will be displayed on the tab of the window is handled. Currently, it's simply the path to the PNG set in the TAB_IMAGE env var, therefore it will only work if the user sets a remotely served PNG. However, that means it will fetch the PNG every time it updates the DOM. The current implementation is temporary.
In the future, we want to drop our images in the docker volume and the app to retrieve it. This would make the flow way smoother, since the PNG would be available locally. I plan to do just this right after this PR is merged, because this problem is shared for the main app customized logo. We can set it in the UI, but we want to make everything settable as conf, so we don't have to do UI actions every time the container is wiped and re-run. I'll make an issue for this.
Developer Validations

[x] I ran yarn lint from the root of the repo & committed changes
[x] Relevant documentation has been updated
[x] I have tested my code functionality
[x] Docker build succeeds locally

From the current WIP - this breaks the share image and content when the website is shared as a link in most application since the content would need to render first for the new tab content to show

From the current WIP - this breaks the share image and content when the website is shared as a link in most application since the content would need to render first for the new tab content to show

Fair enough. Didn't think it would behave like this.  Does the share feature only relie on <meta> tag ? If it's the case, it would mean i still can set the <title> and <link> as dynamic to custom the visual part. Wht do you think ?

In specific these tags render the share card and image stuff
  <title>AnythingLLM | Your personal LLM trained on anything</title>

  <meta name="title" content="AnythingLLM | Your personal LLM trained on anything">
  <meta name="description" content="AnythingLLM | Your personal LLM trained on anything">

  <!-- Facebook -->
  <meta property="og:type" content="website">
  <meta property="og:url" content="https://useanything.com">
  <meta property="og:title" content="AnythingLLM | Your personal LLM trained on anything">
  <meta property="og:description" content="AnythingLLM | Your personal LLM trained on anything">
  <meta property="og:image"
    content="https://raw.githubusercontent.com/Mintplex-Labs/anything-llm/master/images/promo.png">

  <!-- Twitter -->
  <meta property="twitter:card" content="summary_large_image">
  <meta property="twitter:url" content="https://useanything.com">
  <meta property="twitter:title" content="AnythingLLM | Your personal LLM trained on anything">
  <meta property="twitter:description" content="AnythingLLM | Your personal LLM trained on anything">
  <meta property="twitter:image"
    content="https://raw.githubusercontent.com/Mintplex-Labs/anything-llm/master/images/promo.png">

The reason this issue has not been done yet is that basically the root content header needs to be server-side rendered because link preview will not run client-side JS (which this code currently does correctly right now!) so what happens is when you visit the page in browser the tab and content look OK, but that all goes away when the link is pasted into a DM, Slack message, etc - anywhere link previews are supported

Alright, i figured out another way to achieve this customization (overwrite the index.html with a custom one inside the container), i'll go with that for the moment. Closing the PR.

@doumencp I can look into this as well. I think just serves the index.html from the server / in production mode will allow a rewrite of the header objects and since on the server side we have the DB - we can query + serve and cache it for faster page responses


Action generators

Convert @middleware decorated functions to monadic action generator style middleware. Generators are more Pythonic and produce simpler Tracebacks
@middleware
def duplicate(store, next_dispatch, action):
     next_dispatch(action)
     next_dispatch(action)

Can now be written:
def duplicate(store, action):
    yield action
    yield action

This syntax is more familiar to Pythonistas and has less boilerplate code, no decorator or explicit usage of the "next_dispatch" function in the middleware pipeline

Another major improvement with this change is the ability to test middleware directly
def test_middleware():
    assert list(middleware(store, action)) == []

Excellent \o/


core.link: Streamline counters [DRAFT]

Reduce the amount of counter-related code and data in struct link.
The ring indexes (read, write) and now upgraded to 64-bit integers that do not wrap around [*]. This means that we don't need to use separate packet tx/rx counters because the ring indexes serve that purpose directly. (We do have to be careful to handle wrap-around when actually looking up elements in the ring based on the index.)
The byte counters are simply removed. Do we really need them? (How about if we tracked only packets-per-second plus perhaps a system-wide average packet size?)
Dropped packets are still counted directly because this is important information and I don't immediately see a way to derive it indirectly.
[*] 64-bit packet counters really should not wrap around. Even at one billion packets per second it would take more than 500 years. However, if this is not a safe assumption then we could detect wrap around and account for it on the counters.
(Google: "2^64/(1 billion per second)" => "584.554531 years".)
@eugeneia @javierguerragiraldez @alexandergall what do you guys thing about this idea? I mean in terms of information loss, impact on the code, and conflicts with other open PRs?

I think full() is wrong when read >= size-1.  this should work:
function full (r)
    return r.write == r.read + size - 1
end

But in all, is there a measurable difference?  I remember doing some benchmarks and finding no difference between
while not empty(in) and not full(out) do ... end

and
for _ = 1, min(nreadable(in),nwritable(out)) do ... end

even if the first form does a check on every iteration, while the second one does only at the start and it's an integer count from there on.  I guess the index pointers become essentially free once they get cached in.

I am always for simplicity but I wonder what is the benefit here? Did rx/txbytes actually cost us much? It seems like an interesting metric to me so I would consider keeping it.

@eugeneia That is a fair point. I see no urgent need to make this change now. We could revisit this idea in the future if/when we are trying to squeeze more performance out of the basic app network and see if it really helps.
I close this PR for now.


General decoupling in functions

The functions of the main class are decoupled into modules to make it convenient for development.

Hi, Lorenzo. We correct the linter issues, can you check it please 👍


Not able to connect using the generated URL in browser

I am a novice in bot programming.
I followed all  the steps mentioned following URL:
https://docs.botframework.com/en-us/csharp/builder/sdkreference/gettingstarted.html
Once i run the program in visual studio, it gives a browser output mentioning the URL and ideally it should connect to port 3978, but in my case it deviates to 3979.
In bot emulator i paste the url i received in browser, i.e:
http://localhost:3979/api/messages
I hit the enter key and it displays an error:
[12:01:10] -> POST connect ECONNREFUSED <IP_ADDRESS>:3979
What should i rectify this error?

This is the error you would see if your bot was not running, or was running but not listening on port 3979. If you haven't already done so, could you try again, making certain your bot is indeed running at the moment you try to connect from the emulator?

Hi rahulswimmer,
Change in Visual Studio in the project properties the running port for the Bot: under the Web tab of the properties change the port into 3978 and click "Create virtual directory". Run the project and reconfigure the Emulator.
It worked for me.

@rahulswimmer Closing due to inactivity. If you're still seeing this issue, please open a new one. Thank you.

I am unable to connect to localhost. I changed the port into 3978 and created virtual directory. can you tell me how to reconfigure the emulator? When I run the project I get debugger not installed properly. I am using windows 7.

I am also getting an error while in log while installing emulator

i had the same issue and i followed the instruction here and it worked fine:
https://docs.microsoft.com/en-us/azure/bot-service/bot-service-debug-emulator?view=azure-bot-service-3.0

Hi, I think the problem is caused when we start to change things to configure ngrok. For me it worked correctly when returning to Settings and assigning the values as they are in the image.


Fix SimpleKVDiskStore pipeline when expirations are set, for Python 3.12

The only real change here is in cassiopeia-diskstore/cassiopeia_diskstore/common.py, updating it to work on Python 3.12.
The problem only appeared if you were to specify expirations for data.
The linked documentation should have been here instead.

I'm not clear on if this worked in older versions, it doesn't look like it did even though it looks like there was an attempt at making it work with wrapping the parameter in dict() to get a copy.
(580312a was already merged, see: https://github.com/meraki-analytics/cassiopeia-datastores/pull/27). Will have to look at cleaning up this branch later)

Thank you for the PR!


sure fire plugin error when running tests with mvn

The backend-service tests inside IDEA work fine.
However when you try to run tests using mvn package --file backend-service/pom.xml or any mvn build cmd clean install ect ...
You get the following error.
Error:  Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project annotation: There are test failures.
Error:
Error:  Please refer to /home/runner/work/occurrence-annotation/occurrence-annotation/backend-service/target/surefire-reports for the individual test results.
Error:  Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
Error: [ERROR] There was an error in the forked process
Error:  org/junit/platform/engine/EngineExecutionListener.NOOP
Error:  org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
Error:  org/junit/platform/engine/EngineExecutionListener.NOOP

Here is an example from a github actions run.
https://github.com/gbif/occurrence-annotation/actions/runs/6970047769/job/18967176203#step:6:129

Can you please check this works for you now @jhnwllr and close this if so?
Thanks

Thanks @timrobertson100
It works now!
https://github.com/gbif/occurrence-annotation/actions/runs/6971226073/job/18970802024


Improvisation in UI/UX of help page

Is your feature request related to a problem? Please describe.
Currently the help page UI is quite plain.
Describe the solution you'd like
Contributors are welcome to come up with ideas to improve help page UI
Additional context
none

we need to improve the UI Of the Help Page? Can You Explain a bit more!

Ya @skpandey885 it's kinda open ended issue where you need to propose your design for help page and after confirmation you can go on implementing it and raise a pr.

@QAZIMAAZARSHAD please do review this issue.

@skpandey885 if you have any ideas to improve the design of the page then go ahead.

@skpandey885 if you have any ideas to improve the design of the page then go ahead.

OK!

Hey! In The Reach Us Section Should I change the Address /phone no. to some real random phone number and address or should I keep the same!

Sure.

@skpandey885 any updates?

Im on it bhaiya! Actually my final exams are going on so i am a bit busy with it!

@nlok5923 @QAZIMAAZARSHAD  Bhaiya I've Been Tested Positive (Covid) Today! So I won't be able to work on these issues! Sorry For The Inconvenience Caused!

@skpandey885 no problem just take care .

@nlok5923 @QAZIMAAZARSHAD Bhaiya I've Been Tested Positive (Covid) Today! So I won't be able to work on these issues! Sorry For The Inconvenience Caused!

Get well soon


Remote WinAppDriver with Firewall Disabled

Hi,
I am trying to run WinAppDriver on a remote (windows10) machine that has firewall disabled. I have followed the instructions here (https://github.com/Microsoft/WinAppDriver) for running on a remote machine and even though firewall is disabled, my connection would still get denied. So I've created the TCP rule mentioned in the tutorial but I still get the following message when I try to run it from another machine:
"ERROR
======================================================================
ERROR: setUpClass (main.ZuneTest)
Traceback (most recent call last):
File "C:\Python36\lib\urllib\request.py", line 1318, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "C:\Python36\lib\http\client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Python36\lib\http\client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Python36\lib\http\client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Python36\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\Python36\lib\http\client.py", line 964, in send
self.connect()
File "C:\Python36\lib\http\client.py", line 936, in connect
(self.host,self.port), self.timeout, self.source_address)
File "C:\Python36\lib\socket.py", line 724, in create_connection
raise err
File "C:\Python36\lib\socket.py", line 713, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_zune.py", line 30, in setUpClass
desired_capabilities= desired_caps)
File "C:\Python36\lib\site-packages\appium\webdriver\webdriver.py", line 36, in init
super(WebDriver, self).init(command_executor, desired_capabilities, browser_profile, proxy, keep_alive)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 154, in init
self.start_session(desired_capabilities, browser_profile)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 243, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 310, in execute
response = self.command_executor.execute(driver_command, params)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 466, in execute
return self._request(command_info[0], url, body=data)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 528, in _request
resp = opener.open(request, timeout=self._timeout)
File "C:\Python36\lib\urllib\request.py", line 526, in open
response = self._open(req, data)
File "C:\Python36\lib\urllib\request.py", line 544, in _open
'_open', req)
File "C:\Python36\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\Python36\lib\urllib\request.py", line 1346, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Python36\lib\urllib\request.py", line 1320, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10061] No connection could be made because the target machine actively refused it>

Ran 0 tests in 1.160s
FAILED (errors=1)"
After enabling the Firewall, I am still not able to run my test, but i'm getting the below error message instead:
File "C:\Python36\lib\urllib\request.py", line 1320, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>
My Questions are:

Are there any additional logs that WinAppDriver.exe spits out where I could potentially get more information on the failures?
Are there any registry settings that would prevent this connection that I need to change?

Both Machines, test and host are connected to the same LAN network, but this is a secure network as it is company network, so i'm not sure what could be the settings that might be blocking this call. Any help is appreciated.
Thanks!

I didn't need to have wd/hub;
Below worked just fine, the ip and port (be mindful winapp driver needs to be started elevated)
Example:  "c:\Program Files (x86)\Windows Application Driver\WinAppDriver.exe" <IP_ADDRESS> 4723

I didn't need to have wd/hub;
Below worked just fine, the ip and port (be mindful winapp driver needs to be started elevated)
Example: "c:\Program Files (x86)\Windows Application Driver\WinAppDriver.exe" <IP_ADDRESS> 4723

this solution works


Fix. can't enter the wrong environ container

When I enter the nginx container it raise an expection
env: master: No such file or directory

then I cat /proc/$PID/environ, I find nginx set it's environ master process nginx,  it's a wrong env value

I think this is no longer necessary, thanks to importenv (3d5dfca). Thank you!


Warning message in secret store token metadata file

🐞 Bug Report
Affected Services [REQUIRED]
security-secretstore-setup (security-file-token-provider)
Description and Minimal Reproduction [REQUIRED]
The following non-fatal warnings are logged in the create token response from security-file-token-provider.  This may be due to drift in the create token parameters over time.
🔥 Exception or Error
The following warning appears in secrets-token.json:

"warnings":["Endpoint ignored these unrecognized parameters: [meta]"]

🌍  Your Environment
Tip version of EdgeX as of the date of this issue, on Docker, on Ubuntu.

Linked issue: https://github.com/hashicorp/vault/issues/18550

According to linked issue and https://github.com/hashicorp/vault/blob/468f66bf386147002c55389b2e01516556781b22/changelog/18556.txt
auth/token: Fix parsing of `auth/token/create` fields to avoid incorrect warnings about ignored parameters

This error is spurious and does not indicate a real issue.
Will not fix.


Unexpected token : in JSON at position 426

After running the following command :
npm start -- --key="sk-_your-token_"  -o deobfuscated.js obfuscated.js
I get the following error:
SyntaxError: Unexpected token : in JSON at position 426     at JSON.parse (<anonymous>)     at codeToVariableRenames (file:///C:/Users/Alexander/Documents/GitHub/humanify/src/openai/openai.ts:76:59)     at processTicksAndRejections (node:internal/process/task_queues:95:5)     at async file:///C:/Users/Alexander/Documents/GitHub/humanify/src/openai/openai.ts:21:23     at async Promise.all (index 4)     at async mapPromisesParallel (file:///C:/Users/Alexander/Documents/GitHub/humanify/src/openai/run-promises-in-parallel.ts:17:5)     at async client.createChatCompletion.model (file:///C:/Users/Alexander/Documents/GitHub/humanify/src/openai/openai.ts:20:5)     at async file:///C:/Users/Alexander/Documents/GitHub/humanify/src/index.ts:68:25

I'm pretty sure this is since openai does not guarantee that the function calls are valid json. Should probably implement a quick retry logic for it 🤔

Can you use the new JSON mode or tool choice to force it?

https://platform.openai.com/docs/guides/text-generation/json-mode

A common way to use Chat Completions is to instruct the model to always return a JSON object that makes sense for your use case, by specifying this in the system message. While this does work in some cases, occasionally the models may generate output that does not parse to valid JSON objects.
To prevent these errors and improve model performance, when using gpt-4o, gpt-4-turbo, or gpt-3.5-turbo, you can set response_format to { "type": "json_object" } to enable JSON mode. When JSON mode is enabled, the model is constrained to only generate strings that parse into valid JSON object.

https://platform.openai.com/docs/guides/function-calling

https://platform.openai.com/docs/guides/function-calling/function-calling-behavior

Function calling behavior
The default behavior for tool_choice is tool_choice: "auto". This lets the model decide whether to call functions and, if so, which functions to call.
We offer three ways to customize the default behavior depending on your use case:

To force the model to always call one or more functions, you can set tool_choice: "required". The model will then select which function(s) to call.
To force the model to call only one specific function, you can set tool_choice: {"type": "function", "function": {"name": "my_function"}}.
To disable function calling and force the model to only generate a user-facing message, you can set tool_choice: "none"

Potentially related:

https://github.com/jehna/humanify/issues/17

Can you use the new JSON mode or tool choice or similar to force it?

I'm not sure what version of the SDK tool_choice became available in. This project seems to currently be using openai 3.3.0, whereas the latest version is 4.52.3 (at time of writing). I created a more specific issue about upgrading the library, which may end up being a prerequisite to using tool_choice/similar:

https://github.com/jehna/humanify/issues/19

OpenAI API now guarantees structured output:
https://openai.com/index/introducing-structured-outputs-in-the-api/
(should fix this issue properly)

This should now be fixed in v2 since there's the long awaited JSON mode with the new structured outputs.  Please take a look and repoen if anything comes up
Originally posted by @jehna in https://github.com/jehna/humanify/issues/22#issuecomment-2282876269


remove includes which are never used

This will allow people to run gbans on non-TF2 games

Why this is not merged?
On Sat, Jul 16, 2022, 11:41 PM Leigh MacDonald @.***>
wrote:

Closed #198 https://github.com/leighmacdonald/gbans/pull/198.
—
Reply to this email directly, view it on GitHub
https://github.com/leighmacdonald/gbans/pull/198#event-7007031268, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AF5NYR5V47VI3ZT3WAWIC3TVUOTHXANCNFSM5FHFYPSQ
.
You are receiving this because you authored the thread.Message ID:
@.***>


Error running on 8xH100, works on 4xH100?

Am attempting to reproduce your results, as was wondering if you have hit the following issue before. I am using a Lambda Labs 8xH100 SXM5 instance, and run the following commands from a fresh instance of the repository:
python -m venv nanogpt_env
source nanogpt_env/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
python data/cached_fineweb10B.py 18
./run.sh

On a 4xH100 SXM5 instance from Lambda, everything runs fine and I can successfully reproduce everything. Similarly on an 8xA100 instance, it works as well. However, solely on the 8xH100 instances, training fails in the child processes with:
[I1107 01:20:27.770733060 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file

Have you seen this issue before? I suspect a NCCL issue but I have never run across this issue before. Here is the full set of logs with detailed debugging:
[I1107 01:20:06.503069695 debug.cpp:49] [c10d] The debug level is set to DETAIL.
I1107 01:20:06.697000 7974 torch/distributed/run.py:897]
I1107 01:20:06.697000 7974 torch/distributed/run.py:897] **************************************
I1107 01:20:06.697000 7974 torch/distributed/run.py:897] Rendezvous info:
I1107 01:20:06.697000 7974 torch/distributed/run.py:897] --rdzv-backend=c10d --rdzv-endpoint=localhost:0 --rdzv-id=c0d436eb-46c3-4d17-8205-20cd784b1819
I1107 01:20:06.697000 7974 torch/distributed/run.py:897] **************************************
I1107 01:20:06.697000 7974 torch/distributed/run.py:897]
W1107 01:20:06.697000 7974 torch/distributed/run.py:793]
W1107 01:20:06.697000 7974 torch/distributed/run.py:793] *****************************************
W1107 01:20:06.697000 7974 torch/distributed/run.py:793] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
W1107 01:20:06.697000 7974 torch/distributed/run.py:793] *****************************************
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194] Starting elastic_operator with launch configs:
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   entrypoint       : train_gpt2.py
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   min_nodes        : 1
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   max_nodes        : 1
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   nproc_per_node   : 8
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   run_id           : c0d436eb-46c3-4d17-8205-20cd784b1819
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   rdzv_backend     : c10d
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   rdzv_endpoint    : localhost:0
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   rdzv_configs     : {'timeout': 900}
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   max_restarts     : 0
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   monitor_interval : 0.1
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   log_dir          : /tmp/torchelastic_jjl_2a2_
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]   metrics_cfg      : {}
I1107 01:20:06.698000 7974 torch/distributed/launcher/api.py:194]
[I1107 01:20:06.063724695 TCPStore.cpp:298] [c10d - debug] The server has started on port = 38343.
[I1107 01:20:06.063773511 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:06.063792375 TCPStoreLibUvBackend.cpp:1100] [c10d - debug] Uv main loop running
[I1107 01:20:06.063888473 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:06.064337111 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=11, addr=[localhost]:58102, remote=[localhost]:38343).
[I1107 01:20:06.064459433 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
I1107 01:20:06.712000 7974 torch/distributed/elastic/agent/server/api.py:845] [default] starting workers for entrypoint: python
I1107 01:20:06.712000 7974 torch/distributed/elastic/agent/server/api.py:662] [default] Rendezvous'ing worker group
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525] [default] Rendezvous complete for workers. Result:
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   restart_count=0
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   master_addr=localhost
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   master_port=38343
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   group_rank=0
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   group_world_size=1
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   local_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   role_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   global_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   role_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]   global_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:525]
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/api.py:670] [default] Starting worker group
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/local_elastic_agent.py:291] use_agent_store: True
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/local_elastic_agent.py:192] Environment variable 'TORCHELASTIC_ENABLE_FILE_TIMER' not found. Do not start FileTimerServer.
I1107 01:20:06.852000 7974 torch/distributed/elastic/agent/server/local_elastic_agent.py:229] Environment variable 'TORCHELASTIC_HEALTH_CHECK_PORT' not found. Do not start health check.
[I1107 01:20:07.736028383 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:07.737035524 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:07.737814512 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:07.738539896 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:07.738685015 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:07.738767458 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:07.766481046 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:07.770846199 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.510568923 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.512194377 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.512360585 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.512420116 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.515876087 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.556883751 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.632797551 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:09.680358065 debug.cpp:49] [c10d] The debug level is set to DETAIL.
[I1107 01:20:17.526118201 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.526276937 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.526641718 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38136, remote=[localhost]:38343).
[I1107 01:20:17.526769432 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.527218210 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL initialization options: size: 8, global rank: 0, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.527230319 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
[I1107 01:20:17.669525188 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.669655222 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.669907427 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38142, remote=[localhost]:38343).
[I1107 01:20:17.670060963 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.670453419 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 3] ProcessGroupNCCL initialization options: size: 8, global rank: 3, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.670464962 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 3] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
[I1107 01:20:17.721125753 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.721298489 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.721657406 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38148, remote=[localhost]:38343).
[I1107 01:20:17.721805128 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.722298066 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 7] ProcessGroupNCCL initialization options: size: 8, global rank: 7, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.722310536 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 7] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
[I1107 01:20:17.771529419 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.771664852 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.771896000 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38160, remote=[localhost]:38343).
[I1107 01:20:17.772031020 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.772475984 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 4] ProcessGroupNCCL initialization options: size: 8, global rank: 4, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.772488115 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 4] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
[I1107 01:20:17.851667526 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.851804393 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.852143269 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38168, remote=[localhost]:38343).
[I1107 01:20:17.852291631 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.852724194 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 6] ProcessGroupNCCL initialization options: size: 8, global rank: 6, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.852736725 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 6] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
[I1107 01:20:17.003884475 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.004021299 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.004267540 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38172, remote=[localhost]:38343).
[I1107 01:20:17.004397260 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.004723396 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 5] ProcessGroupNCCL initialization options: size: 8, global rank: 5, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.004735292 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 5] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
[I1107 01:20:17.121139412 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.121259659 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.121498873 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38174, remote=[localhost]:38343).
[I1107 01:20:17.121628754 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.121965262 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 1] ProcessGroupNCCL initialization options: size: 8, global rank: 1, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.121977717 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 1] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
[I1107 01:20:17.177078606 socket.cpp:773] [c10d - debug] The client socket will attempt to connect to an IPv6 address of (localhost, 38343).
[I1107 01:20:17.177210465 socket.cpp:847] [c10d - trace] The client socket is attempting to connect to [localhost]:38343.
[I1107 01:20:17.177452073 socket.cpp:938] [c10d] The client socket has connected to [localhost]:38343 on SocketImpl(fd=47, addr=[localhost]:38182, remote=[localhost]:38343).
[I1107 01:20:17.177575988 TCPStore.cpp:334] [c10d - debug] TCP client connected to host localhost:38343
[I1107 01:20:17.178051212 ProcessGroupNCCL.cpp:905] [PG ID 0 PG GUID 0 Rank 2] ProcessGroupNCCL initialization options: size: 8, global rank: 2, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0
[I1107 01:20:17.178064033 ProcessGroupNCCL.cpp:914] [PG ID 0 PG GUID 0 Rank 2] ProcessGroupNCCL environments: NCCL version: 2.21.5, TORCH_NCCL_ASYNC_ERROR_HANDLING: 1, TORCH_NCCL_DUMP_ON_TIMEOUT: 1, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 1, TORCH_NCCL_ENABLE_TIMING: 1, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: DETAIL, TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK: 0, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1
using device: cuda:0
Training DataLoader: total number of tokens:<PHONE_NUMBER>3 across 103 files
Validation DataLoader: total number of tokens: 100000000 across 1 files
using device: cuda:5
using device: cuda:6
using device: cuda:7
using device: cuda:1
using device: cuda:4
using device: cuda:2
using device: cuda:3
[rank0]:[I1107 01:20:21.406778430 ProcessGroupWrapper.cpp:587] [Rank 0] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
[rank1]:[I1107 01:20:21.078965995 ProcessGroupWrapper.cpp:587] [Rank 1] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
[rank5]:[I1107 01:20:21.081068938 ProcessGroupWrapper.cpp:587] [Rank 5] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
[rank3]:[I1107 01:20:21.099637725 ProcessGroupWrapper.cpp:587] [Rank 3] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
[rank2]:[I1107 01:20:21.108116100 ProcessGroupWrapper.cpp:587] [Rank 2] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
[rank4]:[I1107 01:20:21.109547102 ProcessGroupWrapper.cpp:587] [Rank 4] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
[rank7]:[I1107 01:20:21.117719132 ProcessGroupWrapper.cpp:587] [Rank 7] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
[rank6]:[I1107 01:20:21.143227051 ProcessGroupWrapper.cpp:587] [Rank 6] Running collective: CollectiveFingerPrint(SequenceNumber=0, OpType=ALLGATHER, TensorShape=[1], TensorDtypes=Long, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (default), pinned_memory=false (default), memory_format=(nullopt)))
192-222-52-234:8043:8043 [0] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8043:8043 [0] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8043:8043 [0] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8043:8043 [0] NCCL INFO NET/Plugin: Using internal network plugin.
[rank0]:[I1107 01:20:21.159238472 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL broadcast unique ID through store took 0.039149 ms
192-222-52-234:8043:8043 [0] NCCL INFO cudaDriverVersion 12040
NCCL version 2.21.5+cuda12.4
[rank7]:[I1107 01:20:21.159378611 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 7] ProcessGroupNCCL broadcast unique ID through store took 13.4969 ms
[rank6]:[I1107 01:20:21.159392707 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 6] ProcessGroupNCCL broadcast unique ID through store took 13.4834 ms
[rank2]:[I1107 01:20:21.159397933 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 2] ProcessGroupNCCL broadcast unique ID through store took 13.4608 ms
[rank4]:[I1107 01:20:21.159425767 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 4] ProcessGroupNCCL broadcast unique ID through store took 13.4601 ms
[rank1]:[I1107 01:20:21.159422084 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 1] ProcessGroupNCCL broadcast unique ID through store took 13.482 ms
[rank5]:[I1107 01:20:21.159443397 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 5] ProcessGroupNCCL broadcast unique ID through store took 13.496 ms
[rank3]:[I1107 01:20:21.159466617 ProcessGroupNCCL.cpp:2262] [PG ID 0 PG GUID 0 Rank 3] ProcessGroupNCCL broadcast unique ID through store took 13.4963 ms
192-222-52-234:8046:8046 [3] NCCL INFO cudaDriverVersion 12040
192-222-52-234:8046:8046 [3] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8046:8046 [3] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8046:8046 [3] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8046:8046 [3] NCCL INFO NET/Plugin: Using internal network plugin.
192-222-52-234:8050:8050 [7] NCCL INFO cudaDriverVersion 12040
192-222-52-234:8050:8050 [7] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8050:8050 [7] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8050:8050 [7] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8050:8050 [7] NCCL INFO NET/Plugin: Using internal network plugin.
192-222-52-234:8044:8044 [1] NCCL INFO cudaDriverVersion 12040
192-222-52-234:8044:8044 [1] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8044:8044 [1] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8044:8044 [1] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8044:8044 [1] NCCL INFO NET/Plugin: Using internal network plugin.
192-222-52-234:8049:8049 [6] NCCL INFO cudaDriverVersion 12040
192-222-52-234:8049:8049 [6] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8049:8049 [6] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8049:8049 [6] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8049:8049 [6] NCCL INFO NET/Plugin: Using internal network plugin.
192-222-52-234:8045:8045 [2] NCCL INFO cudaDriverVersion 12040
192-222-52-234:8045:8045 [2] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8045:8045 [2] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8045:8045 [2] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8045:8045 [2] NCCL INFO NET/Plugin: Using internal network plugin.
192-222-52-234:8047:8047 [4] NCCL INFO cudaDriverVersion 12040
192-222-52-234:8047:8047 [4] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8047:8047 [4] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8047:8047 [4] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8047:8047 [4] NCCL INFO NET/Plugin: Using internal network plugin.
192-222-52-234:8048:8048 [5] NCCL INFO cudaDriverVersion 12040
192-222-52-234:8048:8048 [5] NCCL INFO Bootstrap : Using eno1:<IP_ADDRESS><0>
192-222-52-234:8048:8048 [5] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
192-222-52-234:8048:8048 [5] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
192-222-52-234:8048:8048 [5] NCCL INFO NET/Plugin: Using internal network plugin.
192-222-52-234:8043:8718 [0] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8043:8718 [0] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8043:8718 [0] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8043:8718 [0] NCCL INFO Using network Socket
192-222-52-234:8046:8719 [3] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8046:8719 [3] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8046:8719 [3] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8046:8719 [3] NCCL INFO Using network Socket
192-222-52-234:8050:8720 [7] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8050:8720 [7] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8050:8720 [7] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8050:8720 [7] NCCL INFO Using network Socket
192-222-52-234:8044:8721 [1] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8044:8721 [1] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8044:8721 [1] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8044:8721 [1] NCCL INFO Using network Socket
192-222-52-234:8049:8722 [6] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8049:8722 [6] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8049:8722 [6] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8049:8722 [6] NCCL INFO Using network Socket
192-222-52-234:8045:8723 [2] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8045:8723 [2] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8045:8723 [2] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8045:8723 [2] NCCL INFO Using network Socket
192-222-52-234:8048:8725 [5] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8047:8724 [4] NCCL INFO NCCL_IB_DISABLE set by environment to 1.
192-222-52-234:8047:8724 [4] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8048:8725 [5] NCCL INFO NET/Socket : Using [0]eno1:<IP_ADDRESS><0>
192-222-52-234:8047:8724 [4] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8047:8724 [4] NCCL INFO Using network Socket
192-222-52-234:8048:8725 [5] NCCL INFO Using non-device net plugin version 0
192-222-52-234:8048:8725 [5] NCCL INFO Using network Socket
192-222-52-234:8044:8721 [1] NCCL INFO ncclCommInitRank comm 0x5bc336192880 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 62000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8043:8718 [0] NCCL INFO ncclCommInitRank comm 0x5c753efeb670 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 61000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8049:8722 [6] NCCL INFO ncclCommInitRank comm 0x5ae4fcb47b70 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId 6c000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8048:8725 [5] NCCL INFO ncclCommInitRank comm 0x627a26551700 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 6b000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8047:8724 [4] NCCL INFO ncclCommInitRank comm 0x5d393ac0a2c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 6a000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8050:8720 [7] NCCL INFO ncclCommInitRank comm 0x57011fd97d00 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId 6d000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8046:8719 [3] NCCL INFO ncclCommInitRank comm 0x6453031ada30 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 64000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8045:8723 [2] NCCL INFO ncclCommInitRank comm 0x579679f0ff50 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 63000 commId 0x479ea98eb36a3aad - Init START
192-222-52-234:8047:8724 [4] NCCL INFO MNNVL busId 0x6a000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
192-222-52-234:8044:8721 [1] NCCL INFO MNNVL busId 0x62000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
192-222-52-234:8049:8722 [6] NCCL INFO MNNVL busId 0x6c000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
192-222-52-234:8043:8718 [0] NCCL INFO MNNVL busId 0x61000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
192-222-52-234:8050:8720 [7] NCCL INFO MNNVL busId 0x6d000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
192-222-52-234:8045:8723 [2] NCCL INFO MNNVL busId 0x63000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
192-222-52-234:8046:8719 [3] NCCL INFO MNNVL busId 0x64000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
192-222-52-234:8048:8725 [5] NCCL INFO MNNVL busId 0x6b000 fabric UUID 0.0 cliqueId 0x0 state 3 healthMask 0x0
[I1107 01:20:27.770733060 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:27.772453248 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:27.785201265 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:27.796137449 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:27.796739768 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:27.798358238 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:27.804740529 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:27.804893941 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
W1107 01:20:27.791000 7974 torch/distributed/elastic/multiprocessing/api.py:897] Sending process 8044 closing signal SIGTERM
W1107 01:20:27.794000 7974 torch/distributed/elastic/multiprocessing/api.py:897] Sending process 8045 closing signal SIGTERM
W1107 01:20:27.794000 7974 torch/distributed/elastic/multiprocessing/api.py:897] Sending process 8046 closing signal SIGTERM
W1107 01:20:27.796000 7974 torch/distributed/elastic/multiprocessing/api.py:897] Sending process 8047 closing signal SIGTERM
W1107 01:20:27.797000 7974 torch/distributed/elastic/multiprocessing/api.py:897] Sending process 8048 closing signal SIGTERM
W1107 01:20:27.800000 7974 torch/distributed/elastic/multiprocessing/api.py:897] Sending process 8049 closing signal SIGTERM
W1107 01:20:27.800000 7974 torch/distributed/elastic/multiprocessing/api.py:897] Sending process 8050 closing signal SIGTERM
E1107 01:20:28.349000 7974 torch/distributed/elastic/multiprocessing/api.py:869] failed (exitcode: -11) local_rank: 0 (pid: 8043) of binary: /home/ubuntu/envs/nanogpt/bin/python
I1107 01:20:28.501000 7974 torch/distributed/elastic/multiprocessing/errors/__init__.py:368] ('local_rank %s FAILED with no error file. Decorate your entrypoint fn with @record for traceback info. See: https://pytorch.org/docs/stable/elastic/errors.html', 0)
Traceback (most recent call last):
  File "/home/ubuntu/envs/nanogpt/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/envs/nanogpt/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 355, in wrapper
    return f(*args, **kwargs)
  File "/home/ubuntu/envs/nanogpt/lib/python3.10/site-packages/torch/distributed/run.py", line 919, in main
    run(args)
  File "/home/ubuntu/envs/nanogpt/lib/python3.10/site-packages/torch/distributed/run.py", line 910, in run
    elastic_launch(
  File "/home/ubuntu/envs/nanogpt/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 138, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/ubuntu/envs/nanogpt/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 269, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
======================================================
train_gpt2.py FAILED
------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-11-07_01:20:27
  host      : 192-222-52-234
  rank      : 0 (local_rank: 0)
  exitcode  : -11 (pid: 8043)
  error_file: <N/A>
  traceback : Signal 11 (SIGSEGV) received by PID 8043
======================================================
[I1107 01:20:28.897510354 TCPStoreLibUvBackend.cpp:119] [c10d - debug] Read callback failed. code:-4095 name:EOF desc:end of file
[I1107 01:20:28.897597960 TCPStoreLibUvBackend.cpp:1033] [c10d - debug] Store exit requested

[I1107 01:20:28.897612013 TCPStoreLibUvBackend.cpp:1103] [c10d - debug] UV main loop done: res:1
[I1107 01:20:28.897621027 TCPStoreLibUvBackend.cpp:1109] [c10d - debug] Walking live handles prior to closing clients
[I1107 01:20:28.897626752 TCPStoreLibUvBackend.cpp:1090] [c10d - debug] UV live handle type 12 active:1 is-closing:0
[I1107 01:20:28.897642077 TCPStoreLibUvBackend.cpp:1119] [c10d - debug] Walking live handles after closing clients
[I1107 01:20:28.897649821 TCPStoreLibUvBackend.cpp:1090] [c10d - debug] UV live handle type 12 active:0 is-closing:1
[I1107 01:20:28.897653480 TCPStoreLibUvBackend.cpp:1128] [c10d] uv_loop_close failed with:-16 errn:EBUSY desc:resource busy or locked
[I1107 01:20:28.897679564 TCPStoreLibUvBackend.cpp:1138] [c10d] uv_loop cleanup finished.
(nanogpt) ubuntu@192-222-52-234:~/thinky-north-texas/git_clones/modded-nanogpt$ pip list packages
Package                  Version
------------------------ ----------
certifi                  2024.8.30
charset-normalizer       3.4.0
filelock                 3.16.1
fsspec                   2024.10.0
huggingface-hub          0.26.2
idna                     3.10
Jinja2                   3.1.4
MarkupSafe               3.0.2
mpmath                   1.3.0
networkx                 3.4.2
numpy                    2.1.3
nvidia-cublas-cu12       <IP_ADDRESS>
nvidia-cuda-cupti-cu12   12.4.127
nvidia-cuda-nvrtc-cu12   12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12        <IP_ADDRESS>
nvidia-cufft-cu12        <IP_ADDRESS>
nvidia-curand-cu12       <IP_ADDRESS>
nvidia-cusolver-cu12     <IP_ADDRESS>
nvidia-cusparse-cu12     <IP_ADDRESS>
nvidia-nccl-cu12         2.21.5
nvidia-nvjitlink-cu12    12.4.127
nvidia-nvtx-cu12         12.4.127
packaging                24.1
pip                      24.3.1
PyYAML                   6.0.2
requests                 2.32.3
setuptools               59.6.0
sympy                    1.13.1
torch                    2.5.1
tqdm                     4.67.0
triton                   3.1.0
typing_extensions        4.12.2
urllib3                  2.2.3

I haven't run into this, although I'm not using lambda instances right now. Maybe try downgrading PyTorch to 2.5.0?

Tried a few different torch versions, but same issue. It also repro's in a minimal torch.distributed example, so I am going to chalk this up to Lambda 8xH100's being janky, and will close it out. Thanks for posting your results, this is fun to watch!

FWIW, this does appear to be a NCCL issue on these machines. Upgrading nvidia-nccl-cu12 using pip install --upgrade nvidia-nccl-cu12, even though this will complain about incompatible torch version (5.1.0 fwiw), works and allows training to both complete and reproduce your training speed results.


python3Packages.amqtt: 0.10.0 -> unstable-2022-01-11

Motivation for this change
Fix build (https://hydra.nixos.org/build/164497056)
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 155008 run on x86_64-linux 1

17 packages built:

home-assistant
python310Packages.amqtt
python310Packages.consonance
python310Packages.dissononce
python310Packages.junos-eznc
python310Packages.pytest-logdog
python310Packages.roombapy
python310Packages.transitions
python310Packages.yowsup
python39Packages.amqtt
python39Packages.consonance
python39Packages.dissononce
python39Packages.junos-eznc
python39Packages.pytest-logdog
python39Packages.roombapy
python39Packages.transitions
python39Packages.yowsup


chore: Colin Review Changes 1

Description
Changes from first review of @colinnielsen
Checklist:

[x] My commit message follows the Conventional Commits specification
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have tested my code
[x] My changes generate no new warnings
[x] My PR is rebased off the most recent main or appropriate feature branch
[x] My PR is opened against the main or appropriate feature branch

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

17 of 17 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.002%) to 98.481%

Totals

Change from base Build<PHONE_NUMBER>:
0.002%

Covered Lines:
615

Relevant Lines:
618

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

17 of 17 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.002%) to 98.481%

Totals

Change from base Build<PHONE_NUMBER>:
0.002%

Covered Lines:
615

Relevant Lines:
618

💛  - Coveralls


Possible to drag freeform poly?

Looks amazing so far.  Kudos on achieving accessible sophistication.  I noticed that a freeform poly doesn't display a drag node in edit mode like circles and squares.  Any way to enable that?

Hi, thanks. However, my part is only a small addition to Leaflet.Draw; namely the possibility to load/edit svg images. The drawing/editing of polylines, polygon, square and circle are part of the main Leaflet.Draw. Therefore I think main credit, but also this issue is related to that library. My code does not contain anything specific regarding the polyline/polygon. Therefore I will close this issue, as I cannot fix it. I suggest you open the issue in Leaflet.Draw instead here https://github.com/Leaflet/Leaflet.draw


Callbacks (hasPermission, register) are not called on iOs 13

Expected Behaviour
The promise hasPermission must at some point answer something, but nothing happens.
Actual Behaviour
console.log('before permission');

try {
       this.push.hasPermission()
            .then((res: any) => {
                if (res.isEnabled) {
                    console.log('We have permission to send push notifications');
                } else {
                    console.log('We do not have permission to send push notifications');
                }
            })
            .catch(error => {
                console.error(error);
            })
            .finally(() => {
                console.log('has permission finally block');
            });
}
catch(e){
   console.log('error', e);
}

console.log('after permission');

This code works on Android, but on iOs the only output is 'before permission' and 'after permission'.
I also tried this (it works on Android):
        const pushObject: PushObject = this.push.init(options);

        pushObject.on('notification').subscribe((notification: any) => this.displayError('Received a notification', [JSON.stringify(notification)]));

        pushObject.on('registration').subscribe((registration: any) => this.displayError('Device registered', [JSON.stringify(registration)]));

        pushObject.on('error').subscribe(error => this.displayError('Error with Push plugin', [JSON.stringify(error)]));

In Xcode I see that the line: https://github.com/phonegap/phonegap-plugin-push/blob/master/src/ios/PushPlugin.m#L545  is invoked, but the if statement is false.
>>> Do you why this code if condition is false ?
if ([appDelegate respondsToSelector:@selector(checkUserHasRemoteNotificationsEnabledWithCompletionHandler:)]) {

Platform and Version (eg. Android 5.0 or iOS 9.2.1)
iOs env:

iPhone 8, iOs 13.4.1 simulator
iPhone 8, iOs 13.3.1 real device
push notification enabled

Cordova CLI version and cordova platform version
Using ionic 5
npx cap doctor output:

💊   Capacitor Doctor  💊
Latest Dependencies:
@capacitor/cli: 2.1.0
@capacitor/core: 2.1.0
@capacitor/android: 2.1.0
@capacitor/ios: 2.1.0
Installed Dependencies:
@capacitor/cli 1.4.0
@capacitor/core 1.4.0
@capacitor/ios 1.5.2
@capacitor/android 1.5.2

Plugin version
"phonegap-plugin-push": "^2.3.0",

having same problem

having same problem

I also tried using the previously initialized object "this.push.hasPermission()" to do call hasPermission() also failed like you do.
I tried this approach "PushNotification.hasPermission(callBackFunction)" and it success call and return the callback function as expected.

Does anyone have a solution for this? I'm having the same problem.


Remove isparta note and document options

After upgrading to istanbul-lib-instrument, also the documentation needs to be updated.
Mentioning the supported options and remove the note about the need for isparta loader if ES6 code should be instrumented

btw why do we have autoWrap: true by default?

Hmm.. I think that was a mistake I made when upgrading... The option was called noAutoWrap before... The new options is called autoWrap and it looks like i just merged those two options... leading exactly to the opposite...


Avoid lock challenges through context management for Kuzu database connections

Thanks again for the awesome work in this project! This issue expands upon topics brought up in #3145 to suggest a kuzu.Connection().close() be somehow implemented to ensure locks don't interrupt possibly concurrent use of single Kuzu databases. It's possible the behavior is better suited for kuzu.Database().close(), please feel free to correct!
Generally I've experienced that open Python processes through Jupyter notebooks which have connected to Kuzu databases will continue to lock the database. This becomes problematic when modularizing work through multiple notebooks that possibly make use of the same database (resulting in errors: RuntimeError: Could not set lock on file : ..). As a workaround, one can interrupt or restart Jupyter kernels associated with a notebook to unlock the database, but this takes time and manual action.
It is common for databases in Python to use context management for connections to help ensure they close properly without impacting other work. This also applies to DuckDB through Python, which enables this type of functionality (e.g. with duckdb.connect() as ddb: ...). It may be that contextlib.closing() is a more appropriate match (similar to sqlite3 connection management in Python), and if that's the case, I suggest updating the documentation to reflect this.
Suggested demo of how this could work:
import kuzu

# init a Kuzu database and connection
db = kuzu.Database(target_database_path)
kz_conn = kuzu.Connection(db)

# run a query
kz_conn.execute("RETURN 1;")

# close the connection manually
kz_conn.close()

Similarly, with a context manager:
import kuzu

# init a Kuzu database and connection
db = kuzu.Database(target_database_path)

# form a context manager for the kuzu connection
with kuzu.Connection(db) as kz_conn:
    # run a query
    kz_conn.execute("RETURN 1;")

# the connection closes to anything outside the context

In both of the above cases, my hope would be that the lock is also removed on the database, meaning other processes may do work on the database without running into a RuntimeError.

Hi @d33bs , thanks for the issue! this makes good to me. we will look into this 😄

Hi @d33bs, yes, other users have also noted that working with a Kùzu database from a Jupyter notebook exhibits this behaviour. From an analysis perspective, it would make sense that more than one user might want to connect to a database at the same time from a Jupyter notebook. It's worth looking into for sure, as @ray6080 mentioned.
In the meantime, as a workaround, there shouldn't be any issues with multiple users connecting to a single database via Python scripts (though it's not ideal from an exploratory standpoint). Hope this helps!

Thanks @ray6080 and @prrao87 for the replies here!

In the meantime, as a workaround, there shouldn't be any issues with multiple users connecting to a single database via Python scripts (though it's not ideal from an exploratory standpoint). Hope this helps!

Thank you for the additional context here. Just to make sure I understand: would one be able to use multiple Python scripts in parallel to connect with a single Kuzu database? Or would this encounter the same locking challenges mentioned?

I think my earlier statement was incorrect, see the best practices in the docs: https://docs.kuzudb.com/client-apis/#best-practices
If you run multiple Python scripts in parallel, you'd have multiple Database instances that would conflict with each other for the afore mentioned reasons, leading to the same issue that you face with Jupyter notebooks. However, within a single Database instances, you're free to open multiple Connection objects that can read from the DB concurrently. I believe this is the approach we use when wrapping Kùzu as an API server..
How to use concurrency is something we've been meaning to clarify better in our docs, and hopefully the user experieonce should also improve with time 😅. Let us know if you have any more questions!

Now that I think about it a bit more, as long as the state of the DB doesn't change (i.e., another process isn't writing to the DB), you should be able to run multiple read query scripts in Python concurrently, even if they have different Database instances. @ray6080 should be able to clarify this once he has a moment, but feel free to give it a try via scripts for now!

I have implemented a Database.close() method which should release the lock. A test case is added here: https://github.com/kuzudb/kuzu/blob/master/tools/python_api/test/test_database.py
I am closing this for now. If you still encounter issue releasing the lock, feel free to reopen this. Thanks.


Fluctuation in Nuxt app

Thank you for this awesome component 👍🏼👏🏼, I'm currently using it on my nuxt app and I noticed that it will work well whenever I import paystack directly from the page and after a while it throw a "Window is not defined" error. Please advice.

Duplicate issue. Follow the thread @dedribble  #26

pls help out on step by step on how to use this paystack on nuxt app


Administration: Sync AHelp chat history

About the PR
AHelps (and their read status) are now persisted on the server, allowing admins to see the goings-on as soon as they join.
Screenshots
N/A
Changelog
no

cc @PaulRitter since I can't test the discord webhook tweaks


Tyler - Individual Review Ticket Classy Functions

Individual GitHub pages (3 points)

[ ] GitHub page that documents key learnings and code snippets using GitHub and replit (See Challenges at the bottom of  TT2 (Links to an external site.) for details)

[ ] Organize files, directories and menus for the first 3 weeks.

[ ] Write Factorial function using classes, providing implementation of call.

[ ] Select your own Math function. Write it in Imperative and OOP form.

[ ] Extra Credit.  Write Palindrome function using classes, must provide implementation of call.  Must indicate where error occurs on front and back comparison.

Score: 2.7/3
Comment: Unorganized files, some github page files not shown
Individual GitHub pages (3 points)

[x] GitHub page that documents key learnings and code snippets using GitHub and replit (See Challenges at the bottom of  TT2 (Links to an external site.) for details)

[ ] Organize files, directories and menus for the first 3 weeks.

[x] Write Factorial function using classes, providing implementation of call.

[x] Select your own Math function. Write it in Imperative and OOP form.

[ ] Extra Credit.  Write Palindrome function using classes, must provide implementation of call.  Must indicate where error occurs on front and back comparison.


hostname with -\d+ crash the installation

because of the following rule https://github.com/ideascube/ansiblecube/blob/oneUpdateFile/roles/set_custom_fact/tasks/main.yml#L91
(but there other same replacement)
the hostname is modified unexpectedly and so not found in device list

can't we use ideascube_project_name instead of hostname

I don't understand how the line https://github.com/ideascube/ansiblecube/blob/oneUpdateFile/roles/set_custom_fact/tasks/main.yml#L91 could replace unexpectedly the hostname.
{{ ansible_hostname }} variable is used here specifically to ensure that the hostname is correctly recorded, I use it against device_list.fact to ensure the device we are going to setup has a json declaration.
Further in the playbook I use {{ ansible_hostname }} mainly for convenience.
ideascube_project_name is only used with a master tag and can not be used with other tags.
Does this answer help you ?

I don't understand how the line https://github.com/ideascube/ansiblecube/blob/oneUpdateFile/roles/set_custom_fact/tasks/main.yml#L91 could replace unexpectedly the hostname.

let's say the hostname: '-123', then ansible_hostname would be something like '-123-999'. after replace it would be ''

The regex is loose because it worked for us and our undefined-but-consistent naming scheme, and we don't plan to ship any box named r2d2-6po - yet.


Start script recursion

Describe the bug
npm run start:ios seems to cause flickering. My Mac crashed!

Fix
The script is wrong. I missed this when I reviewed @saihaj's PR and am realising now.
the start script calls itself. it should call emulator

When I had dev script that was working fine. I just renamed when it was suggested in #103  and didn't check it : )


Exception in UWP project, and problem with binding

If you try to change Position (binded to the viewmodel's property) after setting BindingContext of page that  will crash application.
Test project and screenshoot attached.
In test app you can see Image that has similar binding like in data template for CarouselView item, it's just for reference to show that binding works on simple image and don't work on carousel control.
Click on "Add image" button - Image source of Image control will changed, ItemsSource of Carousel will be updated, but control not.
Click again - app will crash.
Tested on Simulator, Desktop and Phone.

TestCarousel.zip

Will take a look.

Any news about this issue?

Is taking me lot of time to understand what are you trying to do :(

First, CarouselView doesn't support ObservableCollection callback events. To add/remove pages from it you have to use InsertPage and RemovePage.

You can use ObservableCollection and CollectionChanged for your own logic but inserting an element in the collection wont refresh UI.

Actually it needs to be done in the other way. You call InsertPage(bindingContext, position) and the element will be added to UI and to the collection, then CollectionChanged will be called and you do your own stuff there.

I fixed your project: https://drive.google.com/file/d/0B6hZDQudMRBUblJzTXdBNnVXc1k/view?usp=sharing

Thanks, it works, but what about MVVM pattern? Will be better to have support of CollectionChanged  events.

Its not possible in Android.

You can still use an ObservableCollection as source and use CollectionChanged for your own business logic but not to add/remove pages from the carousel.

If my work is helping you, please help me back: https://xamarinhq.wufoo.com/forms/nominate-a-xamarin-mvp/
This is what i have done that is community visible:

Alex has been pretty active sharing his knowledge and answering questions in StackOverflow, GitHub and Xamarin Forums.
Alex has been open sourcing Xamarin components and plugin with the community, with more of 10 repositories being maintained on GitHub.
One of Alex's Xamarin.Forms plugin, the CarouselView, is currently being pulled into the Xamarin.Forms core (https://github.com/xamarin/Xamarin.Forms/pull/853). As an stable widget, this control has become the standard for Xamarin.Forms carousel screens with more that 16K downloads in NuGet and more than 100 solved and not open issues.
Another Alex's plugin, the SegmentedControl (https://github.com/alexrainman/SegmentedControl), is a big candidate to be also pulled into Xamarin.Forms core.
Plus Xamarin.Forms UI related plugins, Alex shares plugins to set app icon Badge, Google Analytics and KeyChain access.
Alex shares his knowledge about cutting edge technologies like Couchbase NoSQL and MobileIron integration with Xamarin Apps.


Fix Gotchas notebook regarding control flow differentiation.

Fix the notebook to report on current state of affairs of cond, while, scan, etc.

looks great, thanks!


Implement TF-IDF and BM25 Algorithms from Scratch

Share functionality between them as much as possible
Output is a Document Term Matrix with one row per document and one column for each embedding dimension. The number of columns coincides with the number of words (distinct, without stopwords) in the learned vocabulary

Note that the Document Term Matrix is necessary to compute cosine similarities between document embeddings (i.e. all embedding vectors must have the same length)!
Thus, generating document embeddings for each document separately is not sufficient!


feat(test): add unit test for function

What this PR does:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

Codecov Report
Patch coverage has no change and project coverage change: -0.26% :warning:

Comparison is base (c80f220) 36.58% compared to head (0f8410f) 36.33%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #726      +/-   ##
==========================================
- Coverage   36.58%   36.33%   -0.26%
==========================================
  Files         247      247
  Lines       26157    26157
==========================================
- Hits         9570     9503      -67
- Misses      15519    15616      +97
+ Partials     1068     1038      -30

see 4 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[06] Changelog for 0.62 - [merged]

Merges cherry-pick-f9c061fc -> 0.6
changelog for 0.6.2
See merge request LMSAL_HUB/aia_hub/aiapy!134
(cherry picked from commit f9c061fc09e67cea380a8be516fcde3ca5a0a222)
798c4c6e changelog for 0.6.2

approved this merge request

mentioned in commit dec600487e8a53259db66db05753ec966aeb5ce9


Faster default K8s configs

Background: https://airtable.com/tblD3oZPLJgGhCmch/viwINAgXA5on9pr2T/rec9FGqIHzffYFXZg
PRs:

@a-robinson, I know you're working on this already. When you have notes/guidance, feel free to record them here.

@a-robinson, any updates on this?

Nothing yet, sorry.

No problem, @a-robinson. Plenty of other things to work on. Just let me know.


Async connection throws exception when Redis returns error

Using pipelining with RedisAsyncCommands I get a RedisCommandExecutionException when one or more commands result in an error. Documentation mentions that

Synchronous methods will throw a RedisCommandExecutionException in case Redis responds with an error. Asynchronous connections do not throw exceptions when Redis responds with an error.

The code below replicates the issue:
RedisClient client = RedisClient.create(DefaultClientResources.create(), RedisURI.create("localhost", 6379));
StatefulRedisConnection<String, String> connection = client.connect();
List<RedisFuture<?>> futures = new ArrayList<>();
RedisAsyncCommands<String, String> commands = connection.async();
commands.setAutoFlushCommands(false);
for (int index = 0; index < 10; index++) {
	RedisFuture<String> future = commands.set("key" + index, "sdfdsf");
	if (future != null) {
		futures.add(future);
	}
	RedisFuture<Long> incrFuture = commands.incr("key" + index);
	if (incrFuture != null) {
		futures.add(incrFuture);
	}
}
commands.flushCommands();
try {
	boolean result = LettuceFutures.awaitAll(5, TimeUnit.SECONDS,
			futures.toArray(new RedisFuture[futures.size()]));
	if (result) {
		for (RedisFuture<?> future : futures) {
			if (future.getError() != null) {
				System.err.println("Future error: " + future.getError());
			}
		}
	}
} catch (RedisCommandExecutionException e) {
	System.err.println(e.getMessage());
}

I think the expected behavior would be to be able to handle the exception by iterating over futures:
for (RedisFuture<?> future : futures) {
	if (future.getError() != null) {
		System.err.println("Future error: " + future.getError());
	}
}

Lettuce version: 5.1.7.RELEASE
Redis version: 5.0.5

Looking at the RedisFuture.awaitAll() code I realize this might be the expected behavior of that method. I can get all errors by iterating over the futures and calling get(). Would that be the recommended approach?

Alright, spent a bit more time on this and it definitely looks like expected behavior. I'll go ahead and close this issue.


Revert "Annotate RBD storageclasses for keyrotation"

This reverts commit e079940750ee2596cc8060fe286b768f14e65ea0.
We are dropping support for automation on SCs in this release.

/retest

@malayparida2000 PTAL.

@malayparida2000 PTAL.

Can you add a bit more details about the reason for the revert both to the commit message/PR desc.
And the failing workflows I think is a github issue may be.

/hold

@malayparida2000 shall I create a BZ for this?

/cherrypick release-4.17