Not support .net4.5 now ?

We will close your issue without comment if you delete or do not fill out the issue checklist and provide ALL the requested information. This repository's issues are reserved for feature requests and bug reports. For support questions, please ask on StackOverflow: https://stackoverflow.com/questions/tagged/reactiveui
Please delete this note before pressing the submit button
Do you want to request a feature or report a bug?
What is the current behavior?
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
What is the expected behavior?
What is the motivation / use case for changing the behavior?
Which versions of ReactiveUI, and which platform / OS are affected by this issue? Did this work in previous versions of ReativeUI? Please also test with the latest stable and development snapshot
Other information (e.g. stacktraces, related issues, suggestions how to fix)

See the release notes at https://reactiveui.net/blog/2018/05/reactiveui-v8.0.0-released for the v8.0.0
The node in particular is:
The minimum version of .NET framework is now net461 which means you'll need to adjust your unit test projects and WinForms/WPF applications. We chose net461 as it's in alignment with Microsoft's official support policy and net461 is now the default option provided in Visual Studio 2017 when scaffolding File -> New Project.


Smakk'd

Business site says it is undergoing maintenance: https://www.smakkdrefreshers.com/

Also: photographer Instagram link isn't working: https://www.instagram.com/jwphotndesign/


Upgrade scala dependency

Code looks good.  Based on the travis config I don't think we're exercising the jvm code?

@jamescway I think the _jars file isn't getting generated properly on JRuby 1.7.19 which doesn't include jar-dependencies in the standard distribution like later versions of JRuby

Hrm, scratch that, I cannot reproduce this locally with the same version of JRuby and Bundler >_<


Image is not preloaded

When the extension is deployed on a real system, the image for the selected characteristic value does not appear on the robot_commonData view until the value help button is pressed. The image appears immediately when the view is tested locally.
Is there any way to force the system to load the image immediately on the robot_commonData view ?

I guess you are referring to characteristic 'AVC_RBT_APPLICATION'? ... As none of the values (01 - Sealing / 02 Painting) is flagged as 'default', initially no value is assigned - hence no image is shown.

@florianstephan01 even when the value for the characteristic has previously been set, the image is not preloaded on the extension view. You have to press the value help button (without changing the value) for the images to be preloaded. When you close the value help widget, the image is now visible in the extension view.

@mathieulawnik: thanks for clarification. Now i could reproduce it. I created an internal incident for this. If you face a similar issue on an UI Extension you use productively, please open an incident via https://support.sap.com/.
Thanks.


Small clean up and improvements to MNit from WBTW

The new OpenGL parameters affect the display of images for a better pixelated look:

When zoomed out, the image is smoother, the color values are interpolated from the nearest source pixels.
When zoomed in, the image is displayed pixelated.

You may recognize the Int::next_pow method, inkscape_tools use something similar for the same purpose. Actually, with this PR, inkscape_tools may not need it anymore and produce smaller PNG files.

+1 c'est cool

Je me demande si next_pow ne pourrai pas aller dans standard, dans math par exemple.

@privat  asked for some love for you, so there it is ... :heart: !
Moreover, this PR looks great, so you can get a +1 too !


when cdc is hanging, the changefeed status in monitor is sometime normal, sometime error

What did you do?

Create a database a in upstream tidb before creating a changefeed
Creating a changefeed
Create a table in database a in the upstream tidb

What did you expect to see?

the cdc will be hanging, and the status of the changefeed is always error Unless the changefeed is really become normal.

What did you see instead?
1.when cdc is hanging, the changefeed status in monitor is sometime normal, sometime error

Versions of the cluster
Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):
(paste TiDB cluster version here)

tidb 5.3.0
TiCDC version (execute cdc version):
v5.3.0
(paste TiCDC version here)

We will adjust changefeed retry backoff mechanism as described in https://github.com/pingcap/tiflow/issues/3329#issuecomment-966980048


Alert styling

There are still some small changes I want to make to styling, but here are some of the more recent changes i've made to our UI. I've moved a lot of repeated code into a parent class.

Merged into James' Login branch


Do not use max allowed ingress expiry to avoid error

Due to time drift, signed messages prepared by quill may have a max expiry time that is greater than what is allowed by IC nodes, which would lead to submission failures of HTTP response code 400.
The workaround is to set the ingress expiry to be 4 minutes 30 seconds, less than the max allowed 5 minutes.

Hi Paul, can you give some context please? I do not understand how a reduction by fixed 30s is helpful given that the clock drift can be arbitrary and into both directions? Why not adjusting the clocks on the AG device instead? If your AG device's clock get slower over time and you do not adjust it, you will also run into the same problem with his fix, just later, right?


Simplify the In-cluster secrets example

Simplify the In-cluster secrets example to work out-of-the-box and ease on-boarding.

/approve

/merge


⚠ Bofrost.at has degraded performance

In e90e886, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1906 ms

Resolved: Bofrost.at performance has improved in 548b4cf after 6 minutes.


Support for events in EDF files

I ran into two issues related to loading EDF files.

The default for stim_channel is -1, but what if there is no stim channel in the file? I was expecting that this should be detected, because other toolboxes (such as Biosig) automatically detect if there is a stim channel in the file. An example EDF file without a stim channel can be found here.
It seems that events are not loaded/parsed correctly in some EDF files. For example, this EDF+ file contains 2 events (at the very first and last samples). mne.find_events doesn't find any events - the stim channel signal contains a value of 2.0 for the first sample, but all other samples are 0.0. So it can't be just using the wrong arguments for mne.find_events.

It seems like pyedflib correctly reads these files.

The default for stim_channel is -1, but what if there is no stim channel in the file? I was expecting that this should be detected, because other toolboxes (such as Biosig) automatically detect if there is a stim channel in the file. An example EDF file without a stim channel can be found here.

I don't think there's a way of automatically detect the stim channel in edf, since this info does not exist in the meta data. The only way I see is to guess using the ch_name or unit, but that seems dangerous. So changing stim_channel default to None would fix this, but in my experience -1 works quite often and changing it would require deprecation. Maybe adding a warning in the docs would suffice.

It seems that events are not loaded/parsed correctly in some EDF files. For example, this EDF+ file contains 2 events (at the very first and last samples). mne.find_events doesn't find any events - the stim channel signal contains a value of 2.0 for the first sample, but all other samples are 0.0. So it can't be just using the wrong arguments for mne.find_events.

Since the tal_channel gets parsed to construct a stim channel, and find_events detects rising edges, they aren't detected. The start is not detected because the stim channel starts at non zero, and the end is not detected because the rising edge would be at sample n+1. We could change find_events to detect non zero starts, but that could have side effects. Another option would be to have get_edf_events method that would return the parsed tal_channel.

I don't think there's a way of automatically detect the stim channel in edf, since this info does not exist in the meta data. The only way I see is to guess using the ch_name or unit, but that seems dangerous.

But how are other libs such as Biosig or pyedflib handling this? They seem to be able to detect events automatically. This would be very convenient, because right now the user has to jump through some hoops to find out if there are events in the file.

Since the tal_channel gets parsed to construct a stim channel, and find_events detects rising edges, they aren't detected.

OK, I have no idea what the tal_channel is, but it seems event information is present in this special(?) channel. Is this channel always present? Can it be addressed independent of its name? I'm sorry, but as I said I don't know much about the EDF format.

The start is not detected because the stim channel starts at non zero, and the end is not detected because the rising edge would be at sample n+1. We could change find_events to detect non zero starts, but that could have side effects.

Plus it would only detect the first event. How can the second event be detected at all?

Another option would be to have get_edf_events method that would return the parsed tal_channel.

If this is a more generic solution and if this solves the issue of automatically detecting if there are events at all, then this would be perfect!

But how are other libs such as Biosig or pyedflib handling this? They seem to be able to detect events automatically. This would be very convenient, because right now the user has to jump through some hoops to find out if there are events in the file.

AFAIK the old EDF format has no events encoding as such. I suppose the other libs just read the data as it is, not even trying to find the events (bullet 1. in the original post).

OK, I have no idea what the tal_channel is, but it seems event information is present in this special(?) channel. Is this channel always present? Can it be addressed independent of its name? I'm sorry, but as I said I don't know much about the EDF format.

In EDF+ the tal channel contains the annotations (events, bullet point 2.).

Plus it would only detect the first event. How can the second event be detected at all?

Yes. I don't think there is a clean solution for this without breaking something else.

If this is a more generic solution and if this solves the issue of automatically detecting if there are events at all, then this would be perfect!

I'll take a stab at this.

AFAIK the old EDF format has no events encoding as such. I suppose the other libs just read the data as it is, not even trying to find the events (bullet 1. in the original post).

OK, so is there a flag somewhere that tells me that what I've loaded is an old EDF file? That way I could call mne.find_events with stim_channel=None.

I'll take a stab at this.

Cool - is using pyedflib an option to simplify things and get maximum EDF compatibility?

I guess the only way is to check whether 'EDF Annotations' exists in the channels. I'm thinking that maybe we should switch to stim_channel='auto' by default like @Eric89GXL suggested earlier. The only problem is that if there's a channel with that name, we have a conflict. I guess we could just raise an error in that case.

Would this solve the issue of automatically detecting events in any EDF file (old or new)?
If there really is a channel called 'auto' and you want to designate it as the stim channel, how would you do it?

Would this solve the issue of automatically detecting events in any EDF file (old or new)?

If 'auto', it would assign a stim channel if 'EDF Annotations' or GDF events exist. Otherwise it would not assign a stim channel.

If there really is a channel called 'auto' and you want to designate it as the stim channel, how would you do it?

You could pass the index of the channel.

If 'auto', it would assign a stim channel if 'EDF Annotations' or GDF events exist. Otherwise it would not assign a stim channel.

Nice, so this solution would be independent of specific names for stim channels. The behavior would correspond to what Biosig or pyedflib are doing. I like that a lot!

You could pass the index of the channel.

Fair enough, if this is mentioned in the docs then I don't see any problem with it!

Come to think of it, GDF stores events in an event table (annotations). Does that mean that MNE always converts events first into a stim channel and then parses that stim channel to get events?

Yes.

Thanks @jaeilepp, this is a great improvement! Just to make sure, this fixes the first issue I mentioned above, but what about parsing the stim channel correctly in the case of this EDF+ file? Should I open a new issue since this is separate from the current fix?

Yeah, in that case the stim channel should actually be appended, not assigned. I'll fix it.

Or actually, I'm not sure. I don't see why someone would want to have the 'EDF  Annotations' channel present, since the data does not make any sense without parsing.
Regarding the start and stop events, I thought the PR took care of it. I'd prefer not to mess with find_events function, since that'll break things. Maybe just mention something in the docstring.

@jaeilepp I'm not sure what you meant with your last comments regarding the stim channel (appended vs. assigned). Having the events is important, the annotations channel is just an internal representation of events that the user doesn't need to see.
Re start and stop events, no, mne.find_events still doesn't find any of these two events. Actually, the stim channel still looks the same, with all zeros except for the first sample. With this information, I think the last event cannot even be reconstructed. Since the events are clearly present in the file (as e.g. loaded by other toolboxes), mne.find_events is already broken in this case.

Re start and stop events, no, mne.find_events still doesn't find any of these two events. Actually, the stim channel still looks the same, with all zeros except for the first sample.

Yes, the events are outside the data range. This is what the new function is for.

Nice! Sorry, I somehow didn't see the new function! Thanks @jaeilepp!

I'm sorry I have to revive this issue, but I'm still running into trouble regarding EDF events. I thought that the new argument stim_channel='auto' correctly handled EDF files with events stored as annotations. However, this file still gets an extra stim channel when loaded with read_raw_edf even though it has event annotations. In this specific case, this means that mne.find_events reports fewer events than there are actually present - the first event is missing (because it occurs at time 0).  The new find_edf_events method correctly returns the events (but in a different format not compatible with the array returned by mne.find_events).
In short, loading EDF files and parsing events is still a mess. It would be nice if there was one consistent workflow to get events. Maybe an option for mne.find_events to consider a non-zero first sample (so without a rising/falling edge) as an event would be a good start.
@larsoner WDYT?

Update: On second thought, I think it is OK that such files get an added stim channel, but the problem is still in parsing this stim channel.

The new find_edf_events method correctly returns the events (but in a different format not compatible with the array returned by mne.find_events).

IIRC the events that are stored in EDF contain extra information (maybe duration?) so it necessitated using a different format. Maybe a doc update to say that this is intentional and giving a little 3-line NumPy recipe for how to convert to standard events array would be useful.

Maybe an option for mne.find_events to consider a non-zero first sample (so without a rising/falling edge) as an event would be a good start.

Sure, a initial_event=False (backward-compat default) or so seems reasonable, I think this has come up before.

IIRC the events that are stored in EDF contain extra information (maybe duration?) so it necessitated using a different format. Maybe a doc update to say that this is intentional and giving a little 3-line NumPy recipe for how to convert to standard events array would be useful.

You're right, it seems like EDF events have durations, so it is good to be able to get this information somehow. I'll try to come up with a short recipe, and we should definitely update the docstring to describe the output of find_edf_events.

Sure, a initial_event=False (backward-compat default) or so seems reasonable, I think this has come up before.

OK, I'll create a PR soon.
But for the user who wants to load an EDF file and has no idea how events are stored - how do we make sure that MNE makes doing the right thing as simple as possible? It would be nice to just say "use mne.find_events to find events, no matter what the file format is". But I guess we could issue a warning (or info?) if read_raw_edf detects event annotations?

It would be nice to just say "use mne.find_events to find events, no matter what the file format is". But I guess we could issue a warning (or info?) if read_raw_edf detects event annotations?

Ideally we can say just use mne.find_events.Users who do know about the events they expect from an EDF specifically might be disappointed because there are no durations, which is why find_edf_events exists.
We could do initial_event=True by default via deprecation, but how common are such events in EDF files? My initial idea is to have False by default, and document it properly in the docstring. Then hopefully people will realize that if they tend to have events right at the recording onset, they should use True.

Ideally we can say just use mne.find_events. Users who do know about the events they expect from an EDF specifically might be disappointed because there are no durations, which is why find_edf_events exists.

Can we log an info if read_raw_edf detects event annotations?

We could do initial_event=True by default via deprecation, but how common are such events in EDF files?

Not that uncommon I'd say. How much would it break if we really set that to True? We could implement a check that even if initial_event=True but the first sample of the stim channel is actually 0, no event would be detected.
We could also issue a warning/info if find_events detects the first sample to be not equal to 0.

My initial idea is to have False by default, and document it properly in the docstring. Then hopefully people will realize that if they tend to have events right at the recording onset, they should use True.

Maybe, if we add these warnings/info messages. Depending how much existing stuff we'd break, I'd tend to set it to True by default though.

Can we log an info if read_raw_edf detects event annotations?

Yes, we should add it if we don't already

How much would it break if we really set that to True?

Users could easily know the structure of their data (and the behavior of find_events) and reasonably do:
events = find_events(...)
# fix event 1
events[0, 2] = 20
...

And now we've broken their code. So we'd need a deprecation cycle to avoid it. The question is, is it worth it? find_events is a very commonly used function (git grep "find_events" | wc -l in the git root itself gives 213 results) so I'm inclined to avoid a cycle if we can.

We could implement a check that even if initial_event=True but the first sample of the stim channel is actually 0, no event would be detected... We could also issue a warning/info if find_events detects the first sample to be not equal to 0.

My idea was that it would just enable finding events at t=0, not force an event at t=0 (I'm imagining an EDF user with events sometimes at zero and sometimes not). So I don't think we'd need any warning for this mode if we document it properly.

[ ] issue info if read_raw_edf detects event annotations

In your example, we would only break things if there was indeed an event at sample 0.
I know that find_events is often used, but how often in combination with EDF? Not 213 times I imagine.
So the question for me is still how much would that really break, but maybe this is difficult to answer so I'm also fine with setting the new arg to False (at least for now).

My idea was that it would just enable finding events at t=0, not force an event at t=0.

Agreed.

In your example, we would only break things if there was indeed an event at sample 0.

Yes but we don't know how common this is. I suspect it's not totally uncommon. For example, a subject holding a button when you press "record" during Neuromag acquisition would be enough to do it. The risk of silent breakage is too high.

I know that find_events is often used, but how often in combination with EDF? Not 213 times I imagine.

Yes but we wouldn't just be breaking EDF users' code. Data from all systems get channeled through find_events, so a deprecation cycle there means emitting warnings to almost every user of MNE.

Of course, I'd never silently break anything, but flooding users with warnings is also not ideal. OK, so do you agree with my checklist above?

Yeah it looks good to me

Regarding the problem with events at t=last_samp, I'm not sure if the stim channel gets constructed correctly. Shouldn't the last sample in the converted stim channel have the event value? Why did @jaeilepp say that the rising edge would be at sample n+1? I think there might be something wrong with this.

Or in this example, are the event annotations incorrect? They are:
[[0.0, 0, 'RECORD START'], [600.0, 0, 'REC STOP']]

But the total length is 600s (120000 samples), which means that if time starts from 0 the last sample is actually 600-1/200=599.995 (because the sampling frequency is 1/200=0.005). This of course would mean that everything MNE does is correct.

Maybe he meant that the falling edge would be at sample n+1. But yes I agree that output looks fine (thinking of STOP as the first time point to exclude at least)

Well this is certainly a valid explanation for a Pythonista, but I doubt that this is true in general. What if the event at t=600s was called something else? Do you agree that t=600s is an invalid time for an event in this data set?

Do you agree that t=600s is an invalid time for an event in this data set?

As on onset time, yes. As an offset ... depends on how we want to define onsets and offsets. I guess we usually use inclusive numbering, which would argue for 599.995.
Do we choose to set it as 600s (e.g., by doing samp_start + duration + 1) or is it encoded that way in the file? If the latter, then I don't think we should change it.

The events from EDF annotations are always given in terms of onsets (so 600s is invalid). I'm pretty sure it's encoded like this in the file, and I wouldn't change anything in this case either because it's not MNE's fault if an invalid event cannot be recovered. But I'll try to see how other tools (pyedflib, biosig) handle this.

Yeah, there's something wrong with this last event; the BioSig toolbox assigns sample 12001 to this event, but at the same time the signal is indexed from 1 to 12000 (remember we're in MATLAB 1-based indexing). So I guess EDF events are not validated and can contain any value.

It's also possible that the STOP event is special

Sure, but then BioSig also handles it incorrectly, and pyedflib also says that the event occurs at t=600s in a file of duration 600s (or 12000 samples), which means that t=600s does not exist if we start at t=0s. I think it's more likely that this event is simply wrong and should really be 599.995s.


Nuget not finding the built file

Hi i just tried the action,  but i had an weird error:
  Restore completed in 1.9 sec for /home/runner/work/EasyConfig/EasyConfig/EasyConfig/EasyConfig.csproj. /usr/share/dotnet/sdk/3.1.101/Sdks/NuGet.Build.Tasks.Pack/build/NuGet.Build.Tasks.Pack.targets(198,5): error NU5026: The file '/home/runner/work/EasyConfig/EasyConfig/EasyConfig/bin/Release/netstandard2.1/EasyConfig.dll' to be packed was not found on disk. [/home/runner/work/EasyConfig/EasyConfig/EasyConfig/EasyConfig.csproj]
It is not finding the file in the bin folder, and even though it failed to publish to Nuget the action was marked as a success, took me a while to find out it was not published/uploaded correctly.

I agree it's confusing 😕 & that's because the action wasn't expecting this behavior with dotnet pack. The fix was simple, do an explicit build & pack
The cause of this error was your project having the GeneratePackageOnBuild set to true. There's a bug in dotnet sdk with GeneratePackageOnBuild & dotnet pack combo. A fix (dotnet/sdk#3473) is targeted for 3.1.1xx


[AIRFLOW-2108] Fix log indentation in BashOperator

Make sure you have checked all steps below.
JIRA

[x] My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-2108

Description

[x] Here are some details about my PR, including screenshots of any UI changes:
Changed from strip to rstrip to retain leading whitespaces. Check the Jira issue for more details.

Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:
Minor Change

Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

[x] Passes git diff upstream/master -u -- "*.py" | flake8 --diff

Codecov Report

Merging #3045 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3045      +/-   ##
==========================================
+ Coverage   73.22%   73.23%   +<.01%
==========================================
  Files         178      178
  Lines       12472    12472
==========================================
+ Hits         9133     9134       +1
+ Misses       3339     3338       -1

Impacted Files
Coverage Δ

airflow/operators/bash_operator.py
91.22% <100%> (ø)
:arrow_up:

airflow/models.py
87.19% <0%> (+0.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0742951...f311734. Read the comment docs.

cc @Fokko PTAL

Thanks @kaxil
I went back in history, and I don't see any particular reason why this strip is applied:
https://github.com/kaxil/incubator-airflow/commit/46f927faab53f4f32f50e460d3ea79f029bfb257
So changing it to .rstrip() seems fine to me.


Skip two new int8 models in the model zoo

The ONNX model zoo recently added 4 models in which two models are integer models that we need to skip testing.
Signed-off-by: Tung D. Le<EMAIL_ADDRESS>

Jenkins Linux amd64 Build #6393 [push] Add new int8 models in t... started at 08:32

Jenkins Linux ppc64le Build #5531 [push] Add new int8 models in t... started at 09:34

Jenkins Linux s390x Build #6409 [push] Add new int8 models in t... passed after 1 hr 25 min

Jenkins Linux ppc64le Build #5531 [push] Add new int8 models in t... passed after 1 hr 27 min


V11

Will you work this so that it works on v11?

If you are still looking for a V11 version, I have it working in DnD5e and will support it going forward until the dev returns to claim it. You can find it here:
https://foundryvtt.com/packages/all-goblins-have-names-reborn

Oh, wow. Thanks for reaching out, I will check it out for sure!

Från: Martin Noreke @.>
Skickat: den 18 december 2023 16:57
Till: jsabol/all-goblins-have-names @.>
Kopia: Brynhold @.>; Author @.>
Ämne: Re: [jsabol/all-goblins-have-names] V11 (Issue #16)
If you are still looking for a V11 version, I have it working in DnD5e and will support it going forward until the dev returns to claim it. You can find it here:
https://foundryvtt.com/packages/all-goblins-have-names-reborn
—
Reply to this email directly, view it on GitHubhttps://github.com/jsabol/all-goblins-have-names/issues/16#issuecomment-1860888427, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVLA5N6JCJQLAVR66ZVHG6TYKBRVHAVCNFSM6AAAAAA3FI3K76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRQHA4DQNBSG4.
You are receiving this because you authored the thread.Message ID: @.***>


Fix git authentication on vscode (locally, remote container and Codespaces)

Out of scope: signed commits on remote container and Codepaces (separate issue for that: #92)
Think we need to add this setting: "remote.SSH.showLoginTerminal": true,

When developing locally in vscode and using the terminal, commits were failing because of a gpg problem.
The fix, as per https://github.com/microsoft/vscode/issues/43809#issuecomment-589868396, was:
brew install pinentry-mac
echo "pinentry-program /usr/local/bin/pinentry-mac" >> ~/.gnupg/gpg-agent.conf
killall gpg-agent


i got error for login after some times

i got an error like:
`Traceback (most recent call last):
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\site-packages\instagram_private_api\client.py", line 506, in _call_api
response = self.opener.open(req, timeout=self.timeout)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 532, in open
response = meth(req, response)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 570, in error
return self._call_chain(*args)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ".\MyTest.py", line 10, in
api = Client(user_name, password)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\site-packages\instagram_private_api\client.py", line 197, in init
self.login()
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\site-packages\instagram_private_api\endpoints\accounts.py", line 50, in login
'accounts/login/', params=login_params, return_response=True)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\site-packages\instagram_private_api\client.py", line 510, in _call_api
ErrorHandler.process(e, error_response)
File "C:\Users\Saeed\AppData\Local\Programs\Python\Python36-32\lib\site-packages\instagram_private_api\errors.py", line 125, in process
error_response=json.dumps(error_obj)
instagram_private_api.errors.ClientCheckpointRequiredError: checkpoint_challenge_required`
and when i use to go pnline with my browser, check my identity with sending security code, how to solve it that do not check my identity every time?

@Painkiller1991
Your issue report does not conform to the issue template that has been specified for this repo:
https://raw.githubusercontent.com/ping/instagram_private_api/master/.github/ISSUE_TEMPLATE.md
Please edit your issue to comply with the template requirement.
This issue will be closed after 24 hours if no followup action is taken.

[This comment is auto-generated. ref=notemplate]


python scikitlearn: fails to build since openblas update

Since openblas 2.14 -> 2.15 scikitlearn fails with a segmentation fault. While I am not entirely sure it is due to openblas it seems the most likely cause.
The log shows a failure with scikit-learn 0.17b1, the same occurs when using the final 0.17.0.  Upstream tests using Travis with openblas 2.14 that is provided via conda.
Could the issue be on our side, or would this be an upstream issue? (None such issues reported there yet).
Any ideas what is causing this? @ttuegel @datakurre

Segmentation faults with OpenBLAS are usually caused by using openblas instead of openblasCompat or vice versa. Packages that can use any BLAS implementation usually need openblasCompat, whereas packages that need OpenBLAS specifically usually need openblas. The first thing I would try is switching out the OpenBLAS build.

I took master and reverted 3d87c5f9c3104739c5c47c2d387101afba2b0089 and the segmentation fault did not occur. Let's see what upstream thinks about this.
For now scikitlearn can be fixed by overriding openblas to 0.2.14.

scikitlearn checks which version of openblas numpy uses, and sets the path to that one.
If I am correct, scikitlearn should indeed also have the same version of openblas.
Using 0.2.14 just for scikitlearn therefore does not work. An openblas 0.2.14 is needed for numpy/scipy/scikitlearn.
#10928

I've built scikitlearn successfully locally so I'm not sure what's going on at hydra.

Seems to segfault on Hydra again. Only with Python 2.7 though.
http://hydra.nixos.org/build/28584750

scikitlearn builds again for some time so closing.

Just as I had closed this issue a segfault occurred again, this time with Python 3.4
I will have a look whether using Atlas instead resolves this issue. The upcoming scipy 0.17 also fails with OpenBlas.

building scikitlearn with openblasCompat like done with numpy and scipy does not solve the issue. However, downgrading to openblas 0.2.14 does solve the issue. Furthermore, scipy 0.17.0rc2 also builds with openblas 0.2.14 while it doesn't with 0.2.15.
@ttuegel
Since upstream also tests against 0.2.14 (Travis) I propose we add openblas 0.2.14 and use it for numpy/scipy/scikitlearn. Shall I just make a copy of default.nix named 0.2.14.nix or add a version parameter to openblas?
Note to self: scikitlearn 0.17 is incompatible with scipy 0.17 because of deprecated function lena(), therefore we need to keep scipy 0.16 as we were planning already anyway.

Shall I just make a copy of default.nix named 0.2.14.nix

This one. I don't like to add version parameters to expressions because it gets complicated if we need to add version-specific fixes later.

closed with #12826


feat: block known sensitive environment variables

Uses the list from https://github.com/Puliczek/awesome-list-of-secrets-in-environment-variables/blob/main/raw_list.txt
but allows for the configuration of custom block lists.
https://github.com/testifysec/witness/issues/170

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

98 of 104 (94.23%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+2.8%) to 40.88%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/attestation/environment/environment.go
17
23
73.91%

Totals

Change from base Build<PHONE_NUMBER>:
2.8%

Covered Lines:
948

Relevant Lines:
2319

💛  - Coveralls

variables are also exposed when --trace=true

variables are also exposed when --trace=true. Should we remove those for now?

the command run attestor probably should have been using an embedded environment attestor so it would get this benefit without maintaining two sections of code.
i'll take a look and see if it can be arranged that way

variables are also exposed when --trace=true. Should we remove those for now?

the command run attestor probably should have been using an embedded environment attestor so it would get this benefit without maintaining two sections of code.
i'll take a look and see if it can be arranged that way

that is a great idea.

I think we should also have a way to pass a list to the witness binery.  The use case I am thinking about is reading GitLab's masked variable list.  I have it available in when I create the witness command.

I think we should also have a way to pass a list to the witness binary. The use case I am thinking about is reading GitLab's masked variable list. I have it available in when I create the witness command.
key could be something like
run:
  attestors:
    env:
      blocklist:
        - item

Might be out of scope for this PR.

Attestor specific config from the CLI is definitely something on the short list to tackle.  I've added the functions to this attestor to support that level of configuration but adding the wiring from the CLI to the attestors is something that will be tackled in a separate PR.
I thought it over last night and I think we can hash blocked environment variables fine.  However, I think we need to re-work the current format of the attestor a bit to accomplish this.  Currently the environment attestor out puts these in a format similar to:
{
  "VARIABLE": "VALUE"
}

If we're going to start hashing protected values I have concerns with this representation as there would be no way to discern that the value was the actual value or a hash of the blocked value.
Changing the format to something like
{
   "VARIABLE": {
      "type": "plaintext",
      "value": "VALUE",
  },
  "BLOCKED_VARIABLE": {
    "type": "digest",
    "value": {
        "sha256": "HASH(BLOCKED_VALUE)"
    }
  }
}

This breaks backward compatibility and would require us to create a new version of the attestor.

Im good with either way.
What use cases would we have for the hashed secrets?  If we cant think of anything, let's avoid the code.

Im good with either way.
What use cases would we have for the hashed secrets? If we cant think of anything, let's avoid the code.

The only thing I can really think of is policy that the expected secret was used by comparing hashes, but I don’t know if that’s truly a policy that would be used.

@colek42
So re-using the environment attestor would require some re-work of the both the command run and environment attestors so for now i've currently exported the blocklist functionality from the environment package.

LGTM -- please add "SNYK_TOKEN" to the list, as we use it internally merge and cut a release.

alright, added.  can merge and release after your review


2 instances of this module can not share the same cache-bucket

Hi,
when I use your module twice, so I can use different sizes of instances for different jobs, I see no possibility to use the same bucket as cache.
The reason for this, is the fact that the module will always try to create a bucket, but if this bucket already exists, terraform will fail.
Maybe it makes sense to remove the cache-bucket from this module and better inject a cache_bucket-name into the module as parameter to be able to get the required information of the bucket inside of the module.
I tried to work around this issue, by introducing a count-variable for the bucket in conjunction with an extra parameter, but that forced me to add count-variables to most of the other resources too, which enforces a lot of duplication.

Thanks for creating the issue. Yep I am aware of the limitation. It also crossed my mind. I had the thought not to remove the bucket but made a more a submodule. Which you have to instantiate specific. The output of this submodule can then be based to the runner module it self.
Maybe in such way that by default the bucket is created and it is the the user to to disable the creation and control the management of the the bucket outside het module.
module "bucket" {
...
}

module "runner1" {
...
   bucket.arn
   bucket.policy
}

module "runner2" {
...
   bucket.arn
   bucket.policy
}

@solutionDrive-Alt I have a first "early" setup working for 2 nodes, one cache. Feel free to have a look. The public example contains a sample setup. I will update the sample later so the cache will be create separated (outside) the runner. And policy and bucket details will be injected.

@npalm I checked it out and this looks very promising. I will have a closer look in the next days and will try it with my setup.
Thanks for working on this already!

@solutionDrive-Alt thanks for the quick answer, it as already on my list for some time.


Revamp the Stats API

Including direct accessors for the different statistics since it's a closed enum.
Make stats access a dynamic trait so we can implement it for ArrayViews.

Fixed in #237


Video Generation model

What a fantastic great work!
When will the Video Generation model be released?

What a fantastic great work!
When will the Video Generation model be released?

Hi,
Thanks for your interest! For the video-related codes, we need to go through the company's legal check and wait for approval....

Thank you for answer!
I hope to hear a good decision. :)

we need to go through the company's legal check

Is it a matter of training data used or regarding the code for the video generation?

Thanks for your interest! For the video-related codes, we need to go through the company's legal check and wait for approval....

so no?

Thanks for your interest! For the video-related codes, we need to go through the company's legal check and wait for approval....

so no?
ELLA SDXl got blocked, there is no way it's making it through.

we need to go through the company's legal check

Is it a matter of training data used or regarding the code for the video generation?

Hi,
thanks for your interest! The main issue is regarding the data we have used for model training.

Hi @zhoudaquan,
Thank you for releasing StoryDiffusion! If the main issue is the training data, is there any possibility that the code could be released first, so if people want to train models independently they can do so?
Thank you!

long live the open source

Any updates?

About 9 month I waiting somebody to release self attention open source model. Maybe some day that will happened. SORA making the same and training they own models but maybe with longer token limits.

We should probably set up the whole thing by our selves on Comfy..

We should probably set up the whole thing by our selves on Comfy..

I just learning but will try to use this new AGI ) https://github.com/Doriandarko/maestro

@zhoudaquan when will release video code?

No, I suppose then. Guess we are not gonna get a good open-source video generation model anytime soon.


BELs bound to multiple pads on Ultrascale+

Some designs cause the following error to be generated by nextpnr:
Assertion failure: tile_status.boundcells[bel.index] == nullptr ([..]/work/nextpnr/fpga_interchange/arch.h:256)

For the context, see #93
The error originates from an assert line 256 of https://github.com/YosysHQ/nextpnr/blob/master/fpga_interchange/arch.h
The bindBel call is the one in line 331 of https://github.com/YosysHQ/nextpnr/blob/master/fpga_interchange/arch_pack_io.cc
Looks like there are two ports that get somehow get the same BEL bound. I've made an unordered set to keep track of cells visited in the loop that start at fpga_interchange/arch_pack_io.cc:212 (I was keeping only a track of pointers, assuming there's only one immovable object backing the a cell!) and none of them seemed to repeat. So it looks like there's a BEL that spans over two cells that different ports are mapped to.

Turns out that was just a typo in XDC which resulted in binding two ports to the same pad. nexpnr doesn't seem to have a clear massage for such case which got me confused for quite some time.


Parse geofence to internal model first

Don’t rely on googles internal ParcelableGeofence. Instead map the json from the server to a GeofenceRegion
Closes #140

🚢


Added support for RFC 5424 (new Syslog standard)

This pull request adds support for RFC 5424 of the Syslog protocol. It maintains backwards compatibility with v2 of this project (hopefully, not actually tested, but reviewed).
I did resolve the async issue that was reported (no longer using async). This might be a problem in high volume systems, along with the ti24horas concern about re-instantiating the UDP client for every log request. I did this work in order to evaluate a possible logging solution that supported Syslog, so didn't put much effort into improving the performance.
If you do look into the async issue more, check out this thread: https://social.msdn.microsoft.com/Forums/vstudio/en-US/cbdd9818-00f0-499f-a935-d8e555899d64/use-net-udpclient-in-a-multithreaded-env?forum=wcf

Thanks for your contribution!


spin out and upgrade Lambda plugin

The plugin now lives in its own repo.
New features (see changelog):

feat: if no credentiuals are provided, the plugin will automatically fetch EC2 or ECS credentials and use the AWS IAM roles retrieved for accessing the Lambda.
feat: new option awsgateway_compatible to make the serialized request compatible with the AWS gateway format, making the plugin a drop-in replacement
feat: new option skip_large_bodies to enable really large bodies (that have been cached to disk) to also be sent to the Lambda. Use with care!
feat: added the ability to connect to the Lambda through a proxy

Note on versioning: the update was developed separately, hence the updates are in the changelog under 0.1.0. After renaming the external plugin to aws-lambda the version was bumped to 3.0.0 to supersede the existing (internal Kong) version at 2.0.0.

CI fails on a LuaRocks error;

Error: Build error: Failed installing kong/plugins/aws-lambda/v4.lua in /home/travis/install-cache/f99b7d5b0a5f6c9d070ff6ce527637ac/luarocks/lib/luarocks/rocks-5.1/kong/1.3.0-0/lua/kong/plugins/aws-lambda/v4.lua: The source and destination are the same files

@murillopaula good catch! updated

@Tieske rebase current 'next' branch to make travis happy

rebase current 'next' branch to make travis happy

had already done so, but needed a few job-restarts to finally pass

@Tieske FYI the plugin's CI tests pass when pointing to the 'lambda' branch: https://github.com/Kong/kong-plugin-aws-lambda/pull/7

Hi, I have approved this change on this repo. However, I believe that some work is needed on the new aws-lambda repo. In particular, these:
* feat: if no credentiuals are provided, the plugin will automatically fetch EC2 or ECS credentials and use the AWS IAM roles retrieved for accessing the Lambda.
* feat: new option awsgateway_compatible to make the serialized request compatible with the AWS gateway format, making the plugin a drop-in replacement
* feat: new option skip_large_bodies to enable really large bodies (that have been cached to disk) to also be sent to the Lambda. Use with care!
( feat: added the ability to connect to the Lambda through a proxy

Seem to be missing.

@kikito the changelog was updated (and merged)


How to follow up after Submit a template to the Power Automate gallery

Hi team,
How can I follow up if my submission has been approved or pending after Submit a template to the Power Automate gallery?
Do we have any point of contact for this?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3bf8d0d2-3a6f-c3f1-494d-b0ea73c22f9b
Version Independent ID: 41bc35b2-5a4d-7406-1ae7-27b292033fcd
Content: Submit a cloud flow template - Power Automate
Content Source: articles/publish-a-template.md
Service: power-automate
Sub-service: cloud-flow
GitHub Login: @sunaysv
Microsoft Alias: sunayv

@thopt -
Thanks for your feedback. We will soon look into this issue on articles/publish-a-template.md.

@thopt There's no way for you to contact the team. The team will communicate with you as needed about your template. Thanks for using Power Automate.


Force that there always has to be an admin user

Imagine the following situation:

There are no users in our system, and we create a new admin.
Go to the admin page and click the "Users" container.
Update the current admin user to not be an admin user.
Now there are no admin users.

Even though this might be proper behavior, it's unexpected. I propose two solutions:

Don't allow to remove admin rights to the last admin.
Cleaner solution imho: don't allow a user to "un-admin" itself. This would protect the user from making any mistakes. Therefore, it should be another admin user the one to "un-admin" this user.

I have slight preference for the 2nd one.


HF web service streaming response differs from OpenAI, breaking clients

System Info
Attempting to reuse an existing OpenAI client to stream responses from HF endpoint doesn't work due to a couple of differences. In my case the differences break the .NET client in Azure AI SDK, though I suspect it might affect other clients too.
Differences found:

When streaming response tokens, OpenAI terminates the stream with a final [DONE] string, while HF simply stops sending tokens. Clients expecting [DONE] get stuck waiting either for another token of for the termination string.
OpenAI supports '0.0 <= top_p <= 1.0', while HF supports only '0.0 < top_p < 1.0'
When sending top_p = 0 to HF endpoint, the service replies 200 OK with an error {"error":"Input validation error: top_p must be > 0.0 and < 1.0","error_type":"validation"} and no final [DONE]. Given the status code and the lack of a termination, the error is parsed as data and causes a client to hang, waiting for the next token.

Information

[ ] Docker
[ ] The CLI directly

Tasks

[ ] An officially supported command
[ ] My own modifications

Reproduction
Example 1: error with top_p = 0
Request:
curl -v -X POST https://api-inference.huggingface.co/v1/chat/completions \
    -H "Authorization: Bearer ${HF_KEY}" \
    -H "Content-Type: application/json" \
    -d '{"messages":[{"content":"how much is 1+1","role":"system"}],
      "max_tokens":50,
      "temperature":0,
      "top_p":0.0,
      "presence_penalty":0,
      "frequency_penalty":0,
      "stream":true,
      "model":"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO"}'

Response:
< HTTP/2 200
< date: Tue, 14 May 2024 22:33:46 GMT
< content-type: text/event-stream
< x-compute-type: 2-a100
< x-request-id: ...
< cache-control: no-cache
< access-control-allow-credentials: true
< vary: origin, Origin, Access-Control-Request-Method, Access-Control-Request-Headers
< x-accel-buffering: no
< access-control-allow-origin: *
< x-compute-characters: 67
< x-sha: ...
<
data:{"error":"Input validation error: `top_p` must be > 0.0 and < 1.0","error_type":"validation"}

OpenAI returns a response instead (see next).
Example 2: OpenAI response includes '[DONE]`
Request:
curl -X POST https://api.openai.com/v1/chat/completions \
    -H "Authorization: Bearer ${OPENAI_KEY}" \
    -H "Content-Type: application/json" \
    -d '{"messages":[{"content":"how much is 1+1","role":"system"}],
      "max_tokens":5,
      "temperature":0,
      "top_p":0,
      "presence_penalty":0,
      "frequency_penalty":0,
      "stream":true,
      "model":"gpt-3.5-turbo"}'

Response:
data: {"id":"chatcmpl-9Ov2YsBUxDJmdENVKFxdZfLMIEJCt","object":"chat.completion.chunk","created":1715726162,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9Ov2YsBUxDJmdENVKFxdZfLMIEJCt","object":"chat.completion.chunk","created":1715726162,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9Ov2YsBUxDJmdENVKFxdZfLMIEJCt","object":"chat.completion.chunk","created":1715726162,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" +"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9Ov2YsBUxDJmdENVKFxdZfLMIEJCt","object":"chat.completion.chunk","created":1715726162,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9Ov2YsBUxDJmdENVKFxdZfLMIEJCt","object":"chat.completion.chunk","created":1715726162,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9Ov2YsBUxDJmdENVKFxdZfLMIEJCt","object":"chat.completion.chunk","created":1715726162,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" equals"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9Ov2YsBUxDJmdENVKFxdZfLMIEJCt","object":"chat.completion.chunk","created":1715726162,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"length"}]}

data: [DONE]

Example 3: HF response is missing '[DONE]`
Request:
curl -v -X POST https://api-inference.huggingface.co/v1/chat/completions \
    -H "Authorization: Bearer ${HF_KEY}" \
    -H "Content-Type: application/json" \
    -d '{"messages":[{"content":"how much is 1+1","role":"system"}],
      "max_tokens":5,
      "temperature":0,
      "top_p":0.01,
      "presence_penalty":0,
      "frequency_penalty":0,
      "stream":true,
      "model":"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO"}'

Response:
data:{"id":"","object":"text_completion","created":1715726245,"model":"text-generation-inference/Nous-Hermes-2-Mixtral-8x7B-DPO-medusa","system_fingerprint":"2.0.2-sha-dccab72","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"logprobs":null,"finish_reason":null}]}

data:{"id":"","object":"text_completion","created":1715726245,"model":"text-generation-inference/Nous-Hermes-2-Mixtral-8x7B-DPO-medusa","system_fingerprint":"2.0.2-sha-dccab72","choices":[{"index":0,"delta":{"role":"assistant","content":" result"},"logprobs":null,"finish_reason":null}]}

data:{"id":"","object":"text_completion","created":1715726245,"model":"text-generation-inference/Nous-Hermes-2-Mixtral-8x7B-DPO-medusa","system_fingerprint":"2.0.2-sha-dccab72","choices":[{"index":0,"delta":{"role":"assistant","content":" of"},"logprobs":null,"finish_reason":null}]}

data:{"id":"","object":"text_completion","created":1715726245,"model":"text-generation-inference/Nous-Hermes-2-Mixtral-8x7B-DPO-medusa","system_fingerprint":"2.0.2-sha-dccab72","choices":[{"index":0,"delta":{"role":"assistant","content":" the"},"logprobs":null,"finish_reason":null}]}

data:{"id":"","object":"text_completion","created":1715726245,"model":"text-generation-inference/Nous-Hermes-2-Mixtral-8x7B-DPO-medusa","system_fingerprint":"2.0.2-sha-dccab72","choices":[{"index":0,"delta":{"role":"assistant","content":" mathematical"},"logprobs":null,"finish_reason":"length"}]}

Expected behavior
Would be great if it was possible to reuse OpenAI clients (and apps built on these clients) simply by pointing them at https://api-inference.huggingface.co.
While it's possible to workaround the different range of top_p changing the code (if apps allow for it), the lack of termination strings makes it impossible to use these clients.

@github-actions bump

The top_p issue seems unresolved still?


Implement BitmapData.perlinNoise

Was trying to port a threshold-effect snippet I found online to HaxeFlixel that involved using perlinNoise, but...
"Lib.hx:113: Warning: BitmapData.perlinNoise is not implemented"
;-;

#913 Basically we're not sure how flash's works and if we should use simplex noise instead.

Duplicate, closing.


sidebar toggle icon doesn't work

Hi everyone,I am working on angular2 and the toggle icon of the side bar menu doesn't work .It seems as if Jquery has difficulties to acces the DOM(it is just a tought) , but I do not really get the issue.PLEASE SAVE ME!! THANKS ALOT

I had solved the issue myself the problem was that the external JS file was not properly read so it was in typescrtip the the DOM manipulation and all the rest was supposed to be done,so I used the controllers in the template at once .Thanks for the help and the great work done on this template


Activating Open Collective

Hi, I'm making updates for Open Collective. Either you or a supporter signed this repo up for Open Collective. This pull request adds backers and sponsors from your Open Collective https://opencollective.com/visifile ❤
It adds two badges at the top to show the latest number of backers and sponsors. It also adds placeholders so that the avatar/logo of new backers/sponsors can automatically be shown without having to update your README.md. [more info]. See how it looks on this repo.
We have also added a postinstall script to let people know after npm|yarn install that you are welcoming donations (optional). [More info]
You can also add a "Donate" button to your website and automatically show your backers and sponsors there with our widgets. Have a look here: https://opencollective.com/widgets
P.S: As with any pull request, feel free to comment or suggest changes. The only thing "required" are the placeholders on the README because we believe it's important to acknowledge the people in your community that are contributing (financially or with code!).
Thank you for your great contribution to the open source community. You are awesome! 🙌
And welcome to the open collective community! 😊
Come chat with us in the #opensource channel on https://slack.opencollective.com - great place to ask questions and share best practices with other open source sustainers!

I messed up the top link - should be https://opencollective.com/visifile-55


[MLE] OPENTHREAD_CONFIG_ENABLE_DYNAMIC_MPL_INTERVAL could not be enable after #2678

[Enhancement] When set OPENTHREAD_CONFIG_ENABLE_DYNAMIC_MPL_INTERVAL to 1, the compile could not pass since commit #2678.

Resolved by #2739.


Increment version number

My bad. This should be good to go. I'll push a minor release out once this goes through.


"dove prove" does not accept z3 (nor cvc4) executable path correctly anymore

Hello again!
While dove prove works fine with 1.3.0-44c7e88, the --z3-exe switch seems broken in 1.3.0-9283954.
One used to be able to run dove prove --boogie-exe ~/.dotnet/tools/boogie --z3-exe ~/bin/z3 successfully, but with the current version, the output is as follows:
➜  taohe git:(master) ✗ dove prove --boogie-exe ~/.dotnet/tools/boogie --z3-exe ~/bin/z3

ERROR: z3 executable not found in PATH. Please install it from https://github.com/Z3Prover/z3

The only way I got this to work was setting up PATH correctly, and running dove prove without arguments. (With the arguments above, I would still get errors: cvc4 cannot be found, and --cvc4-exe does not seem to fix the problem.)

Added the ability to set paths starting with ~/.


How to use Java SDK to check app service name availability

Query/Question
We are wanting to use the Java SDK to check app service name availability. The REST API has this feature but we are not sure how to use the SDK to accomplish the same.
Why is this not a Bug or a feature Request?
As the REST API has this feature, we believe it is possible in the SDK as well, but are unable to find out how it is accomplished.
Setup (please complete the following information if applicable):

OS: any
IDE: eclipse
Library/Libraries: com.azure.resourcemanager:azure-resourcemanager-appservice:2.14.0

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Query Added
[x] Setup information Added

@l3ender
This would do
            var availability = azure.webApps().manager().serviceClient().getResourceProviders()
                    .checkNameAvailability(new ResourceNameAvailabilityRequest()
                            .withName("funweidxu")
                            .withType(CheckNameResourceTypes.MICROSOFT_WEB_SITES));

@weidongxu-microsoft Thank you!


Improve query cache hit rate

Codecov Report

Merging #194 (04d2996) into main (9144913) will increase coverage by 0.02%.
The diff coverage is 93.93%.

@@            Coverage Diff             @@
##             main     #194      +/-   ##
==========================================
+ Coverage   94.17%   94.19%   +0.02%
==========================================
  Files          65       65
  Lines        4549     4566      +17
==========================================
+ Hits         4284     4301      +17
  Misses        204      204
  Partials       61       61

Impacted Files
Coverage Δ

internal/promapi/metrics.go
100.00% <ø> (ø)

internal/promapi/errors.go
80.64% <50.00%> (ø)

internal/promapi/config.go
100.00% <100.00%> (ø)

internal/promapi/query.go
100.00% <100.00%> (ø)

internal/promapi/range.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9144913...04d2996. Read the comment docs.


[deployment] Add documentation for continuous deployment to now

https://github.com/probot/probot/blob/master/docs/deployment.md#now
instructions are at https://github.com/hoodiehq/first-timers-bot/pull/89

@gr2m Thanks for the guide, that was helpful.
I think a thing that's missing is scaling the new instance so it never goes to sleep.
I've fixed a bug in now-cli that makes CI deployments easier: https://github.com/zeit/now-cli/pull/940 (released in canary) as well as a small utility that makes replacing an alias easier: https://github.com/rtsao/now-replace
The script I've been using is:
yarn global add now@canary now-replace
APP_URL=$(now -t $NOW_TOKEN --public -e PRIVATE_KEY=@private-key -e APP_ID=@app-id -e WEBHOOK_SECRET=@webhook-secret -e NODE_ENV="production")
now scale $APP_URL 1 -t $NOW_TOKEN
now-replace graphene-probot $APP_URL -t $NOW_TOKEN

It's a bit annoying the base64 workaround is required for usage with now. I wonder if it would be a good idea to implement direct support for this in Probot, so if a PRIVATE_KEY_BASE64 env variable is set, it would use that and automatically base64 decode.

And... what's the "ugly workaround"? I'm using latest canary and there are no errors, it is deployed, but seems to not receive the webhook events?
I deployed testing app here https://semantic-release.now.sh and 1) for some reason redirects to /probot 2) from where is that index coming really, huh? 3) nothing happens. And it's not a problem in the bot or in my side, cuz locally it works.

@olstenlarck I don't know enough about Now to tell you why webhooks aren't being received, but the /probot redirect exists for when your app's web server doesn't have its own homepage. This is mostly for you to know that something is working (your server is running).

It's okey. But should my webhook url include the /probot or not? ;d

@olstenlarck your webhook URL should not include Probot; if you'd like, you can join the Probot Slack and I can help you out 😸

Yea, got it :D Found another host temporary, it's live now https://github.com/apps/semantic-release-app :)

And... what's the "ugly workaround"? I'm using latest canary and there are no errors, it is deployed, but seems to not receive the webhook events?

How are you supplying the private key? Maybe it's fixed in the newest canary, but basically now.sh wouldn't handle the newlines well as either an environment variable or secret.

mmm.. using cat, in latest canary.

Just spent hours trying to get private key to work on now and it turns out a base64 hack is necessary. I would suggest this hack to be mentioned on the deployment page : )

Sorry for the frustrating experience :( could you start a PR to add information on it and we discuss it there?

@gr2m there you go.

that would be great because this issue is aways useful


Thor dependency problem

Because I'm running foreman in my app (which requires thor 0.19.1) I can't use shopify_api versions after this commit: https://github.com/Shopify/shopify_api/commit/a56318ab185f1f9a8b323da97ecd16bfb0436b26
So versions > 3.2.1 give the following error:
Bundler could not find compatible versions for gem "thor":
  In Gemfile:
    shopify_api (~> 3.2.7) ruby depends on
      thor (~> 0.18.1) ruby

    foreman (~> 0.78.0) ruby depends on
      thor (0.19.1)

Using gem 'shopify_api', '~> 3.2.0' in my gemfile loads 3.2.1 instead of the latest version because of this issue.
Anyone deploying a Shopify app to Heroku will run into this issue.

Version 4.0.0 will completely removes the thor dependency from the project. If you upgrade to that this should be resolved.


Running React Native app with mattermost-redux fails.

Summary
Running React Native app with mattermost-redux fails.
Steps to reproduce

npm i  mattermost-redux --save
react-native run-ios

Expected behavior
The React Native app should start without error.
Observed behavior

error: bundling failed: Error: Unable to resolve module core-js/modules/web.dom-collections.iterator from /Users/node_modules/mattermost-redux/store/configureStore.dev.js: Module core-js/modules/web.dom-collections.iterator does not exist in the Haste module map

Possible fixes
Install additional deps

@react-native-community/netinfo
core-js
remote-redux-devtools

@avegrv as you can see in https://github.com/mattermost/mattermost-mobile that is a react native app and the use of mattermost-redux just works, perhaps those deps should be included in your own project?

Correct, those deps should be included in my own project, but unfortunately, it's hard to guess.
It seems to me the best solution would be to exclude these dependencies from the configureStore call.

Possibly. Perhaps they can be added as peerDependencies in this project. Would you mind submitting a PR?

Yes, Elias, I will try tomorrow. Thank you.

@enahum Fixed at #981. I found that the problem was with core-js.
Also, I will create a separate issue to remove offline features of library. I think that code that depends on:

redux-offline
@react-native-community/netinfo

must be on the mobile version of mattermost. Because working with offline is the responsibility of the mobile application only. Let's disccuss it in separate issue – #982.


unable to log in to PNC banking on Firefox

Description
 cannot log in to www.pnc.com; will get an error message. Turning of the extension fixes the issue.
Steps to Reproduce

step 1 use your username and password to log in to www.pnc.com
step 2 behold the error message.
...

Expected behavior:
 should be able to log in to my account
Actual behavior:
 error message
Versions

Extension: 20xx-xx-xx
Browser:
OS:

Additional Information

This happens on multiple websites, including chipotle.com and walgreens.com. The issue seems to be related to CORS. When I attempt to log in with the extension, I get the error CORS Missing Allow Origin

Versions:
OS: Ubuntu 20.04.2 LTS
Browser: Firefox 87.0

Thanks for the reports. We have heard that PNC are now only supporting Chrome and Edge browsers, and blocking logins from other browsers. @ranimal19 which browser are you using?
@kevinrtai Thanks for reporting those issues. I was able to reproduce the chipotle issue, and we'll fix that ASAP. Could you perhaps describe the issue you're having with walgreens.com so we can look into that one too?

@sammacbeth I am using firefox. I can login to PNC when I turn the DuckDuck extension off.

@ranimal19 are you able to check again if our list update fixed the issue?
If not we can disable all protections for a short while on this site, whilst we debug further.


Remove redundant app labels in control plane.

The app label should be reserved for end-user applications and we
shouldn't use it ourselves. We already have a Conduit-specific label
we can use instead, so just use that one.
Signed-off-by: Brian Smith<EMAIL_ADDRESS>

This PR is based on #365. I will rebase it onto master before merging.


Missing floor & ceil

If division is possible, floor and ceil should be present
granted rounding has some difficulty (what to do at xxx.5 and whether to do account rounding)

Hi @djsegal!
What do you mean? You'd like to see floor and ceil helpers added or you'd like to see rounding automatically applied when doing division?

Floor and ceil helpers


--scope machine for zip install type incorrectly inherits permissions

Brief description of your issue
Using zigtools.zls as an example.
Folder where winget unpacks files itself have correct permissions, but all files inside that folder, doesn't.
Files and folders inside will incorrectly inherit (or do not inherit at all, IDK) permissions.
That will cause for non admin users inability to access/use anything what was installed in “machine” scope, which breaks whole purpose of installing machine-wide
Steps to reproduce
winget install --scope machine -e --id zigtools.zls
Expected behavior
Full permission inheritance (achieved manually by disabling and enabling back inheritance):

Actual behavior
Partial permission inheritance:

Environment
Windows Package Manager v1.7.11261
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22631.3593
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.22.11261.0

Winget Directories
-----------------------------------------------------------------------------------------------------------------------
Logs                               %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\Diag…
User Settings                      %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\sett…
Portable Links Directory (User)    %LOCALAPPDATA%\Microsoft\WinGet\Links
Portable Links Directory (Machine) C:\Program Files\WinGet\Links
Portable Package Root (User)       %LOCALAPPDATA%\Microsoft\WinGet\Packages
Portable Package Root              C:\Program Files\WinGet\Packages
Portable Package Root (x86)        C:\Program Files (x86)\WinGet\Packages
Installer Downloads                %USERPROFILE%\Downloads

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Admin Setting                             State
--------------------------------------------------
LocalManifestFiles                        Disabled
BypassCertificatePinningForMicrosoftStore Disabled
InstallerHashOverride                     Disabled
LocalArchiveMalwareScanOverride           Disabled

How to fix (manually)
Steps (using example):

Right click on folder C:\Program Files\WinGet\Packages\zigtools.zls_Microsoft.Winget.Source_8wekyb3d8bbwe.
Click Security tab.
Click Advanced button.
Click Change permissions button.
Check checkbox Replace all child object permission entries with inheritable permission entries from this object.
Click OK button.
Confirm by clicking Yes.
Click OK button.

How to fix (manually) for those who wonder
Steps (using my example):

Right click on folder C:\Program Files\WinGet\Packages\zigtools.zls_Microsoft.Winget.Source_8wekyb3d8bbwe.
Click Security tab.
Click Advanced button.
Click Change permissions button.
Check checkbox Replace all child object permission entries with inheritable permission entries from this object.
Click OK button.
Confirm by clicking Yes.
Click OK button.

Same or similar issue with OpenTofu.Tofu. "C:\Program Files\WinGet\Packages\OpenTofu.Tofu_Microsoft.Winget.Source_8wekyb3d8bbwe\tofu.exe" has the following permissions:

Install command: winget install -e --id  OpenTofu.Tofu --scope machine --verbose
Winget version: v1.8.1911


New score request: Fractions Skill Score

Scale-Selective Verification of Rainfall Accumulations from High-Resolution Forecasts of Convective Events in: Monthly Weather Review Volume 136 Issue 1 (2008) (ametsoc.org)
https://www.researchgate.net/publication/269222763_Fast_calculation_of_the_Fractions_Skill_Score
Scores users have requested the Fractions Skill Score to be added. The above two references provide information on the definition of the score and how it could be implemented.

We can confirm that the implementation works by comparing it to Nathan Faggian's implementation copied over at https://github.com/nathan-eize/fss/blob/master/fss_core.py using xr.apply_ufunc

The obs & fcst fields are compared against a threshold field. The result is a binary field for each.
The alg mainly focuses on a 2-D binary field (but can probably be generalized). The cumulative sum (or integral image) of the 2-D binary field is a major part of the computation; which is done once, and cached for the obs and fcst binary fields. The cached fields are then used to compute the fractionals over various windows (typically fixed-size sliding windows).
Since addition is associative, the cumulative sum computation can be decomposed quite easily into parallel operations. e.g. row first then columns (looks like this is similar to what Nathan F. has done).
Once that's done the fractional for a window can be computed using 4 reference points e.g.
     0,0
       ┌────┬────────────┐
       │    │            │
       │   a│           b│
       ├────┼────────────┤ ▲
       │    │\\\\\\\\\\\\│ │
       │    │\\\\\\\\\\\\│ │
       │    │\\\\\\\\\\\\│ │ m
       │    │\\\\\\\\\\\\│ │
       │    │\\\\\\\\\\\\│ │
       │    │\\\\\\\\\\\\│ │
       │   c│\\\\\\\\\\\d│ │
       └────┴────────────┘ ▼
            ◄────────────►
                  n
       shaded fractional = (d - b - c + a) / (m x n) for an arbitrary n by m window
       where a,b,c,d are reference values from cumsum field w.r.t the bounding box of the window.

The resulting fractionals for obs and fcst can then can be used to calculate the fractional skill score (fss) for given a set of windows (typically fixed-size, sliding).
The alg shouldn't be too complex:
Cache

Apply threshold over the obs/fst fields to generate the binary fields
Compute cumulative sum over rows (can be optionally be sped up using dask)
Repeat above for columns
We now have the cumulative sum of the obs/fcst binary fields which can be used to derive the fractional fcst/obs (p_f/p_o) for a given window

FSS

Given a set of windows, derive p_f & p_o from the cache i.e. using the bounding box computation described above. (can be optionally be sped up using dask)
Accumulate the fractional scores for each window to compute the final metric.

[optimization ]
This is purely optional and requires some investigation.
A potential improvement to the above algorithm, is to utilize sparsity of the binary fields. Assuming that the choice of threshold results in a sparse field; i.e. count of ones << n x m note that the sparsity assumption also holds if  count of ones ~ n x m => count of zeros << n x m.
In such a scenario we can retrieve a semi-sorted set of sparse points k << n x m in the N-D space and construct a data structure (e.g. kd-tree) for quick insert and search. Each data node will contain the coordinate as well as the cumulative sum which can be computed by the same method above using d = b + c - a + v, where d is the cumulative sum and v is the value at the coordinate.
When computing the fractional for the windows, we just take the closest data node to the each point on the bounding box of each window, and use them instead. Given that the data is sparse, these computations can also be cached; as neighbouring windows are likely to re-use the same set of four reference points.
While I believe this will greatly speed up computations for sparse fields, I'm not sure how it will perform if p(1) ~= p(0), but this should be a trivial investigation using binomial sampling. Assuming that the above only works well for sparse matrices, we could attempt to infer it. For inference, random sampling approach could be used which is computationally cheap; alternatively we could pass in an optional argument for the user to decide.

Implementation considerations
After some research, I believe that the python implementation referenced above provides a good trade-off between various approaches. The only thing to note is that if we intend to use dask - e.g. via xarray, it may potentially hinder in-built numpy parallelisation, and would have to be used with care.
Some notes here:

Current numpy versions have dist wheels that come with OpenBlas. This means the operations automatically use multiple threads. In theory we need not worry about low level optimizations like parallelising cumsum. If dask is being used for higher level parallelisation, we should limit the numpy threads. While this comes at a trade-off, it maybe required in certain environments. We may need a separate issue to investigate this if it becomes an issue. Details here: https://numpy.org/install/#numpy-packages--accelerated-linear-algebra-libraries
[optional] For stride-lengths of 1, the Integral image calculation is often not necessary, since we can compute the 2-D sliding window directly by decomposing it into sliding 1-D prefix sums, with no trade-off in performance. While this reduces the need to cache/compute the integral image, it does not expand nicely if we want to consider arbitrary stride lengths.
[beyond scope] While the kd-tree algorithm above is interesting in theory, and is used in other similar applications (sliding window template detection, partial summed volume tables in sparse volumetric data etc.); performance gains requires reasonable sparsity, with 1000x1000 my rough calculation shows that the data populace should be <5%, which may not be very often in this application. There are partial trees, and sqrt trees that improve upon this, but probably non-trivial for this exercise.
[beyond scope] Note that GPU implementations further improve on computations by using work efficient algorithms for computation of prefix sums, but this is beyond scope of this issue. See: https://developer.nvidia.com/gpugems/gpugems3/part-vi-gpu-computing/chapter-39-parallel-prefix-sum-scan-cuda

numba has a stencil(neighbourhood=...) convenience decorator we can also try out, which makes the implementation look straightforward: https://numba.readthedocs.io/en/stable/user/stencil.html#neighborhood. It may be worth benchmarking against a systems language like rust. I think we need to investigate the following:

baseline with rust
numba
numpy + blas
dask + numpy (this should implicitly test xarray as well)

As a side note, numba has the additional benefit of being able to interface to cuda quite easily. A lot of metrics in theory would benefit from accumulation operations (e.g. CDFs), so findings from this may apply to other parts of scores too.

Sum area table
Some brief timings show that multi-threading only really provides benefits at around 1000x1000, where the implementations are tied at around 5-20ms.
For 4000x4000 however, we have:

rust/c: 50ms
numba: 120ms
numpy: 800ms

I still need to profile the entire sliding algorithm, however this should be around the same ballpark as its roughly the same order of operations.

Was out of action last week. Here are some benchmarks.
Meanwhile:

tried out opencl implementation
tabulated the bench results

Conclusions:

I don't think it's worth adding in a rust/opencl implementation at the moment. I believe we're not likely to go to resolutions 4000x4000 and beyond to reap the benefits. See table at the end.
numba performs really well for this operation
numbagg for some reason doesn't perform any faster than plain numpy, despite using the in-built decorators. But that may be due to my ineptitude in using it. Will reinvestigate later...
While I didn't explicitly investigate dask, I believe that within a singular compute device dask may not provide significant advantages, as threads are very saturated already during the benchmarks. However, for HPC/multi-device situations, it maybe useful for async optimizations and scheduling.

Here's the implementation plan:

Most of the initial implementation in numpy + numba as a extra dependency.
Initial notebook showing how to aggregate results e.g. across time/levels using dask.
Add in the benchmarks for rust/opencl for the curious - though will not integrate it as an extra dependency.

Device
------
11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
Cores: 4 x 2
Mem: 4GB
OS: WSL-2 Ubuntu 22.04 Linux
GPU: Intel Integrated Graphics (Shared memory with host cpu)

Bench
-----
                +-----------+-----------+-----------+
                | 1000x1000 | 2000x2000 | 4000x4000 |
                +===========+===========+===========+
          numpy | 40ms      | 200ms     | 1.2s      | **
        numbagg | 200ms     | 400ms     | 1.2s      | **
          numba | *7ms      | *35ms     | 170ms     | <1>
  rust (native) | 15ms      | 46ms      | 160ms     | <2>
  rust (opencl) | 12ms      | 36ms      | *120ms    | <3>
                +-----------+-----------+-----------+
                **  - estimated using sum area table computations and factored by 3, due to
                      speed reasons at higher resolutions
                <1> - note: native numba - cached jit parallelization
                <2> - note: unoptimized - uses data copying liberally
                <3> - note: kernel building not optimized - can potentially be sped up with
                      caching

I just realised, that I've done fss scoring computations per window as opposed to accumulating the mean square of the window populace.
This adds an additional O(N), N = num windows, which is, in general, less than the image data size, so the final timings should not be more than double what's shown in the table above, in-fact we'll be avoiding extra operations per window. In any case, It might be a good time to compare against Nathan's code to verify - I'll add this to the benchmarking table.

Further note: since this is a summation operation as opposed to a prefix sum (which is more complex to vectorize), it can be trivially reduced further to O(N log2(P + 1) / P), P = num threads.

Ok so I compared it against Nathan's code (fast-fss). The results are below.

As a side (but maybe important) note: I realized after re-doing the benches with Nathan's code that numba may be actually caching some of the results (I suspected this earlier). This isn't inherently a bad thing, but the benchmarks in the previous comment above may be under-estimating numba timings, due to the way timeit was used. The following timings however, are accurate and make sure to use uniquely generated inputs and local variables every iteration.

Conclusions:

Numba is ~15-20x faster
EXCEPT for the first fss computation, which probably initializes the kernel. It roughly this takes 4s in this scenario due to the relatively large size of the intermediate buffers. (This can be circumvented by using a static build and accounting for a maximum possible buffer size). Subsequent computations of fss of the same image dimensions (but different input values) are much faster.
The scores are the same to at least 6 decimal places on visual inspection (but only 4 shown in table below), for the 100 x 100 sliding window.
Odd image dimensions were used in the tests, since not all inputs will be nice NxN multiples of 1000.
However, window sizes were restricted to square size as this is also the case in Nathan's code.

obs_dist=N(0,1)
window_size=(100,100)

tests verified with flipped images as well; image dimensions
(2429,1513) and (1513,2429) have the same results.

-----------------------------------------------------------
TEST 1: standard fcst
-----------------------------------------------------------
Setup:
fcst_dist=N(0,1)
window_size=(100,100)
threshold=0.5
image_size=(2429,1513)

         +----------+-----------+
         | fast-fss | numba <1> |
         +==========+===========+
fss      | 0.9998   | 0.9998    |
         +----------+-----------+
walltime | 2.1s     | 135ms     |
         +----------+-----------+

<1> kernel initialization: 4.7s

The above result is expected since they have the same pdf.

-----------------------------------------------------------
TEST 2: biased fcst
-----------------------------------------------------------
Setup:
fcst_dist=N(1,1)
window_size=(100,100)
image_size=(2429,1513)
threshold=0.5

         +----------+-----------+
         | fast-fss | numba <1> |
         +==========+===========+
fss      | 0.7444   | 0.7444    |
         +----------+-----------+
walltime | 2.05s    | 134ms     |
         +----------+-----------+

<1> kernel initialization: 4.8s

-----------------------------------------------------------
TEST 3: variant fcst
-----------------------------------------------------------
Setup:
fcst_dist=N(0,4)
window_size=(120,80)
image_size=(2429,1513)
threshold=0.5

         +----------+-----------+
         | fast-fss | numba <1> |
         +==========+===========+
fss      | 0.9327   | 0.9327    |
         +----------+-----------+
walltime | 1.98s    | 111ms     |
         +----------+-----------+

<1> kernel initialization: 4.3s

Am I reading correctly that the kernel initialization overhead is more than the time taken by numpy to do the work? Doesn't that mean that ultimately numpy is just faster? I'm not sure how often people will be doing the calculations on very large data sizes either. I am wondering if having the standard implementation be numpy, then putting another implementation into scores.optimised for the numba implementation would make sense. Maybe I have misunderstood what is going on with the kernel initialisation.

I was planning on base implementation be numpy, numba will be a optimised version.
Re: kernel initialization. That’s a one time thing. The timings reported are an average over 20 iterations of unique input data excluding initialisation (I should have probably mentioned that). If the user is planning on processing any more than 1 field, numba is faster. The other thing is, even that can be preemptively avoided as mentioned earlier; as we can do a static compile rather than jit on import. Other imports also take a few seconds to initialize anyway.

I've added all my initial implementations in this PR: https://github.com/nci/scores/pull/266
I fully expect to be told to re-arrange most of it. I'm just keeping it there for reference atm. The numpy and numba implementations will be brought into the main scores package. The examples can be run using the instructions here: https://github.com/nci/scores/tree/188-fractions-skill-score/rust
On another note, I noticed that the reference fss implementation accumulates the sum of squares before computing the scores. This can potentially overflow for larger fields - probably no one tried anything much larger than 1000x1000. In any case this doesn't affect the numpy/pure python implementations since they automagically handle big ints. We do have to be careful with using numba etc. though.

Following up from discussions, this issue will now only focus on the numpy implementation.
#266 contains lumped examples as an initial version so the work is in version control.
The idea is that #266 will only contain the numpy implementation.

The numba implementation will be cherry picked out and dealt with in #269 and it's associated branch
The rust implementation(s) will be cherry picked out and dealt with in #270  and it's associated branch (note: this will be merged back against an experimental branch first)

Aggregation across non-spatial dims
@nicholasloveday pointed me to this: https://journals.ametsoc.org/view/journals/mwre/149/10/MWR-D-18-0106.1.xml.
From what I understand, aggregation across other dimensions, is not dissimilar to the 2-D method. However, N-D "volumetric" images or convolutions are highly non-trivial to compute in reasonable time. However, if we make the assumption that only the spatial dimensions are aggregated using the sliding window, and every other dimension that is being reduced is just summed up to a scalar, we can use the following simplified algorithm.
Let's assume our data set is of the form lat x lon x time x lead_time => N x M x T x L
where,
    lat x lon = spatial dims

1. [Map] Map the input field into a list of fields to be aggregated over. i.e.

        lat x lon x time x lead_time -> [lat x lon] x lead_time,
        where,
            [lat x lon] is a list of T spatial fields.

2. [Compute] For each lead time compute the decomposed fss score for [lat x lon] (len = T).

        The decomposed FSS score is a tuple containing the components required
        to formulate the inal score:

        (obs_sum / T, fcst_sum / T, (obs_sum - fcst_sum) / T)

        The division by T is optional, but good to have for larger datasets to
        avoid integer overflow issues when aggregating.

3. [Reduce] Sum the decomposed score across the accumulated dimension.

        fss <- [0; 1 x L]
        for each leadtime, l {
            obs_sum <- sum_T(fss_decomposed[l][0])
            fcst_sum <- sum_T(fss_decomposed[l][1])
            diff_sum <- sum_T(fss_decomposed[l][2])
            fss[l] <- dff_sum / (obs_sum + fcst_sum)
        }
        return fss :: size := 1 x L

The reason why we need to keep track of the decomposed score instead of simply
aggregating the final score is because:

1/x + 1/y != 1 / (x+y)


Prefer blackbox macros

I noticed this while porting the macros to Dotty, and found this comment:

Also can't remember why these had to be whitebox instead of blackbox.

…but these definitely shouldn't need to be whitebox. Maybe there was an issue on 2.11? In any case tests pass with this change now.

Just noticed that illTyped can also be blackbox.


Default (nonces, threads) = (288, CPU*3/4) for Mainnet

Perhaps we should set threads to all cores on low end CPU with not many cores?

It will be hard to predict and still some spare for the system needed imho:)

bors merge

bors merge

bors merge

bors merge

bors merge


Prime Offload not working after switching to hybrid mode  - Ubuntu 22.04

Running __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia vkcube still uses intel iGPU:
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

Selected GPU 0: Intel(R) UHD Graphics (CML GT2), type: 1

neofetch:

I've run strace on sudo envycontrol --switch hybrid (if that's of any help):
strace: Process 102252 attached
read(0, "y\n", 1024)                    = 2
newfstatat(AT_FDCWD, "/lib/udev/rules.d", {st_mode=S_IFDIR|0755, st_size=135, ...}, 0) = 0
openat(AT_FDCWD, "/lib/udev/rules.d/80-nvidia-pm.rules", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_EMPTY_PATH) = 0
ioctl(3, TCGETS, 0x7ffe51ec9710)        = -1 ENOTTY (Inappropriate ioctl for device)
brk(0x55cd7174a000)                     = 0x55cd7174a000
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
write(3, "# Automatically generated by Env"..., 1177) = 1177
close(3)                                = 0
newfstatat(AT_FDCWD, "/etc/modprobe.d", {st_mode=S_IFDIR|0755, st_size=16, ...}, 0) = 0
openat(AT_FDCWD, "/etc/modprobe.d/nvidia.conf", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_EMPTY_PATH) = 0
ioctl(3, TCGETS, 0x7ffe51ec9710)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
write(3, "# Automatically generated by Env"..., 176) = 176
close(3)                                = 0
newfstatat(AT_FDCWD, "/etc/debian_version", {st_mode=S_IFREG|0644, st_size=13, ...}, 0) = 0
write(1, "Rebuilding initramfs...\n", 24) = 24
openat(AT_FDCWD, "/dev/null", O_RDWR|O_CLOEXEC) = 3
pipe2([4, 5], O_CLOEXEC)                = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
vfork()                                 = 106238
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
close(5)                                = 0
close(3)                                = 0
read(4, "", 50000)                      = 0
close(4)                                = 0
wait4(106238, 0x7ffe51ec8f1c, 0, NULL)  = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
wait4(106238, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 106238
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=106238, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
write(1, "Successfully rebuilt initramfs!\n", 32) = 32
write(1, "Graphics mode set to: hybrid\nPle"..., 78) = 78
write(1, "\n", 1)                       = 1
rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK, sa_restorer=0x7ff645c84520}, {sa_handler=0x55cd706c9990, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK, sa_restorer=0x7ff645c84520}, 8) = 0
munmap(0x7ff64531e000, 299008)          = 0
munmap(0x7ff645367000, 1048576)         = 0
exit_group(0)                           = ?
+++ exited with 0 +++

I've run prime-select on-demand and systemctl disable gpu-manager.service before switching to the hybrid mode.
xrandr --listproviders:
Providers: number : 2
Provider 0: id: 0x42 cap: 0x9, Source Output, Sink Offload crtcs: 3 outputs: 1 associated providers: 1 name:modesetting
Provider 1: id: 0x1fa cap: 0x2, Sink Output crtcs: 4 outputs: 8 associated providers: 1 name:NVIDIA-G0

any suggestions where the problems may reside will be welcomed with a lot of gratitude

Hybrid mode works out of the box without any tool, the only thing
Envycontrol does is enabling modeset and Preserve VRAM kernel driver
options.
On Sun, Jan 29, 2023, 09:00 danielkrajnik @.***> wrote:

I can't run __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only
__GLX_VENDOR_LIBRARY_NAME=nvidia glxgears - still uses intel iGPU.
I've run strace on sudo envycontrol --switch hybrid (shown below).
Noticed that xorg.conf isn't created and /usr/share/sddm/scripts/Xsetup
isn't modified either - I thought that they need to be changed? I haven't
found anywhere in the repo any suggestion that users should do it manually?
I was hoping that it will work out of the box :). Then again, I'm using
Ubuntu...
strace: Process 102252 attached
read(0, "y\n", 1024)                    = 2
newfstatat(AT_FDCWD, "/lib/udev/rules.d", {st_mode=S_IFDIR|0755, st_size=135, ...}, 0) = 0
openat(AT_FDCWD, "/lib/udev/rules.d/80-nvidia-pm.rules", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_EMPTY_PATH) = 0
ioctl(3, TCGETS, 0x7ffe51ec9710)        = -1 ENOTTY (Inappropriate ioctl for device)
brk(0x55cd7174a000)                     = 0x55cd7174a000
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
write(3, "# Automatically generated by Env"..., 1177) = 1177
close(3)                                = 0
newfstatat(AT_FDCWD, "/etc/modprobe.d", {st_mode=S_IFDIR|0755, st_size=16, ...}, 0) = 0
openat(AT_FDCWD, "/etc/modprobe.d/nvidia.conf", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_EMPTY_PATH) = 0
ioctl(3, TCGETS, 0x7ffe51ec9710)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
write(3, "# Automatically generated by Env"..., 176) = 176
close(3)                                = 0
newfstatat(AT_FDCWD, "/etc/debian_version", {st_mode=S_IFREG|0644, st_size=13, ...}, 0) = 0
write(1, "Rebuilding initramfs...\n", 24) = 24
openat(AT_FDCWD, "/dev/null", O_RDWR|O_CLOEXEC) = 3
pipe2([4, 5], O_CLOEXEC)                = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
vfork()                                 = 106238
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
close(5)                                = 0
close(3)                                = 0
read(4, "", 50000)                      = 0
close(4)                                = 0
wait4(106238, 0x7ffe51ec8f1c, 0, NULL)  = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
wait4(106238, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 106238
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=106238, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
write(1, "Successfully rebuilt initramfs!\n", 32) = 32
write(1, "Graphics mode set to: hybrid\nPle"..., 78) = 78
write(1, "\n", 1)                       = 1
rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK, sa_restorer=0x7ff645c84520}, {sa_handler=0x55cd706c9990, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK, sa_restorer=0x7ff645c84520}, 8) = 0
munmap(0x7ff64531e000, 299008)          = 0
munmap(0x7ff645367000, 1048576)         = 0
exit_group(0)                           = ?
+++ exited with 0 +++
—
Reply to this email directly, view it on GitHub
https://github.com/bayasdev/envycontrol/issues/77, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALY7IYJEHHCKLJQA4NOSAOTWUZZZPANCNFSM6AAAAAAUKIM4L4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

thanks, hmm I'm just not sure which steps I'm missing to get prime offload to work - sorry, I'm sure it's a really basic question, but I'm digging through strace logs on vkcube command just to figure out what's missing.
Envycontrol has helped me a lot already, but any suggestion how to make the prime offload to work would be really appreciated.

Seriously, this makes no sense. There is no information how to set it up at all

PRIME offloading works, try running Firefox with __NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia firefox and go to about:support it should say EGL Vendor is Nvidia Corporation.

thanks for checking this again, but it trying to run firefox with these variables didn't work - please see excerpt from the output of about:supprt here:

Try the same command after envycontrol --reset and rebooting
On Sun, Jan 29, 2023, 13:22 danielkrajnik @.***> wrote:

thanks for checking this again, but it trying to run firefox with these
variables didn't work - please see excerpt from the output of about:supprt
here:
[image: image]
https://user-images.githubusercontent.com/15096799/215347643-b90a7f5e-d2af-49eb-8a06-57c81f3b4aab.png
—
Reply to this email directly, view it on GitHub
https://github.com/bayasdev/envycontrol/issues/77#issuecomment-1407735093,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALY7IYPW6CPVHDYZ3E6INBDWU2YPBANCNFSM6AAAAAAUKIM4L4
.
You are receiving this because you commented.Message ID:
@.***>

@bayasdev Sorry, it yields the same results:

xrandr --listproviders
Provider 0: id: 0x42 cap: 0x9, Source Output, Sink Offload crtcs: 3 outputs: 1 associated providers: 1 name:modesetting
Provider 1: id: 0x1fa cap: 0x2, Sink Output crtcs: 4 outputs: 8 associated providers: 1 name:NVIDIA-G0

Sorry, I can't reproduce your problem but it sounds to me like some config in your setup is conflicting.
Since you have a Legion, you can disable Optimus from your BIOS and enjoy a better experience on Linux.

No worries, but I may try to install an entire system from scratch then, hopefully it will eventually click.

solved, I didn't install the nvidia driver properly. Running sudo apt install nvidia-driver-515 repaired it.