Tensor memory management and memory hierarchy redesign

From various discussions initial research was carried out to explore better and more optimized ways to handle memory management both on the CPU (heap) and GPU (host/device memory), mainly from points summarised in the discussion in #130 and #36.
This issue will mean we will close #113 and #36
The proposed design encompasses the following changes (which would be carried out one after the other):

Removing the concept of staging type tensors in favour of hostVisible type tensors and using two buffers per tensor
Removing the concept of persistent anonymous sequences (in conjunction with optensor not owning tensor memory)
Amending the memory hierarchy such that Manager owns tensors directly instead of OpCreateTensor
Remove std::vector mData; completely from Tensor in favour of always storing data in hostVisible buffer memory (TBC)

Below each of the points are covered in more details:
1) Removing the concept of staging type tensors in favour of hostVisible type tensors and using two buffers per tensor
This includes removing the concept of a "staging tensor". Instead we will now just have hostVisible, deviceOnlyVisible and storage (which is deviceVisible without src / dst copy). This also encompasses amending tensors to always have two vkBuffers and two vkMemories. This will optimise memory requirements and will reduce complexity from OpTensorCreate to need to hold a separate set of staging vectors only to perform the transfer.
2) Removing the concept of persistent anonymous sequences  (in conjunction with optensor not owning tensor memory)
This basically encompasses simply removing the concept of anoymous sequences to avoid implicit accumulation of memory from every time a new manager operation is run. This will re-use the same DEFAULT operation for anonymous functions. The main challenge will be that the OpCreateTensor will always need to be created in a named sequence (although this will be addressed in the next phase)
3) Amending the memory hierarchy such that Manager owns tensors directly instead of OpCreateTensor
This step removes the memory ownership from OpBase towards Tensor. This will require a potentially large breaking change, as the most intuitive way in which this can be added is by ensuring that always Tensors have to be created via the manager, which would need to be via one of the two options:

Running the mgr.buildManagedTensor(....) function, or;
Creating a new t = std::shared_ptr<Tensor(...)> and then "initialise/register it" with the manager via mgr.buildManagedTensor(t)

This will ensure that the Manager keeps track of all the relevant Tensors created, and ensures they are destroyed when the manager is destroyed. Of course, this wouldn't be required if you initialise the tensor with your own vulkan components, such  as:

Create a new t = Tensor(...)
Initialise it manually with t.init(vkdevice, vkinstance)

4) Remove std::vector mData; completely from Tensor in favour of always storing data in hostVisible buffer memory (TBC)
After doing some resarch  I was able to confirm:

All GPU devices will always have both HostVisible and DeviceVisible memory available for use
Performance of GPU hostVisible memory access from CPU is relatively efficeint (even if not as efficient as RAM), this AMD document mentions "up to 64 GB/s peak theoretical transport data bandwidth from CPU to GPU per card." - still needs to be further confirmed
It is possible to leverage std-compatible containers such as std:span-like containers (or msft's gsl/span) to have a container with non-owning reference to the host-visible memory

Due to these points, it may be possible to completely remove the std::vector mData attribut completely whilst still keeping the best of both worlds - namely 1) access to the memory that was added to the GPU, and 2) optimum memory usage (mainly by not storing an extra copy of the data).
This point would still need to be explored further as tehre are some nuances in regards to how this would be implemented in a memory-safe way, as well as ensuring that the python interface can be exposed efficiently, as ideally it can be built in a way in which it can be passed to and from python with minimal amount of copies (as it's not clear what the support for non-owning containers like span is)

This also encompasses amending tensors to always have two vkBuffers and two vkMemories.

If I'm not mistaken one buffer will be allocated on the CPU and one on the GPU, is that correct? Does this also apply to storage tensors? For them the CPU buffer seems redundant and with something like DNNs one would have a lot of storage tensors.

@alexander-g no, vkBuffer and vkMemory is always vRAM (GPU memory), the only difference is whether the memory is CPU / Host visible or GPU-Device-visible-only. You can have a look at the PR #137 which implements it

The main thing here is that instead of having the concept of Staging tensors, now there would be the concept of host-visible-gpu-memory tensors, and device-only-memory-tensors. For host visible it would only use a single buffer for host memory. For device-only in the case of TensorType::eDevice it would create an extra buffer for staging the data into the device, and for the case of TensorType::eStorage, it would only use one buffer. It could also be explored for functionality that only creates the staging the buffer for data transfer into the device - this would add an overhead when reading / writing into the device, but could be a handy option actually (which we can explore once the PRs are merged)

Both of the main PRs including phase 1, 2 and 3 have been merged. Closing this issue and adding an extra issue to explore #4 separately


Feature request: Explicit confirmation before experiment is started

For some worker types, the link expires after it has been started. In my experience, users do this all the time. You can tell them very clearly to not click the link unless they actually want to launch the experiment, but many of them do it anyway.
Would it be possible to have an explicit confirmation button that users need to click before the experiment is launched? I'm aware of the preview functionality, and I suspect that it can be made to work with OSWeb, but it strikes me as an unnecessarily complicated way to avoid the simple issue of users clicking on the link.

Hmm, that sounds feasible.

worker uses study link
if link is legit, JATOS shows page with confirmation button
only when clicked JATOS starts study

In the prefs we could have an option to show or not to show this confirmation button.
Let us think about it.

@smathot, we thought about it again and would propose a slightly different approach. We would change the preview feature.
Right now a study with preview is set up during link creation. For OpenSesame/OSWeb this has the problem that you can't set it up via export/import of a study. The checkbox has to be checked by the user of JATOS and they might not be aware of it.
We propose an additional setting in the study properties. Those you can write from OSWeb (you do already). This field could be something like a boolean 'isPreviewAllowed'. Although this means that you would need an additional JATOS component, additionally to the OSWeb one. This component would contain the confirmation button you are looking for and could potentially contain other stuff like introduction or consent.
The advantage is that you can determine this confirmation page completely by yourself (layout, text etc.) - the disadvantage is that you have to define an additional JATOS component (although this should be a one time thing).

This would be acceptable. It strikes me as a bit more complicated then necessary, but if you feel that this is the best route then we can certainly build this into OpenSesame.

It is versatile. You (and others) could define their own layout and
wording, using their own language of this confirmation page.
On Thu, Apr 18, 2019, 15:49 Sebastiaan Mathot<EMAIL_ADDRESS>wrote:

This would be acceptable. It strikes me as a bit more complicated then
necessary, but if you feel that this is the best route then we can
certainly build this into OpenSesame.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/JATOS/JATOS/issues/164#issuecomment-484517511, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB6V4BVFNEBSNBS5EZEJAKTPRB36PANCNFSM4HFRFV2Q
.

@smathot Are you still interested in this feature? I would implement this the next then.

Feature is part of 3.7.1


C++11 on Windows

Read this
http://stackoverflow.com/questions/23343080/c-to-c11-in-gcc-mingw-missing-function

This is a bit lazy.
Rename the issue to something reasonable, supply proper details and a since
you've obviously already got a fixed for it, supply a pull request.
Thanks.
On 13 May 2015 at 23:54, 1Conan<EMAIL_ADDRESS>wrote:

Read this
http://stackoverflow.com/questions/23343080/c-to-c11-in-gcc-mingw-missing-function
—
Reply to this email directly or view it on GitHub
https://github.com/brofield/simpleini/issues/14.

Well its 12AM when i posted it i couldnt make a proper post that ime

This fix won't work. It changes defines that should not be changed and breaks the Linux build. It appears to be just a blanket _WIN32 to _MSC_VER search and replace. I expect that you would at least critically evaluate each change to ensure that it is relevant and correct. Feel free to submit again when you have done so.


For Developers: Potential Addition to "op" for developers: download repository

Basically, Shimmering Obsidian already has all groundwork of replicating the functionality of this useful small shell script.
I am considering to add such a function as a modifier for op, e.g. fn + return.
Any devs think this would be useful, or have ideas/suggestions concerning this?

implemented with release 1.8


Update ns-schannel-sch_credentials.md

Added links to TLS_PARAMETERS structures.

@fran-ki : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@GrantMeStrength : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Thank you!


Unable to use package

I'm getting this error message after installing midjourney-api package, what might be a problem, and how to solve it?

#69


setState has problem

Hi
I got this:
Error - typescript - src/webparts/annualLeaveForm/components/AnnualLeaveForm.tsx(242,22): error TS2345: Argument of type '{ leaveType: any; }' is not assignable to parameter of type '{ leaveType: any; vacationRequest: any; }'.
export default class AnnualLeaveForm extends React.Component<IAnnualLeaveFormProps, { leaveType: any, vacationRequest: any }> {
this.setState({ vacationRequest: value[item] });
If i have two variables in state, i can't set only one? i am using node v8.11.4.
thanks
Peter

Has nothing to do with the Node version, we think you need to use the setState as official docus

in windows, compile is ok, but in mac fail. I solved it by adding ? after variable name. Now it can be compile in both windows and mac. thanks


Provide actionable help when get error of "provider not yet registered"

It is possible the provider commands work with is not yet registered, say acr
2 options:

Polish the error and tell users to run az prvoider register. This is xpalt does
Auto register it, and output a warning which tells users about this and prepare the command might still fail if the region was not propagated in time

Either way, we need to have a common step to capture the error.

@yugangw-msft : As discussed we should definitely do #1.
We can discuss #2 with ACR guys as the ARM team's recommendation is to avoid this.

We don't this auto register is a good option. Submitting a request on users behalf unintentionally seems wrong. But this applies broadly to the CLI and hence we will probably take your recommendation on this.


network veth interface leak in docker

Hi all,
When I run docker test in my local environment, the docker daemon would be killed while testing process, But after all the test stopped and stopped all the containers, I found that there are some network interfaces existing on host.
So I supposed there is network interface leak in docker. BTW, I use the bridge mode in docker.
After checking the libnetwork code, I found that if we killed docker after libnetwrok has created endpoint before saving it in blob, and the leak will be existing.
If we add time.Sleep(10*time.Second) before saving it to blob, after creating endpiont, it will be reproduced.  e.g.  add here: https://github.com/docker/libnetwork/blob/master/network.go#L1135
I have checked the libnetwork cleanup sandbox/network/endpoint mechanism. I think if we save the interface data into blob done, It won't leak at all.
So I wonder if we could update store first, and then create the endpoint ? Or any other good solution about this?
Thanks!

Reproduce steps:

call netowrk.CreateEndpoint
create Endpoint using bridge driver on host.
kill docker
should update store, but docker was killed.

===> after docker restart. will sync and remove the stopped container endpoints, but the veth pair was not saved in db. libnetwork could not remove it. the device will leak on host.
Actually, In my environment, we run docker for long time for testing (maybe docker crashed or killed at that time). Then I cleanup the environment I found the leak.
I have submit a patch. To save endpoint first, then create the interfaces. But I am not familiar with libnetwork, and do not know if it is a good way.
Is this solution ok for libnetwork?  Or is there any other ideas to fix this?

I  see different behavior when I  kill dockerd with two different way ( Ctrl C  and "kill -9 dockerd")
when I tried with Ctrl C, I see below output and veths are getting cleaned up properly.
DEBU[2017-11-17T22:35:12.179551174Z] daemon configured with a 15 seconds minimum shutdown timeout
DEBU[2017-11-17T22:35:12.179565223Z] start clean shutdown of all containers with a 15 seconds timeout...
DEBU[2017-11-17T22:35:12.179583726Z] stopping e48bf83d2f75b03441646a57df8413ff436b6737d860ba9c82835785eefc8f0f
DEBU[2017-11-17T22:35:12.179607669Z] Sending kill signal 15 to container e48bf83d2f75b03441646a57df8413ff436b6737d860ba9c82835785eefc8f0f
DEBU[2017-11-17T22:35:12.179721392Z] stopping e3aadcd6ed86df17f17fc00131e66770d08890981feba68aff9bbf201caaef34
DEBU[2017-11-17T22:35:12.179745774Z] Sending kill signal 15 to container e3aadcd6ed86df17f17fc00131e66770d08890981feba68aff9bbf201caaef34
DEBU[2017-11-17T22:35:12.179799428Z] stopping 03a4756b999baeccd1015fd7113d2e60994725df221aaabc3e896f6402a9fa41
DEBU[2017-11-17T22:35:12.179815334Z] Sending kill signal 15 to container 03a4756b999baeccd1015fd7113d2e60994725df221aaabc3e896f6402a9fa41
DEBU[2017-11-17T22:35:12.179863821Z] stopping cafa5f23efc00ff6698d3ace198b2b632e262ccf2dc889aa00b4c96df492e5c8
DEBU[2017-11-17T22:35:12.179879796Z] Sending kill signal 15 to container cafa5f23efc00ff6698d3ace198b2b632e262ccf2dc889aa00b4c96df492e5c8
INFO[2017-11-17T22:35:22.212720832Z] Container 03a4756b999baeccd1015fd7113d2e60994725df221aaabc3e896f6402a9fa41 failed to exit within 10 seconds of signal 15 - using the force
DEBU[2017-11-17T22:35:22.212763749Z] Sending kill signal 9 to container 03a4756b999baeccd1015fd7113d2e60994725df221aaabc3e896f6402a9fa41
INFO[2017-11-17T22:35:22.226901451Z] Container e3aadcd6ed86df17f17fc00131e66770d08890981feba68aff9bbf201caaef34 failed to exit within 10 seconds of signal 15 - using the force
DEBU[2017-11-17T22:35:22.227031710Z] Sending kill signal 9 to container e3aadcd6ed86df17f17fc00131e66770d08890981feba68aff9bbf201caaef34
INFO[2017-11-17T22:35:22.227830661Z] Container e48bf83d2f75b03441646a57df8413ff436b6737d860ba9c82835785eefc8f0f failed to exit within 10 seconds of signal 15 - using the force
DEBU[2017-11-17T22:35:22.227893245Z] Sending kill signal 9 to container e48bf83d2f75b03441646a57df8413ff436b6737d860ba9c82835785eefc8f0f
INFO[2017-11-17T22:35:22.239082131Z] Container cafa5f23efc00ff6698d3ace198b2b632e262ccf2dc889aa00b4c96df492e5c8 failed to exit within 10 seconds of signal 15 - using the force
DEBU[2017-11-17T22:35:22.239218178Z] Sending kill signal 9 to container cafa5f23efc00ff6698d3ace198b2b632e262ccf2dc889aa00b4c96df492e5c8
But when I use "Kill -9 dockerd"
none of the veth interfaces are getting cleaned up when I run dockerd again.
I also introduced 20 seconds delay after creating end endpoint but before saving into store. In this case also behavior with Ctrl C and Kill command is different.
Could you pls confirm how did you kill dockerd while testing ?  by looking at the logs and code, I believe Ctrl C takes proper clean up code path way and veth  is not  getting cleaned up when it crashes before store (blob) happens. Pls provide your input.

May I know which version are you using ? is it CE or EE ?
The reason I am asking is, when I kill dockerd with EE and My private image (pulled from moby/moby), I see two different behavior on host. with EE, I see containers are getting stopped and veth is getting removed .  But with my Private image , I  see containers are still running even after I kill dockerd and veth still present. I showed the behavior to TEC engineer Trapier and he was surprised to see the behavior . Pls let me know which version of docker  did you use ?
In the mean time I will pull EE source and try to make changes and see if I can reproduce the issue.
Earlier I was testing Docker-in-Docker setup but now now I am using Ubuntu VM as host .

Hi @selansen  sorry for late reply.
I used kill -9 <docker pid>, the reproduce process is as below:

add some sleep after creating the veth, and before storing it to db. (you have done it, good).
ip addr to check the network devices on host,
start docker, (you'd better add some logs, after creating veth, before storing it in db.)
kill -9 dockerd
ip addr to check to the veth,  they must be existing on host, docker does not have chance to delete them.
startup docker again. ( to check the veth again, if you do not see them, this means the veth has been saved in db, you should kill dockrd more earlier.  docker (or we can say libnetwork) will remove the veth which do not connected to container but record in db at startup).

I was using 1.11.2 version of docker, but the logic of the libnetwork code is the same. ( not docker in docker.  Ubuntu VM should be OK)

I Moved to EE latest version and I am able to reproduce the issue in Ubuntu host.
I faced different issue with CE edition (containers were running even after I kill dockerd). I checked the docker inspect and made sure all the parameters are correct.
Will continue to debug on EE version.

I just tried 1.11.2 version, it is very old. but for this issue, I think, it is not relative with docker version.
not familiar with latest version of docker.

Below is my testing and looks good to me.  can you pls confirm if I am testing it correctly?
Before killing dokcer D, VETH 373 & 374 got created for attaching network to a container
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet <IP_ADDRESS>/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
link/ether 12:da:01:fa:7d:1a brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/20 brd <IP_ADDRESS> scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::10da:1ff:fefa:7d1a/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:d3:19:e8:b4 brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:d3ff:fe19:e8b4/64 scope link
valid_lft forever preferred_lft forever
8: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:c2:c1:2e:09 brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global docker_gwbridge
valid_lft forever preferred_lft forever
inet6 fe80::42:c2ff:fec1:2e09/64 scope link
valid_lft forever preferred_lft forever
333: veth372a389@vethce2a52c: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default
link/ether 6a:92:a9:2a:83:99 brd ff:ff:ff:ff:ff:ff
334: vethce2a52c@veth372a389: <NO-CARRIER,BROADCAST,MULTICAST,UP,M-DOWN> mtu 1500 qdisc noqueue master br-48c75de92d13 state LOWERLAYERDOWN group default
link/ether d6:61:51:f2:d2:68 brd ff:ff:ff:ff:ff:ff
372: veth4f34f95@if371: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
link/ether b6:e2:8a:f3:59:03 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::b4e2:8aff:fef3:5903/64 scope link
valid_lft forever preferred_lft forever
373: vethb1b9eaf@veth4ac0e51: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default
link/ether a2:88:a5:29:26:e0 brd ff:ff:ff:ff:ff:ff
374: veth4ac0e51@vethb1b9eaf: <NO-CARRIER,BROADCAST,MULTICAST,UP,M-DOWN> mtu 1500 qdisc noqueue master br-48c75de92d13 state LOWERLAYERDOWN group default
link/ether 82:5f:78:75:6e:e8 brd ff:ff:ff:ff:ff:ff
170: br-dae4b713d51b: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:5f:df:c3:97 brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global br-dae4b713d51b
valid_lft forever preferred_lft forever
inet6 fe80::42:5fff:fedf:c397/64 scope link
valid_lft forever preferred_lft forever
250: br-48c75de92d13: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:d7:ec:f6:2b brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global br-48c75de92d13
valid_lft forever preferred_lft forever
inet6 fe80::42:d7ff:feec:f62b/64 scope link
valid_lft forever preferred_lft forever
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$ sudo kill -9 14073
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-dev/binary-daemon$
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/bundles/17.06.0-
After dokcerD comes back VETH 373 & 374 got deleted
====================================================================dev/binary-daemon$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet <IP_ADDRESS>/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
link/ether 12:da:01:fa:7d:1a brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/20 brd <IP_ADDRESS> scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::10da:1ff:fefa:7d1a/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:d3:19:e8:b4 brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:d3ff:fe19:e8b4/64 scope link
valid_lft forever preferred_lft forever
8: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:c2:c1:2e:09 brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global docker_gwbridge
valid_lft forever preferred_lft forever
inet6 fe80::42:c2ff:fec1:2e09/64 scope link
valid_lft forever preferred_lft forever
333: veth372a389@vethce2a52c: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default
link/ether 6a:92:a9:2a:83:99 brd ff:ff:ff:ff:ff:ff
334: vethce2a52c@veth372a389: <NO-CARRIER,BROADCAST,MULTICAST,UP,M-DOWN> mtu 1500 qdisc noqueue master br-48c75de92d13 state LOWERLAYERDOWN group default
link/ether d6:61:51:f2:d2:68 brd ff:ff:ff:ff:ff:ff
380: veth193ad10@if379: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
link/ether 4a:c8:13:f7:64:7f brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::48c8:13ff:fef7:647f/64 scope link
valid_lft forever preferred_lft forever
170: br-dae4b713d51b: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:5f:df:c3:97 brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global br-dae4b713d51b
valid_lft forever preferred_lft forever
inet6 fe80::42:5fff:fedf:c397/64 scope link
valid_lft forever preferred_lft forever
250: br-48c75de92d13: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:d7:ec:f6:2b brd ff:ff:ff:ff:ff:ff
inet <IP_ADDRESS>/16 scope global br-48c75de92d13
valid_lft forever preferred_lft forever
inet6 fe80::42:d7ff:feec:f62b/64 scope link
valid_lft forever preferred_lft forever
docker@ELANGO-TESTKIT-ubuntu-2:~/docker-ee/components/engine/

code merged, closed by https://github.com/docker/libnetwork/pull/2024


[Snyk] Upgrade @material-ui/core from 4.9.5 to 4.12.4

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade @material-ui/core from 4.9.5 to 4.12.4.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 24 versions ahead of your current version.
The recommended version was released 2 years ago, on 2022-04-03.

The recommended version fixes:

Severity
Issue
PriorityScore (*)
Exploit Maturity

Incomplete List of Disallowed Inputs SNYK-JS-BABELTRAVERSE-5962462
572/1000  Why? Proof of Concept exploit, CVSS 9.3
Proof of Concept

(*) Note that the real score may have changed since the PR was raised.

Release notes

Package name: @material-ui/core

4.12.4 - 2022-04-03

4.12.3 - 2021-07-30

4.12.2 - 2021-07-19

4.12.1 - 2021-07-07

4.12.0 - 2021-07-06

4.11.4 - 2021-04-27

4.11.3 - 2021-01-24

4.11.3-deprecations.1 - 2021-01-25

4.11.3-deprecations.0 - 2021-01-24

4.11.2 - 2020-12-02

4.11.1 - 2020-11-24

4.11.0 - 2020-06-30

4.10.2 - 2020-06-11

4.10.1 - 2020-06-01

4.10.0 - 2020-05-23

4.9.14 - 2020-05-11

4.9.13 - 2020-05-04

4.9.12 - 2020-04-26

4.9.11 - 2020-04-18

4.9.10 - 2020-04-11

4.9.9 - 2020-04-04

4.9.8 - 2020-03-28

4.9.7 - 2020-03-18

4.9.6 - 2020-03-18

4.9.5 - 2020-02-29

    from @material-ui/core GitHub release notes

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.536%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
637

Relevant Lines:
640

💛  - Coveralls


Unable to install

ParserError: D:\Download\install.ps1:322
Line |
 322 |          <svg class="octicon octicon-sign-out v-align-middle" viewBox= …
     |                      ~~~~~~~
     | Unexpected token 'octicon' in expression or statement.

That's probably because you downloaded the wrong file.
You should either clone this repo to your local disk and run it, or download the raw file from https://raw.githubusercontent.com/lextm/windowsterminal-shell/master/install.ps1

Oh, thanks!


API: Edit session identifier provider

Problem
There is currently support for Continue Working On with Edit Sessions from web to Codespaces, desktop to vscode.dev, desktop to Codespaces. This works by propagating an edit session identifier (a guid) through query parameters and passing it into the workbench creation options.
We would now also like to have Edit Sessions support for Continue Working On from web to desktop, desktop to desktop, etc. This would allow us to have full feature parity with the existing Continue Working On experience, as well as the ability to bring uncommitted changes with you. The following existing scenarios are to be supported:

github.dev -> clone locally
RemoteHub desktop -> clone locally
RemoteHub desktop -> clone in a container volume (Remote Containers)

To do this we have a few options:

Propagate the edit session identifier in all entrypoints to desktop, i.e.

Protocol handler for extensions (web -> desktop)
vscode.openFolder command args (desktop -> desktop)
This has the advantage of being extremely unambiguous about which edit session should be associated with the current workspace

Enable VS Code to figure out the relevant edit session identifier

Ultimately an edit session is an opaque JSON object consisting of some data to be applied to a particular VS Code state
We want enough metadata associated with this data so that we don't apply the wrong state
The relevant state here is repository identity
As initial customers, this API should be implemented by Remote Repositories and the builtin git extension
This API would also eventually be needed to enable the scenario where you never lose your RemoteHub changes, i.e. edit sessions are saved without explicitly invoking the store command or the Continue Working On command, and they are automatically resumed without explicitly invoking resume. That will be a complicated scenario to get right so we don't want to go after it yet
This proposal lets us have increased confidence that the latest edit session is relevant to the current workspace, without requiring us to introduce the edit session identifier into more places in VS Code (eventually potentially allowing us to remove our reliance on the query parameter approach in the web embedder case). It also potentially improves the reliability of the edit session resume mechanism, because we are not at the mercy of transient faults like the embedder dropping query parameters resulting in the edit session association being lost, for example

Proposal
Reference implementation: https://github.com/microsoft/vscode/pull/157733
Here are some examples of the actual structure of data that we'd put into the returned workspace identifier:

For https://github.dev/microsoft/vscode, https://vscode.dev/github/microsoft/vscode, as well as clones of https://github.com/microsoft/vscode:
{
    "remote": "https://github.com/microsoft/vscode",
    "ref": "main",
    "sha": "0eac052eb7f7bcd42c587f91b300fd07b89a3ba2"
}

For https://vscode.dev/azurerepos/monacotools/Monaco/Monaco?path=%2F&version=GBmaster and clones of https://dev.azure.com/monacotools/_git/Monaco?path=%2F&version=GBmaster:
{
    "remote": "https://dev.azure.com/monacotools/_git/Monaco?path=%2F&version=GBmaster",
    "ref": "master",
    "sha": "10c286cfdf73ea7429df686fa5a382d87777f99c"
}

@TylerLeonhardt also proposed that we could use this API to disambiguate repositories cloned across different machines, so that a user would only have to trust a given repository once across all their clones of the same repo.

@jrieken @bpasero and I chatted about this API proposal today since we ran out of time during the usual API call. The major feedback was:

In future if this API is used for workspace trust or roaming workspaces, it should probably be renamed from EditSessionIdentity to WorkspaceSessionIdentifier or similar
The current implementation assumes that there can only be one valid provider for a workspace with file scheme, however it should eventually be possible that a given workspace may have multiple providers e.g. if it is backed up to multiple remote hosting services. In this case:

the API implementation needs to either tolerate multiple providers registering for the same scheme, or otherwise introduce more expressive registration so that I can have identity providers from the builtin git extension and OneDrive extension for example
the edit sessions implementation and UI should support displaying edit sessions associated with multiple sources

If a user wants to continue working on a workspace that currently lacks an identifier, there should be an onboarding path for them to use edit sessions, which would likely create additional API needs
In future when we support auto-backup of edit sessions, we should see if edit sessions can benefit from the work that local history has done to make periodic backups performant
There may be UX implications of the autoStore setting since communicating with a cloud service before window unload is likely to be long-running
We could support multiple edit sessions available for a given identifier (e.g. originating from multiple machines) without needing to modify this API (e.g. by storing a manifest of all edit session GUIDs associated with a given identifier)

Please feel free to edit if I missed anything!


Adding MockFileData overload to serialize objects

Added a simple overload to include object serialization to the MockFileData class. I moved the common code to a private method and called it from both the old constructor and the new overload.

Thanks for your work, but I don't think this is something that needs to be in System.IO.Abstractions for the following reasons:

the discoverability of the new overload is low, I wouldn't intuitively know what it does
this is a low-level library for IO operations that should stay focused
it is easy (and IMO more flexible) to do the serialization on top of S.IO.A

So, I'm sorry, but I'll reject this PR.
For the future I'd suggest you to create an issue that allows discussing the idea before starting a PR.

Quick question: If you wanted discussion is there any reason it couldn't be done... right here? In the PR. Where GitHub has provided a space for it?

Sure, PR is also fine, but needs more effort on your end (writing the code) and also for the maintainers (reviewing the changes). For new things it’s IMO worthwhile to discuss the idea first (only providing pseudo code if necessary).


Removed minimist dependency

This closes #618.
I couldn't find any uses of arguments in the installation/usage guides. I also couldn't find the dependency being included in this project.
Projects depending on this project should still be checked of course. I'm not aware of the breadth or complexity of this search.
I know that it's a small PR but I hope this helps!

Hi @dylmye thanks for raising this pull request. As mentioned in the linked, our users might be using this dependency, so it might be a breaking change. We'll discuss this and determine when we can merge it. We'll update with a comment here.,
Thanks again


GOOGLE DOC LINK WITH EACH TEST

https://docs.google.com/document/d/1UAZYvAZMBhmQF52q1DtX7g8f4lYsf17AjEIIr0MFezI/edit?usp=sharing

@jalen941 Your group must submit the full document via D2L, rather than posting it here (per the instructions).


Gitlab permissions are too open for source checkout

[WARNING]: Ansible is being run in a world writable directory
392(/builds/crc4/um6p/infrastructure/um6p-kayobe-config), ignoring it as an
393ansible.cfg source. For more information see
394https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-
395world-writable-dir

Options:

Copy it else where and change permissions
Change permissions in place
Get gitlab to checkout with different permissions (nothing immediately obvious in the docs)
set umask in image to something more restrictive?

Looks like it uses a volume, docker inspect output:
        "Mounts": [
            {
                "Type": "volume",
                "Name": "runner-bkqvjp4m-project-21118777-concurrent-0-cache-c33bcaa1fd2c77edfc3893b41966cea8",
                "Source": "/var/lib/docker/volumes/runner-bkqvjp4m-project-21118777-concurrent-0-cache-c33bcaa1fd2c77edfc3893b41966cea8/_data",
                "Destination": "/builds",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            },

Directory is owned by root:
[stack@runner-bkqvjp4m-project-21118777-concurrent-0 um6p-kayobe-config]$ stat .
  File: .
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: fd03h/64771d    Inode: 198967306   Links: 7
Access: (0777/drwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-02-11 14:22:20.024386985 +0000
Modify: 2021-02-11 14:22:12.590730648 +0000
Change: 2021-02-11 14:22:12.590730648 +0000
 Birth: -

Turns out there is a feature flag for this (only recently added): https://gitlab.com/gitlab-org/gitlab-runner/-/blob/d1ad54103350cacf52f2e1134edbfb836914dc83/docs/configuration/feature-flags.md


minor issue: pure middleware can return nil (reframe 0.4.1)

I'm just starting to use re-frame (the pattern and implementation), and it looks terrific.
The documentation is also really helpful (and rather inspiring to boot).
In looking over the built-in middleware as a model for my own, I noticed a minor issue
and thought I'd mention it.  The end of pure (along the main code path) looks like
(if-not (identical? db new-db)
  (reset! app-db new-db))

which will return nil if the handler does not change the database.
I realize that pure is applied as the outermouse layer with register-handler,
so this causes no problems in typical use. But on the chance that someone
builds a different registration path using register-base or something else
unusual, it seems reasonable (and cost free) to have pure conform to
the expected behavior, with:
(if-not (identical? db new-db)
  (reset! app-db new-db)
  db)

As I said, a minor issue at most.
Thanks again.

Hey, thanks for the report,  but I'm not sure this is a bug or a required change.  pure is always the first middleware. No need for it to return anything.

Hi Mike, I did address that issue explicitly in my comment.  But I agree
that that is an unlikely and minor case. No worries. And thanks again.
On Wed, Nov 4, 2015 at 7:48 PM, Mike Thompson<EMAIL_ADDRESS>wrote:

Hey, thanks for the report, but I'm not sure this is a bug or a required
change. pure is always the first middleware. No need for it to return
anything.
—
Reply to this email directly or view it on GitHub
https://github.com/Day8/re-frame/issues/127#issuecomment-153917788.


[Error tooltip] why element add？it is Influencing layout

class name conflict

any workaround for this one?

This lassName is very prone to conflict

any workaround for this one?

This was really a problem in our application so I made a fork to fix it, you can find it here
https://github.com/mcbattirola/vue-wordcloud-tooltip-fix
But I think the best solution is a PR to fix this here

I agree with this idea and we will do it later, sorry


export xstyled/packages/emotion/src/create.ts

Summary
If you'd like to import cssCreate function with xstyled/emotion, xstyled/packages/styled-components/src/create.ts module should be exported in the index file.
Related Issues

https://github.com/gregberge/xstyled/issues/295

Thanks @iktakahiro


See hotwired/turbo#684

I've opened https://github.com/hotwired/turbo/pull/684, which changes the way that custom actions are defined from directly accessing StreamActions to defining a turbo:before-stream-render event listener.
If that change ships before a 7.2.0 release, this package will need to change how its morphdom integration is defined.

Thank you for the heads-up!

Is this issue related to this error I'm seeing?
js     | ✘ [ERROR] No matching export in<EMAIL_ADDRESS>for import "StreamActions"
js     |
js     |     ../../node_modules/turbo-morph/dist/index.js:1:7:
js     |       1 │ import{StreamActions as e}from"@hotwired/turbo";var t;var n="undefi...
js     |         ╵        ~~~~~~~~~~~~~
js     |
js     | /Users/drnic/Projects/insurance-policies/insurance-policies/node_modules/esbuild/lib/main.js:1624
js     |   let error = new Error(`${text}${summary}`);
js     |               ^
js     |
js     | Error: Build failed with 1 error:
js     | ../../node_modules/turbo-morph/dist/index.js:1:7: ERROR: No matching export in<EMAIL_ADDRESS>for import "StreamActions"

I've on turbo-rails 7.1.3 / turbo 7.1.0.

Ah, I see in the readme that I need to wait for turbo 7.2.0.

I think we can close this one since Turbo 7.2 ships with the StreamActions export and the turbo:before-stream-render event listener.


Autoscroll when opening submenus in sidebar

If the user opens a submenu in the sidebar located at the bottom of the viewport, he will need to manually the opening menu into view.
I implemented a quick autoscroll functionality which scrolls the submenu into view after opening it.
This is not an optimal solution as it has to wait for the opening animation to complete.
The slideToggle function is hard to modify as the function code is minified.

That's great! Thank you very much for your contributions 😁


update fails on 23.04 Lunar Lobster

Reporting a bug 🐛
Expected behavior
The local files are updated.
Packages that do not support the current release should be either skipped or updated as is.
Actual behavior
Update fails because the current version is not supported by all packages.
Steps to reproduce

run deb-get update
bug reproduces for mergerfs and stops the update process

System information
/etc/os-release
cat /etc/os-release

PRETTY_NAME="Ubuntu 23.04"
NAME="Ubuntu"
VERSION_ID="23.04"
VERSION="23.04 (Lunar Lobster)"
VERSION_CODENAME=lunar
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=lunar
LOGO=ubuntu-logo

/etc/lsb-release
cat /etc/lsb-release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=23.04
DISTRIB_CODENAME=lunar
DISTRIB_DESCRIPTION="Ubuntu 23.04"

deb-get version
deb-get version

0.3.7

deb-get update
deb-get update

  [+] Updating /etc/deb-get/01-main
/etc/deb-get/01-main.repo             100%[======================================================================>]   2.70K  --.-KB/s    in 0s
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100  102k    0  102k    0     0  49912      0 --:--:--  0:00:02 --:--:-- 49912
/home/gal
  [+] Updating cache of supported apps in the background
  [!] ERROR! Missing required information of github package mergerfs:
URL=
VERSION_PUBLISHED=

This should already be fixed by #774
Seemingly related to/duplicate of #756 and #763

The same error reproduces when installing the latest release from this repository.
sudo dpkg -i deb-get_0.3.7-1_all.deb


Adding more languages for translation.

Salam Alaikum brother
I added an extra English translation (Clear Quran), alongside 4 new languages (Farsi + Portuguese + Italian + Dutch). Please review and see if it looks fine.
Thank you

Jazakallah Khair

Salam brother
Thanks for accepting the PR, just wanted to enquire on how does it get updated on pub.dev?
Jazakallah

Wa alaikumussalam wa rahmathullah,
Updated! v1.2.6


Php 8.0

Quick test regarding the issue: https://github.com/thephpleague/oauth2-facebook/issues/62
I will add a longer comment to the mentioned issue :)

I believe php 5.6 should be thrown out now plus it's not supported by this version of phpunit anyway. If someone is still on php5.6 it has older version, besides that version is not supported for long time already...

I would also drop 7.1 and 7.2 as the are EOL

Hi, can I ask how is it going? Do you need any help? Thank you.

Hi, can I ask how is it going? Do you need any help? Thank you.

We probably want to change the tested php versions in .travis.yml as well. Do you plan to update this PR @lbacik?
@shadowhand Would you be willing to merge this once the tests pass? Are there any other issues that need to be resolved? I'm happy to help in any way to get a release going.
I'm a former user of https://github.com/facebookarchive/php-graph-sdk, which seems completely abandoned at this point (and, sadly, has the same primary author as this library), but I'm more hopeful that this library can be updated. Fingers crossed! :)

We probably want to change the tested php versions in .travis.yml as well. Do you plan to update this PR @lbacik?
@shadowhand Would you be willing to merge this once the tests pass? Are there any other issues that need to be resolved? I'm happy to help in any way to get a release going.
I'm a former user of https://github.com/facebookarchive/php-graph-sdk, which seems completely abandoned at this point (and, sadly, has the same primary author as this library), but I'm more hopeful that this library can be updated. Fingers crossed! :)

I would be also thankful for this update.

I would be also thankful for this update.

We probably want to change the tested php versions in .travis.yml as well. Do you plan to update this PR @lbacik?

@hemberger - yep, of course - done :)

We probably want to change the tested php versions in .travis.yml as well. Do you plan to update this PR @lbacik?

@hemberger - yep, of course - done :)

I've merged this, and I am going to release it as a hotfix, but I think we should do a major release that makes a hard requirement on PHP 7.4+.

I've merged this, and I am going to release it as a hotfix, but I think we should do a major release that makes a hard requirement on PHP 7.4+.

Released in version 2.0.4.

Released in version 2.0.4.


Transform-origin breaks the transform animation

🐛 Bug Report
I recently updated react-spring (was on beta22) and some of my transforms animations are broken.
To Reproduce

Create a useSprings

Use interpolate to create a transform

This should work perfectly

Add  transformOrigin: 'what ever you want'

The animation will not fire
Expected behavior
The animation should work with the correct transform origin
Environment

react-spring ^9.0.0-beta.33
react v16.8.6

This should the black flagged like that other prop which caused problems I think it was zIndex. Thanks for the report 👍

Fixed in @react-spring/web v9.0.0-beta.34
Run yarn --force to use it

Thank you, that was fast !


calamares installer: quality of life improvements

Description of changes
Since the release of NixOS 22.05 I have been monitoring the community response to the calamares installer, and any sore points that could be easily fixed to improve the experience. This PR addresses some of those little things that could make the default installation experience better
Things done

Remove idle screen off while installing with the GNOME ISO
Increase default verbosity, allowing for users to view nixos-install installation output in the calamares log view

Updated calamares-nixos-extensions to improve log output

Update to the latest release of calamares: 3.2.59

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Just watched https://youtu.be/ck4J2Faa7Fc and my main takeaway was that the option to enable/disable "strong passwords" should be mentioned before the actual password form.
(I'm sorry this is completely unrelated to your other changes)

I saw that too, but the position is handled by calamares itself, so probably better as an upstream issue: https://github.com/calamares/calamares/issues. Thanks for bringing it up though!

Is there a nice place to document some comments I had? I ran through it and had a few but wasn't sure if I should make issues, or if there was a thread somewhere.

Is there a nice place to document some comments I had? I ran through it and had a few but wasn't sure if I should make issues, or if there was a thread somewhere.

Try it here or in the Discourse

I tried installing twice and gave up because it seemed stuck at 46%. I see reddit comments about this and will give it another go tonight. Is there a way to indicate progress during this period? (I'm still not sure what it's doing for 20 minutes).

I noticed that in the mentioned video  the firefox package is still part of environment.systemPackages instead of in user.users.foo.packages. We also added thunderbird.
We made that change in https://github.com/NixOS/nixpkgs/commit/6051f8028dd0487d73a657b524e4247934d34350

Ok, I'll make appropriate changes to the config generated by the installer to best match that format. Right now Firefox is installed by default on all desktops, I'm thinking leave that, but add thunderbird commented out

I noticed that in the mentioned video the firefox package is still part of environment.systemPackages instead of in user.users.foo.packages. We also added thunderbird.
We made that change in 6051f80

I changed the configuration to match that in the commit. As for the rest of the issues, we should probably move them to separate issues so it's easier to track individually.


⚠ SceneTime has degraded performance

In 9f8cfcb, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1942 ms

Resolved: SceneTime performance has improved in 93b7e2b after 22 minutes.


33 implement navbar

I think hi-fi's have changed a bit, there should be the name and the selection color should be yellow:

here is the link to hi-fi figma for reference:
https://www.figma.com/file/bneJmgZkkoiyKaSZPGilwj/Prototypes?type=design&node-id=309-341&mode=design&t=drGBdylavKxuj76d-0

also order should be convo push contacts, I think you have it as convo contacts push

Looks good to me, same comments as Jerry's. Also if you don't want to pass navigation as a prop between components, it is also possible use to the useNavigation hook but either way works.


Add release notes and dev notes to develop branch.

Reformat README file, add Release notes, add top level license file.

:+1:


openfl_pool_events shouldn't work anymore, but is still a valid define

I'm making a list containing all openfl's compile arguments (or defines).
While checking the purpose of openfl_pool_events, i noticed those instances all point to Event.__pool and do their "event stuff" on there:

Problem is: this field has been commented about 1.5 years ago (!)

(I know i only gave one example, but from the various instances i checked, all event classes have __pool commented out, and the openfl_pool_events option)
Should this be removed?

nice, ill close this then


Do not crash when running in node.js

The node.js environment has no concept of a document origin.
Therefore, the secure-origin test crashes on an undefined variable.
If we see this case, we can assume the origin is secure.
This is maybe a better way to solve vibornoff/asmcrypto.js#100

Could you, please, pay attention on this PR? Seems it is needed for my project as well.

Could you please provide a suitable replacement for weak seeding function too? It uses global.location.href as a salt value at seed hardening steps.

In fact you could approve the PR and plus introduce the same check into weak seeding function. Instead of location.href you could generate any fake string for the seed (just a proposal).


clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9

clang: warning: libstdc++ is deprecated;
move to libc++ with a minimum deployment target of OS X 10.9

What did you do to see this warning?
I don't see where Cython would reference this library directly, so maybe there's something wrong with your local C++ compiler setup?

Apple LLVM version 8.0.0 (clang-8<IP_ADDRESS>)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

https://github.com/websockets/utf-8-validate/commit/a87dd40953077ad02f7621e91c3c7d342276879e

Closing as it's not a problem in Cython. Probably needs to be fixed in distutils or setuptools.

did you solve this problem? if so, how did you do?


delete duplicate 'events' resource type from list

the resource type 'events' is listed twice in the list of resources. Remove the second occurrence. Not in this pull request but perhaps also consider ordering the resource types list alphabetically by name

the list is ordered by API group names rather than resource types.
there are actually two Event types exposed from different API groups, for historical reasons.
/hold


Can we default configDir to src/main/liberty/config?

Gradle plugin equivalent of https://github.com/WASdev/ci.maven/issues/427
Lets keep all discussion to the other repo, but I just wanted to open an issue here as a reminder to keep the maven and gradle plugins in-sync.

Added this change in #312

This is done.


What's meaning of wh_offset_base ?

It seems that the model will converge much more slower if wh_offset_base =1, especially the wh_loss. Besides, compared with centernet, the loss weight of wh_loss is much more larger than hm_loss（centernet: wh_loss_weight=0.1, hm_loss=1),why?

The wh_offset_base enables smaller, more easily predicted logits for wh.  Wh logits are more similar to hm logits using this method.
The loss values after weighting are balanced for wh and hm.  TTFNet uses a different wh loss than centernet requiring a different loss re-weighting to keep the two losses in approximate balance.


While creating category if In display mode "only products" is selected then description field should not be required.

Framework version: 0.1.5

Framework Version: 0.1.5
Fixed


Should builtin functions (those written in Go) support the pseudo-map interface?

See issue #1122 for background. This issue is for discussion whether builtin commands should also allow introspection via the pseudo-map interface. In other words why should this work:
> keys { }
▶ arg-names
▶ body
▶ def
▶ opt-defaults
▶ opt-names
▶ rest-arg
▶ src

But this fails:
> keys $echo~
Exception: cannot iterate keys of fn
[tty 14], line 1: keys $echo~

That is, ideally we would be able to use the pseudo-map interface for functions written in Elvish to do the same thing for builtins written in Go. Note that some of these attributes for builtins (e.g., def and body)  might be empty strings, $nil, or some other special value since these are not written in Elvish script. But being able to introspect the arg-names, opt-names, etc. could be useful. And, really, whether the command is written in Go or Elvish shouldn't affect their use as read-only pseudo-maps.

I have a type function and wanted to modify it to support the &all option if the search-external builtin supported that option; otherwise display an error or workaround the fact search-external &all isn't supported. That would be relatively easy if I could introspect $search-external~[opt-names].

Closing as duplicate of issue #680.


Thank you for your hard work!

This tool is really great, and I reverse-searched the URL to find the repo. Thanks a bunch, sorry for the junk issue!

Thanks for feedback. I will put the link to sources.


[incubator/kafka] Add same secrets the brokers get to the job so it can setup topics

when using SASL.

Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.
No.

What this PR does / why we need it:
When you setup the brokers to use login modules the job will fail when communicating as it doesn't have the correct credentials.
Which issue this PR fixes
Didn't open an issue.
Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[X] DCO signed
[X] Chart Version bumped
[X] Variables are documented in the README.md
[X] Title of the PR starts with chart name (e.g. [stable/mychartname])

@faraazkhan
@h0tbird

Can anyone take a look at this?


Create a signet setup

So far, we used regtest or testnet for testing.
We should evaluate how easy it is to setup a bitcoin signet including additional services like a blockexplorer and electrs.

A custom signet is quite easy to setup, however we should think about how we want to use it? After looking into it I see the following options
Local nigiri fork
Fork nigiri and add support for a custom signet chain. I got this working prototypical, but there is a lot of stuff in nigiri we do not really need for a custom signet. However if we plan to work with regtest and signet (and potentially liquid) in parallel that might be the way to go.
e.g.
nigiri start -signet
nigiri faucet 0.1

However, we would need to maintain a fork of nigiri and nigiri-chopsticks
Local custom
Similar to nigiri - create a docker-compose with electrs, esplora and bitcoin (signet).
e.g.
docker-compose up
curl --data-binary '{"jsonrpc": "1.0", "method": "sendtoaddress", "params": ["tb1qlsp47mzmzzncwrp0gr4wvrqhn3s0dupf8hnwr4", 0.1]}' -H 'content-type: text/plain;' http://admin1:123@<IP_ADDRESS>:38332/

Remote
This could be done with nigiri fork or a custom setup, but on a remote server accessible from the outside. We setup the signet node only once and point our configuration to signet statically.

I am rather in favor of the remote option as I see signet more as an alternative to testnet than to regtest.

I am rather in favor of the remote option as I see signet more as an alternative to testnet than to regtest.

I agree, it would replace the testnet setup.
I'm in favor of docker-compose or vanilla docker containers as we can't use nigiri on our VMs.

https://github.com/holzeis/signet


Improve RandomWalk

This is very expensive. How about making vector_bundle a set (or dict), and popping random items from it?
_Originally posted by @pckroon in https://github.com/render_node/MDE3OlB1bGxSZXF1ZXN0UmV2aWV3NDE2ODY3OTAx/pull_request_reviews/more_threads

This is out of date and has been taken care of too large extend, at least it is not the bottle-neck in the current RW algorithm.


Keyword "trait" is deprecated in traitlets 5.0

DeprecationWarning: Keyword trait is deprecated in traitlets 5.0, use value_trait instead
Fixes the following pytest warnings seen in our GitHub Actions:
=============================== warnings summary ===============================
bqplot/marks.py:167
  /home/runner/work/bqplot/bqplot/bqplot/marks.py:167: DeprecationWarning: Keyword `trait` is deprecated in traitlets 5.0, use `value_trait` instead
    scales = Dict(trait=Instance(Scale)).tag(sync=True, **widget_serialization)

bqplot/interacts.py:157
  /home/runner/work/bqplot/bqplot/bqplot/interacts.py:157: DeprecationWarning: Keyword `trait` is deprecated in traitlets 5.0, use `value_trait` instead
    scales = Dict(trait=List(trait=Instance(Scale)))\

bqplot/figure.py:135
  /home/runner/work/bqplot/bqplot/bqplot/figure.py:135: DeprecationWarning: Keyword `trait` is deprecated in traitlets 5.0, use `value_trait` instead
    title_style = Dict(trait=Unicode()).tag(sync=True)

Thanks, this looks good to me!

Visual regression tests are failing on master as well, so let's merge. Thanks


Reduce BLE service discovery time

In the below packet capture:
Wallet side:
Frame 1 : Advertisement data frame
Frame 2 : Scan Response data frame
Frame 77: MTU exchange starts

In between the device connection and MTU exchange, there is a time delay of ~2.5 secs (please refer to the wallet.log)
There is a time difference of  ~1.5 sec between the frame 24 and frame 25.
There are some frames : Sent Read by type request which has either error response frame or Recd Read by
type response  frame. We need to investigate these frames(frame 34 to frame 74).

wallet-inji-issue.log
verifier-inji-issue.log

I'll check if anything needs to be done from iOS side regarding this


Pre-migration to Numba 0.50

Hello @kozlov-alexey! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file sdc/tests/test_series.py:

Line 3013:44: W605 invalid escape sequence '('
Line 3013:46: W605 invalid escape sequence ')'
Line 3013:48: W605 invalid escape sequence '.'
Line 3014:33: W605 invalid escape sequence '\s'
Line 3015:33: W605 invalid escape sequence '\s'
Line 3032:44: W605 invalid escape sequence '('
Line 3032:46: W605 invalid escape sequence ')'
Line 3032:48: W605 invalid escape sequence '.'
Line 3033:33: W605 invalid escape sequence '\s'
Line 3034:33: W605 invalid escape sequence '\s'
Line 3091:51: W605 invalid escape sequence '('
Line 3091:53: W605 invalid escape sequence ')'
Line 3091:55: W605 invalid escape sequence '.'
Line 3092:37: W605 invalid escape sequence '\s'
Line 3093:37: W605 invalid escape sequence '\s'
Line 3528:46: W605 invalid escape sequence '('
Line 3528:48: W605 invalid escape sequence ')'
Line 3528:50: W605 invalid escape sequence '.'
Line 3529:33: W605 invalid escape sequence '\s'
Line 3530:33: W605 invalid escape sequence '\s'
Line 3877:24: E128 continuation line under-indented for visual indent
Line 3877:40: W605 invalid escape sequence '('
Line 3877:42: W605 invalid escape sequence ')'
Line 3877:44: W605 invalid escape sequence '.'
Line 3878:24: E128 continuation line under-indented for visual indent
Line 3878:26: W605 invalid escape sequence '\s'
Line 3879:24: E128 continuation line under-indented for visual indent
Line 3879:25: W605 invalid escape sequence '\s'
Line 3880:24: E128 continuation line under-indented for visual indent
Line 3881:24: E128 continuation line under-indented for visual indent
Line 3886:24: E128 continuation line under-indented for visual indent
Line 3886:40: W605 invalid escape sequence '('
Line 3886:42: W605 invalid escape sequence ')'
Line 3886:44: W605 invalid escape sequence '.'
Line 3887:24: E128 continuation line under-indented for visual indent
Line 3887:26: W605 invalid escape sequence '\s'
Line 3888:24: E128 continuation line under-indented for visual indent
Line 3888:25: W605 invalid escape sequence '\s'
Line 3889:24: E128 continuation line under-indented for visual indent
Line 3890:24: E128 continuation line under-indented for visual indent
Line 4002:24: E128 continuation line under-indented for visual indent
Line 4002:41: W605 invalid escape sequence '('
Line 4002:43: W605 invalid escape sequence ')'
Line 4002:45: W605 invalid escape sequence '.'
Line 4003:24: E128 continuation line under-indented for visual indent
Line 4003:26: W605 invalid escape sequence '\s'
Line 4004:24: E128 continuation line under-indented for visual indent
Line 4004:25: W605 invalid escape sequence '\s'
Line 4005:24: E128 continuation line under-indented for visual indent
Line 4006:24: E128 continuation line under-indented for visual indent
Line 4011:24: E128 continuation line under-indented for visual indent
Line 4011:41: W605 invalid escape sequence '('
Line 4011:43: W605 invalid escape sequence ')'
Line 4011:45: W605 invalid escape sequence '.'
Line 4012:24: E128 continuation line under-indented for visual indent
Line 4012:26: W605 invalid escape sequence '\s'
Line 4013:24: E128 continuation line under-indented for visual indent
Line 4013:25: W605 invalid escape sequence '\s'
Line 4014:24: E128 continuation line under-indented for visual indent
Line 4015:24: E128 continuation line under-indented for visual indent
Line 4788:45: W605 invalid escape sequence '('
Line 4788:47: W605 invalid escape sequence ')'
Line 4788:49: W605 invalid escape sequence '.'
Line 4788:67: W605 invalid escape sequence '.'
Line 4788:73: W605 invalid escape sequence '.'
Line 4789:33: W605 invalid escape sequence '\s'
Line 4790:33: W605 invalid escape sequence '\s'

In the file sdc/tests/test_utils.py:

Line 236:1: E302 expected 2 blank lines, found 1
Line 237:25: W605 invalid escape sequence '\s'
Line 237:37: W605 invalid escape sequence '('
Line 237:48: W605 invalid escape sequence ')'

@kozlov-alexey conflict


Tree view BG color makes it unreadable

It seems that if you modify a TreeView while the program is running (ex. progressively loading a remote file tree), as soon as you deselect one of the newly updated nodes, the coloring changes to white, making it unreadable until you select it again.

In this case, I currently have "firmware" selected, and the white blocks are ones that were previously selected. I tried manually setting the background color to something else (ex. Blue). When it loads the first time, it's blue as expected, but as soon as you deselect it, it turns white.
Reapplying the theme, or using Refresh() and Update() functions on the TreeView doesn't appear to help.

I use a couple of Treeviews that change dynamically but i have never had this issue. Are you sure you don't interfere with the style in your code? Would you mind sending your Treeview's initialization code from there .Designer file, so that i can compare it with mine?

Actually... I realized I always deselect the current node before changing the tree and then select it again. That's probably why i never run into this issue.

Another difference night be that my changes to the tree are always synchronously triggered from the UI, never by any other thread. Are you using Invoke when updating the tree?

Plus I use BeginUpdate/EndUpdate , i don't know if it may play any role...

Unfortunately i have not been able to reproduce this behavior:
i am dynamically loading the structure of the 'My documents' folder in the TreeView using this code:
		//Agregar estructura de archivos y directorios recursivamente:
		private void ListDirectory(TreeView treeView, string path, string filter)//<- filter = "*.exe;*.jpg"
		{
			treeView.Nodes.Clear();
			DirectoryInfo rootDirectoryInfo = new DirectoryInfo(path);
			treeView.Nodes.Add(CreateDirectoryNode(rootDirectoryInfo, filter));
		}
		private static TreeNode CreateDirectoryNode(DirectoryInfo directoryInfo, string filter)
		{
			TreeNode directoryNode = new TreeNode(directoryInfo.Name);
			foreach (DirectoryInfo directory in directoryInfo.GetDirectories())
			{
				try
				{
					directoryNode.Nodes.Add(CreateDirectoryNode(directory, filter));
				}
				catch { }
			}

			foreach (FileInfo file in directoryInfo.GetFiles(filter))
			{
				try
				{
					directoryNode.Nodes.Add(new TreeNode(file.Name));
				}
				catch { }
			}
			return directoryNode;
		}

string myDocumentsPath = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
ListDirectory(this.treeView1, myDocumentsPath, "*.*");

But the control behaves as intended:

Would you mind to share your code? or a way to replicate this?
Thanks.

I was working on an example program for you, but found that I was unable to duplicate the issue. While copying chunks of code over, I found a stray node.BackColor = Color.White; that was leftover from when I was doing things with the default theme. I apologize for wasting your time trying to duplicate the issue, but it would appear that the issue was me. (I spent way too much time trying to track this down the other day, but never noticed that line of code.)


Test build pr outside container

This PR adds a new options custom_container to build PR in the CI in a custom container. By default, workflow runs inside "huggingface/transformers-doc-builder" which is the container needed for transformers and that has all dependencies like torch, tensorflow, etc. However, it takes quite some time (>2min) to initialize the container in the CI given its size. In theory, most libraries don't need this container so we should be able to bypass this step and run directly on ubuntu:latest.
This PR is related to https://github.com/huggingface/huggingface_hub/pull/2140 in which I tested the idea.

As discussed on slack (internal), plan is:

we merge https://github.com/huggingface/doc-builder/pull/487 in its current state
we test to disable custom_container (i.e. custom_container: "") on several repos, including huggingface_hub (already done), datasets, diffusers, hub-docs, etc.
for the repos that needs the custom container, we set custom_container: "huggingface/transformers-doc-builder" in their CI. This should not change anything since it is the default value for now.
Once we fixed the repos in which we think transformers image is needed, we make a new PR on doc-builder to change the default value of custom_container to ""

This way we should be able to make the transition between "all of them use transformers image" to "most of them use a bare ubuntu" without a breaking change in the process.

lgtm, why it is still in draft mode ?

lgtm, why it is still in draft mode ?

Lol, you're too fast. Just wanted to update description first :smile: Should be ready now :)


Clear Interval if the component doesn't have the $el property

Using the vuelidate plugin creates components that do not have an $el property, but still have _hasMetaInfo set to true, which results in the setInterval never being cleared. There may be other plugins that create similar components.
Changing the line to if (this.$el && this.$el.offsetParent !== null) return would resolve this issue.

Fixes https://github.com/declandewet/vue-meta/issues/187

@paulgv Should I do something regarding to the Codecov Report

@eddiesigner I think it's all right since the coverage diff is at 0% but I guess it wouldn't hurt to add some tiny tests to this module. I opened a PR to add tests to your repo so we can add them to this PR: https://github.com/eddiesigner/vue-meta/pull/1


Added STL random number distribution wrappers

The last two, piecewise_constant_distribution and piecewise_linear_distribution, were really messy, though.
Thanks!

Ahahaha on it.

@yasamoka I added support for MSVC 16.7. Unfortunately, some unit tests for random generators fail there. Do you have the possibility to verify and possibly fix this?

For sure, I'll get right on it as soon as I'm done with some tasks I have these couple of days.

Thanks!


3.0.2 is incompatible with RHF 7.13.0

When trying to use<EMAIL_ADDRESS>with<EMAIL_ADDRESS>throws an error:
TypeError
Cannot read property 'current' of undefined

CSB with reproduction
https://codesandbox.io/s/bold-silence-grwfc?file=/src/index.js

Closed in favor of https://github.com/react-hook-form/react-hook-form/issues/6318.


docs: Makefile, check-build.sh clean-ups and perf improvements

Some users find that make render-docs or make test-docs are too slow for quickly iterating over local changes. This PR contains a few commits to improve things. The first run of either command will still take time, but provided the docs-builder image is present on the system and up-to-date, subsequent runs of make render-docs SKIP_BUILDER_IMAGE=1 or make test-docs SKIP_LINT=1 INCREMENTAL=1 SKIP_BUILDER_IMAGE=1 should go faster. See individual commits for details.
This PR also contains some fixes and clean-ups, in particular for the output of make -C Documentation help.

docs: Add SKIP_BUILDER_IMAGE to Makefile to skip rebuilding docs-builder
docs: Allow incremental builds for "make html" via env variable
docs: Remove dependency on update-helm-values for building HTML
docs: Add headings and missing descriptions to "make help" output
docs: Reorder targets in Makefile
docs: Do not remove _api on "make clean"

Cc @networkop

/test

/test


Fix broken link to post about ES2015 modules

Hello!
What kind of change does this PR introduce?:
Small fix for README. I noticed that link is broken due to extra line break.
Did you add tests for your changes?:
Nope (:
Does this PR introduce a breaking change?:
No

Will definitely work better now. Thanks :)


[Doc] Describe how to manage multithreading

Spent a bit of time on figuring this out (after accidentally running on all 256 cores of our shared login server 😅) so figured it is worth a bit of documenting.
Please LMK if there's a more appropriate place for this.

Might also be worth pointing to
https://pytorch.org/docs/stable/generated/torch.set_num_threads.html
(Since that ought to keep up with whatever library PyTorch uses under the hood)


Add List component

Suggest change <UnorderedList /> to <List />, and <OrderedList /> to <List ordered />.

@J-F-Liu What benefits would this change have?

Easier to read and write, and to toggle a list is ordered or unordered.

Easier to read and write

I don't see how is it easier to read and write, it's just two different names. I would actually argue that <List> is worse, because without looking at documentation, how do you know if this <List> is ordered or unordered?

toggle a list is ordered or unordered.

That sounds like a unique case, to be honest. When would you need to change the type of the same list between ordered and unordered?

List is a higher level of abstraction, we usually don't care so much about whether it is ordered or unordered, so List is fine and can distinguished by look at the ordered prop. The only concern is breaking change which can be avoided by still keep UnorderedList and OrderedList.
This is only my personal opinion, you are free to close this issue or accept it.

Sorry, I don't think there are enough benefits to make this breaking change.

@J-F-Liu for the time being you could easily add a wrapping component that achieves what you want:
import { UnorderedList, OrderedList } from 'evergreen-ui'

export default function List({ ordered, ...props }) {
  return ordered
    ? <OrderedList {...props} />
    : <UnorderedList {...props} />
}


Rails 6 test issue

An error occurred in a `before(:suite)` hook.
Failure/Error: ThinkingSphinx::Test.start_with_autostop

Riddle::CommandFailedError:
  Sphinx command failed to execute

I'm seeing this after our Rails 6 upgrade. I'm wondering if this deprecation warning finally turned into an error?
https://github.com/pat/thinking-sphinx/issues/1165

My initial guess is that the issues aren't related - I wouldn't expect callbacks to be involved when starting Sphinx.
Is this issue just in the test environment? i.e. do TS/Sphinx work fine in dev (i.e. using the standard rake tasks, etc)?

It's just in the test environment. Starting sphinx in development works. I just rebuilt and got some errors:
ERROR: index 'wireless_module_core': sql_range_query: Unknown column 'pcms_wireless_modules.id' in 'field list'

wireless module model has:
  has_one :pcm

There isn't a pcms_wireless_modules table. I'm looking to see if there was a change to has_one in rails 6.

What does the index definition for the WirelessModule model look like?

has pcm(:id), as: :pcm_id
has pcm(:serial_number), as: :pcm

Hmm… I've no idea if this is related, and the syntax you've got should still work. But also, you can use the method notation rather than arguments without any issues (it was only a problem back before TS v3 and MRI 1.9):
has pcm.id, as: :pcm_id
has pcm.serial_number, as: :pcm

Tried that in no luck. I commented out the offending lines in the indexes but I'm still seeing the error in rspec unfortunately.

Weird, when I'm running rake db:test:prepare I'm getting:
ActiveRecord::StatementInvalid: Mysql2::Error: Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs

<Riddle::CommandResult:0x00007fb5d0dd0580 @output="Sphinx 2.2.11-id64-release (95ae9a6)\nCopyright (c) 2001-2016, Andrew Aksyonoff\nCopyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)\n\nusing config file 'config/test.sphinx.conf'...\nERROR: section 'activation_description_core_0' (type='source') already exists in config/test.sphinx.conf line 5670 col 1.\nFATAL: failed to parse config file 'config/test.sphinx.conf'\n", @status=1, @command="indexer --config \"config/test.sphinx.conf\" --all", @successful=false>

I had to comment out some indexes. And fix some other issues. I'll follow up when I'm more sure about the cause.

ERROR: section 'activation_description_core_0' (type='source') already exists in config/test.sphinx.conf

This part of the output you shared makes me wonder if it's actually the same issue as #1191 - which I still haven't had the chance to look into deeply (sorry @thebravoman!). But it's good to know that this seems to have started with your Rails 6 upgrade, and it looks like you're perhaps both using SQL-backed indices rather than real-time indices (which is maybe why I've not seen it in my own apps, but helps narrow down the focus).
Something seems to be evaluating index definitions multiple times… no idea if it's thread-related. Perhaps that helps your own debugging, but I'll try to get a test app set up soon and see if I can reproduce with SQL-backed indices + Rails 6 + test environment.

My offer for reproducing it on my machine and getting together over a video
call still stands, if this could help you narrow It down.
On Tue, Jun 8, 2021, 07:42 Pat Allan @.***> wrote:

ERROR: section 'activation_description_core_0' (type='source') already exists in config/test.sphinx.conf
This part of the output you shared makes me wonder if it's actually the
same issue as #1191 https://github.com/pat/thinking-sphinx/issues/1191

which I still haven't had the chance to look into deeply (sorry
@thebravoman https://github.com/thebravoman!). But it's good to know
that this seems to have started with your Rails 6 upgrade, and it looks
like you're perhaps both using SQL-backed indices rather than real-time
indices (which is maybe why I've not seen it in my own apps, but helps
narrow down the focus).

Something seems to be evaluating index definitions multiple times… no idea
if it's thread-related. Perhaps that helps your own debugging, but I'll try
to get a test app set up soon and see if I can reproduce with SQL-backed
indices + Rails 6 + test environment.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pat/thinking-sphinx/issues/1195#issuecomment-856434231,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKAGBOYX7ZFQRGJ7GYT5ADTRWNTLANCNFSM4546YFJA
.

Appreciated! Let me try and reproduce within a test app first - I'm hoping that would at least help give me a bit more context so any video calls are more productive :)

Oh, and: how are your index files named? e.g. app/indices/article_index.rb containing the index for the Article model - or a different pattern?

Oh, and: how are your index files named? e.g. app/indices/article_index.rb containing the index for the Article model - or a different pattern?

Exactly that.

I commented out a line in our code that calls ThinkingSphinx::Configuration.instance.indices thinking maybe it could be responsible. No luck. I'll keep working with it.
I also added a puts "ABC" in one of our indexes. It's being loaded twice. I'm going to check the configuration files and see if we're doing anything we shouldn't be in rails 6. Autoload maybe? Hrm.

If you feel comfortable diving into the TS code: it might be worth looking at how the preload_indices and preload_index methods are behaving.
And I'm not sure if it's related, but the mutex (since Rails 5.1.5) is not a standard Ruby one, but rather something custom for Rails/Active Suppoort, that permits dependency loading while locked (see #1132).
With both of those things in mind, I'm wondering if shifting @preloaded_indices = true from line 96 up to 88 could help? Thus even if preload_indices is being invoked more than once, the second call should stop at the boolean check on line 87. Also, maybe within that method it's worth resetting the sources/indices (which are initially set in the superclass, Riddle::Configuration)… so:
def preload_indices
  @@mutex.synchronize do
    return if @preloaded_indices
    @preloaded_indices = true

    @indices = []
    @sources = []

    index_paths.each do |path|
      Dir["#{path}/**/*.rb"].sort.each { |file| preload_index file }
    end

    normalise
    verify
  end
end

(For what it's worth: I tried creating a sample app using Rails 6 and SQL-backed indices, and the super simple test suite was fine… also: it's interesting to note that this sounds like it's a consistent problem for @atomical, but intermittent for @thebravoman 🤔 - I'm really hoping it's the same cause though!)

are you using the old-style classic loader, or Zeitwork's new approach?

config/application.rb:    config.autoloader = :classic

how are your index files named? e.g. app/indices/article_index.rb containing the index for the Article model - or a different pattern?
The model is app/models/course.rb

$ tree app/indices/
app/indices/
├── course_index.rb

It loads the indexes twice when config.eager_load = true in config/environments/test.rb

Should an index file be require'd by Rails before Sphinx loads it with preload_index()? Would that be a problem?

Thanks so much for that debugging! With those clues, I was able to reproduce the problem in my test app, with eager_load set to true and using the classic autoloader (either scenario separately is fine, though).
The fix I've added to my test app is putting the following line in config/application.rb:
config.eager_load_paths.reject! { |path| path == Rails.root.join("app", "indices").to_s }

I'll look into making Thinking Sphinx do this automatically (and thus, we don't need that check against which files are loaded).
@thebravoman: would love to know if this also fixes things for you? The fact that your issue has been intermittent makes me unsure if this is the exact same issue, but perhaps it's related?

Thanks @pat
$ bin/rspec spec/system/search_spec.rb

/search
    search doesn't show content that has no image

Finished in 3.62 seconds (files took 0.9671 seconds to load)
1 example, 0 failures

Tests passed on my side. My current rails is 6.1  just to let you know.

Just pushed a fix via 02bbb5071482cab8c91b956d2f6437b4cded3cce - if either/both of you are able to confirm the latest in develop works well in your apps, that'd be great, and I'll get a new release cut shortly.
gem "thinking-sphinx",
  :git    => "https://github.com/pat/thinking-sphinix.git",
  :branch => "develop"

All good on my side.
The gitrepo has a typo - it should be thinking-sphinx and not thinking-sphinix.

My laptop keyboard is a little flaky :/  fixed - and especially, thank you for testing!

I just thought of something.
I have a rails engine and it is located in
/gems/my-engine/
There I have a model
gems/my-engine/app/models/user.rb
Currently my indices are in
app/indices/user_index.rb
but I would like to move them to
gems/my-engine/app/indices/user_index.rb
My point is - is it possible to have inidices in many folders, not only app/indices but also gems/my-engine/app/indices?

Indices in engines should get picked up automatically, provided they're in the engine's app/indices folder. If you find that's not the case, happy to discuss further, but perhaps in a separate issue :)

Of course. I will try it and if there are issues I will open a separate issue here. I just wanted to make sure the case of engines (as it is supported) is not forgotten with this change.

Ah… I guess that depends on whether Rails adds an engine's app/* directories to eager_load_paths 🤔 I look forward to hearing what you discover!

Just released v5.2.0 which includes this fix (and a further improvement that should ensure engine app/indices paths are also removed from eager_load_paths). So, I think this issue is all resolved - certainly, feel free to re-open the issue if I've got that wrong!