[Discussion] Spleeter for real-time applications

Hi !
This issue is a follow-up of a gitter discussion.
I'm developing a C++ port of spleeter here as a side project. My goal is to give people the opportunity to use the spleeter technology within plug-ins.
Lately, I realized that the architecture is built to run in batches of size 'T' which is 512 fft frames in the pre-trained models (~12seconds). This kind of latency isn't suitable for real-time processes.
To check if the architecture is suitable for my needs, I need to evaluate the quality loss of changing that value to something lower.
My plan is to train models for 2 and 4 stems with multiple values of T and compare their quality.
I created a repository to report on that, as I assume it may be of interest to others. And also to get some help in the process.
I already have a couple of questions:
First:
I am using MUSDB18. I noticed that the training configuration needs the description of the evaluation set. Considering that I will be using that set to evaluate the trained models, I would like to make sure that training does not use the validation set as it would mean that I am evaluating on data the network already knows. Is the evaluation_csv parameter used during training ?
Second:
I am struggling with computation power. For my first test, it took almost 4hours to run 100 000 steps on a p2.xlarge AWS instance (GPU Tesla k80). Is this expected ? Do you think that, considering my problem I could lower the train_max_steps ?
Third:
This one is more of a note than a discussion but I ran the evaluation using python -m spleeter evaluate -p spleeter:2stems --mus_dir [exported db dir] -o [local path] and got NaNs. Is the evaluation system broken atm ? I changed it in a fork here to fix it for my case. Should I do a pull request for that ?
Thank you for your help and for publishing your work !

How many files are in the MUSDB18 dataset?
I have a dataset of 300 files and I can train it in 6-7 hours on an RTX2080.

Hi @gvne!
First, be aware that you don't necessarily need to retrain the system. You can change the size 'T' at test time as long as it remains a multiple of 64 (which corresponds to 1.5s). That will change a bit the performances especially at the segment borders, but if you want to do realtime separation (then causal separation), you won't be able to take advantage of the future audio samples in anyway and then you'll need to cope with it.
You can evaluate straightforwardly spleeter with shorter T by just updating the corresponding config json file.
If you want to go below T=64, you'll need both to retrain the system and to change the architecture of the network (reduce the number of layers) and you won't be able to take advantage of the data we used for training. And I suspect that you may not be able to use much shorter segments without strongly degrading separation results.
What I would recommend is to perform overlap and add with a short T that does not need retraining (for instance T=64). You would then process 1.5s of the past audio and just output the as short as you want current frame of audio, the size of which would be mainly set by the computation power of your system. Do you have an idea of a many T-long segment you can process per second ? (note that doing STFT/iSTFT outside tensorflow may make things faster on CPU).
Regarding your questions:

the validation_csv is used for monitoring validation cost. While it has no impact on the final model (there is no early stopping mechanism), it is useful to check that your model is not overfitting (which would results in a validation cost going up at some point). The easiest way to visualize it is to use tensorboard.

Training is quite intensive. 4 hours for 100000 steps does not seem that much to me. You can monitor the GPU your using to make sure it is actually well exploited.

The evaluation system is not supposed to be broken. I need to check it then :).

Hey @romi1502, nice to read from you around here :)

You can change the size 'T' at test time as long as it remains a multiple of 64

Well, I surely didn't get that ! I was sure it was a training parameter that configured the network input size. My bad. At least it means I don't need to retrain the models which is a very good news. Those AWS GPU instances aren't cheap...

If you want to go below T=64, you'll need both to retrain the system and to change the architecture of the network (reduce the number of layers) and you won't be able to take advantage of the data we used for training. And I suspect that you may not be able to use much shorter segments without strongly degrading separation results.

That matches what mmoussallam said on gitter. I'll keep that option for later. I'm no Deep Learning expert and that's not what I was expecting to do when I started the project.

What I would recommend is to perform overlap and add with a short T that does not need retraining (for instance T=64). You would then process 1.5s of the past audio and just output the as short as you want current frame of audio, the size of which would be mainly set by the computation power of your system. Do you have an idea of a many T-long segment you can process per second ? (note that doing STFT/iSTFT outside tensorflow may make things faster on CPU).

I'll try that out ! I think that it's the right first step anyway.
I'll keep the T and overlap rate between windows of T frames as high level parameters. I want that tool to work on any machine so I can't really force a user into using a specific setup.
Thank you so much for your time. You answered my questions perfectly !
I'll close the issue for now.

Hi there, just wanted to follow up on that matter.
I followed @romi1502 's suggestion and successfully implemented a very simple volume control VST3 plugin running in real time with spleeter !
You can find the plug-in code right here. I also provide a pre-build for OSX (tested on 10.14 and 10.15) here.
As expected the latency is the worst. 64 frames for spleeter and a couple of extra frames to leave enough time for the process to run properly (setup to 10 on the pre-built if I'm not mistaking). That leads to a latency close to 2seconds... But still, playing with those sliders is so much fun ! 😃
As a side note, I didn't release a new spleeterpp version that includes the on-line processing yet (the code is available in the develop branch though). I still need to update the documentation with details about the algorithm. There are quite a few parameters after all. I also need to assert the equality with the classic process.
Anyway, thank you once again for releasing your work. If you ever have further suggestions to improve this integration, I'd be very happy to read them !

@gvne I'm trying the VST3 file on MacOS on Studio One as my DAW, it does not seem to be recognized. Do you know what could be the issue?

Hi @gvne
That's just great! We'll take a look at the VST!

I think it might be better to go algorithmic instead of ML for realtime.
https://github.com/tachi-hi/euterpe/ is realtime for vox, but unsure of latency. (I think it's algorithmic)
https://www.yellownoiseaudio.com/  is realtime for drum,  latency is .5s. (maybe ML, idk)

@gvne if you have some time to assist me in getting vstSpleeter running on MacOS using JUCE, please do help me.
I've been trying to build it all day.

@aidv I answered in the issue you created on the vst repository. That is probably the best place to discuss build details. That would help keeping this issue as a place for discussing potential algorithm improvements.
@junh1024 Thank you for your suggestion. I agree that overall the spleeter architecture isn't suitable for real-time applications. We discussed that with @mmoussallam on gitter a couple of weeks back. I still decided to give it a try as I figure some people may not care that much about latency. And some other may accept it as long as it is free and open source 😄
My goal here really is to find the best approach to twist spleeter into a real-time application. Even if it wasn't designed for it.

@gvne Yes I think that it's best for the project. Thank you.

Hi,
I think this is really interesting & I applaud the work already done for real-time processing!
As for the user's hardware, I suggest you let them decide (like how Isola Pro Fx did on their VST Gui) - Because some of us would value the benefit of this plugin enough to buy 1 (or 2, 3, etc) GPU's just to run it at a lower, usable latency (~50ms or less) ... For instance: Virtual DJ supports VST plugins/effects. I could pull this up on deck A playing a track, and fade out drums, bass, etc to have an INSTANT acapella version to mix it with another, say, INSTANT instrumental on deck B, creating a live mashup, etc... So no need of rendering files! :-)  If I need 4x Radeon VII's to achieve this, I'll get them!!!
So basically, thank you and keep up the good work! We are waiting for this to drop to usable latencies - Even if we need Titans & Threadrippers to get it done lol!

@spereree VST technology doesn't work like that.
The time it takes for data to move from CPU to GPU and back to CPU is too much for it to work, that's why there's no GPU VST's on the market.
So no amount of GPU's would fix that problem.
I have looked in to @gvne code and I have managed to get the latency down to 8ms, but I still experience a delay in my DAW.
I'm still inspecting the issue to try to find a good solution.

Oh, didn't know that about VSTs & GPUs, thanks for the clarification!
So you managed to get it down to 8ms? That's great!
What about the audio quality, how does it compare to non-realtime?
Besides the DAW delay, were there any other issues you came across? Does CPU core count help?
Can you post what you have done so far so I also give it a try on my DAW or VDJ?
Many thanks again!

@spereree Audio quality is not great due to the model being used. It isn't using the 16kHz models, which I've asked @gvne to have a look at.
I have not yet doe any tests on different machines with different cores, but I'm assuming that multicore is not used for a single VST. Maybe I'm wrong. However multiple VST's will most likely be assigned to their respective cores.
Here's the VST: https://filebin.net/kjzyn56tqsm9r08a/spleeter-vst.vst3.zip?t=qmfhgehq

the 16khz is a fake extension https://github.com/deezer/spleeter/issues/2#issuecomment-548798493 . I suggest removing the separated stem(s) (excl. others) from the input, that way you have unity.
DAWs may not support PDC properly so it's important to say which DAW you use if you have latency problems. Another possibility is that the VST doesn't implement PDC properly. 3rd possibility: offline & online behaviors differ. I have seen 2-3 happen & 1 is widely reported with Live8-.

@junh1024 I'm on Studio One.
Do you have any suggestions on how better PDC can be achieved in JUCE? Would love to implement a fix if possible.
I don't think that other algorithms are relevant here.
@gvne project vstSpleeter is in my opinion a great proof of concept on how Tensorflow models could be turned into VST's.
If anyone fails to see how this will revolutionize the VST music production industry, then take notes, you don't want to fall behind on this opportunity.

S1 should be OK. Accordingly to  https://github.com/deezer/spleeter/blob/master/configs/5stems/base_config.json#L8 , the FFT size is 4096, so the latency should be 4096sa or 4096/44100 = 92ms. Can you explain how you got it to 8ms?
I;m not experienced with PDC in juice/c++, but in JSFX, PDC is very manual. You set the PDC & the DAW will give you x sa in advance, and you delay your output, and give out 1 sa at a time.
I don't think it's a revolution, more like workflow enhancement. BTW, izotope had a AAX "RX7 music rebalance" in 2018 & VST "Ozone 9 master rebalance " in 2019.

Since spleeter U-Net is so simple, I do a full rewritten version in C for real time lag-less VST inference.
https://github.com/james34602/SpleeterRT

I realise various companies have implemented real time separation already so just wondering if they are using a variation of spleeter or there's other ai projects out there? Just tested the free new Virtual DJ version and it allows real time separation, with tracks loading instantly. So surely there is a way to improve the current spleeter? My pc with gpu takes quite some seconds but these new apps are doing it instantly, even works on an ipad for the djay Pro app.

Probably just Spleeter with audio buffer streamed to a background service.
But I could be wrong.
sön 12 juli 2020 kl. 11:11 skrev Scylla2020<EMAIL_ADDRESS>
I realise various companies have implemented real time separation already
so just wondering if they are using a variation of spleeter or there's
other ai projects out there? Just tested the free new Virtual DJ version
and it allows real time separation, with tracks loading instantly. So
surely there is a way to improve the current spleeter? My pc with gpu takes
quite some seconds but these new apps are doing it instantly, even works on
an ipad for the djay Pro app.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/deezer/spleeter/issues/276#issuecomment-657195112,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIREEEHMDA5DLVABD43XOTR3F43FANCNFSM4KZ2E4XA
.

--
Sincerely, Aid vllasaliu

@aidv is right.
SpleeterRT is continuous STFT buffered, lag-less(no frame drop for modern CPU).
But not real-time algorithmically.
@Scylla2020
I'll tell you what, true real-time is not possible without tricks, even all commercial high-quality separator is not real-time, iZotope Music Rebalance probably use a high buffer pre-cache to give you the illusion of real-time process, but what's the tricks, I can't reveal.,

@aidv @james34602 Oh i see. So this buffer trick would just have been reimplented in different languages as per the target environment, for example Virtual Dj could have rewritten the functionality in C++ for speed and to avoid using python

So really there is no escaping the real time it actually currently takes to separate a track from start to end?

@Scylla2020
Avoid using Python is totally the point, beside Python, SpleeterRT offer flexible intrinsic latency adjustment(How many time frame for the neural network to inference).
The main advantage C neural network is mainly make you easier to debug and integrate your software, link and loading DLL of libtensorflow can be very slow.
The buffering and STFT system already written for real time buffering system that require.

@james34602 So SpleeterRT will take just as much time if it was processing a track just like how Spleeter does? Or is it just for real time applications. Im not too familiar with C

@Scylla2020 In short yes.
SpleeterRT offer background processing, corresponding to Spleeter4Stems.c.
If you use spleeter.c directly for offline inference, you may see sightly faster inference compared to tensorflow-cpu.

@Scylla2020 I haven't used the DJ software, but I suspect they are misusing the word "realtime".
Obviously as a DJ you don't stream any audio, you just load the file, right? The file is then processed through Spleeter and then the stems are made available.
It's not realtime. They just say that for marketing. It takes probably 30 seconds to process a 3 minute song on a laptop, and thus in theory it could be considered realtime, actually faster than realtime, but I doubt they stream the audio buffer through Spleeter.
It just doesn't make any sense why they would do that. Just separate the stems once in one go and save the stems.

@aidv
@Scylla2020
DJ software marketing "real time" is not good, but not wrong in definition, "real time" definition first appear in computer as a measures of reaching deadline, however, the definition in computer doesn't suit audio, because audio system is streaming, mean that if you can't provide enough samples for a system to process, the system simply can't output anything.
In Spleeter cases, if you don't provide enough T frames, the neural network won't inference anything.

@Scylla2020 I believe VDJ/djay are using classic FFT algorithmic things to do realtime separation. It's been possible for a few years, but no-one's interested because it's hard, it actually required understanding of instruments, and you can quickly achieve results for ML given enough GPUs.
Nothing spleeter or ML based, that's too slow for 4x songs on an ipad, and not full bandwidth.

@junh1024 The site you provided sound like HPSS algorithm, which is almost real time in algorithmic sense.
For real time-ness:
I guess you understand "Processing time" and "Algorithmic latency".
Assume a supercomputer can compute anything in the world in 1 ns, it going to run a algorithm with algorithmic latency of 3 seconds, supercomputer going to process recorded signal from microphone, the supercomputer still need at least 3 seconds real-world clock to output result, no matter how fast your supercomputer is.

@aidv @junh1024 I doubt they using old tools as all this has come about soon after spleeter so I believe their solution is based on reimplementing spleeter differently. Also the results sound exactly like using spleeter with its default parameters. Since you mention the djying software would have already loaded and processed the track to separate stems, there must be a new super fast solution then thats way better than current spleeter. My computer takes about 10-15 seconds for a 3min track yet on virtual dj this separation seems to be almost instantaneous and I can randomly skip to various sections of the track with no issues

So, like I’ve said, I don’t own that software and I’m only speculating,
chopping up the song into smaller pieces and processing only the parts
relevant to where you’re at in the timeline surely will feel like
instantaniously.
It’s not rocket science.
If 3 minutes take you 10 seconds to process using Spleeter on the cpu, that
would be 18x faster than realtime, so while DJing you’re probably looking
at 2.5s before and 2.5s after the transport position.
So 5s in total to process through spleeter would take  0.27 seconds, and
that will feel like instant.
mån 13 juli 2020 kl. 07:35 skrev Scylla2020<EMAIL_ADDRESS>
@aidv https://github.com/aidv @junh1024 https://github.com/junh1024 I
doubt they using old tools as all this has come about soon after spleeter
so I believe their solution is based on reimplementing spleeter
differently. Also the results sound exactly like using spleeter with its
default parameters. Since you mention the djying software would have
already loaded and processed the track to separate stems, there must be a
new super fast solution then thats way better than current spleeter. My
computer takes about 10-15 seconds for a 3min track yet on virtual dj this
separation seems to be almost instantaneous and I can randomly skip to
various sections of the track with no issues
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/deezer/spleeter/issues/276#issuecomment-657368656,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIREEE2O3F2WMGWQNM4J53R3KMLDANCNFSM4KZ2E4XA
.

--
Sincerely, Aid vllasaliu

@aidv Ok thanks I got it now. It wasnt clear from your previous comment what you meant about loading it did seem like you meant the whole song gets loaded at once. Everyone is just trying to learn here no one said its rocket science. Be gentle

What I mean with it’s not rocket science is that it’s not as complicated as
you might think. There is a very logical explanation as to how they might
have achieved their performance, thus, no need to overthi nk it, hence,
it’s not rocket science.
It’s not even machine learning at that point.
mån 13 juli 2020 kl. 08:06 skrev Scylla2020<EMAIL_ADDRESS>
@aidv https://github.com/aidv Ok thanks I got it now. It wasnt clear
from your previous comment what you meant about loading it did seem like
you meant the whole song gets loaded at once. Everyone is just trying to
learn here no one said its rocket science. Be gentle
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/deezer/spleeter/issues/276#issuecomment-657378301,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIREEB53KNNREEOSWLN65TR3KP7XANCNFSM4KZ2E4XA
.

--
Sincerely, Aid vllasaliu

Also the results sound exactly like using spleeter.
@Scylla2020 Not necessary, given enough dataset to train, most STFT-based BSS sound similar, you can check your DJ software program files to see the model file, file size tell if they are actually Spleeter behind the scene.

@Scylla2020

I doubt they using old tools as all this has come about soon after spleeter so I believe their solution is based on reimplementing spleeter differently.

They are making NEW tools, which I believe are based on OLD ideas. Just because Y came after X, it doesn't mean that Y HAS to be based on X. As I said, i believe it's not ML, nor spleeter. Maybe something like https://github.com/tachi-hi/euterpe/

Hi @gvne!
What I would recommend is to perform overlap and add with a short T that does not need retraining (for instance T=64). You would then process 1.5s of the past audio and just output the as short as you want current frame of audio, the size of which would be mainly set by the computation power of your system. Do you have an idea of a many T-long segment you can process per second ? (note that doing STFT/iSTFT outside tensorflow may make things faster on CPU).

Regarding the solution for short latency, I'm wonder, what will happens if I will take long T (512) but will use only the last vector of the mask to reconstruct one frame from the input spectrogram  (this should occurs every frame_step=1024 samples). of course, this will consumes a lot of CPU. but basically is it should gives with the same performance?
If it works the latency will be (frame_length)/fs = 4096/44100 ~=90ms + processing time depends on the CPU/GPU.
Actually, is right now there is an overlap between the frames of the spectrogram (in the inference)? or each input spectrogram
produces a mask without depending the neighbor spectrograms?

Hi @gvne!
What I would recommend is to perform overlap and add with a short T that does not need retraining (for instance T=64). You would then process 1.5s of the past audio and just output the as short as you want current frame of audio, the size of which would be mainly set by the computation power of your system. Do you have an idea of a many T-long segment you can process per second ? (note that doing STFT/iSTFT outside tensorflow may make things faster on CPU).

Regarding the solution for short latency, I'm wonder, what will happens if I will take long T (512) but will use only the last vector of the mask to reconstruct one frame from the input spectrogram (this should occurs every frame_step=1024 samples). of course, this will consumes a lot of CPU. but basically is it should gives with the same performance?
If it works the latency will be (frame_length)/fs = 4096/44100 ~=90ms + processing time depends on the CPU/GPU.
Actually, is right now there is an overlap between the frames of the spectrogram (in the inference)? or each input spectrogram
produces a mask without depending the neighbor spectrograms?

Quality will be worsen, since the information you use are only from the past, waste too much computation that does not contribute to the output.
The CPU usage will going to be impractical, approximately, 20 minutes processing time for 1 song on CPU. 1 minutes processing time on GPU for a 3 minutes song.

Hi @james34602  ,
Thanks for the quick reply.
What I try to do is also implement Spleeter for real-time with low latency.
May you tell me how much information from the future does Spleeter takes right now? is it an half length of the spectrogram (256 frames ~=6 sec)?
I'm trying to understand it from the code but it's a bit hard for me to follow the process , since it's written in TensorFlow  .
Thank you very much.

Hi @james34602 ,
Thanks for the quick reply.
What I try to do is also implement Spleeter for real-time with low latency.
May you tell me how much information from the future does Spleeter takes right now? is it an half length of the spectrogram (256 frames ~=6 sec)?
I'm trying to understand it from the code but it's a bit hard for me to follow the process , since it's written in TensorFlow .
Thank you very much.

Hello.
Default buffering mode assume to take full future information, e.g. T=64, 63 future frames, 1 present frame.
Ok, well, it's not about Tensorflow, the way Tensorflow behave is the same as Matlab and many others.
T in Spleeter mean time step of frame "jumping".

@james34602  what you say is that the decision for  a fft frame (the present one) relays only on the future frames (the next 63 frames in your example), right?
It's a bit incomprehensible to me. because if the decision is for a frame from the edge of the spectrogram figure, why it does matter if we take the rest frames from the future (say 'right' side) or from the past (the 'left' side of the figure)?
I thought that the decision is for the middle frame, such that we are utilizing the neighbor frames both from the past and the future.
(I mean, for T=64, the decision is for the 32th frame, by utilizing the next 32 frames and the last 31 frames).

T in Spleeter mean time step of frame "jumping".
I think T mean the number of fft frames in the spectrogram. but not the "jumping". T maybe 512, with jumping of one frame, no?   or I'm missing something?

Thanks

@james34602 what you say is that the decision for a fft frame (the present one) relays only on the future frames (the next 63 frames in your example), right?
It's a bit incomprehensible to me. because if the decision is for a frame from the edge of the spectrogram figure, why it does matter if we take the rest frames from the future (say 'right' side) or from the past (the 'left' side of the figure)?
I thought that the decision is for the middle frame, such that we are utilizing the neighbor frames both from the past and the future.
(I mean, for T=64, the decision is for the 32th frame, by utilizing the next 32 frames and the last 31 frames).

T in Spleeter mean time step of frame "jumping".

I think T mean the number of fft frames in the spectrogram. but not the "jumping". T maybe 512, with jumping of one frame, no? or I'm missing something?
Thanks

T = 64 is one "jump" in my definition, I'm not a trained english speaker.
64 spectral frames == one shot for the inference.
You can experiment yourself, the future and past doesn't matter a lot, the system is non-causal, this is the way network was trained, the left hand side and right hand side happen all at once, what you going to do is assume right hand side frame is present, so you will get real time response, which is true, but like I said the system(network itself) uses information from the left hand side contribute to the right hand side.
What your neural network can do when left hand side is empty(Black)?
This is why quality must drops, because there have no non-causal information.
This is very common you got either 1 empty stem track.
Or, what if spectral image of left hand side generates from different vocal singer?
The past spectral distribution is no more similar to present.
This happens when you change song or the song switching vocal player or any form of spectral distribution changes.

How about this realtime project. I really hook on it. Thanks so much.


[Auto-generated] Update dependencies

Update dependencies
Automatically created PR from ci/dependabot-updates.
The PR is based on the 'CI - Update dependencies PR' workflow in SINTEF/ci-cd.
To do

[ ] Check that the diff is sensible, and that tests and builds pass with the new dependency versions.
[ ] Make sure that the PR is squash merged, with a sensible commit message.

Codecov Report

Merging #520 (abc89c6) into master (53979dc) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #520   +/-   ##
=======================================
  Coverage   65.46%   65.46%
=======================================
  Files          16       16
  Lines        3081     3081
=======================================
  Hits         2017     2017
  Misses       1064     1064

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Upgrade to Homestead 2.0

@railto Give this a spin before we merge. It works for me but I want to make sure we're not breaking anything.
I'm using Vagrant 1.7.2 but want to make sure this works for 1.6.x

Tested this on Vagrant 1.6.5 and everything runs as expected.


[stable/prometheus-operator] Ignore coredns version

What this PR does / why we need it
When prometheus-operator was copied from coreos the was a rule that didn't applied to Google Cloud managed kubernetes cluster.
It seem that the job kube-dns is called coredns in Google Cloud, here are my metrics in prometheus:
kubernetes_build_info{buildDate="1970-01-01T00:00:00Z",compiler="gc",endpoint="http-metrics",gitCommit="$Format:%H$",gitTreeState="not a git tree",gitVersion="v1.5.2-beta.0+$Format:%h$",goVersion="go1.10.3",instance="<IP_ADDRESS>:10055",job="coredns",major="1",minor="5+",namespace="kube-system",platform="linux/amd64",pod="kube-dns-78d5dcc986-7b6cs",service="prometheus-coredns"} | 1
-- | --
kubernetes_build_info{buildDate="1970-01-01T00:00:00Z",compiler="gc",endpoint="http-metrics",gitCommit="$Format:%H$",gitTreeState="not a git tree",gitVersion="v1.5.2-beta.0+$Format:%h$",goVersion="go1.10.3",instance="<IP_ADDRESS>:10055",job="coredns",major="1",minor="5+",namespace="kube-system",platform="linux/amd64",pod="kube-dns-78d5dcc986-wwrpj",service="prometheus-coredns"} | 1

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

not sure why it says DCO not signed... I did it

/assign @vsliouniaev

@bclermont the rules are synced from upstream repos using the scripts in the /hack folder. Please submit your PR there and then sync to here

@vsliouniaev Just did. Someone fixed that bug a few days ago in upstream
But there had been no releases for the last few months, so I just did it on master (not sure if that will be approved).


Keep dates of creating/publishing vocabularies

We could use dct:created for the ontology

Before
<http://w3id.org/neologism/FE_Ontology#> a owl:Ontology;
    dcterms:creator<EMAIL_ADDRESS>    dcterms:title "FE_Ontology";
    dcterms:description "".

After:
<http://w3id.org/neologism/FE_Ontology#> a owl:Ontology;
    dcterms:creator<EMAIL_ADDRESS>    dcterms:title "FE_Ontology";
    dcterms:created "YYYY-MM-DD";
    dcterms:description "".


Removing the isDefined check for scope.resultImage

Removing the isDefined check for scope.resultImage. I should not be required to define the property I bind to the resultImage object to. This is a very unnecessary requirement to place upon your users and it can cause issues when using more complex javascript objects which do not have their entire structure predefined in code, as is my case in a lot of situations.

Oh yeah, accept this push request. I've just passed 30 minutes trying to make this module works and it was all because of an undefined variable...


[Don't Merge] Toast stack resize

Example of sizing down large toasts when they are under a smaller one. Also, reduced the opacity of the bottom toast to give the impression of more toasts behind.
I had to turnoff some animation timings for this POC that were interfering with getting the right container sizes. I couldn't find a suitable solution in the time left, so that is something that needs a solution.
This is also missing an obvious indication that a toast was added to the back of the stack.
✅ Pull Request Checklist:

[ ] Included link to corresponding React Spectrum GitHub Issue.
[ ] Added/updated unit tests and storybook for this change (for new code or code which already has tests).
[ ] Filled out test instructions.
[ ] Updated documentation (if it already exists for this component).
[ ] Looked at the Accessibility Practices for this feature - Aria Practices

📝 Test Instructions:

🧢 Your Project:

Closing since Spectrum guidance is now that toasts should stacking vertically


Use mapbox gl instead of homegrown setoptions

https://github.com/mapbox/mapbox-gl-js/blob/master/js/util/util.js#L327-L345
This is exported by mapbox gl - we should use this instead of homegrowing setOptions

is it? https://github.com/mapbox/mapbox-gl-js/blob/master/js/util/util.js#L335

it's not publicly documented, but it's exported under mapboxgl.util

:+1: -> https://github.com/mapbox/gl-draw/commit/f56048bdee079bb6425eca0c7a6d7810a2dd7b1b

Please don't do this. If it's explicitly marked as a private API, it's just that -- not meant to be used by other libraries, could be removed or changed at any time.


Adjust icon sizing in app icon

Currently we use the same size as e.g. mobile safari. However, it's too visually large, and most icons sit within a smaller inner circle bound. We should do a survey of other icons that have circular/close-to-circular marks, and align with the rough bounds of those app icons.

Some circle icon examples from my phone. Looks like most apps are quite consistent with apple's inner circle radius.

Subconscious matches Safari/Spotify/Settings/etc. The distinction is that for most of these icons the circle acts as a platform for the real icon.
This leaves Subconscious icon feeling big and cramped to me as compared to others. I sense that a radius closer to GH would feel mor comfortable.
IIRC Apple has some guidelines about this that we should dig up.


Exit components on OutOfMemory exceptions

Scala's idiomatic error handling makes it "hardish" to exit the JVM on fatal errors (for example if a Future fails with a fatal exception). The added flags just ignore error handling and exit the JVM, which we can recover by a docker auto-restart for example.
I bumped the Java versions because the flag was added in 8u92. Figured we might as well go to latest. It got released in January, so it should be stable.

Unfortunately, yes. Other oracle JDK dockerhub images do the same.


tests: refactored fs watch_dir tests for stability

fs_event_watch_dir and fs_event_watch_dir_recursive could fail randomly
due to the way in which the tests were written.  Originally timers were
used to create, remove and recreate the test files but this could lead
to a race condition if the timeout used to delete the test files ran
before all file creation fs events were handled.  On top of that, the
file removal timer scheduled another timer to recreate the test files
and that timer's timeout could also lead to the same condition.
The refactoring removed timers for the removal/recreation of the test
files and instead the fs event callback was updated to have the
necessary logic to drive the test file removal.  We no longer recreate
the test files since it appears to be unnecessary.
With the current PR, I can run 1000 tests for fs_event_watch_dir and
fs_event_watch_dir_recursive without a single failure, where prior to this
PR they would fail 14/100 on average.
Fixes #30

/cc @saghul @indutny @misterdjules

One thing I forgot to mention, or maybe it wasn't clear, is that the original test appeared to do two cycles of creating files and then removing the files.  This current PR only does one cycle but updating it to do two should be trivial.  It's your call as to whether I add the extra cycle back or not.

I'm afk until Monday. Will take a look then! Thanks!
On Aug 15, 2015 00:09, "Jeremy Whitlock"<EMAIL_ADDRESS>wrote:

/cc @saghul https://github.com/saghul @indutny
https://github.com/indutny @misterdjules
https://github.com/misterdjules
—
Reply to this email directly or view it on GitHub
https://github.com/libuv/libuv/pull/480#issuecomment-131252036.

CI: https://jenkins-iojs.nodesource.com/view/libuv/job/libuv+any-pr+multi/130/

Not sure if it was failing before, but SmartOS and FreeBSD are failing:
SmartOS:
`fs_event_watch_dir` failed: exit code 134
Output from process `fs_event_watch_dir`:
Assertion failed in test/test-fs-event.c on line 175: events == UV_RENAME

FreeBSD:
%  86|+ 216|-   7|T   0|S  12]: fs_event_watch_dir
`fs_event_watch_dir` failed: timeout
Output from process `fs_event_watch_dir`:
fs_event_cb_dir_multi_file called: 1
  created: 16
  removed: 0

I will remove the unnecessary stuff and test on the other environments.  I will update you.

Thanks! Ping me me if you need me to trigger the CI.

I have updated the PR.  I removed my debugging output (sorry), removed the unnecessary assertions @saghul mentioned and updated the assertion in the fs_event_cb_dir_multi_file callback so we do not have assertion failures on SmartOS.  Fixing the assertion on SmartOS does not fix the tests completely as it will timeout like it does on FreeBSD but these tests were failing prior to my PR and appear to be a deeper libuv issue than a test race condition, since that is now gone.  (A quick debugging session shows that on FreeBSD and SmartOS that the fs_event_cb_dir_multi_file callback is only invoked once instead instead of once per created/removed file.)

CI: https://jenkins-iojs.nodesource.com/view/libuv/job/libuv+any-pr+multi/132/

I can confirm I've somehow broken something.  I don't know how but it seems that way.  If I build v1.x on FreeBSD without my change, all 16 events are processed prior to the timeout whereas with my change, I only see 1 event processed prior to the timeout.  I'll get this sorted and will update the PR.

I've found something.  On FreeBSD and SmartOS, the tests were failing prior to this PR.  But after looking at the CI environment, it seems something changed for fs_event_watch_dir because it use to pass on CentOS 7 but now it's failing there.  I did notice on FreeBSD that prior to this PR, all create events were handled but that is not the case now.  That being said, I'll chase this down and update the PR.

I've cleaned up the sources a little to remove some compiler warnings on non-OSX/Windows.  Progress:
FreeBSD
FreeBSD was failing prior to this PR but I wanted to see if there was something I could do about it.  What I've noticed is that the fs event callback fs_event_cb_dir_multi_file is called only once even though the test creates 16 files, which should result in 16 separate fs_event_cb_dir_multi_file invocations.  dtruss -e -d -f ./test/run-tests fs_event_watch_dir shows only 3 kevent calls and that doesn't seem right but it seems to match up with what I'm seeing in my debugging.  On a whim, I removed the EV_ONESHOT from https://github.com/libuv/libuv/blob/v1.x/src/unix/kqueue.c#L97 and the test passes but I know that is not the fix, and it could very well be a red herring.  I'm working with @indutny on this but I don't know how much of his time I'll have.
CentOS
Building an environment as we speak.  No idea why it would stop passing due to these changes but I'll find out soon enough.
SmartOS
SmartOS was failing prior to this PR as well and it has similar symptoms to FreeBSD.  I will circle around to SmartOS when I make progress on one of the environments above.

I've merged in v1.x changes and resolved conflicts.  I'm testing on CentOS right now.

I just tested locally on CentOS 7 and fs_event_watch_dir passes without issue.  Testing FreeBSD again.

I think we're back in the state we were prior to this PR.  FreeBSD and SmartOS are failing as they were before and I think it might be a bigger issue.
@saghul Can you kick off a new CI build to make sure CentOS7 is passing again?

Thanks again for the effort you're putting  on this. I'll trigger the CI as
soon as I'm back home.
On Aug 18, 2015 5:33 PM, "Jeremy Whitlock"<EMAIL_ADDRESS>wrote:

I think we're back in the state we were prior to this PR. FreeBSD and
SmartOS are failing as they were before and I think it might be a bigger
issue.
@saghul https://github.com/saghul Can you kick off a new CI build to
make sure CentOS7 is passing again?
—
Reply to this email directly or view it on GitHub
https://github.com/libuv/libuv/pull/480#issuecomment-132251855.

No problem.  If you get a few minutes when you get home, ping me in IRC so we can chat.

I have investigated the FreeBSD side. I'm afraid this is just how it works: it coalesces events into one knote and returns them at kevent() call.
I think we could do following thing - make it create and unlink files in different event-loop ticks. Does it make sense?

Yes.  I will update the PR when I've got that approach implemented.  Thanks for your help.

The latest version of the PR has fs_event_watch_dir passing on FreeBSD, which should allow us to check that one off of #128 if/when this gets merged, and it is also passing on SmartOS now.  (Both of these were failing prior to this PR.)
Once @indutny verified that the fs event callback fs_event_watch_dir was only being called once and was by design (expected), we realized the only way for this test to pass as written was to create one file per event loop tick and remove one file per event loop tick to make sure that the proper number of fs events were emitted for the created and removed files.
As it stands, this PR not only makes the fs_event_watch_dir and fs_event_watch_dir_recursive stable, it also fixes fs_event_watch_dir on FreeBSD, SmartOS and possibly others where it might had been failing.  Thanks a lot @indutny and @misterdjules.

@whitlockjc sorry, yesterday ended up not working for me. CI: https://jenkins-iojs.nodesource.com/view/libuv/job/libuv+any-pr+multi/133/

@whitlockjc can you please rebase again? I pushed a fix for the broken build on Windows.

Done.

CI: https://jenkins-iojs.nodesource.com/view/libuv/job/libuv+any-pr+multi/135/

Great job @whitlockjc! LGTM :+1:

Thanks to all that helped: @indutny and @misterdjules  Two excellent mentors.

Landed in :sparkles: 8326470 :sparkles: Much obliged, @whitlockjc!

WOOHOO!


Metadata on uploadFile

Metadata param on uploadFile
Ability to send custom metadata on uploadFile, which is available on API (https://uploadcare.com/api-refs/upload-api/#operation/baseUpload)
Metadata is helpful to identity the images/files and match in database. Informations like email, user id, etc.
It could be something like:
client.uploadFile(file, { metadata: { user_email: 'some@email' } });

Hey @jonatasdaniel, It's WIP, see PR #368

Hey @jonatasdaniel, It's WIP, see PR #368

is there any example?


Feat/opening metrics

Description
Added elements to the Opening Screen
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Docker Compose

Checklist

[x] I have read the CONTRIBUTING doc
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have already been accepted and merged

Further comments

Thanks for the PR!
Any successful deployments (not always required) will be available below.
Backend: https://nr-silva-249-backend.apps.silver.devops.gov.bc.ca/actuator/health
Frontend: https://nr-silva-49-frontend.apps.silver.devops.gov.bc.ca
Once merged, code will be promoted and handed off to following workflow run.
Main Merge Workflow

Stale.  Closing.  Can be revisited.


Could not find devs.mulham.horizontalcalendar:horizontalcalendar:1.3.4.

Execution failed for task ':app:mapDebugSourceSetPaths'.

Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Could not find devs.mulham.horizontalcalendar:horizontalcalendar:1.3.4.
Searched in the following locations:
- https://repo.maven.apache.org/maven2/devs/mulham/horizontalcalendar/horizontalcalendar/1.3.4/horizontalcalendar-1.3.4.pom
- https://dl.google.com/dl/android/maven2/devs/mulham/horizontalcalendar/horizontalcalendar/1.3.4/horizontalcalendar-1.3.4.pom

Steps to Reproduce the Problem (sample code if possible)
Upgrade android to sdk 34
Specifications

Android Version: 34
Horizontal-Calendar Version: 1.3.4

same thing happening with me too

Hello, any updates here?
Google has forced an upgrade to target SDK 34 as of August 31st, 2024. I guess that means this library can no longer be used in releasable apps on Google Play?

Hi @MontePablo  and @jessica-muremure ,
I just configure my settings.gradle as foolowing:
I keep it the jcenter() as the mentioned library is hosted there,please try this approach and let me know if it works for you
pluginManagement {
repositories {
google()
mavenCentral()
gradlePluginPortal()
jcenter()  //keep jcenter as the devs.med lib is hosted in jcenter!!!!
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
jcenter() //keep jcenter as the devs.med lib is hosted in jcenter!!!!
}

It did not work for me. Jcenter link is getting 404 error.
https://jcenter.bintray.com/devs/mulham/horizontalcalendar/horizontalcalendar/1.3.4/horizontalcalendar-1.3.4.pom

thank you a lot Lukeng, I appreciate your contribution a lot. Im realy
thankyou to you. bytheby I can't test this repo because I have switched to
something else and skipped it , there willbe definitely someone out there
who is still searching for the solution to this problem :-) . have a
happy life Lukeng.
On Thu, Sep 5, 2024 at 10:27 PM lukeng4 @.***> wrote:

It did not work for me. Jcenter link is getting 404 error.
https://jcenter.bintray.com/devs/mulham/horizontalcalendar/horizontalcalendar/1.3.4/horizontalcalendar-1.3.4.pom
—
Reply to this email directly, view it on GitHub
https://github.com/muraee/Horizontal-Calendar/issues/144#issuecomment-2332220030,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWVEWOARUJXVVYHWOT2XLEDZVCENXAVCNFSM6AAAAABNBLGFXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZSGIZDAMBTGA
.
You are receiving this because you were mentioned.Message ID:
@.***>

The same issue here, but i download AAR file from this location :

and build successful without using any repository
dependencies {
implementation files('libs/horizontalcalendar-1.3.4.aar')
}

I hope this will help

thank you so much Stefan for giving time on this issue. definitely it will
solve the problem for lots of people
On Wed, Sep 11, 2024 at 3:33 PM Stefan Kanev @.***>
wrote:

The same issue here, but i download AAR file from this location :
http://maven.wafour.com:8081/artifactory/libs-release/devs/mulham/horizontalcalendar/horizontalcalendar/1.3.4/
and build successful without using any repository
dependencies {
implementation files('libs/horizontalcalendar-1.3.4.aar')
}
I hope this will help
—
Reply to this email directly, view it on GitHub
https://github.com/muraee/Horizontal-Calendar/issues/144#issuecomment-2343199223,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWVEWOAZO6PNGWC5LXIAHF3ZWAIQFAVCNFSM6AAAAABNBLGFXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBTGE4TSMRSGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

Here is forked version working and published on jitpack

Github
Jitpack


Code Coverage not 100%

Is there a reason why code coverage is not 100% or was it simply not completed for the starter code?

You can always submit PR if you feel something is missing or coverage could be higher.

100% coverage for this project is really not all that important to me, and there are a few issues with the coverage reporting anyways, esp. now that isparta is apparently deprecated (I'm looking into this). It's more to just demo the test suite rather than test the project itself, if that makes sense.


DistilBERT: Issue with Multi-Device Tensor Handling in Data Parallel Implementation

We are working on Data Parallel implementation of DistilBERT model taking reference from (#9507)
We encountered the issue: Cannot mix single and multi-device tensors when calling launch op!.
We could find a workaroud with the following changes in ttnn/cpp/ttnn/operations/matmul/matmul.hpp
Changed:
constexpr auto matmul = ttnn::register_operation<"ttnn::matmul", operations::matmul::MatmulOperation>();
constexpr auto linear = ttnn::register_operation<"ttnn::linear", operations::matmul::LinearOperation>();

To:
constexpr auto matmul = ttnn::register_operation_with_auto_launch_op<"ttnn::matmul", operations::matmul::MatmulOperation>();
constexpr auto linear = ttnn::register_operation_with_auto_launch_op<"ttnn::linear", operations::matmul::LinearOperation>();

Is this change valid?
These changes allowed us to run the Data Parallel implementation for the DistilBERT model on n300.

However, the implementation might be incomplete as we couldn't get complete output.
With a batch size of 8, the output shape from the ttnn_distilbert model was (4, 384) instead of the expected (8, 384). We checked the PCC between ttnn_output and torch_output[:4, :] and found it to be greater than 0.99, indicating that the output from the second device is missing.
By analysing the perf sheet here, we conclude that all ops are not in data parallel. Most of the ops aren't running on both the devices.
We are working to resolve these issues, and any suggestions or guidance would be help us with data parallel implementation. current branch is  here
Distilbert_data_parallel_n300_2024_09_13_13_30_25.csv

cc @boris-drazic , @jvasilje

@bbradelTT, The changes mentioned above were made in the ttnn/cpp/ttnn/operations/matmul/matmul.hpp  file to address the Cannot mix single and multi-device tensors when calling launch op! issue.
Could you please confirm if these changes are appropriate and expected on the main branch?

@Sudharsan-V we explicitly removed register_operation_with_auto_launch_op  and do not want to go back to it. Also, what you are trying to do is not supported directly.
You would need to use CCL to run matmul across multiple devices.
@SeanNijjar would be able to provide more information about CCL.
cc @TT-BrianLiu

@Sudharsan-V  For CCL you can also see https://github.com/tenstorrent/tt-metal/blob/main/tech_reports/CCL/CclDeveloperGuide.md

I want to confirm the issue here.
If the model is purely data parallel across some number of devices - in this case I assume 8, then we shouldn't need any explicit CCLs on the model or op side because for data parallel we don't need to move data between the different model instances running across the chips - they should all be able to run independently (except for perhaps splitting and merging inputs/outputs respectively).
To support this use case, @cfjchu has been working on the multi-device tensor infrastructure. I'd recommend checking out the doc found here: https://github.com/tenstorrent/tt-metal/blob/main/tech_reports/Programming Mesh of Devices/Programming Mesh of Devices with TT-NN.md
More specifically, from a quick glance it looks like https://github.com/tenstorrent/tt-metal/blob/main/tech_reports/Programming Mesh of Devices/Programming Mesh of Devices with TT-NN.md#3-distributing-tensor-to-meshdevice has the relevant information.

@SeanNijjar is right. There shouldn't be any need for explicit CCL operations for data-parallel. @Sudharsan-V Please review relevant docs in multi-device tensor infra. Make sure all inputs to the model use a mesh_mapper / i.e. shard activations and replicate any model parameters

The issue has been resolved.
The problem was that one of the input tensors, position_ids, had a batch size = 1 (regardless of the input batch size). Since we were slicing along axis 0, this tensor wasn't sharded across devices, causing the issue.
The position_ids tensor has now been updated to have a batch size greater than 1, which resolves the problem.
As the issue has been resolved, closing it.
Thank you @bbradelTT , @SeanNijjar , @cfjchu


Access to the data

Hello,
Firstly, thank you for this. Amazing work!
Hello, I'm a PhD student and I have applied to get access to your dataset. I haven't received any reply yet, could you please give me access.
<EMAIL_ADDRESS>Best regards,
Markus Ekvall

Just accepted all pending valid requests, I apologize for the delay.


Enable absolute paths for target folder

Currently not supported (see #2), will be added with the next patch

Thank you again for your help and work on this tool.  Good stuff.

Hi @agentx216, I've started implementing this.
More information can be found in #39.


The osf.io website is too slow, takes minitues to load for simple webpages

My location is inside China. The datacenter selected is in Australia. When I share my project in cell phone apps, it will take minutes to load the home wiki page. This will make it extremely difficult to share anything on OSF. Why does OSF has to be soooo sloooow compared to most other websites? It seems that a lot of elements need to be loaded sequentially before any real contents can be loaded.
Additionally, when I open it in cell phone, the " Project Wiki Pages" box takes the whole screen even though I only put two entries there, making people assume that there is no more content and abandon the page.

@yjiangnan Do you use a VPN service to connect to our website?

Why not put a server inside China or somewhere close to China (such as Google cloud datacenter in Taiwan)? For users in China, it is known that there is the Great Fire Wall that slows everything down. The best choice is to put a server inside China. Otherwise, a server with CN2 GIA network is required for good Internet connections inside China.

@adlius
No. VPN only makes things worse for OSF. I connect to OSF directly.

A server in Hong Kong may also improve the situation. But the software side may also need improvements.


Package request: Graphite

Graphite is an in-development raster and vector graphics package that's free and open source. It is powered by a node graph compositing engine that fuses layers with nodes, providing a fully nondestructive editing experience.

Metadata

homepage URL: https://github.com/GraphiteEditor/Graphite
source URL:
license: Apache 2.0
platforms: linux

Add a :+1: reaction to issues you find important.

AFAICT the wasm32-unknown-unknown target is not yet supported by nixpkgs's rust tooling.
Upstream has a shell.nix that uses an external overlay for wasm support. You might be able to use that to build the package yourself (not as a derivation).
If there's more interest I can try to create an out-of-tree package definition, using that overlay.


Checksum offloading has been enabled.

What about checking whether VM requests checksumming via VIRTIO_NET_HDR_F_NEEDS_CSUM? Raja please advise if we close this with the current solution or we should add support for the flag above before?

That can be done in another commit in agile-style development. Currently the primary functionality requested have been provided and I find no reason to hold it back (as it has already taken too much) for the time that I would need - with my lack of knowledge about dpdk and virtio - to figure out how to condition the CUM offloading basing on VIRTIO_NET_HDR_F_NEEDS_CSUM. Better to have the feature slowly advancing and working...

Dominic,
If the guest did not request checksum offload (say, for a non TCP/UDP packet such as ping), we shouldn't tell the NIC to do checksum offload. So I think there might be a problem if we always assume that the NIC should do checksum offload. You can revisit this after the rebasing is done, so let's not commit this change for now.

The checksums will be fixed on master.


The font name from an embedded font contains a strange prefix

Describe the bug
This is a follow up of #72. When converting the test.pdf to html, the font is not recognized because it is named incorrectly.
There are actually two issues here:

[ ] The font-name has an strange prefix, e.g. "VZWISY+Georgia" instead of "Georgia"
[ ] The font-family css attribute has a binary string as value, including the b'' prefix.

To Reproduce
Convert the test.pdf to html:
pdf2txt.py test.pdf -t html -o test.html

Observe the span element:
The Portable Document Format (PDF) is the world’s leading language for describing

the printed page

Expected behavior
Extract font-name correctly, or rename it in HTMLConverter such that Georgia is recognized by the browser.

I figured out what the strange prefix is. According to Section 5.5.3 from the PDF Reference:

For a font subset, the PostScript name of the font - the value of the font's BaseFont entry and the font descriptor's FontName entry - begins with a tag followed by a plus sign (+). The tag consists of exactly six uppercase letters; the choice of letters is arbitrary, but different subsets in the same PDF file must have different tags. For example, EOODIA+Poetica is the name of a subset of Poetica, a Type 1 font.


Bulk Client login ignores provided endpoint value

Forcex.Bulk.Client.login ignores the provided endpoint in the conf if passed in, and instead always makes a HTTPoison.post! to "https://login.salesforce.com/services/Soap/". See here:
https://github.com/jeffweiss/forcex/blob/master/lib/forcex/bulk/client.ex#L58
This means I cannot make Bulk requests against test.salesforce.com or other endpoints.

Fixed with #46


[VisualDebugging] Cannot retain Systems game object after switching to new scene

When switching to whole new scene, the Systems game object will just getting destroyed as it does not call Object.DontDestroyOnLoad. I suggest make Systems game object still retain after switching to new scene or provide an API to do the setting. For now, I just use DontDestroyOnLoad(GameObject.Find("Root Systems")); to fix this issue but it's kind of ugly.

The destruction of systems is an essential part of their usage. This is what we use ITearDownSystem for.
You basically use it to cleanup the current state of the game - unlink entities that have their GameObject views destroyed, etc.
If you need a notion of common systems that don't get destroyed for the whole lifetime of a game, you should create them from an already DontDestroyOnLoad object.
Systems themselves should know nothing about whether they are used from Unity or from pure C# code, it's just an unnecessary coupling.

As every game is different, sometimes you want the the VisualDebugging Systems Object to be destroyed when changing scenes, sometimes not. I recommend using DontDestroyOnLoad as you suggested if your systems instance needs to stay alive.


Bug: lt command illegal option

Hello,
I just installed localtunnel through npm on my machine (mac OS).
When I try tun run the command lt --port 3000, I have a strange error message:
ls: illegal option -- -
usage: ls [-SOMEKEY] [file ...]

I dont get it, when I only run lt I have the same result as ls -l or ll commands.
Any idea on how I could fix this issue ?
Thanks a lot

it's because lt is aliased to ls -tr
i fixed this by doing unalias lt
it will work for the current session but the next time that you open the terminal it will be aliased again
so I also added unalias lt to my .zshrc

Hello @defunctzombie, 4 years later I still had this issue whit zshell.
> which lt
> lt: aliased to ll -tr

Adding unalias lt in my .zshrc fixed this issue.
Do you think it still adds value to put a troubleshooting section for this specific issue in the README?


Don't use master for heroku deployment

In light of #13 , we might want to point the "deploy to heroku" button to a recent, tagged github release, as opposed to always building directly from master. We should make this change on both README.md as well as on the lightning-viz.github.io "getting started" page.

I've updated the heroku button to point to the lastest release on npm, and +1 to continue doing this going forward


Display meaningful error message when manual configuration checkbox fails to be changed

The issue is also happen on charging station online: the SAP Keba for example on testcharger tenant on QA. So it's also a bug.

Hello Jerome could you give me a bit more information, what needs to be done?
I had a look at the logs from the station and it seems, that it does not find a template for it:

Right.
But no error popup message is display, even stating to check the logs, the configuration switch is reverted without explanation.

Okay you are right. I think we have a bug in the front end, that we cannot uncheck the manual configuration box.
I will have a look.

And I will fix the templates to add that model.
With the simulator, you can easily have a charging station without matching template to reproduce.

@jerome-benoit The bug is happening because the charging station has never been saved before, so the manual config parameter is not yet in the database.
So once the station was saved it should be possible to switch it to automatic.

It should work without requiring to save it ...

Normally yes, but in my opinion we will never have this specific case at a customer.
To run into this issue the following must happen:
The customer onboards a station without a template and then switch it immediately to automatic (even if still no template is available). Normally he should fill the parameters of the station manually and save it.

We have it on on several customer tenant that do their own charging stations: exadys for example.

Normally yes, but in my opinion we will never have this specific case at a customer.
To run into this issue the following must happen:
The customer onboards a station without a template and then switch it immediately to automatic (even if still no template is available). Normally he should fill the parameters of the station manually and save it.
After this he can switch to automatic anytime.

Sorry for the intrusion :-) but the switch from manual to automatic should not never be displayed when no template is available for the charging station.

The REST API for charging stations does not give that information.

The REST API for charging stations does not give that information.

Make it happen.


Move "Daily Rate" to separate panel

On the Validator Overview page, move "Daily Rate" to a separate panel.
https://www.figma.com/file/nUhBBfzmAynizCEymRR01G/Validators?node-id=234%3A2110

@angle943 @buckyroberts  I just made a PR for this issue. https://github.com/thenewboston-developers/Account-Manager/pull/401


glitch in sync

I see several cases of out-of-sync of same table in several browsers.  No clear issues but we need to investigate.

We found an obvious case about simultaneous moving components and removing kit.  Removing kit causes table load and it is very likely to cause out-of-sync situation.
Removing kit will not be used while playing games under normal circumstances, this issues will have lower priority.

This is easily recreated with tests/performance/move_and_remove_kit.py

Situation is significantly improved.  Done.


SSL: No credentials available in the security package

I try to use SSL for my WebSocket connection and therefore i use this setup:
var server = new WebSocketServer("wss://<IP_ADDRESS>:8181");
var config = BuildConfig();
server.Certificate = X509Certificate2.CreateFromPemFile("fullchain.pem","privkey.pem");

Everything seems fine log says:
Using default TLS 1.0 security protocol.
The problem is, that as soon as a client tries to connect an exception is thrown:
System.ComponentModel.Win32Exception (0x8009030E): No credentials available in the security package.
I use .NET 5.0, is there any known fix for this?

Might have to use .pfx certificate file.
var certificate = new X509Certificate2("path-to-certificate.pfx", "certificate-password");

Might have to use .pfx certificate file.
var certificate = new X509Certificate2("path-to-certificate.pfx", "certificate-password");

Yes that fixed it indeed. I wonder if there is a way to make it work with .pem certificates, because i use Certbot to autoamtically recreate the certificates for my domain and now i have to manually convert it to a pfx cert each time a new certificate is generated.

Yes that fixed it indeed. I wonder if there is a way to make it work with .pem certificates, because i use Certbot to autoamtically recreate the certificates for my domain and now i have to manually convert it to a pfx cert each time a new certificate is generated.


Lokalise: Translations update

This pull request was initiated by Lokalise (user Lutz Biedinger) at 2023-06-21 16:44:41

This has encoding issues on the slash escaping.


Logo creation

1st suggestions

maybe different colors (blue is present in our vue application)
one version without the caption
maybe without the clip above

I uploaded a folder of suggestions (SVG format) on our Discord server
Preview:


CookieValuePrefix patch broke test requests ->withCookie()

Laravel Version: 7.22.4
PHP Version: 7.4.6

Steps To Reproduce:
The following test fails:
Route::middleware([EncryptCookies::class])->group(function () {
    Route::get('/test', function (Request $request) {
        return $request->cookie('test');
     });
});

$response = $this
    ->withCookie('test', 'value')
    ->get('/test');

$response->assertSeeText('value');

Fix
The bug seems to be in MakesHttpRequests.
The line should be:
return encrypt(CookieValuePrefix::create($key, app('encrypter')->getKey()).$value, false);

Works fine for me in the version you're reporting.

@driesvints Okay, it seems to work if the APP_KEY is base64 encoded. This bug happens if the key is just a random string:
config(['app.key' => Str::random(32)]);

@josiasmontag behaviour updated to handle your case here - https://github.com/laravel/framework/commit/84765753ee164e7b0978021c4d42711011aed340


using Nanoid; won't work

Hi there. First off, thank you for this package.
It seems like it's not possible to do a using Nanoid; at the top of a .cs file and then reference the Nanoid class in the code. Later Nanoid references point to the namespace and not the class:
using Nanoid;

string id = Nanoid.Generate(); // ERROR: "The type or namespace name 'GenerateAsync' does not exist in the namespace 'Nanoid'"

Minimal repro: https://github.com/aradalvand/nanoid-namespace-test
Having a type with the same name as its containing namespace is officially considered bad practice. More importantly in this case, it's basically unusable. You always have to use a fully-qualified name, which is ugly (e.g. Nanoid.Nanoid.Generate())
Please consider changing the name of the namespace to something else, for example, NanoidNs, NanoidGenerator or anything that would make it different from the class name.

Please use version 3.0.0, thank you.

Thanks, but I think it would've made more sense to keep tbe namespace as Nanoid (to match the NuGet package name) and instead change the class name to something like NanoidGenerator.


testnet: Assertion `cs.is_satisfied(primary_input, auxiliary_input)' failed.

Snark workers are immediately dying upon receiving work on current testnet:
Code Version Under Test:
0.1.94495-CI
https://circleci.com/gh/CodaProtocol/coda/94495
https://github.com/CodaProtocol/coda/commit/d6ea7f39092f208225ca8391ed30cd769fb4de0a
Error:
coda: /home/opam/project/src/_build/default/lib/snarky/src/camlsnark_c/libsnark-caml/./libsnark/reductions/r1cs_to_qap/r1cs_to_qap.tcc:216: libsnark::qap_witness libsnark::r1cs_to_qap_witness_map(const libsnark::r1cs_constraint_system&, libsnark::r1cs_primary_input&, libsnark::r1cs_auxiliary_input&, const FieldT&, const FieldT&, const FieldT&) [with FieldT = libff::Fp_model<5l, ((const libff::bigint<5l>&)(& libff::mnt46_modulus_A))>; libsnark::r1cs_primary_input = std::vector<libff::Fp_model<5l, ((const libff::bigint<5l>&)(& libff::mnt46_modulus_A))>, std::allocator<libff::Fp_model<5l, ((const libff::bigint<5l>&)(& libff::mnt46_modulus_A))> > >; libsnark::r1cs_auxiliary_input = std::vector<libff::Fp_model<5l, ((const libff::bigint<5l>&)(& libff::mnt46_modulus_A))>, std::allocator<libff::Fp_model<5l, ((const libff::bigint<5l>&)(& libff::mnt46_modulus_A))> > >]: Assertion `cs.is_satisfied(primary_input, auxiliary_input)' failed.
Is it possible to add some additional context to help identify what's going on here?

This looks like it might be related to a bug in how reduce_to_prover interacts with one of the Coda proofs which I've been trying to track down. PR #2229 disables it.

Now I get a trace.
{"timestamp":"2019-04-12 20:30:10.465441Z","level":"Trace","source":{"module":"Snark_worker_lib__Worker","location":"File \"lib/snark_worker_lib/worker.ml\", line 110, characters 60-67"},"message":"No work received from <IP_ADDRESS>:8301 - sleeping 6.4603s","metadata":{"pid":16221}}
{"timestamp":"2019-04-12 20:30:16.930317Z","level":"Info","source":{"module":"Snark_worker_lib__Worker","location":"File \"lib/snark_worker_lib/worker.ml\", line 117, characters 59-66"},"message":"Received work from <IP_ADDRESS>:8301","metadata":{"pid":16221}}
(monitor.ml.Error
 (Failure
   "Constraint unsatisfied:\
  \nfield_inverse\
  \nChecked.inv\
  \nassert_non_zero: File \"lib/snarky/src/snark0.ml\", line 877, characters 4-114\
  \nany: File \"lib/snarky/src/snark0.ml\", line 1023, characters 8-101\
  \nmodify_req: File \"lib/snarky/src/merkle_tree.ml\", line 399, characters 2-817\
  \nmodify_account: File \"lib/coda_base/ledger_hash.ml\", line 90, characters 0-466\
  \napply_tagged_transaction: File \"lib/transaction_snark/transaction_snark.ml\", line 342, characters 2-4174\
  \nmain: File \"lib/transaction_snark/transaction_snark.ml\", line 469, characters 2-1672\
  \n")
 ("Raised at file \"stdlib.ml\", line 33, characters 17-33"
  "Called from file \"lib/snarky/src/snark0.ml\", line 476, characters 14-173"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 493, characters 14-68"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 493, characters 14-68"
  "Called from file \"lib/snarky/src/snark0.ml\", line 731, characters 13-27"
  "Called from file \"lib/snarky/src/snark0.ml\", line 1476, characters 12-122"
  "Called from file \"lib/transaction_snark/transaction_snark.ml\", line 529, characters 6-74"
  "Called from file \"lib/transaction_snark/transaction_snark.ml\", line 1250, characters 6-164"
  "Called from file \"lib/snark_worker_lib/prod.ml\", line 73, characters 14-291"
  "Called from file \"lib/snark_worker_lib/worker.ml\", line 43, characters 10-116"
  "Called from file \"src/container.ml\", line 32, characters 12-22"
  "Called from file \"list.ml\", line 117, characters 24-34"
  "Called from file \"src/container.ml\", line 31, characters 11-142"
  "Called from file \"src/with_return.ml\", line 20, characters 12-24"
  "Re-raised at file \"src/with_return.ml\", line 27, characters 11-20"
  "Called from file \"lib/snark_worker_lib/worker.ml\", line 122, characters 16-45"
  "Called from file \"src/deferred0.ml\", line 61, characters 64-69"
  "Called from file \"src/job_queue.ml\" (inlined), line 120, characters 2-5"
  "Called from file \"src/job_queue.ml\", line 159, characters 6-47"))

Here's another one, but in this one snarky was able to emit it's info too.. (new)
{"timestamp":"2019-04-12 21:15:28.882103Z","level":"Info","source":{"module":"Snark_worker_lib__Worker","location":"File \"lib/snark_worker_lib/worker.ml\", line 117, characters 59-66"},"message":"Received work from <IP_ADDRESS>:8301","metadata":{"pid":27328}}
(monitor.ml.Error
 (Failure
   "Constraint unsatisfied:\
  \nfield_inverse\
  \nChecked.inv\
  \nassert_non_zero: File \"lib/snarky/src/snark0.ml\", line 877, characters 4-114\
  \nany: File \"lib/snarky/src/snark0.ml\", line 1023, characters 8-101\
  \nmodify_req: File \"lib/snarky/src/merkle_tree.ml\", line 399, characters 2-817\
  \nmodify_account: File \"lib/coda_base/ledger_hash.ml\", line 90, characters 0-466\
  \napply_tagged_transaction: File \"lib/transaction_snark/transaction_snark.ml\", line 342, characters 2-4174\
  \nmain: File \"lib/transaction_snark/transaction_snark.ml\", line 469, characters 2-1672\
  \n")
 ("Raised at file \"stdlib.ml\", line 33, characters 17-33"
  "Called from file \"lib/snarky/src/snark0.ml\", line 476, characters 14-173"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 493, characters 14-68"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 472, characters 24-58"
  "Called from file \"lib/snarky/src/snark0.ml\", line 493, characters 14-68"
  "Called from file \"lib/snarky/src/snark0.ml\", line 731, characters 13-27"
  "Called from file \"lib/snarky/src/snark0.ml\", line 1476, characters 12-122"
  "Called from file \"lib/transaction_snark/transaction_snark.ml\", line 529, characters 6-74"
  "Called from file \"lib/transaction_snark/transaction_snark.ml\", line 1250, characters 6-164"
  "Called from file \"lib/snark_worker_lib/prod.ml\", line 73, characters 14-291"
  "Called from file \"lib/snark_worker_lib/worker.ml\", line 43, characters 10-116"
  "Called from file \"src/container.ml\", line 32, characters 12-22"
  "Called from file \"list.ml\", line 117, characters 24-34"
  "Called from file \"src/container.ml\", line 31, characters 11-142"
  "Called from file \"src/with_return.ml\", line 20, characters 12-24"
  "Re-raised at file \"src/with_return.ml\", line 27, characters 11-20"
  "Called from file \"lib/snark_worker_lib/worker.ml\", line 122, characters 16-45"
  "Called from file \"src/deferred0.ml\", line 61, characters 64-69"
  "Called from file \"src/job_queue.ml\" (inlined), line 120, characters 2-5"
  "Called from file \"src/job_queue.ml\", line 159, characters 6-47"))
* Elements of w skipped: 71410 (30.08%)
* Elements of w processed with special addition: 56197 (23.67%)
* Elements of w remaining: 109811 (46.25%)
* Elements of w skipped: 65986 (38.03%)
* Elements of w processed with special addition: 51269 (29.55%)
* Elements of w remaining: 56268 (32.43%)
* Elements of w skipped: 71410 (30.08%)
* Elements of w processed with special addition: 56196 (23.67%)
* Elements of w remaining: 109810 (46.25%)
* G1 elements in proof: 2
* G2 elements in proof: 1
* Proof size in bits: 1195
* G1 elements in proof: 2
* G2 elements in proof: 1
* Proof size in bits: 1493

repro without ANY transactions being made (just blocks)
occurs at block 6
spot 34 in the snarkwork tree is the snark that kills
coda.log.save.gz
Around "2019-04-13 02:04:46.582847Z" in the log file

@jkrauska can we run a test with https://circleci.com/gh/CodaProtocol/coda/97501 (ab3de49 from PR #1261)? That should give us the actual backtrace through the checked computation.

Snarks are working now. And we've heavily polished the error reporting. No more C++ assertions!


chore: Adjust changelog to include changes from #730

This is a follow-up PR to add an entry to the changelog for changes introduced in #730. The previous PR accidentally didn't include this entry.

Easily missed.
Is there a way in Github Actions to warn but not block?
Would be nice to have a check for changelog edits in PRs but not enforce it.

We could use conventional commits for auto-generation of the change log. But that would require some more tooling and research. Happy to explore this further, as I already suggested it a few months back.

We could use conventional commits for auto-generation of the change log. But that would require some more tooling and research. Happy to explore this further, as I already suggested it a few months back.

I'm keen to automate this, but wonder how much friction is could generate? I'm a fan of conventional commits (for this reason, but also search-ability) but others might find it a bit of a showstopper to their flow. Especially due to the whole squash thing where commit messages are quite free-style.
I know you said you'd explore it further, but my question is: would it be a pre-commit/pre-push hook run locally? Or something that runs in Github Actions which then pushes a commit?
Either way, I'm pro-exploration.


Run tests on JRuby on Travis CI

The tests were disabled a long time ago when JRuby 9000 wasn’t released yet, but I haven’t gone back to re-enable them yet.

#832 is an initial attempt, although potential issues with Bundler 1.11 make this tricky.

This is now done.


Add 100+ tactics for 3+ domains

Goal is to have 2000+ tactics added by 2025 to facilitate LemiTree's essential intent "Supporting 1 million ambitious global citizens self-develop by 2025". If I were to go about this alone it implies adding just over 60 per month on average. This process would be greatly facilitated if we can answer the following questions about copyright laws:
I would be willing to also pay a certified copyright lawyer to get an official answer to these questions. To just give a contextual sketch of the situation: we’re looking to centralize the best content for individual self-development and -empowerment from the internet in one open-sourced database. We wish to do this solely from a non-profit incentive, using only crowdfunding via Patreon to support the maintenance and expansion of this project. We have every incentive to show our collective social intelligence by sourcing where the content and claims come from.

Is it allowed to copy paste (pieces of) text from a different website (while also sourcing this website and possibly using quotation marks)? How can we make it legally feasible?
How can we make it legally feasible to embed other people’s videos and graphics in our database?
Does the fact that it is freely available content make it different?
Are we required to have explicit consent from the content writers of the other websites before we can do this?
Does a small disclaimer on top of a page stating we are willing to remove any quoted pieces of copied text prevent us from getting into legal issues?

We are definitely willing to rewrite all the pieces of text, recreate the graphics and videos from the different brilliant sources on the internet. It would just take much more time and would go against our strategy of ‘more impact & less effort’.


feat: dynamic MultiSelect widget standardisation

Description

Passed the contentRef to MultiSelectComponentV2 from MultiSelectWidgetV2.
Wrapped the MultiSelectComponentV2 functional, component in the React.forwardRef.
Set the final ref from React.forwardRef to MultiSelectContainer which is a styled div.

Fixes #13056
Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Manually

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=0d8de70

@Tooluloope Could you review this?

Completed testing this on the provided vercel link. Seems to be working fine.


Documentation Clarifications

https://github.com/mtlynch/tinypilot/wiki/Troubleshooting-and-Diagnostics#browser-console-logs  shouldn't suggest to use F12 to open the console, as that is captured for input

https://github.com/mtlynch/tinypilot/wiki/Troubleshooting-and-Diagnostics#browser-console-logs  does not specify where to update the environment variables, should add they are in  /lib/systemd/system/tinypilot.service

Not clearly mentioned of where tinypilot is installed by default: /opt/tinypilot    - would suggest adding to https://github.com/mtlynch/tinypilot#simple-installation

Thanks for reporting these!

https://github.com/mtlynch/tinypilot/wiki/Troubleshooting-and-Diagnostics#browser-console-logs shouldn't suggest to use F12 to open the console, as that is captured for input

Good point, fixed!

Not clearly mentioned of where tinypilot is installed by default: /opt/tinypilot - would suggest adding to https://github.com/mtlynch/tinypilot#simple-installation

True. Fixed in #608.

https://github.com/mtlynch/tinypilot/wiki/Troubleshooting-and-Diagnostics#browser-console-logs does not specify where to update the environment variables, should add they are in /lib/systemd/system/tinypilot.service

Is that the correct URL? I don't see the mention of environment variables you're talking about in that section.

Whoops second one was a bad double paste. Under the Options on the Readme for environmental vars.


logs of edits for voucher specimen

This is just to report you a small bug re tax. updating on VoSeq - when I updated Order name, it turns out "orden" edited by admin in record history instead of "order" - you got what I mean, ehh? Here is an example to understand better what was I mean:
Genus, orden, tribe, collection date edited by admin  on 2013-05-14 17:07:38

The logging of edits is being done by Django admin by default.


Makes tinygo tests run in verbose mode

The helps ensure folks running tests also on their workstations are actually running the same tests.

@dgryski @deadprogram do you know if it is intentional for tinygo to not run tests in the GOROOT, and only those in the TINYGOROOT, or am I missing something? I can understand maybe short-circuiting being intentional (e.g. if a test exists in TINYGOROOT's package, don't run same from GOROOT).
I'm trying to understand the mechanics as there are some a lot of tests in the net package in the go root and the only ones run here are in the TINYGOROOT. https://github.com/tinygo-org/tinygo/tree/dev/src/net

fwiw our GOOS=js impl doesn't pass everything, but mainly the net package requires reading /etc which is an edge case. One test that rarely passes is TestPipe/RacyWrite, but in any case I expect some tests from net should also be run on tinygo...
$ GOOS=js GOARCH=wasm bin/go test -c -o net.wasm net
$ wazero run -mount=src/net:/ -mount=/etc:/etc net.wasm -test.v -test.short
=== RUN   TestDNSSRVUniformity
    dnsclient_test.go:31: actual: 5035 diff: 35 e: 5000 m: 0.05
    dnsclient_test.go:31: actual: 3380 diff: 46.666666666666515 e: 3333.3333333333335 m: 0.1
    dnsclient_test.go:31: actual: 1007 diff: 7 e: 1000 m: 0.2
    dnsclient_test.go:31: actual: 6014 diff: 14 e: 6000 m: 0.05
--- PASS: TestDNSSRVUniformity (0.06s)
=== RUN   TestWeighting
    dnsclient_test.go:31: actual: 5929 diff: -71 e: 6000 m: 0.05
--- PASS: TestWeighting (0.01s)
=== RUN   TestSpuriousENOTAVAIL
--- PASS: TestSpuriousENOTAVAIL (0.00s)
=== RUN   TestLookupStaticHost
--- PASS: TestLookupStaticHost (0.00s)
=== RUN   TestLookupStaticAddr
--- PASS: TestLookupStaticAddr (0.01s)
=== RUN   TestHostCacheModification
--- PASS: TestHostCacheModification (0.00s)
=== RUN   TestAddrList
    ipsock_test.go:219: both IPv4 and IPv6 are required
--- SKIP: TestAddrList (0.00s)
=== RUN   TestAddrListPartition
--- PASS: TestAddrListPartition (0.00s)
=== RUN   TestParseMAC
--- PASS: TestParseMAC (0.00s)
=== RUN   TestReadLine
--- PASS: TestReadLine (0.18s)
=== RUN   TestDtoi
--- PASS: TestDtoi (0.00s)
=== RUN   TestParsePort
--- PASS: TestParsePort (0.00s)
=== RUN   TestPipe
=== RUN   TestPipe/BasicIO
=== RUN   TestPipe/PingPong
=== RUN   TestPipe/RacyRead
=== RUN   TestPipe/RacyWrite
(0x13128e0,0xc0011ad5c0)
=== RUN   TestPipe/ReadTimeout
--snip--


Error while applying .transform()

Same error that has been documented in here #56.
Tried downgrading the version to 0.7.0 through the repository that was linked in that thread. Still showing the same dimensionality error.
Here its the code:
import numpy as np
import pandas as pd
X_n = pd.DataFrame(data=np.random.rand(10000,2),columns=list('AB'))
X_c =pd.DataFrame(np.random.choice(list('abcde'),size=(10000,4),replace=True),columns =list('CDEF'))
X=pd.concat([X_n,X_c],axis=1)
from prince import FAMD
famd = FAMD(n_components = 6, n_iter = 100)
famd.fit(X)
famd.transform(X.iloc[1:10,:])
I got the same error in version 0.7.0 and 0.7.1
ValueError: shapes (9,20) and (22,6) not aligned: 20 (dim 1) != 22 (dim 0)

I've run into this issue a few times and it looks like it's based on how dummies are generated in _build_X_global. When the dataset you are transforming does not have examples of all the categorical variables from the larger original dataset, the resulting dummified dataset has fewer columns (in this case, 20 rather than 22).
Suggested fix for this (and #56 and #116) is to store the dummified columns in the famd and mfa models. If a new dataset being transformed only has a subset of categorical values, then its dummified dataset should have the right number of columns and one or more columns will be all zeroes. If a new dataset being transformed has new categorical values, should probably throw an error.

Had the same issue, so had to make sure my train, validation, and test have examples of all the categorical variables, before fitting MCA. And dump columns where they don't:
for clmn in X_train_cat.columns:     train_cats = set(X_val_train_cat[clmn].unique())     val_cats = set(X_val_test_cat[clmn].unique())     test_cats = set(X_test_cat[clmn].unique())     keep.append(train_cats == val_cats == test_cats) keep_columns = X_train_cat.columns[keep]
But that's obviously an awkward temp solution, just to make it work. The dummy matrix workaround @christophe-williams mentioned would be nice to have.

Hello there 👋
I apologise for not answering earlier. I was not maintaining Prince anymore. However, I have just refactored the entire codebase. This refactoring should have fixed many bugs.
I don’t have time and energy to check if this fixes your issue, but there is a good chance it does. Feel free to reopen this issue if the problem persists after installing the new version — that is, version 0.8.0 and onwards.


populate the v1alpha1.Hostname from the URI as per the spec

As discovered as part of the move to v1beta1 reconcile work, the v1alpha1 down conversion is dropping the Hostname from the Status.
https://prow.knative.dev/view/gcs/knative-prow/pr-logs/pull/knative_eventing/3070/pull-knative-eventing-integration-tests/1255611066595938304
    --- FAIL: TestChannelStatus/TestChannelStatus-{Channel_messaging.knative.dev/v1alpha1} (2.34s)
        test_runner.go:95: namespace is : "v1alpha1-lhmqk"
        channel_status_test_helper.go:50: Running channel status conformance test with channel {"Channel" "messaging.knative.dev/v1alpha1"}
        channel_status_test_helper.go:54: Creating channel {Kind:Channel APIVersion:messaging.knative.dev/v1alpha1}-channel-req-labels
        creation.go:52: Creating channel &TypeMeta{Kind:Channel,APIVersion:messaging.knative.dev/v1alpha1,}-channel-req-labels
        channel_status_test_helper.go:78: No hostname found for {"Channel" "messaging.knative.dev/v1alpha1"}

Spec bit:
Channel Status
v1alpha1
When the channel instance is ready to receive events status.address.hostname and status.address.url MUST be populated and status.addressable MUST be set to True.

https://github.com/knative/eventing/blob/master/docs/spec/channel.md#spec-requirements

This failed?
--- FAIL: TestMemStatsMetrics (1.31s)
/test pull-knative-pkg-unit-tests

/hold
Holding if you want to address @vagababov's comment.

/unhold
talked with Victor and he's fine with this going in this way.
Thanks!!

@dprotaso @mattmoor it looks like Serving conformance tests don't catch this?


Unreachable 'for' clause doesn't correctly narrow

nightly build Oct 24, 2016
// works as expected
function shouldNodeBeConsidered(ruleKey: string, node: ts.Node): boolean {
    for (let current: ts.Node | undefined = node; current !== undefined; current = current.parent) { // works
    }
}

// weirdly doesn't work
function shouldNodeBeConsidered(ruleKey: string, node: ts.Node): boolean {
    for (
        let current: ts.Node | undefined = node;
        current !== undefined;
        current = current.parent // <-- [ts] Object is possibly 'undefined'.
                                 //          let current: ts.Node | undefined

     ) {
        if (shouldImmediateNodeBeConsidered(ruleKey, current)) {
            return true;
        } else {
            continue;
        }
    }
    return false;
}

@aleksey-bykov Not super familiar with the flow based type checking, but did you mean to have an if statement there?
        let current: ts.Node | undefined = node;
        if(current !== undefined) {
            current = current.parent;
        }

@masaeedu the comparison is the middle clause of a for loop, not an expression statement

@aleksey-bykov I can't reproduce this. As usual, we'd really prefer a self-contained repro and a descriptive title on bugs

@RyanCavanaugh My bad. I can reproduce this, you need to enable strictNullChecks. I'll set up a repo.

@RyanCavanaugh https://github.com/masaeedu/typescript11821

@RyanCavanaugh updated the title,
what is a self contained repro?

it's like you are giving me a favor of considering bugs that i found in your product, sorry for doing it and making you upset

function foo() {
    for (let current: string | undefined = 'oops'; true; current = current.substr(1)) {
    // Incorrect error in third clause
        return;
    }
}

@aleksey-bykov apology accepted

@RyanCavanaugh Isn't the error only incorrect in the case where the middle clause narrows the loop variable?

The narrowing is irrelevant because we never assign anything possibly-undefined to current

@RyanCavanaugh Well in the original case current.parent, and in your case current.substr (I assume), can return undefined. Note that this isn't just a problem with returns, you get the problem if you continue as well, in which case you are assigning a possibly undefined value to the loop var. I added some more scenarios to the repo.

@RyanCavanaugh, you made a repro that doens't really cover the reported case: notice that the first snippet works, the problem reveals itself in the presence of a body

substr can't return undefined.
More accurately, this seems to happen when the end flow point of the for loop body is unreachable.

oh, you used the return for a body, nevermind

@RyanCavanaugh Here's what I'm trying to say
interface Node {
    parent: Node | undefined;
}
// This currently fails to compile. After the fix it SHOULD compile
function foo1(node: Node): boolean {
    for (let current: Node | undefined = node; current !== undefined; current = current.parent) {
        continue;
    }
    return false;
}
// This currently fails to compile. After the fix it SHOULD NOT compile
function foo2(node: Node): boolean {
    for (let current: Node | undefined = node; 1 === 1; current = current.parent) {
        continue;
    }
    return false;
}

The second case should fail because the conditional clause does not do the appropriate narrowing, regardless of whether the bottom of the loop body is reachable.

The second case shouldn't fail. There are no assignments to current which don't have an RHS of a non-undefined type. See how this does compile without error:
interface Node {
    parent: Node;
}
function foo2(node: Node): boolean {
    for (let current: Node | undefined = node; 1 === 1; current = current.parent) {
        // continue;
    }
    return false;
}

Note that there should never be a difference between a for loop like this
for(a; ; c) {
  // code
  // block-terminal continue
  continue;
}

and a for loop like this (all other expressions being equal)
for(a; ; c) {
  // code
}

There are no assignments to current which don't have an RHS of a non-undefined type.

current.parent is Node | undefined in the example I gave you (not Node), so there is one assignment to current which does not have an RHS of a non-undefined type.
We're travelling up the parent chain of a Node: eventually you'll run out of parents and get undefined. If you don't have a condition that ensures you stop iterating at that point, there is no guarantee that current is not undefined when the final clause is evaluated.

@RyanCavanaugh You can verify that the latter fails at runtime where the former does not:
interface N {
    parent?: N;
}

let n1: N = { }, n2: N = {};
n1.parent = n2;

// This currently fails to compile. After the fix it SHOULD compile
function foo1(node: N): boolean {
    for (let current: N | undefined = node; current !== undefined; current = current.parent) {
        continue;
    }
    return false;
}

// This currently fails to compile. After the fix it SHOULD NOT compile
function foo2(node: N): boolean {
    for (let current: N | undefined = node; 1 === 1; current = current.parent) {
        continue;
    }
    return false;
}

console.log('foo1: ', foo1(n1)); // false
console.log('foo2: ', foo2(n1)); // Uncaught TypeError: Cannot read property 'parent' of undefined

Forward-duping this to #26914


Update pipeline-service in production

Added overlay for the performance settings so that default settings
are used in production until we are ready to activate tuned settings
Rename pipelines-scc to appstudio-pipelines-scc
Update pipeline-service-exporter-reader ClusterRole to watch
pipelineruns and task runs, and get customresourcedefinitions
Align Pipelines Controller logs with ADR-6
Use downstream image for tekton chains
Add more ArgoCD sync-wave and phases
Set 'enable-api-fields: beta' in OSP

/test appstudio-e2e-tests

/test appstudio-e2e-tests

@Roming22 please provide the following in the commit message body (and pull request description):
1. Which features are being promoted to production by this change?

2. Why are we providing an overlay for performance settings?

@Roming22 @adambkaplan - I can help with 2) .... it will allow for direct manipulation of the performance settings for the new openshift CI based approach that @pmacik and @jhutar have discussed with me for vetting / confirming in an isolated fashion (i.e. not corrupted by other changes that may have merged into infra-deployments and the staging env) the effects of increasing the pipeline controller's work queue thread count, k8s client tuning.
The CI runs create create an infra-deployment branch with just that change, and they can measure the effects with more certainty.
Their prior attempt using staging was not conclusive because almost another 12 changes were introduced more or less concurrently after the summit freeze was lifted.

@adambkaplan Commit message and description updated

I'll give CI one more pass - otherwise I approve a "red button" merge since these changes have been deployed and vetted in staging, and the changeset includes a security/threat-model related change.

/lgtm


Incorrect GCD uptime calculations

I was sent an example of a SGE log with 100.86% gcd uptime: https://xivanalysis.com/fflogs/pGvYbkW7yPQLrw1j/4/4
The estimated GCD on that log is 2.51s which is clearly an overestimate, but that would only explain up to 100*2510/2500 = 100.4% uptime, and I am considering this effect expected behavior. Beside that, I believe I found a few actual bugs.

Pneuma generates two cast events, one targeted at spell confirm and one untargeted at the actual end of the cast bar. It appears that both are crediting the full gcd (or slightly different partial GCDs for a pre-cast Pneuma) to uptime, so Pneuma casts are effectively double counted. I do not know if any other spells have this behavior.

When considering a pre-cast spell, the gcd uptime credit does not account for the cast event happening at spell confirm rather than the end of the cast bar, and also does not account for animation lock on that spell. As an example, see this BLM log: https://xivanalysis.com/fflogs/Qj87q142PZNAVrcW/1/3 The pre-pull fire 3 cast has the event at -0.403s, and the uptime module does not credit any uptime for it. However, the cast bar actually finishes half a second later at 0.097s, and this lines up as expected with the subsequent cast being a high thunder at 0.222s. I think in lines with the rest of the calculations, this cast should have been credited 197ms of uptime from the cast bar finishing 97ms after pull plus 100ms for the animation lock.

When considering a end-of-pull cast, the gcd uptime credit appears to be incorrectly crediting the out-of-fight part of the gcd instead of the in-fight part. In the same BLM log, the final cast is a Paradox (instant cast) which happens at 436116 ms after the pull, with the GCD ending at 438536 ms. The pull ended at 436661 ms after the pull, so we expect to get<PHONE_NUMBER>16 = 545 ms of uptime credit. However, the calculation actually gives<PHONE_NUMBER>61 = 1875 ms of credit. This effect is especially noticeable on PCT logs where the fight ends with a swiftcast rainbow drip, such as this one: https://xivanalysis.com/fflogs/a:NKCQgknYTM8zJx2H/5/321

PRs opened for points 1 and 3. I haven't wrapped my head around how the code calculating prepull GCD uptime is actually functioning, plus we've had some other "before the start of the pull" shenanigans with fflogs recently, so I might need someone else to take a look at that one...


Add logging/logback documentation

About this change - What it does
This PR adds information about logging in Guardian. It also separates out the application documentation into different sections.
Why this way
Self explanatory

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 75.683%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
277

Relevant Lines:
366

💛  - Coveralls