OverflowError reads_to_bins

Hi Endre,
I sequentially used this call
epic2 -bin 200 -g 3 -fdr 0.05 -t $TREATMENT -c $CONTROL -cs ../4_epic2/Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizesand epic2 stops with an error message for some of my treatments.
Do you have an idea of how to continue?
This is the typical error (console output):

Found a median readlength of 50.0
Using chromosome sizes found in
../4_epic2/Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes.
Using an effective genome length of ~2172 * 1e6
Parsing ChIP file(s):
H3K4me1_MACT_P60_E.sorted.bam
Traceback (most recent call last):
File "/home/hadlich/miniconda/bin/epic2", line 248, in
_main(args)
File
"/home/hadlich/miniconda/lib/python3.7/site-packages/epic2/main.py",
line 40, in _main
args["treatment"], args, "ChIP")
File "epic2/src/reads_to_bins.pyx", line 183, in
epic2.src.reads_to_bins.files_to_bin_counts
File "epic2/src/reads_to_bins.pyx", line 309, in
epic2.src.reads_to_bins.files_to_bin_counts
File "epic2/src/reads_to_bins.pyx", line 61, in
epic2.src.reads_to_bins.remove_out_of_bounds_bins
OverflowError: Python int too large to convert to C long

I checked the chromosome sizes and they are all fine! Maybe the BWA mappings ahead cause the error ... but why this error occurs only for a few treatments?
Hope you can help ... or give an idea to continue.
Greetings from Germany,
Frieder

Thanks for creating an issue. Do you have the possibility to share the files with me?

The error is here:
https://github.com/biocore-ntnu/epic2/blob/master/epic2/src/reads_to_bins.pyx#L61
The error message likely means that either your chromsize or one of the positions in the file are above the uint32 max... Or that they have been a negative number and misinterpreted as a large int.
If you cannot share the files I suggest you go through the starts and ends in you alignment files and see what the max and min value of any read positions are. Sometimes not aligned reads are given a start and end of -1 for example.

I used Bos_taurus.ARS-UCD1.2.dna.toplevel.fa as genomic reference for the BWA mapping. Real data cannot be uploaded ... I will check the mapping results for erroneous positions and let you know. Thanks a lot!

Try converting to bed files with bedtools and see if it works then? Good
luck and feel free to ping me for anything. Without the files it is
impossible for me to debug as I am sure you understand :)
On Mon, Jul 13, 2020 at 2:51 PM friederhadlich<EMAIL_ADDRESS>wrote:

I used Bos_taurus.ARS-UCD1.2.dna.toplevel.fa as genomic reference for the
BWA mapping. Real data cannot be uploaded ... I will check the mapping
results for erroneous positions and let you know. Thanks a lot!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/epic2/issues/36#issuecomment-657541892,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURUXFBZS5Z56PIDPDQXLR3L7OZANCNFSM4OYOFEEQ
.

Unfortunately, bed files aren't the solution for me!
The problem must be in parameter "fs" ... please see these calls:
epic2 -t ../3_bwa/H3K27Ac_MACT_p59_E.sorted.bam -bin 200 -g 3 -fdr 0.05 -fs 300 -egf 0.8 -c ../3_bwa/INPUT_pool_MACT_E.sorted.bam.bed -cs Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes > H3K27Ac_MACT_p59_E.txt

Found a median readlength of 50.0
Using chromosome sizes found in Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes.
Using an effective genome length of ~2172 * 1e6
Parsing ChIP file(s):
../3_bwa/H3K27Ac_MACT_p59_E.sorted.bam
Traceback (most recent call last):
File "/home/hadlich/miniconda/bin/epic2", line 248, in
_main(args)
File "/home/hadlich/miniconda/lib/python3.7/site-packages/epic2/main.py", line 40, in _main
args["treatment"], args, "ChIP")
File "epic2/src/reads_to_bins.pyx", line 183, in epic2.src.reads_to_bins.files_to_bin_counts
File "epic2/src/reads_to_bins.pyx", line 309, in epic2.src.reads_to_bins.files_to_bin_counts
File "epic2/src/reads_to_bins.pyx", line 61, in epic2.src.reads_to_bins.remove_out_of_bounds_bins
OverflowError: Python int too large to convert to C long

epic2 -t ../3_bwa/H3K27Ac_MACT_p59_E.sorted.bam -bin 200 -g 3 -fdr 0.05 -egf 0.8 -c ../3_bwa/INPUT_pool_MACT_E.sorted.bam.bed -cs Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes > H3K27Ac_MACT_p59_E.txt

Found a median readlength of 50.0
Using chromosome sizes found in Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes.
Using an effective genome length of ~2172 * 1e6
Parsing ChIP file(s):
../3_bwa/H3K27Ac_MACT_p59_E.sorted.bam
Valid ChIP reads: 11245276 (11245276 before out of bounds removal)
Score threshold: 19.652
Number of tags in a window: 3
Number of islands found: 39544
Parsing Input file(s):
../3_bwa/INPUT_pool_MACT_E.sorted.bam.bed
Valid Background reads: 23502103 (23502128 before out of bounds removal)

Chromosome positions of the mapping file are fine.
cat ../3_bwa/INPUT_pool_MACT_E.sorted.bam.bed | cut -f 3 | awk 'BEGIN{a=10000;b=0};{if ($0<a) a=$0; if ($0>b) b=$0}; END{print a,b;}'

37 158533984

cat ../3_bwa/INPUT_pool_MACT_E.sorted.bam.bed | cut -f 2 | awk 'BEGIN{a=10000;b=0};{if ($0<a) a=$0; if ($0>b) b=$0}; END{print a,b;}'

0 158533934

One more running example:
epic2 -t ../3_bwa/H3K27Ac_MACT_p59_E.sorted.bam -bin 500 -g 3 -fdr 0.05 -fs 300 -egf 0.8 -c ../3_bwa/INPUT_pool_MACT_E.sorted.bam.bed -cs Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes > H3K27Ac_MACT_p59_E.txt

I will look at it in a few weeks. Taking a vacation now :)
Thanks for providing more details :)
On Wednesday, July 15, 2020, friederhadlich<EMAIL_ADDRESS>wrote:

One more running example:
epic2 -t ../3_bwa/H3K27Ac_MACT_p59_E.sorted.bam -bin 500 -g 3 -fdr 0.05
-fs 300 -egf 0.8 -c ../3_bwa/INPUT_pool_MACT_E.sorted.bam.bed -cs
Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes > H3K27Ac_MACT_p59_E.txt
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/epic2/issues/36#issuecomment-658719246,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURURJZXZ7WV7FYTLFOQTR3WIYPANCNFSM4OYOFEEQ
.

I am also seeing this error -- but I figured out a temporary workaround. I am using hg38 chrom.sizes from UCSC link and a paired-end fastq file which I aligned with bwa and converted to a bed file.
Here is the epic2 command:
epic2 -t RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed -e 100 -fdr .01 --effective-genome-fraction 0.6753009149366569 --mapq 30 --chromsizes RSeq_out/tmp/hg38.chrom.sizes --output RSeq_out/peaks_epic_unstranded/SRX2481503_S96_DRIP_Seq_1_hg38.bed

And the output:
Found a median fragment size of 183.5

Using chromosome sizes found in RSeq_out/tmp/hg38.chrom.sizes.

Using an effective genome length of ~2167 * 1e6

Parsing ChIP file(s):
  RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Traceback (most recent call last):
  File "/home/UTHSCSA/millerh1/miniconda3/envs/RSeqEnv/bin/epic2", line 248, in <module>
    _main(args)
  File "/home/UTHSCSA/millerh1/miniconda3/envs/RSeqEnv/lib/python3.7/site-packages/epic2/main.py", line 40, in _main
    args["treatment"], args, "ChIP")
  File "epic2/src/reads_to_bins.pyx", line 183, in epic2.src.reads_to_bins.files_to_bin_counts
  File "epic2/src/reads_to_bins.pyx", line 309, in epic2.src.reads_to_bins.files_to_bin_counts
  File "epic2/src/reads_to_bins.pyx", line 61, in epic2.src.reads_to_bins.remove_out_of_bounds_bins
OverflowError: Python int too large to convert to C long

The error goes away when I remove all the non-standard chromosomes from the chrom.sizes file, though I don't understand why. Regardless most people will have these non-standard chromosomes so it would be great if using them was supported.
Here's chrom.sizes modified by me:
chr1	248956422
chr2	242193529
chr3	198295559
chr4	190214555
chr5	181538259
chr6	170805979
chr7	159345973
chrX	156040895
chr8	145138636
chr9	138394717
chr11	135086622
chr10	133797422
chr12	133275309
chr13	114364328
chr14	107043718
chr15	101991189
chr16	90338345
chr17	83257441
chr18	80373285
chr20	64444167
chr19	58617616
chrY	57227415
chr22	50818468
chr21	46709983

The output of the same epic2 command as before, but with the updated chrom.sizes:
effective-genome-fraction 0.6753009149366569 --mapq 30 --chromsizes RSeq_out/tmp/hg38.chrom.sizes --output RSeq_out/peaks_epic_unstranded/SRX2481503_S96_DRIP_Seq_1_hg38.bed
Found a median fragment size of 183.5

Using chromosome sizes found in RSeq_out/tmp/hg38.chrom.sizes.

Using an effective genome length of ~2085 * 1e6

Parsing ChIP file(s):
  RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Chromosome chr10_GL383545v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
Chromosome chr10_GL383545v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
...
Chromosome chrUn_KI270757v1 not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
Chromosome chrX_KI270880v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
Chromosome chrX_KI270913v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY

Valid ChIP reads: 24012551 (24017420 before out of bounds removal)

Score threshold: 27.632

Number of tags in a window: 5

Number of islands found: 6736

Empirical estimate of FDR is: 0.014845605700712588 (100/6736)

Hope this is helpful!
Best,
Henry

I will definitely try to fix this soon. Will test with non-standard
chromosomes and see what happens.
On Tue, Jul 28, 2020 at 11:16 PM Henry Miller<EMAIL_ADDRESS>wrote:

I am also seeing this error -- but I figured out a temporary workaround. I
am using hg38 chrom.sizes from UCSC link
https://hgdownload-test.gi.ucsc.edu/goldenPath/hg38/bigZips/hg38.chrom.sizes
and a paired-end fastq file which I aligned with bwa and converted to a bed
file.
Here is the epic2 command:
epic2 -t RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed -e 100 -fdr .01 --effective-genome-fraction 0.6753009149366569 --mapq 30 --chromsizes RSeq_out/tmp/hg38.chrom.sizes --output RSeq_out/peaks_epic_unstranded/SRX2481503_S96_DRIP_Seq_1_hg38.bed
And the output:
Found a median fragment size of 183.5
Using chromosome sizes found in RSeq_out/tmp/hg38.chrom.sizes.
Using an effective genome length of ~2167 * 1e6
Parsing ChIP file(s):
RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Traceback (most recent call last):
File "/home/UTHSCSA/millerh1/miniconda3/envs/RSeqEnv/bin/epic2", line 248, in
_main(args)
File "/home/UTHSCSA/millerh1/miniconda3/envs/RSeqEnv/lib/python3.7/site-packages/epic2/main.py", line 40, in _main
args["treatment"], args, "ChIP")
File "epic2/src/reads_to_bins.pyx", line 183, in epic2.src.reads_to_bins.files_to_bin_counts
File "epic2/src/reads_to_bins.pyx", line 309, in epic2.src.reads_to_bins.files_to_bin_counts
File "epic2/src/reads_to_bins.pyx", line 61, in epic2.src.reads_to_bins.remove_out_of_bounds_bins
OverflowError: Python int too large to convert to C long
The error goes away when I remove all the non-standard chromosomes from
the chrom.sizes file, though I don't understand why. Regardless most
people will have these non-standard chromosomes so it would be great if
using them was supported.
Here's chrom.sizes modified by me:
chr1	248956422
chr2	242193529
chr3	198295559
chr4	190214555
chr5	181538259
chr6	170805979
chr7	159345973
chrX	156040895
chr8	145138636
chr9	138394717
chr11	135086622
chr10	133797422
chr12	133275309
chr13	114364328
chr14	107043718
chr15	101991189
chr16	90338345
chr17	83257441
chr18	80373285
chr20	64444167
chr19	58617616
chrY	57227415
chr22	50818468
chr21	46709983
The output of the same epic2 command as before, but with the updated
chrom.sizes:
effective-genome-fraction 0.6753009149366569 --mapq 30 --chromsizes RSeq_out/tmp/hg38.chrom.sizes --output RSeq_out/peaks_epic_unstranded/SRX2481503_S96_DRIP_Seq_1_hg38.bed
Found a median fragment size of 183.5
Using chromosome sizes found in RSeq_out/tmp/hg38.chrom.sizes.
Using an effective genome length of ~2085 * 1e6
Parsing ChIP file(s):
RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Chromosome chr10_GL383545v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
Chromosome chr10_GL383545v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
...
Chromosome chrUn_KI270757v1 not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
Chromosome chrX_KI270880v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
Chromosome chrX_KI270913v1_alt not in the chromosome sizes: chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY
Valid ChIP reads: 24012551 (24017420 before out of bounds removal)
Score threshold: 27.632
Number of tags in a window: 5
Number of islands found: 6736
Empirical estimate of FDR is: 0.014845605700712588 (100/6736)
Hope this is helpful!
Best,
Henry
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/epic2/issues/36#issuecomment-665287584,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURUULS44QJPLXKM365ZLR5452HANCNFSM4OYOFEEQ
.

I have now started my sleuthing to fix this problem.
There is one thing which I did not think of communicating: if you include chromosomes in the chromsizes file that are not in the reads, the algorithm becomes more conservative. It computes a Poisson parameter based on the number of reads in the genome. With more chromosomes, the genome becomes longer, and hence less sparsely populated.
I have tried:

adding nonexisting chromosomes to the chromsizes file
added nonexisting chromosomes to the treatment file

but have been unable to break epic2. If anyone can upload a breaking example (if the data is semi-sensitive you can 7zip with a pword) I will do my best to fix this problem. But I have not been able to reproduce. Version: 0.0.41.

epic2 -bin 200 -g 3 -fdr 0.05 -t /mnt/cargo/endrebak/control.sorted.filtered.bam -cs Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes > /dev/null
Found a median readlength of 50.0

Using chromosome sizes found in Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes.

Using an effective genome length of ~2344 * 1e6

Parsing ChIP file(s):
  /mnt/cargo/endrebak/control.sorted.filtered.bam

Valid ChIP reads: 1949548 (1949548 before out of bounds removal)

Score threshold: 22.615000000000002

Number of tags in a window: 1

Number of islands found: 16028

Empirical estimate of FDR is: 0.062390816071874224 (1000/16028)

I.e. was not able to reproduce. Tomorrow I will wrap the affected code in a try/catch and print the error with detailed information. Then you can report that info back to me and I will try my best to understand and fix it :)

Hi Endre,
please use treatment and control argument to reproduce the error ... i.e.
epic2 -t treatment.sorted.bam -bin 200 -g 3 -fdr 0.05 -fs 300 -egf 0.8 -c control.sorted.filtered.bam -cs Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes
Good luck, Frieder

epic2 -fs 300 -bin 200 -g 3 -fdr 0.05 -t /mnt/cargo/endrebak/treatment.sorted.filtered.bam -c /mnt/cargo/endrebak/control.sorted.filtered.bam -cs Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes > /dev/null
Found a median readlength of 50.0

Using chromosome sizes found in Bos_taurus.ARS-UCD1.2.dna.toplevel.chromsizes.

Using an effective genome length of ~2344 * 1e6

Parsing ChIP file(s):
  /mnt/cargo/endrebak/treatment.sorted.filtered.bam

Valid ChIP reads: 1185813 (1185813 before out of bounds removal)

Score threshold: 17.337

Number of tags in a window: 1

Number of islands found: 9343

Parsing Input file(s):
  /mnt/cargo/endrebak/control.sorted.filtered.bam

Valid Background reads: 1949548 (1949548 before out of bounds removal)

I still am unable to reproduce :) So I will stick to my plan of implementing good reporting in the case of errors.
I am using version 0.0.41 and linux. What about you people?

Now a 0.0.42 is out which should give a better error message when it happens :).

Strange! All my samples pass epic2 with version 0.0.42! Thx a lot for your support!

Indexplicable weirdness is common 🙈 Thanks for reporting.
It might have been that I forgot to raise an exception if I caught one. Were any error messages?

Look for an error message like this:
    except OverflowError as e:
        print(e)
        print("\nAdditional info:\n")
        print("Chromosome:", chromosome)
        print("Chromsize:", chromsize)

0.0.43 (hotfix) should raise an error if OverflowErrors happen again. These were noisily ignored in 0.0.42

I am still getting this error with epic2 0.0.42... though it finishes and gives some additional info:
(base) millerh1@cbbi16:~/Bishop.lab/Projects/RSeq$ epic2 -t RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed -e 100 -fdr .01 --effective-genome-fraction 0.6753009149366569 --mapq 30 --chromsizes RSeq_out/tmp/hg38.chrom.sizes --output RSeq_out/peaks_epic_unstranded/SRX2481503_S96_DRIP_Seq_1_hg38.bed
Found a median fragment size of 183.5

Using chromosome sizes found in RSeq_out/tmp/hg38.chrom.sizes.

Using an effective genome length of ~2167 * 1e6

Parsing ChIP file(s):
  RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Python int too large to convert to C long

Additional info:

Chromosome: chr2_GL582966v2_alt
Chromsize: 96131

Valid ChIP reads: 24120376 (24125321 before out of bounds removal)

Score threshold: 45.687

Number of tags in a window: 4

Number of islands found: 5165

Empirical estimate of FDR is: 0.01936108422071636 (100/5165)

The chrom.sizes are here: http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.chrom.sizes
I'm hosting the bed file here: https://hem-misc.s3-us-west-2.amazonaws.com/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Thank you!
Henry

I will look at it. Thanks!
On Fri, Aug 14, 2020 at 10:47 PM Henry Miller<EMAIL_ADDRESS>wrote:

I am still getting this error with epic2 0.0.42... though it finishes and
gives some additional info:
(base) millerh1@cbbi16:~/Bishop.lab/Projects/RSeq$ epic2 -t RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed -e 100 -fdr .01 --effective-genome-fraction 0.6753009149366569 --mapq 30 --chromsizes RSeq_out/tmp/hg38.chrom.sizes --output RSeq_out/peaks_epic_unstranded/SRX2481503_S96_DRIP_Seq_1_hg38.bed
Found a median fragment size of 183.5
Using chromosome sizes found in RSeq_out/tmp/hg38.chrom.sizes.
Using an effective genome length of ~2167 * 1e6
Parsing ChIP file(s):
RSeq_out/tmp/pe_beds/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Python int too large to convert to C long
Additional info:
Chromosome: chr2_GL582966v2_alt
Chromsize: 96131
Valid ChIP reads: 24120376 (24125321 before out of bounds removal)
Score threshold: 45.687
Number of tags in a window: 4
Number of islands found: 5165
Empirical estimate of FDR is: 0.01936108422071636 (100/5165)
The chrom.sizes are here:
http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.chrom.sizes
I'm hosting the bed file here:
https://hem-misc.s3-us-west-2.amazonaws.com/SRX2481503_S96_DRIP_Seq_1.hg38.experiment.bed
Thank you!
Henry
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/epic2/issues/36#issuecomment-674265922,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURUWDMIUJYM2YXMY5GBLSAWPGDANCNFSM4OYOFEEQ
.

What about now? 0.0.44 is out :)

Well the number of reads and peaks does not change, but the error disappeared! I'm assuming that this is alright now. Thanks!

No problem. There must have been some weirdness with the Cython and C++ interop is my guess :)


[Feature] Make port definitions on internal LB subnet more restrictive

For the following subnet:

internal LB

security list allows traffic to restricted ports
Direction: ingress
Protocol: TCP
Ports to close: 11, 17 - 19, 21, 23 - 25, 43, 49, 53, 70 - 74, 79 - 81, 88, 111, 123, 389, 445, 500, 636, 3306, 3389, 5901, 5985 - 5986, 7001, 8000, 8080, 8443, 8888

This has to be addressed by introducing more tight security list rules definitions to prevent access to a set of restricted ports.

Closed in #399


[BUG] Argument printing in output no longer shows sufficient digits

Summary
The printing of input arguments in the output now seems to be truncated to a fixed number of digits (possibly rounded as well):
Under cmdstan 2.33.1:
andrew@Andrews-MacBook-Air bernoulli % ./bernoulli optimize algorithm=lbfgs tol_obj=1e-11 data file=bernoulli.data.json
method = optimize
  optimize
    algorithm = lbfgs (Default)
      lbfgs
        init_alpha = 0.001 (Default)
        tol_obj = 9.9999999999999994e-12

Under cmdstan 2.34:
andrew@Andrews-MacBook-Air bernoulli % ./bernoulli optimize algorithm=lbfgs tol_obj=1e-11 data file=bernoulli.data.json
method = optimize
  optimize
    algorithm = lbfgs (Default)
      lbfgs
        init_alpha = 0.001000 (Default)
        tol_obj = 0.000000

Current Version:
v2.34.0

Seems related to https://github.com/stan-dev/cmdstan/issues/1214. My guess is the fix will be similar, but in https://github.com/stan-dev/cmdstan/blob/develop/src/cmdstan/arguments/singleton_argument.hpp
Do you have time today to put in a PR?


Problems in simulation data

Hi, I don't know why I don't do well in simulation data using the base model of scVI and scANVI.  Batch8 and Batch9 are target data, which cannot be well integrated with source data  after fine-tuning.  I have adjusted the learning rate, but it didn't work. Are there any other parameters that need to be adjusted?  I have no idea, can you give me some advice? Thank you in advance！

hi. please start an issue in the reproducibility not here.
thx


[Relax] Implement operators to inspec DLTensor::strides and offset

A follow-up PR to https://github.com/apache/tvm/pull/16563.  This PR implements similar operators to inspect the runtime values of DLTensor::strides and DLTensor::byte_offset.  In addition, while the element offset is not explicitly present in the DLTensor struct, a Relax operator is implemented to infer it from the byte_offset and data_type fields, for use when interacting with the TIR BufferNode::elem_offset field.

This PR branch has been updated to be based on top of the two spun-out PRs.

We may want to revisit the default inferred PrimStructInfo for some of these calls in the future, namely if we handle offsets/strides more systematically later, though the approach here is correct for the present.

Sounds like a plan.  I think the biggest use of strides would be in exposing a view of a tensor to a compute kernel, without requiring the entire tensor to be exposed.  (e.g. Improved R.split legalization)  That said, there's enough kernels that assume contiguous tensors, as are currently provided by Relax, that for now I'd want to keep that requirement.


Add HTML support for MD045: no-alt-text

Issue: https://github.com/DavidAnson/markdownlint/issues/992
This PR adds support to flag missing alt text on HTML images. I wasn't sure if I should hold off on opening the PR until there was more discussion on the issue so this might be premature... I am happy to continue discussing this on this PR or on the issue.
Approach taken
I am new to this repo and followed the approaches I saw in other rules.
I noticed there is parser that returns HTML tokens. I didn't see a way to access the attributes off the HTML. It may not be ideal but I ended up using regex to check if the HTML image token contains alt=.

This looks pretty thorough and I like the test cases I see. Thank you! I will need to review this from a real computer, potentially this evening. I will have a set of nitpicks as always, so be warned. :)
A few things I noticed from a very quick scan on my phone:

Element and attribute comparisons should be case insensitive, this suggests there are no test cases for uppercase
I do not understand how range can be calculated after removing line breaks, this may mean there is something of a gap in the tests
If you convert this to a real PR, we can see the result of CI, you can get close locally with "npm run ci"
There are three copies of the documentation and you modified the correct one, but did not run the npm script to update the other two
I include a space between rule names in test files {MD001} {MD002}
I might choose to add the alternate text tag to the couple of unrelated test files instead of suppressing this rule as you did

Hi @DavidAnson, I appreciate your feedback! I've addressed the following from your first set of feedback:

Element and attribute comparisons should be case insensitive, this suggests there are no test cases for uppercase

There are three copies of the documentation and you modified the correct one, but did not run the npm script to update the other two

I include a space between rule names in test files {MD001} {MD002}

I might choose to add the alternate text tag to the couple of unrelated test files instead of suppressing this rule as you did

With regards to:

I do not understand how range can be calculated after removing line breaks, this may mean there is something of a gap in the tests

I followed the approach taken in MD033.js which was one of the only other rules I could find that sets an error on an HTML tag. I believe this sets the error range on the first line of the tag and accounts for multiline tags. My understanding could be wrong and I am definitely open to your suggestions.

Thanks for the quick activity! I will remind myself how the other range calculation works when I review this.

FYI, GitHub is complaining this PR cannot be merged due to conflicts and I think that may be blocking me from approving a CI run?

FYI now that tests are running, if anything new pops up in TestRepos, I typically verify all new issues are valid before updating those snapshots. There are npm scripts to clone and lint and update.

Hi @DavidAnson! Apologies for the delayed follow-up.
Starting from commit https://github.com/DavidAnson/markdownlint/pull/993/commits/0165cf569e19895ff0eab385dd68761ca7e0a294, I've hopefully addressed all of your feedback, but let me know if I'm missing anything.
Re: https://github.com/DavidAnson/markdownlint/pull/993#discussion_r1348216640

Please let me know if you did not visually verify each one of these and I will.

I've visually verified the repo snapshot flags with the following steps:

In my VSCode editor, I did a regex search of <img and filtered to test-repos/**/*.md. This gives me 19 results across 5 files.
In the snapshots, there are 9 new image violations being flagged.
I've verified that each of the 9 flagged violations are legit violations. I saw that the remaining 10 images all have alt text on it.

Thank you for verifying the new violations - that's exactly what I wanted.

Thanks, I've pushed up a commit(https://github.com/DavidAnson/markdownlint/pull/993/commits/a1db76d26fcbe4462cf9bc31b0010508349bb586) which re-runs the script!

GitHub is being silly, but maybe you can tell me how to work around it. If I make edits and push changes from a Codespace, it fails with HTTP 403 because I do not have write permission to your fork.

That sounds frustrating! I do have this checkbox option toggled on this PR:

This checkbox  should give you permission to commit to my branch. (Related: Allowing changes to a pull request branch created from a fork)
Is your codespaces of my forked repo? This article has some pointers that might be helpful: Committing changes to a pull request branch created from a fork - GitHub Docs.

Thank you! I will do a final review and merge the PR this evening.
Regarding permissions, I think all the preconditions are correct (and my ability to change individual files confirms my permission), but pushing from two different Codespaces created from that pull request failed. I have tried this in the past from the command line and that failed as well. It is possible I am making a mistake every time, but this has never worked for me and I am sure that I created the Codespace from the PR yesterday, so would have expected GitHub to set things up correctly. (I confirm that it created an "origin" and "upstream" remote.) I will try again tonight following the directions you sent exactly to double check myself.

It is possible I am making a mistake every time, but this has never worked for me and I am sure that I created the Codespace from the PR yesterday,

@DavidAnson I was able to repro what you're seeing on my end too. After some investigation, I discovered this is a known issue in Codespaces currently.
Related community issues:

https://github.com/orgs/community/discussions/36357
https://github.com/orgs/community/discussions/36299

For now, you will have to stick with the traditional route of cloning via the steps in Committing changes to a pull request branch created from a fork - GitHub Docs. 😅

Merged, thank you!
Regarding the Codespaces issue, thanks for tracking that down. :)


Added Missing Period + Comma

Description of the change: Added Missing Period + Comma
Reason for the change: reading experience
Link to original source: https://sendgrid.com/docs/ui/analytics-and-reporting/click-tracking-html-best-practices/

Closes #

Hello @shinypenny31,Thanks again for the PR! It's HACKTOBERFEST! We want to show our appreciation by sending                                              you some special Hacktoberfest swag. If you have not already, could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX


derive Debug for SqliteRow

add  Debug trait for SqliteRow.

A derived Debug impl isn't all that useful I think. I'd also like to see either all the row types implement Debug or none of them.

Closing this due to inactivity. If you want to reopen, please clarify on what information you need from the Debug impl of SqliteRow as simply deriving it isn't going to be very useful.


Created general Java interfaces

Third commit (a149b98) is a suggestion and can be ignored

@Propanu or @pylbert can decide when/if to merge it.

@stefan-andritoiu ignoring functions not needed in a particular language is ok.


correct object type of user_attributes

Thanks for the PR. I only can image for what reasons (id_token mapping in proxy case?) but need more context why you need/want to change this. Best with a test.

Correct, this is for adding user_attributes to the access_token in a custom token enhancer. The presence of user_attributes in the list of claims already suggests someone anticipated as much. Just the choice of a String is odd, especially since in the id_token user_attributes is a Map<String,List<String>> (see https://github.com/cloudfoundry/uaa/blob/f8188564facddee35600a23580eb845194337753/server/src/main/java/org/cloudfoundry/identity/uaa/oauth/UaaTokenServices.java#L621-L624). This PR then just brings what is here in line with what is there.

I checked the creation with user_attributes and id_token contains the entries if the user has the scope, see
https://github.com/cloudfoundry/uaa/blob/develop/server/src/main/java/org/cloudfoundry/identity/uaa/oauth/openid/IdTokenCreator.java#L131
The access_token is not designed to have user attributes. The standard code would not serialize the user_attributes in access_token if Claims properties are corrected and you suggest, so please provide a test that it gets clear what you change should correct.

@bruce-ricard  any thoughts about this

@strehle Please explain why you closed this.

Because there is no progress... and until now I dont understand the business context for this, there is no test which could explain it and therefore I closed it now

The use case is, as I mentioned, for adding user_attributes to the access_token. In any event as I pointed out there is an inconsistency in the app since elsewhere user_attributes is of type Map<String,List<String>>.

Or, to put it a different way the whole reason why user_attributes is here is because someone anticipated adding user_attributes to the access_token claims. Yet that is not currently possible to implement because the type is wrong. So either delete user_attributes altogether from here or correct the type.


Backend queries

Hi Team ,
I have added graph_query api for backend visualization queries.
Thanks
Kalluri

From show Graph :

From Table View:


bootstrap: Tolerate a missing lsb_release

Add missing keywords for openSUSE Leap and Tumbleweed
Fixes: ce77b04aabe8e17e6bd70ad5812233b5772c50bc

@zmc the dev_container/docker fail is not related, though, just wondering why it is trying to login somewhere?


[Feature] Jellyfin 10.9.0 related changes

Jellyfin 10.9.0 will use dotnet SDK 8.x
Jellyfin change person definitions from raw strings to Enum values

Solves: #313
PR applies changes to make it compatible with future 10.9.0 release.
Tested locally with default plugin settings.

Thanks for your PR! But I noticed that this PR only supports Jellyfin, lacking the backward support for Emby.

you are right! will try to keep this in mind the future :) Thanks for pushing and merging the changes.

Yea, but still thanks for your contribution!

@rzalawad BTW, can you test it on your Jellyfin 10.9 to see if everything works, including the scheduled tasks?

@xjasonlyu just added new media and ran a scan media library task. The media was populated with the correct metadata and images so everything seems to be working fine on the jellyfin side.

Thanks for the feedback! How about other tasks like organize metadata and generate trailers?

after enabling trailers, I see that the trailer file is created but its only ~75 bytes for different media. Is this because metatube is in the process of creating the trailer files?
for organize metadata, I don't see new collections in the collections tab so i'm not sure if its because I have no collections (is it based on series or actress?) or if metatube is not working properly.
At least based on the logs, I don't see exceptions/tracebacks.
I haven't used either organize metadata or generate trailers before so apologies for not being much help here

For trailers, it's correct. The trailer file is only a text file with a http link to the real trailer video.
If organize metadata runs without any errors, then it's working!

Thanks for your time, and I will release the latest plugin soon.
Cheers.


fix: ensure that -h works for all commands. fixes #506

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

I created this issue of Oclif to look into the base issue with -h not being supported.  https://github.com/oclif/command/issues/256

cool. although I've been waiting for oclif to review/merge https://github.com/oclif/parser/pull/163 for a while, so I think we should workaround this (since unlike that issue this one is pretty non-invasive to address)


Add cilium to the list of network providers for kubeadm

https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/

Also update Section on "Verified HostPort CNI providers" (if applicable) on page:
https://kubernetes.io/docs/setup/independent/troubleshooting-kubeadm/

The PR has been submitted and I am working to get it approved https://github.com/kubernetes/website/pull/9806

Looks like there is a parallel effort https://github.com/kubernetes/website/pull/9889


chore(risedev): add external_labels and remote_write in prometeus config

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
Add external_labels and remote_write labels in  Prometheus config to collect metrics data for multiple risingwave clusters:

external_labels: The labels to add to any time series or alerts when communicating with external systems.
remote_write: Ingest multiple Prometheus instance metrics to the one AWS Prometheus workspace.

Checklist

[x] I have written necessary rustdoc comments
[x] I have added necessary unit tests and integration tests
[x] All checks passed in ./risedev check (or alias, ./risedev c)

One concern: did we set necessary credential and permission for the remote write URL? If not, maybe it would be better to hide it from our repo.

One concern: did we set necessary credential and permission for the remote write URL? If not, maybe it would be better to hide it from our repo.

I create a IAM role to  managed policy of AmazonPrometheusRemoteWriteAccess, and ec2 that include this role can write this workspace.


npm version fails when run in Remote - Containers

npm version fails with errors because VSCode by default runs as root in devcontainer.json.

Thought this would be easy, just changing remoteUser to node in devcontainer.json but that also required setting user  in the docker-compose.yml file and then npm install failed on container creation ☹

Appears the only real solution at the moment is to do npm version --unsafe-perm

Guidance from the VSCode team is to try this approach: https://github.com/microsoft/vscode-dev-containers/blob/dc367e6ce28e228c95d71a335a4331de5918bd7f/containers/javascript-node-12/.devcontainer/Dockerfile#L48
Not sure about the sudo line though since that isn't on node:alpine.

Tried this and it now fails on git commands :(

Suggestion is to chown the .git folder when creating the image before swapping to the node user.


optimize:replace @Deprecated getOwnernName to getOwnerName in druid

Ⅰ. Describe what this PR did
optimize:replace @Deprecated getOwnernName to getOwnerName in druid
Ⅱ. Does this pull request fix one issue?

#3953
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #3954 (7d67674) into develop (e2e5fa6) will decrease coverage by 0.00%.
The diff coverage is 16.66%.

:exclamation: Current head 7d67674 differs from pull request most recent head ab498a1. Consider uploading reports for the commit ab498a1 to get more accurate results

@@              Coverage Diff              @@
##             develop    #3954      +/-   ##
=============================================
- Coverage      40.38%   40.37%   -0.01%
  Complexity      3070     3070
=============================================
  Files            685      685
  Lines          23194    23194
  Branches        2869     2869
=============================================
- Hits            9366     9364       -2
- Misses         12958    12960       +2
  Partials         870      870

Impacted Files
Coverage Δ

...sqlparser/druid/oracle/OracleUpdateRecognizer.java
0.00% <0.00%> (ø)

...r/druid/postgresql/PostgresqlUpdateRecognizer.java
0.00% <0.00%> (ø)

...a/sqlparser/druid/mysql/MySQLUpdateRecognizer.java
67.85% <50.00%> (ø)

...c/main/java/io/seata/config/FileConfiguration.java
43.10% <0.00%> (-1.15%)
:arrow_down:


fix typo in CLI command

@craigshoemaker
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

This change has already been made in the main branch, so I'll close this PR now.


What happened to the click event?

The NPCInteractEvent added in 64ff6ea94d87e27793d34a6176ef3808362b76ab appears to have been removed. Once 3.7 becomes available (see #31) how are developers supposed to check if an NPC has been clicked?

It hasn't been added yet, it will be added before the version is released, for now it's a wip.


Gallery opens when dragging slide

When you use lightGallery() with the slider the gallery is accidentally opened when you drag a slide. When you drag a slide the release click should be ignored.
A few bugs here and there but great script(s) overall, thank you!

When you use lightGallery with lightslider i recommend you to turn off mouse drag option enableDrag: false
Both are independent plugins and this is just a demonstration for how to integrate third party plugins with lightslider. I will work on lightslider and improve this after releasing lightGallery version 1.2
Thanks.

Thank you for the tip of disabling drag.

Hi @sachinchoolur
do you have any solution for solving this issue without turn off the drag option?

It would be great if this can be properly fixed, but in the mean time the following may resolve the problem of having lightGallery opening while lightSlider is being dragged by a desktop mouse.  This works because lightGallery checks for the existing of a class on the body tag before opening (to prevent double-opening), and because the order of lightSlider onBeforeSlide(), the lightGallery opening, and lightSlider onAfterSlide() methods firing in the order needed.   Note - I haven't tested this across all browsers/devices, feel free to respond for others if this doesn't work in all cases.
var myLightGallery;
var myLightSlider = $('#photo-gallery').lightSlider({
    // Your options here
    enableDrag: true,
    onSliderLoad: function (el) {
        myLightGallery = el.lightGallery({
            selector: '#photo-gallery .lslide'
        });
    },
    onBeforeSlide: function (el) {
        // When dragging on a desktop lightSlider runs but lightGallery also gets triggered to open
        // Adding the lg-on class to the body tag is a way of temporarily preventing lightGallery from
        // opening during a lightSlider drag (lightGallery uses this class to prevent double-execution)
        $('body').addClass('lg-on');
    },
    onAfterSlide: function (el) {
        $('body').removeClass('lg-on');
    }
});

Same issue, i cant resolve it

+1
I've got this issue as well. No solution for this?
There should be a setting when initializing lightSlider that lets you setup a lightslider with thumbnails but without lightGallery (I'm using that on the same page but for other content), so that when you slide it doesn't open a empty gallery but still gives the ability to show thumbnails.

Thanks @andyf101 , it may be temporary solution but it works like a charm and it does not need to change the core code. Great!

Great solution @andyf101 ! Works like a charm.

Thank you @andyf101 ! Your workaround is still pretty valid, and it saved my headache ;)

I can confirm that @andyf101 solution works fine in 2021 /clap /clap


update: added otel fix

coverage: 68.65% (+0.1%) from 68.54%
when pulling a608ecf6c03d057a11de454dfb9191b96b435236 on fix/otel
into 85aacf622c85ce27593418ecdc178a539ae58654 on main.


Support strokeAlpha/fillAlpha when creating a fallback appearance stream (issue 6810)

Fixes most of issue #13447; with only two things remaining (that I won't attempt here):

Support for non-default /LE-entries (line endings) in LineAnnotations, assuming we care about that.
The "weird" StrikeOutAnnotations with vertical lines, again assuming we want to fix that.

With this patch, while we still don't match Adobe Reader perfectly, we're doing a much better job overall than e.g. PDFium since all Annotations are now at least visible in the viewer.
For some Annotation-types without an included appearance stream, it definitely feels like we're getting into the territory of diminishing returns w.r.t. adding even more special-cases. At some point, it's feels very tempting to simply blame PDF generators for using more "exotic" Annotations and failing to including an appearance stream.

/botio test

/botio-windows preview

Thank you for fixing this, also for the other markup annotation types that according to the spec also support the CA property! I forgot about CA before since I had no test cases that actually used it and I also postponed custom line endings because they seemed very rare.
/botio makeref


No /usb commandline on stable-2.0 branch (missing URBDRC channel)

Hi, I think there is a problem on cmake configuration when compiling stable-2.0 branch:
cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_SSE2=ON -DWITH_CUPS=on -DWITH_PULSE=on -DCMAKE_INSTALL_PREFIX:PATH=/opt/remmina_devel/freerdp .

The produced executable does not have usb support:
xfreerdp /? | fgrep usb

outputs nothing.
I can see
CHANNEL_URBDRC:BOOL=ON
CHANNEL_URBDRC_CLIENT:BOOL=ON

on  CmakeCache.txt
But then config.h
config.h:/* #undef CHANNEL_URBDRC */
config.h:/* #undef CHANNEL_URBDRC_CLIENT */
config.h:/* #undef CHANNEL_URBDRC_SERVER */
config.h:/* #undef WITH_DEBUG_URBDRC */

(discovered from remmina user "unable to redirect usb camera" here https://gitlab.com/Remmina/Remmina/-/issues/2265#note_404332434)

Compiling FreeRDP from HEAD resolves this.
Can this fix be release?


Deprecate and remove pc_masker

This was a prototype node early on that should be removed in favor of point_cloud_annotator_node

Removed in #9


Add Electroneum

https://electroneum.com/
https://electroneum.com/press/
Found an SVG here: https://cdn.coinranking.com/HJvOCDlyM/etn.svg
Coin Name is ETN
https://coinmarketcap.com/currencies/electroneum/

Added, thanks for the info!


Ensure objs_read are consistent for all descriptors with a given stream name.

See this comment for context.

I have this half done

closed by #853


feat: add dud components

This wouldn't need any specific handling, just the trigger as ^dud:? and then anything else can be added after.
Would be useful for active demos... without actually needing to do anything.

This trigger would accept any component, as a method of demonstration - rather than restricting by type, and having one for each distinct component type.

tada :tada: (... just imagine I used a wand dammit.)


Add Workspace Managed Network Swagger

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

@joshharrin  Can you please fix the errors introduced by your current PR here : https://github.com/Azure/azure-rest-api-specs/pull/23283/checks?check_run_id=12263002892 for "Swagger LintDiff"

@raosuhas changes have been made to fix certain validation errors. Remaining Lintdiff errors will not be fixed per discussion within our team. Our service will continue to follow the valid ARM pattern of returning 202 with polling uri in location header for LRO operations same as our other APIs. Please help to approve and merge this PR, I have had teammates review as well. Thanks! cc @seanyao1
https://github.com/Azure/azure-openapi-validator/issues/483

Breaking changes previously approved in #22588

@visingla-ms @raosuhas . Thank you for your comments and recommendations, all have been addressed. Please help approve, thanks.

lintdiff errors are either false alarms or not intorduced by this PR


[Tech]: XCM Fee Management

Problem statement

For each incoming XCM message the parachain takes some fees on assets and burns some of them (value in units needed to execute transaction)
Since the parachain has not any tokenomics there are two ways to organize the transaction fee withdrawal

Calculate the rate of each asset on the parachain and take some fees based on that. The fees would be sent to outbound channel to technical account or other treasury account or an entity
Calculate the rate of each asset on Sora network.

The second way is more relevant because it is easier to organize the asset rate calculation on sora side. Bit the biggest drawback of that way - is to leave parachain without transaction fees that leads to security vulnerability. Thus, parachain should have a protection from spamming with low amounts of tokens - so for each registered asset the minimum amount that can be passed through the bridge should exist.
Description
No response
Definition of Done

Create a storage that keeps the information about minimum amount of assets that can be sent to it via XCM
Create a code that checks for this amount and returns an error for each amount
Disable fee collection for incoming XCM messages

Requirements
No response

Sora2-common pr
https://github.com/sora-xor/sora2-common/pull/50


Admin control panel

Implements a prototype user management panel which
lists users in a table
allows you to delete, create new users, update passwords and edit roles (of which there is currently only one 'ADMIN')
Disclaimer I 'co-developed' with Claude AI on this and it's my first time working with Angular, would appreciate a more comprehensive review which points out any anti-patterns/simpler way of doing things.
Cheers.

@arlowhite  when the user is admin the buttons cover the ESRI layers thing - can we move that?

@arlowhite when the user is admin the buttons cover the ESRI layers thing - can we move that?

Resolved


bump lm-coursier-shaded to 2.0.6 (includes coursier 2.0.9)

Fixes: #5200
Coursier 2.0.9 fixes an issue with connecting to repositories that conform to the newer RFC 7617, rather than the older RFC 2617, when challenging the user for credentials.
This caused an inability to resolve artifacts from a private Nexus repository.

Please also consider backporting this into the sbt 1.4.x branch

Please also consider backporting this into the sbt 1.4.x branch


Remove unused script after refactoring

Coverage increased (+0.03%) to 48.552% when pulling 16aa52bdba4a1ba6a68ea2fcaf0b6ad502572438 on yuye/remove_clean_script into 4f9eeecdc77efe4853b98de463cd6dbb9d727c10 on master.

Coverage increased (+0.03%) to 48.552% when pulling 16aa52bdba4a1ba6a68ea2fcaf0b6ad502572438 on yuye/remove_clean_script into 4f9eeecdc77efe4853b98de463cd6dbb9d727c10 on master.

Coverage increased (+0.03%) to 48.552% when pulling 16aa52bdba4a1ba6a68ea2fcaf0b6ad502572438 on yuye/remove_clean_script into 4f9eeecdc77efe4853b98de463cd6dbb9d727c10 on master.

Coverage increased (+0.03%) to 48.552% when pulling 16aa52bdba4a1ba6a68ea2fcaf0b6ad502572438 on yuye/remove_clean_script into 4f9eeecdc77efe4853b98de463cd6dbb9d727c10 on master.


Development to stating

Codecov Report
Base: 66.66% // Head: 61.47% // Decreases project coverage by -5.19% :warning:

Coverage data is based on head (28b3985) compared to base (cdb14cd).
Patch coverage: 25.64% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##           staging      #23      +/-   ##
===========================================
- Coverage    66.66%   61.47%   -5.20%
===========================================
  Files           16       21       +5
  Lines          285      353      +68
  Branches        77       84       +7
===========================================
+ Hits           190      217      +27
- Misses          95      136      +41

Impacted Files
Coverage Δ

src/components/FilterForm.jsx
90.90% <0.00%> (+18.18%)
:arrow_up:

src/components/inventory/ApproveButton.jsx
0.00% <0.00%> (ø)

src/components/inventory/EditedReport.jsx
0.00% <0.00%> (ø)

src/components/inventory/RejectButton.jsx
0.00% <0.00%> (ø)

src/services/user-service.js
31.25% <0.00%> (ø)

src/components/user/UserPage.jsx
61.53% <33.33%> (+2.35%)
:arrow_up:

src/services/inventory-service.js
41.17% <38.46%> (-4.28%)
:arrow_down:

src/components/inventory/EditedInventoryItem.jsx
75.00% <75.00%> (ø)

src/components/Navbar.jsx
100.00% <100.00%> (ø)

src/components/inventory/EditedReportList.jsx
100.00% <100.00%> (ø)

... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Installing Courier New font gives invalid name error

Checklist

[ ] Check other issues and make sure that it is not reported yet.

Versions and Environment
~/elixir -v
Erlang/OTP 21 [erts-10.2.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]
Elixir 1.8.1 (compiled with Erlang/OTP 21)
~/
Scenic:

scenic (Hex package) (mix)
locked at 0.9.0 (scenic) 129594f9

OS:
OSX

Steps to reproduce
https://github.com/QuantumProductions/scenic_font_test
Expected Behavior
I tried to install a Courier New font following https://github.com/boydm/scenic/blob/master/guides/custom_fonts.md
Actual Behavior

In GameOver.ex there is
  @font_folder :code.priv_dir(:play) |> Path.join("/static/fonts")
  @custom_font_hash "UYe2ryfLUpczug9LKyrTxPYGSYTimnPHzpSMbC6UF0M"
  @custom_metrics_path :code.priv_dir(:scenic_example)
           |> Path.join("/static/fonts/Courier New.ttf.metrics")
  @custom_metrics_hash Scenic.Cache.Support.Hash.file!(@custom_metrics_path, :sha)

Stack Trace
==> play
make: Nothing to be done for `all'.
Compiling 4 files (.ex)

== Compilation error in file lib/scenes/game_over.ex ==
** (FunctionClauseError) no function clause matching in IO.chardata_to_string/1

    The following arguments were given to IO.chardata_to_string/1:

        # 1
        {:error, :bad_name}

    Attempted function clauses (showing 2 out of 2):

        def chardata_to_string(string) when is_binary(string)
        def chardata_to_string(list) when is_list(list)

    (elixir) lib/io.ex:461: IO.chardata_to_string/1
    (elixir) lib/path.ex:512: Path.join/2
    lib/scenes/game_over.ex:9: (module)
    (stdlib) erl_eval.erl:680: :erl_eval.do_apply/6
could not compile dependency :play, "mix compile" failed. You can recompile this dependency with "mix deps.compile play", update it with "mix deps.update play" or clean it with "mix deps.clean play"

Name of font does not seem to be accepted. I downloaded from https://www.wfonts.com/font/courier-new

Additional Comments

Custom fonts and loading and all that stuff has been extensively re-written in Scenic v0.10.
Please see the upgrade docs here: https://hexdocs.pm/scenic/upgrading_to_v0-10.html, then let me know if it makes sense. The custom font thing is aa little trick as you will need to use the truetype_metrics tool to generate a metrics file, then store that alongside your custom font.
Is well worth it tho as all sorts of goodness is unlocked. Needs a how-to-do-custom-fonts blog post...

Thanks! Since there is an update I will close the issue for now. Will try again later with latest version.


Separate sequence diagram used in the Implementation section for the …

…Add Lesson Command in the Developer Guide

Codecov Report

Merging #149 (3680c7d) into master (bd3d584) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #149   +/-   ##
=========================================
  Coverage     51.12%   51.12%
  Complexity      528      528
=========================================
  Files           115      115
  Lines          2363     2363
  Branches        259      259
=========================================
  Hits           1208     1208
  Misses         1075     1075
  Partials         80       80

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd3d584...3680c7d. Read the comment docs.


Adds simple Helm chart for Kubernetes deployment

Hi Mailcrab maintainers,
I'm contributing this Pull Request adding a Helm chart to Mailcrab, as I've recently needed to write one. The chart is really simple and differs very little from a clean helm create boilerplate, so should be easily maintainable.
Motivation:

Effortless Kubernetes deployments: Helm streamlines packaging, configuration, and deployment, making it easier than ever to manage Mailcrab on Kubernetes.
Consistent best practices: An official chart ensures everyone leverages tested, optimal configurations for a uniform and reliable experience.
Reduced fragmentation: With a central Helm package, users avoid piecing together deployment scripts, preventing compatibility issues and configuration inconsistencies.

Pull Request contents:

Helm chart: Includes templates for the a simple MailCrab deployment.
Documentation: Provides clear instructions for installation and configuration compatible with helm-docs

Request for review:
I appreciate your time considering this contribution. I'm specially interested in feedbacks about:

Roadmap: Is Kubernetes support something interesting for the MailCrab community?
Guidelines: Does it adhere to the repository best practices and contribution guidelines?
Documentation: Is it concise, easy to understand, and helpful for users to deploy successfully?

Notes:

No deployment method for the chart was added so the recommendation is to clone the repo and run it locally. If this is something you guys are interested in I can help with a future PR.

Thank you for your time and consideration! I'm happy to contribute to making Mailcrab easier to manage on Kubernetes. Feel free to reach out with any questions or if you need further assistance.
Best regards.

Hi @luiseduardobrito - thank you for your contribution!
My only concern is the maintenance of the cart. Since I am not experienced with Helm, I can't really do maintenance on the cart (except for updating version number). Would you be available to do updates to the chart on some reasonable interval?

Hi @marlonbaeten, thanks for the response!
I'm glad to help maintaining and improving it in the future. The idea was never to add another burden to you guys, but ease the process of moving from some alternatives that have Helm charts available today (I used Mailhog for some time before MailCrab). So yes, count on me for helping with it.
The chart is configured to use the latest  Docker tag, so It doesn't need to be updated every time a new Docker image is released, just the version numbers in the Chart.yaml file for best practices. We could automate that with Github Actions.


return addurl --raw as  --raw-except=datalad

will need not yet released git-annex as fixed in 10.20240129-42-g6b38d0c427

10.20240227 was just released. I am looking (again) into updating conda build: https://github.com/conda-forge/git-annex-feedstock/pull/160


bug in date assignment

Prevent Birthday from being set to future date #75 fixation.

Resolve conflcts

Ok. Now can you please review the changes..


Update the loader for the demos

Update the loader so that the demo for emojis are working + changed from rawgit.com as it has been deprecated.

Thanks! Did not see rawgit deprecation message before.


Push Mirrors: Option to disable force-push

Feature Description
Push mirrors currently do force pushes for push mirrors, implying the assumption that the source repo is the single source of truth.
But consider this use case:

find a repo on $nonfederating-forge
fork the repo on $nonfederating-forge
add the repo to gitea, to work in a familiar environment
set up push mirror in gitea to fork on $nonfederating-forge
create PR in source repo on $nonfederating-forge
owner of the source repo makes changes to the PR branch

gitea will overwrite the changes made by the original repo maintainer when mirroring.
This is not desired and causes confusion as can be seen in https://github.com/xperimental/nextcloud-exporter/pull/65#issuecomment-1230026287
Screenshots
No response

This would be useful for us as well. We found that the git config options are ignored so that --mirror and --force are always supplied, regardless of the remote_mirror_$random mirror and push values.
I was thinking we could implement this in two PRs.

Remove the hard coded force and mirror parameters so that the git config is respected. Users that wish to disable mirroring and/or force pushes can edit the git config and things will just work. Existing users will be unaffected.
Extend PushMirror to make the mirror and force flags dynamic. Update API and frontend accordingly.

Is summary, assuming that git v2.0.0 is really the minimum required version, git should should not need to be told explicitly to use --mirror and --force.
If there is a maintainer that thinks this is a reasonable plan, please let me know and we'll get started on it.

waiting for update

Pull Mirrors also need an option to disable force-push. I opened an issue for pull mirrors https://github.com/go-gitea/gitea/issues/29604

Interested in this as well.


Hashtag span?

Markwon version: {4.4.0}

Thanks for this awesome library. My life became so much easier after I wasted 2 days trying to handle bullet points, numbered lists, tables and whatnot manually 😅
However, earlier in my manual parsing, I was adding hashtag parsing (making text bold with some color) to any word preceded by # (without spaces), like "#someWord"
Is there any way to add these custom spans to the already existing markdown text?
I'm using the following code to set the markdown:
var TextView?.markdownText: String?
  @Deprecated(NO_GETTER, level = DeprecationLevel.ERROR) get() = noGetter()
  set(value) {
    this?.let { textView ->
      Markwon.builder(context)
        .usePlugin(StrikethroughPlugin.create())
        .usePlugin(object : AbstractMarkwonPlugin() {

          override fun configureConfiguration(builder: MarkwonConfiguration.Builder) {
            builder.linkResolver { view, link ->  }
          }

          override fun configureTheme(builder: MarkwonTheme.Builder) {
            builder
              .linkColor(Color.parseColor(ACCENT_BLUE_COLOR))
              .bulletWidth(dip(4))
              .bulletListItemStrokeWidth(dip(1))
              .listItemColor(Color.parseColor("#DE2F2F2F"))
              .blockMargin(dip(20))
          }
        })
        .usePlugin(TablePlugin.create {
          it.tableBorderWidth(dip(1))
          it.tableBorderColor(Color.parseColor("#E5E5E5"))
          it.tableCellPadding(dip(4))
        })
        .build().let { markwon ->
          val markdown = markwon.toMarkdown(value.orEmpty())
          markwon.setParsedMarkdown(textView, markdown)
        }
    }
  }

And this is my hashtag parsing function:
  private fun SpannableStringBuilder.withHashTagHandling(): SpannableStringBuilder = apply {
    if (isEmpty()) return this
    Pattern.compile("""#[A-Za-z0-9!]*""").matcher(this)?.let { matcher ->
      while (matcher.find()) {
        matcher.group(0)?.let { hashTagString ->
          setSpan(StyleSpan(Typeface.BOLD), matcher.start(), matcher.end(), SPAN_EXCLUSIVE_EXCLUSIVE)
          setSpan(ForegroundColorSpan(colorCode), matcher.start(), matcher.end(), SPAN_EXCLUSIVE_EXCLUSIVE)
          setSpan(RelativeSizeSpan(1.1F), matcher.start(), matcher.end(), SPAN_EXCLUSIVE_EXCLUSIVE)
        }
      }
    }
  }

I search around in the wiki regarding hashtags, but couldn't find anything, can you please guide me if there is a way to add the spans while setting the text?

For now I'm using a workaround:
Markwon.builder(context)
//...
    .build().let { markwon ->
          val markdown = markwon.toMarkdown(value.orEmpty())
          val span = SpannableStringBuilder.valueOf(markdown)
          span.withHashTagHandling()
          markwon.setParsedMarkdown(textView, span)
     }

If there is a better way to approach this, please suggest

Hello @premacck ,
there are multiple options available:

inline parsing which relies on experimental inline-parser artifact
text processing which does not rely on additional artifacts

Oh so should I create a separate custom plugin for hashtags?

You actual parser should be implemented inside your OnTextAddedListener, which in turn can be placed inside a plugin for better encapsulation/isolation

Okay so this is what I've implemented:
My HashtagTextAddedListener:
  private class HashtagTextAddedListener : CorePlugin.OnTextAddedListener {

    private val hashtagPattern = Pattern.compile("""#[A-Za-z0-9!]*""")

    override fun onTextAdded(visitor: MarkwonVisitor, text: String, start: Int) {
      hashtagPattern.matcher(text)?.let { matcher ->
        while (matcher.find()) {
          SpannableBuilder.setSpans(
            visitor.builder(), arrayOf(StyleSpan(Typeface.BOLD), ForegroundColorSpan(Color.parseColor(ACCENT_BLUE_COLOR)), RelativeSizeSpan(1.1F)), matcher.start(), matcher.end()
          )
        }
      }
    }
  }

And I'm applying this in the configure method like this:
    override fun configure(registry: MarkwonPlugin.Registry) {
      registry.require(CorePlugin::class.java) {
        it.addOnTextAddedListener(HashtagTextAddedListener())
      }
    }

This approach is working, but can it be further improved?

There is always room for improvement, but I find your implementation pretty solid


log entire request url (with search params) in at least some verbose modes

Context

I'm trying to debug query parameter tests...prism mock -v trace doesn't log the search part of the URL so it's harder to see what requests it's getting
Current Behavior

prism mock -v trace logs request paths but not the URL search params
Expected Behavior

prism mock -v trace (and maybe some lower levels) log the URL search params
Possible Solution(s)

Thanks for your report, @jedwards1211 .  We agree that this would be very useful, and welcome PRs to add it.


Fix faultinj spdlog dependency

Fixes #966 by adding the same spdlog dependency used by the RAPIDS build, allowing faultinj to inherit the same spdlog compile definitions.

build


WIP: ssl/ssl_locl.c: switch to time_t time.

This is rather a conversation starter than WIP. Well, it is also a WIP, but it touches too little of the problem. I've ran into this when looking at 32-bit --strict-warning build. What's wrong with long time in ssl_session_st (as well as else-/anywhere)? Problem is that there are platforms where sizeof(long) < sizeof(time_t). [No, not cough-dows alone.] Note that it's kind of loose-loose situation. Currently it's problematic to assign ssl_session_st.time, because of possible truncate, but if it's declared time_t, it becomes problematic to return it with SSL_SESSION_get_time. Also note that it might be undesirable to return time_t from SSL_SESSION_get_time, because there is platform, cough-dows,  where sizeof(time_t) depends on compiler flags. Trouble is that while we have control over flags OpenSSL itself gets compiled with, we don't have control over flags for application that includes our public header. With this in mind you kind of start pondering over  [64-bit] ossl_time_t. But how about this? If we take this path, why not as well choose other units, such as milliseconds...

it might be undesirable to return time_t from SSL_SESSION_get_time,

I naturally mean declare SSL_SESSION_get_time['s replacement] as one returning time_t.

(This is what we ended up doing, but I'm guessing you won't quite match. We really don't like negative numbers. :-)
https://boringssl.googlesource.com/boringssl/+/ad8f5e1de94676fd674178705beb1e9697a67ab3^!/ )
There's actually two 32-bit time nuisances: struct timeval on Windows is also made of longs. This is because it's from Winsock where it is only used as a time interval, not a concrete time. But OpenSSL uses it in the DTLS next_timeout logic, which wants a concrete time.
One thing to be aware of, because the compiler won't catch it (BIO_ctrl): the type is passed into BIO_CTRL_DGRAM_SET_NEXT_TIMEOUT, which is a public API. We ultimately just took it out because no one uses the blocking DTLS retransmit logic, but that's less of an option for you. (Signal both BIO_CTRL_DGRAM_SET_NEXT_TIMEOUT and BIO_CTRL_DGRAM_SET_NEXT_TIMEOUT_EX?)
(I think you don't have to solve the two at the same time. We did because we unified the time handling into a callback for testing at some point.)

why not as well choose other units, such as milliseconds...

I know an operating system that has 100 ns granularity...  just saying ;-)

I know an operating system that uses 100 ns granularity...

Are you sure? Normally it's GetSystemTime -> SystemTimeToFileTime, but GetSystemTime has ... millisecond granularity. Unless of course you're talking about VMS in which case I stand no chance :-) :-) :-) But on serious note. Milliseconds was a "such-as" suggestion, i.e. it doesn't preclude other possibilities. However, I'd say that from human programmer's viewpoint, milliseconds are more suitable, because it's easier to wrap your head around it. I mean human staring at a value would still tend to think in terms of seconds (or fractions of one), and you would have lesser problem disregarding (or concentrating on) three last digits than seven.

Just in case it wasn't clear. This does address --strict-warning problem... Well, last line in "files changed" is actually sufficient, but it's just painful to see now = (uint32_t)time(NULL). Yes, it formally does the trick, but reading code should not be "can-you-do-that?" exercise...

why not as well choose other units, such as milliseconds...

I know an operating system that uses 100 ns granularity...  just saying ;-)

Are you sure? Normally it's GetSystemTime -> SystemTimeToFileTime, but GetSystemTime has ...

Oh, I'm very sure

But on serious note. Milliseconds was a "such-as" suggestion, i.e. it doesn't preclude other possibilities. However, I'd say that from human programmer's viewpoint, milliseconds are more suitable, because it's easier to wrap your head around it. I mean human staring at a value would still tend to think in terms of seconds (or fractions of one), and you would have lesser problem disregarding (or concentrating on) three last digits than seven.

On a serious note, what's important for us is time granularity from a security perspective.  Given that not too many seem to ask much more than second granularity (or like Kerberos, allow for quite a large time difference between systems), milliseconds should be good enough today.  On the other hand, since things like usleep() are fairly common these days, I don't really see an issue with going for microseconds.  After all, there's quite a lot of space in 64 bits...

Oh, I'm very sure

Well, my remark actually targeted difference between granularity of measurement and representation. I mean even if time is expressed in 100ns, it doesn't mean that clock has that accuracy, you simply won't observe specific fractions, sub-millisecond for example. Well, I don't know what exactly happens on VMS(*), but at least I find it extremely hard to believe that VAX (as referred on that page) clock had better-than-millisecond accuracy.
(*) Modern OSes tend to synthesize sub-timer-interrupt intervals, but at least in GetSystemTime -> SystemTimeToFileTime conversion it's out of reach anyway.

We really don't like negative numbers. :-)

You mean it's all about positive thinking ;-) I suppose this is an option too... I personally would lean toward signed, as having same type for specific time and negative intervals can be handy. On the other hand if unsigned would enforce some kind of discipline (implicitly through compiler warning system?), I'd be for it. Is there any better rationale behind your choice? Better than "don't like" that is.

but at least I find it extremely hard to believe that VAX (as referred on that page) clock had better-than-millisecond accuracy.

No, of course not.  However, DEC was quite forward thinking with regards to time, and considering the advances of hardware over time, hardware that has better-than-millisecond accuracy doesn't seem like an impossible thought, wouldn't you say?

Better than "don't like" that is.

Well, referred commit message mentions serialization. Is this it?

hardware that has better-than-millisecond accuracy doesn't seem like an impossible thought, wouldn't you say?

Of course I would [have to]. I suppose one can put it like "human-programmer-friendly milliseconds" vs. "future-oriented micro-/nano-/picoseconds". Either way, bottom line seems to be that switching to alternative representation is not unthinkable. Well, probably more people should weigh in...

On a serious note, what's important for us is time granularity from a security perspective.

Not only, because we do do protocols too, and it's not inappropriate to provide option to specify [at least] sub-second intervals (or time with sub-second granularity) in libssl.

I think we can agree that we agree ;-)

ping.  what's up with this?


Programmatically navigate [Suggestion]

On the router example it would be good to provide an example on how to programmatically navigate. Thanks

Sure! The answer is use_navigate if you're just looking for it for yourself, but I'll add it to the example.

Thanks @gbj for the help!! So when using use_navigate with the default navigateOptions the URL changes but the router does not update to reflect the changes that should be applied to the dom. Do i need to trigger something manually to get the dom to update as per the route? Thanks for your help.

Sounds like either a mistake or a bug. Can you provide an example to reproduce it?


Fee estimation failed. Fallbackfee is disabled. Wait a few blocks or enable -fallbackfee.

Fee estimation failed. Fallbackfee is disabled. Wait a few blocks or enable -fallbackfee

Transaction creation failed!

working with -fallbackfee=0.00001000

https://github.com/bitcoin/bitcoin/pull/16524/files

Fixed https://github.com/sugarchain-project/yumekawa/pull/55/commits/9951250499cff450ff31d98457be07d981eebe01


Dbref的问题？-from china

在测试初期未使用dbref 进行数据保存，之后添加dbref和Include的使用，但原数据无法查询到，这个如何解决

Can you write in english? I'm from Brazil

"At the beginning of the test, data was not saved using dbref, and then the use of dbref and Include was added, but the original data could not be queried. How can this be solved?" - Google Translator

Yes, but i still not understand the real problem. Maybe he was using in wrong way. @happygb21, can you post an simple example to reproduce this error?

When inserting data, dbref is not used. Data is read by adding dbref, unable to query data.

@happygb21 is about this? https://github.com/mbdavid/LiteDB/issues/963

First, Read all old data using old BsonMapper(the one without dbRef)
Then, Save all the data using the new BsonMapper with DbRef
@happygb21 先用旧的mapper全部读出来，在用新的mapper存一遍。 new LiteDataBase(filePath, bsonMapper)


10 out of 10 times it panics

It panics when trying to jailbreak with dopamine 2. If it doesn't panic it will go into a watch dog time out and disables tweak injection. If I try to enable tweak injection it goes back into a watch dog time out and it goes on etc. also phone gets really hot.
iphone 14 pro max iOS 16.6 b1
Older version did the same but got fixed then in latest update it have come back.

tweak issue


[V6 RC3] FieldArray examples don't work

I just tested the V6 RC3 example on FieldArray, and it didn't work out of the box for me.
I am talking about these pages:
http://redux-form.com/6.0.0-rc.3/docs/MigrationGuide.md/ (FieldArray section)
http://redux-form.com/6.0.0-rc.3/examples/fieldArrays/
The example shows:
render() {
  return (

      <FieldArray name="awards" component={awards =>

            {awards.map((name, index) =>

              Award #{index + 1}
              <Field name={name} type="text" component="input"/>

)}

          <button onClick={() => awards.push()}>Add Award</button>

      }/>

  )
}

But when I use this syntax, I get an error:

Uncaught TypeError: awards.map is not a function

I have a similar problem with forEach.
After debugging I found "map" and "forEach" functions were available not in "awards", but in "awards.fields", thus I had to change my code to the following:
render() {
  return (

      <FieldArray name="awards" component={awards =>

            {awards.fields.map((name, index) =>
 // <--- Change here
              Award #{index + 1}
              <Field name={name} type="text" component="input"/>

)}

          <button onClick={() => awards.push()}>Add Award</button>

      }/>

  )
}

And now it works.
Did I do something wrong, or is there a mistake in both examples?

+1 @Sir4ur0n, same thing here using V6 RC3

That was a problem with the docs. awards used to be what we usually call... props!
It was fixed in e21380772781257383ccbdf24dd4c49e1f94092b


refactor(multiple): clean up IE vendor prefixes

Cleans up the styles specifically for IE and Edge.

needs rebase

Rebased.


I18n

Description

New translations ko.json (Korean)
Motivation and Context

I hope Korean can use this program easily more than now.
Screenshots

Checklist:

[ ] My pull request is properly named
[ ] The changes respect the code style of the project ($ npm run lint)
[ ] I tested/previewed my changes locally

Thank you for your contribution and thank you for taking the time to translate Ferdi.
Unfortunately, we do not accept i18n PRs through GitHub. Instead, please submit your translations to our CrowdIn Project. If I would merge this PR, the CrowdIn Bot would overwrite them automatically and remove all changes.


perf(runtime-vapor): use setAttr for attributes that must be set as attirbute

Hi! I'm the autofix.ci troubleshooter bot.
It looks like you correctly set up a CI job that uses the autofix.ci GitHub Action, but the autofix.ci GitHub App has not been installed for this repository. This means that autofix.ci unfortunately does not have the permissions to fix this pull request. If you are the repository owner, please install the app and then restart the CI workflow! 😃


RLC code should stay on the RLC if it is broken and picked up.

It should work like NBT data, where if you pick up the item, the code remains on it instead of getting cleared. This will save many headaches.

Hi, can you double check in survival mode? In creative there is no event to pop off the blocks, it's directly removed, and the mod code bypassed. In survival, it should retain the code, and when placed again, the RLC should be in PAUSE mode.

Let's close this, works in survival mode.


Detect unused Mappings

Show a warning if a Mapping is defined that is not used anywhere within the template (similar to Conditions behavior)

Should be fairly easy to also build an undefined Mapping error...


Fixes #20403 - icon for closing compose box

https://github.com/zulip/zulip/issues/20403
before:

after:

cc @alya (posted #20403 )

Mm, that looks pretty terrible. The ^ and x icons need to be roughly the same size and thickness, as well as being well-aligned with each other.

Ok thx, will revise and push again.
On Thu, Jan 6, 2022, 4:09 PM Alya Abbott @.***> wrote:

Mm, that looks pretty terrible. The ^ and x icons need to be roughly the
same size and thickness, as well as being well-aligned with each other.
—
Reply to this email directly, view it on GitHub
https://github.com/zulip/zulip/pull/20703#issuecomment-1006936410, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AABCFYS5SNHWWMNZLEA2J2TUUYAG7ANCNFSM5LNDUUEA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

how's this look @alya ?  I tried various font-awesome icons, and this is the best match I could find for font-awesome v4.7: chevron-up and close aka times
#compose_top_right {
    button {
         ...
	text-shadow: none;
	font-weight: normal;
	font-size: 15px;

light theme:

dark theme:

rejected: angle-up / times, with angle-up using font-weight: bold, font-size: 22px

@asah This looks great to me! Could you please fix the commit history and post a comment when done? Looks like there's something funny going on.

Slightly edited commit message, LGTM!

@alya merged main including #21708, re-tested, squashed and re-pushed.  new screenshot:

@asah please read https://zulip.readthedocs.io/en/latest/contributing/version-control.html#commit-messages to fix your commit message.

@amanagr sorry to be dense, but can you help me understand what's wrong with the commit message? "compose: Improve icon for closing compose box (Fixes #20403)" I believe conforms to the standard?
https://github.com/zulip/zulip/pull/20703/commits/bcc39c9d436bff575903357bcebde7a5c5b5c218
Thx/Sorry

Writing it like
compose: Improve icon for closing compose box.
Fixes #20403

is preferable.

done - gotcha/thx, doing same for my other PRs. @amanagr

@timabbott ready for your review

Merged, after rebasing to drop the merge commit (You should always do git pull --rebase to avoid creating these. Our git clone instructions have a gitconfig option that makes this to default), and also removing the space inside the button element. Check out our GitHub guide -- it has a bunch of useful workflow advice like this.
I also extended the commit message to explain why changing this icon requires changing the adjacent button's icons as well. (In an ideal world, we'd have had 2 commits: First one that switches to the correct fa-chevron and then this one, but I think the intermediate commit state would have been messy in this case). Take a look -- that sort of explanation in a commit message for why a change is correct dramatically reduces the time required to understand and thus merge something.
If one greps for each of &times, fa-times, fa-close, and fa-remove, one will find we have a lot of variation in how we do these. The last 3 are all the same icon, just different aliases -- can you start a #frontend thread on which of those we want to standardize on?  That bit at least would be easy; migrating the &times is likely a bit more involved just because it'll require CSS work like in this PR, and there's ~20 of them.
Thanks @asah!


[SPARK-39614][K8S] K8s pod name follows DNS Subdomain Names rule

What changes were proposed in this pull request?
This PR aims to fix a regression at Apache Spark 3.3.0 which doesn't allow long pod name prefix whose length is greater than 63.
Why are the changes needed?
Although Pod's hostname follows DNS Label Names, Pod name itself follows DNS Subdomain Names whose maximum length is 253.
Does this PR introduce any user-facing change?
Yes, this fixes a regression.
How was this patch tested?
Pass the CIs with the updated unit tests.

cc @yaooqinn

Thank you, @yaooqinn . K8s UT passed at the first comment and the second commit is only changing comments.
Merged to master/3.3.


mysql_1       | exec /usr/local/bin/docker-entrypoint.sh: exec format error

Describe the bug
A reoccuring error after make install or make restart
mysql_1       | exec /usr/local/bin/docker-entrypoint.sh: exec format error

To Reproduce
Steps to reproduce the behavior:

git clone repo
cp llm-server/.env.example to llm-server/.env
add OPENAI_API_KEY to that llm-server/.env file
make install

Expected behavior
To see localhost:8888 working
Desktop (please complete the following information):

Linux Ubuntu x86_64 GNU/Linux

this usually arises due to a mismatch between the architecture of the Docker image and the architecture of your host system. but as you're using an x86_64 it seems it's something else, can you try to re-build the project one more time? if that does not work I can spend some time debugging it

Thanks for the speedy reply @gharbat
I did the following:
docker-compose down -v
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker container prune
docker volume prune

Then, make install, and it worked.
Thanks again.


[RAC] Unmemoized render values passed to useRenderProps()

Provide a general summary of the issue here
When rendering a RAC with a memoized render callback, the callback is invoked on each render.
This happens regardless of whether the components state or rather its render values have actually changed.
When performing expensive calculations inside a render function, this is especially impactful.
🤔 Expected Behavior?
RAC's should not run a memoized render function twice if the render value hasn't changed.
This would allow the developer to make the decision whether or not to memoize the result of a render.
😯 Current Behavior
RAC's invoke their render callbacks even when the render values haven't changed.
💁 Possible Solution
Memoize the values argument passed to useRenderProps in most RAC's.
Example of refactored Button.tsx:
const values = {
  isHovered,
  isPressed,
  isFocused,
  isFocusVisible,
  isDisabled: props.isDisabled || false
};

let renderProps = useRenderProps({
  ...props,
  // eslint-disable-next-line react-hooks/exhaustive-deps
  values: useMemo(() => values, Object.values(values)),
  defaultClassName: 'react-aria-Button'
});

🔦 Context
No response
🖥 Steps to Reproduce
Example: Codesandbox

Choose any RAC utilizing render props.
Force a re-render of the parent, while the props of the RAC don't change.
Observe another execution of the render function.

Version
latest
What browsers are you seeing the problem on?
Chrome
If other, please specify.
No response
What operating system are you using?
OSX
🧢 Your Company/Team
No response
🕷 Tracking Issue
No response

Thanks for your thoughts! Unfortunately, I don't think this solves it. You'd have to memo the children of the component as well. https://github.com/adobe/react-spectrum/blob/b940126e4d67a11d28b7c0b098eab23205598b6c/packages/react-aria-components/src/utils.tsx#L95
This discussion is related https://github.com/adobe/react-spectrum/discussions/2569#discussioncomment-1672680
Instead of running expensive operations in render, I suggest using an effect.

@snowystinger Hey Robert, thanks for getting back to me.
The scenario with the expensive calculations was meant as an exemplary worst case. I opened this issue because its impossible for the developer to make the choice whether to memoize or not, since the value has a new identity each time.

Desired might be a strong word. I don't think the memo is hurting anything by being there though it'd probably make more sense to memo some of those values individually and then join them.
I also chased down the state returned by useToggleState isn't memoed, so that'd also create a new useRenderProps value. Honestly, I think this is something the compiler should be good at solving.
I'm going to close this issue, we can reopen if it's causing a problem.


feat: rename $ref to ref

So as promised in the Spec 3.0 call today, here's my PR 😂
With this change, I'm trying to prevent a future breaking change if we start accepting references to Protobuf or XSD, or any other non-JSON formats. We will define the behavior of the JSON Reference object. As of today, we keep the same behavior so nothing else changes but the name ($ref -> ref). Whenever we think we're ready to ship non-JSON referencing support, it would be just a matter of augmenting the behavior of the JSON Reference object, making it a non-breaking change.
What do you think?

tbh I do not get how this change is helping and prevents any future breaking changes.
I think it is just a matter of making sure $ref description is clear and do not follow 1:1 JSON Reference.
In OpenAPI 3.1 you can see Reference Object is not limited to $ref
https://github.com/OAI/OpenAPI-Specification/blob/3.1.0/versions/3.1.0.md#reference-object
We could have the same (not summary and description), and in future have another field where one can specify that reference is to proto or xsd, or in general non-json

You're both right. Closing it!

I think it is just a matter of making sure $ref description is clear and do not follow 1:1 JSON Reference.
In OpenAPI 3.1 you can see Reference Object is not limited to $ref
https://github.com/OAI/OpenAPI-Specification/blob/3.1.0/versions/3.1.0.md#reference-object
We could have the same (not summary and description), and in future have another field where one can specify that reference is to proto or xsd, or in general non-json

@fmvilas and what do you think about 👆

Yeah, I like that solution. We just use a different JSON Reference object. That should be fine 👍


textutil: add color map facilities

Introduce SetColorMap()/GetColorMap() API's. With this the user may
change the termbox's output (i.e with Output256) mode and set a broader
color map.
In the future - if needed - we could introduce an API to switch modes
and automatically remap the colors as required. For now, this API's
are good enough.
Signed-off-by: Leandro Dorileo<EMAIL_ADDRESS>

Interesting idea. Like a fine-tuning thing for themes.
I see it is time to update changelog. I'll do it soon


another TypeError in cm_factor attack

[*] Performing cm_factor attack on ./pub.pem.
100%|█████████████████████████████████████████████████| 6/6 [04:20<00:00, 43.39s/it]
/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import fnmatch, glob, traceback, errno, sys, atexit, locale, imp, stat
Traceback (most recent call last):
File "./RsaCtfTool.py", line 283, in
attackobj.attack_single_key(publickey, attacks_list)
File "/home/aditya/ctf/RsaCtfTool/lib/rsa_attack.py", line 194, in attack_single_key
self.priv_key, unciphered = attack_module.attack(
File "/home/aditya/ctf/RsaCtfTool/attacks/single_key/cm_factor.py", line 46, in attack
if sageresult > 0:
TypeError: '>' not supported between instances of 'bytes' and 'int'

Hi @adityabadhiye would you like to share your pub.pem? so we can debug this issue.
Thanks

Hi @adityabadhiye would you like to share your pub.pem? so we can debug this issue.
Thanks

@daedalus i'm on it, last tests before commit :)

@daedalus i'm on it, last tests before commit :)

fixed in latest commit !

fixed in latest commit !

cheers!

cheers!

rsa.zip
@daedalus see this has pub key and enc file

rsa.zip
@daedalus see this has pub key and enc file


Feat: Gateway-specific external models

Add an optional gateway:  key to the external models definitions which allows gateway-specific external models and addresses the problems highlighted in issue #2689
After this change, the behaviour is as follows:
sqlmesh create_external_models - no change, current behaviour. Models get written without the new gateway:  key.
sqlmesh --gateway dev create_external_models - Models get written with gateway: dev
sqlmesh --gateway prod create_external_models - Models get written with gateway: prod. The gateway: dev models remain in the file.
When running plans:
sqlmesh plan - will pick up all external models with no gateway:  set, plus any external models with gateway: <default gateway>
sqlmesh --gateway dev plan - will pick up all external models with no gateway:  set, plus any external models with gateway: dev

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Erin Drummond seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Images coffea-dask-almalinux8:v2024.8.3-py3.* Fail for some Microarchs

When using the above-named images (ending in either py3.11 or py3.10), Dask workers frequently fail. However, when we request workers with microarch "x86_64-v3" (as opposed to "x86_64-v2"), jobs run successfully.
This issue does not seem to happen with coffea-dask-almalinux8:v2024.5.0-py3.*.
Some details:
We are submitting jobs from a host within a container running the same image as we give to the workers. Workers access images as SIFs in CVMFS. Jobs are submitted at Wisconsin with HTCondor, through dask-jobqueue's HTCondorCluster. The only error message we see is a Dask KilledWorker exception.

@rpsimeon34 can you try latest tag again, I merged fix https://github.com/CoffeaTeam/af-images/pull/56/commits/a5944aaf98c5e45f2c98c69ad5eeec63d65c0184. All tags that have a fixed where pushed today: https://hub.docker.com/r/coffeateam/coffea-dask-almalinux9/tags

still not fixed, I will try one more iteration, and let you know when it is ready to test

The reason this is happening is explained here: https://github.com/CoffeaTeam/af-images/issues/58
Please test again with 2024.11.0. For older images, the solution is to install awkward via pip in the image.

Thanks, @ikrommyd and @oshadura ! It looks like v2024.11.0 solves this issue (specifically, I tested with coffea-dask-almalinux9:2024.11.0-py3.11)


Temp param

Dubnium has a temp param that will not do anything.  This was mistakenly left in and will be removed in the next version.

Resolved in v2.3.0


[5.10] Set file-system as device-agnostic in build manifest

Cherry-pick of  #7052.
Motivation:
This is the default mode in XCBuild, and allows incremental builds to work across reboots on APFS. It would also likely address https://github.com/apple/swift-package-manager/issues/4651.
Modifications:
Set file-system as device-agnostic in the generated build manifest
Result:
Restarting your system (or inode changes) doesn't invoke unnecessary builds
Related to rdar://96403757.

@swift-ci test


IDVA6-1936: Fixed tests

Short description outlining key changes/additions
Integration tests were failing in pipeline. Fixed.
JIRA Ticket Number
IDVA6-1936
Type of change

[x] Bug fix
[ ] New feature
[ ] Breaking change
[ ] Infrastructure change

Pull request checklist

[x] I have added unit tests for new code that I have added
[x] I have added/updated functional tests where appropriate
[x] The code follows our coding standards

An exhaustive list of peer review checks can be read here

CI: Security warnings found!

CI: Security warnings found!