About the accuracy on base-to-new generalization setting

Hi,
Thanks very much for your interesting and inspiring work and well-organized codebase! I am trying to reproduce your results in the paper, and here are the results I obtained for the base-to-new generalization setting:
 | Imagenet | Caltech101 | Food101 | Dtd | Ucf101 | Oxford_Flowers | Oxford_Pets | FGVC | Standford_Cars | Sun397 | Eurosat | AVG
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
Base | 77.60 | 98.07 | 90.67 | 83.53 | 86.93 | 98.00 | 95.17 | 42.87 | 78.43 | 82.53 | 92.83 | 84.24
Std | 0.14 | 0.12 | 0.09 | 0.92 | 0.05 | 0.14 | 0.21 | 0.88 | 0.25 | 0.12 | 0.97 |
New | 70.30 | 93.87 | 91.57 | 59.63 | 78.93 | 77.10 | 97.33 | 36.60 | 75.30 | 78.80 | 72.93 | 75.67
Std | 0.08 | 0.05 | 0.09 | 3.52 | 2.07 | 0.14 | 0.49 | 0.42 | 0.59 | 0.41 | 3.92 |
H-mean | 73.77 | 95.92 | 91.12 | 69.58 | 82.74 | 86.30 | 96.24 | 39.49 | 76.83 | 80.62 | 81.69 | 79.73
Basically, I find that the accuracies on base classes are consistent with the results in the paper and their variances are quite small. However, when it comes to the accuracies on the new classes, the accuracies are lower than that reported in the paper (on average 75.67 vs. 76.10). After carefully checking the results, I find that the gap mainly comes from two datasets, DTD and EuraSAT, where the variances are much larger than other classes. Therefore, I wonder whether this phenomenon is normal or not? Did you observe larger variances for the two classes during testing on new classes?
Thanks in advance!

Hi @yxgnahz,
Thank you for showing interest in PromptSRC!
Regarding your query, we also observed a bit larger variances for these datasets, but the obtained results are same as mentioned in our main paper.
So I think, the reproduced performance should roughly match the main paper results. There can be slight variance in the results when reproducing results on different machines, but the shared results gap on average novel class performance is significant (75.67 vs 76.10). From your table, even for base classes, some datasets have different results, such as 78.43 % on Stanford cars vs reported 78.27 %.
In order to resolve the issue, can you kindly verify the following?

You can make sure that you are using the same versions of PyTorch and other packages as mentioned in our installation read-me files shown here. We have noticed in our experiments that using different PyTorch and CUDA version leads to fluctuating results. So this might help in obtaining the exact results.

You may kindly verify if you are using the provided scripts for running your experiments and not changing any hyper-parameters in the config files. For training in base-to-novel setting, we use this script which by default uses the vit_b16_c2_ep20_batch4_4+4ctx config file.

Optionally, can you also re-run your experiments using the commit version of 1af6e40b01e0c0c24fd8b6854be835209a0cde74 for the Dassl.pytorch library. I have noticed that we used slightly older version of the Dassl.pytorch. It should not be a problem, but you may just check this as well in-case.

I hope that would be helpful. Kindly let us know if this resolves your issue!

Thanks for your prompt reply! I followed your installation guide to set up the environments (torch 1.9.0+cu111) and used exactly the same script you provided for the reproduction. All the experiments are conducted on an Nvidia 3090 GPU. I also tried with the provided Dassl version, and I found the results unchanged. Therefore, could you please show me the exact variances you observed for these datasets?

Hi @yxgnahz,
Thank you for verifying the installation.
Actually I do not have access to the machine where all the checkpoints and results log files are stored. I will try to get the access, and share the variance details here as soon as possible.
Lastly, can you clarify if you are training your models from the scratch to reproduce the numbers or are you using the provided pre-trained weights to reproduce the results?
Thank you!

Hi @muzairkhattak ,
The results above are trained from scratch using the scripts you provided in the document. By the way, I just verified the official released checkpoint for base-to-new setting, and here are results (accuracy on new classes) I obtained by running the reproduce inference script:
DTD 63.08+/-3.15, Eurosat: 73.88+/-4.80
The averaged accuracies are very close to the results in the paper (less than 0.1%) and I think the minor gap may come from the difference between our hardware platform. Moreover, the variances from the released checkpoint are  just consistent with what I have got in the models trained from scratch, which indicates that these two datasets indeed have large variances on new classes.

Hi @yxgnahz,
Thank you for sharing this update. Yes, I think these two datasets have higher variance as discussed.
If manageable for you, can you try to reproduce the results from scratch using another machine if that is available? This will be a double check.
Thank you once again!

Hi @yxgnahz,
I have just verified reproducing the main paper results by training models from scratch using this repository. I am able to obtain roughly same results as mentioned in our paper.
For example, the reproduced results for EuroSAT are 92.90% for base classes and 73.83% for novel classes. Similarly on DTD, the base class results are 83.53% and novel class results are 62.97%.
To facilitate the process of reproducing the results on these datasets, I have provided the training and testing log files through this link. You can verify and compare them with your log files.
I hope that would be helpful!
Thank you and kind regards!

Hi @muzairkhattak ,
Sorry I only have 3090 GPUs on my server. If I have chance, I will test this on another machine. I check the logs and I find all the settings are just the same. I believe the large variance leads to the performance gap in my reproduction and the value of the gap is normal considering the variance of the two datasets.
Thanks for your discussion!


Rename CLI folder

https://github.com/prisma/prisma2/tree/master/cli/prisma2
But to what?

We should probably rename it to packages to go with the lerna, yarn and pnpm workspaces convention.

This is done with the latest repo merging changes.


ONNX export does not work for EfficientNet model due to missing operator (AdaptiveAvgPooling2D)

I am trying to convert the 'EfficientNet' pre-trained model (B0 variant) to ONNX format.
See paper https://arxiv.org/abs/1905.11946
Unfortunately, this does not work because it seems that one operator is missing.
The error thrown is 'AttributeError: No conversion function registered for op type _contrib_AdaptiveAvgPooling2D yet'
MXNet version: 1.5.0
Code to reproduce this issue:
import numpy as np
import gluoncv2
from gluoncv2.model_provider import get_model as glcv2_get_model
import mxnet as mx
from mxnet.contrib import onnx as onnx_mxnet
effnetb0 = glcv2_get_model("efficientnet_b0", pretrained=True)
input_shape = (1, 3, 224, 224)
x = mx.nd.zeros(input_shape, ctx=mx.cpu())
effnetb0.hybridize()
y = effnetb0(x)
effnetb0.export('effnetb0')
converted_model_path = onnx_mxnet.export_model('effnetb0-symbol.json', 'effnetb0-0000.params', [input_shape], np.float32, 'effnetb0.onnx', verbose = True)

@szha  Has this issue been solved in 1.x?

yes, this has been partially resolved. we support the output_size==1 case (equivalent to global pooling). Other cases are not supported due to onnx not having a adaptive avg pooling op.
you can find the new operator support matrix here https://github.com/apache/incubator-mxnet/tree/v1.x/python/mxnet/onnx#operator-support-matrix. The new onnx support will be released in mxnet 1.9 soon

Hi!
I'm trying to convert EfficientB0 mxnet model to ncnn, and get the same error:
"_contrib_AdaptiveAvgPooling2D not supported yet!"
I'm using:
mxnet version: 1.5.0
onnx version: 1.10.1
Win10
Could you please with it?


 Appreciate your time!

I try to compile last stable 0.8.8 branch, came with a lot of issues:
go get ./...

# github.com/influxdb/influxdb/protocol
protocol/protocol_extensions.go:18: undefined: Point
protocol/protocol_extensions.go:24: undefined: Point
protocol/protocol_extensions.go:28: undefined: Point
protocol/protocol_extensions.go:32: undefined: Point
protocol/protocol_extensions.go:38: undefined: FieldValue
protocol/protocol_extensions.go:63: undefined: Point
protocol/protocol_extensions.go:73: undefined: Point
protocol/protocol_extensions.go:92: undefined: Request
protocol/protocol_extensions.go:96: undefined: Request
protocol/protocol_extensions.go:102: undefined: Request
protocol/protocol_extensions.go:102: too many errors

Please show your entire build process, including cloning the repo from scratch.

Please be sure to precisely follow the build instructions here:
https://github.com/influxdb/influxdb/blob/v0.8.8/docs/contributing.md

@mikhailov -- executing go get ./... is not part of the 0.8.8 build process.

@otoolep, OK! The current master is under major refactoring, but the latest stable it is 0.8.8 as I could find. Can you share building process specific for 0.8.8 please? :)

@otoolep, OK! The current master is under major refactoring, but the latest stable it is 0.8.8 as I could find. Can you share building process specific for 0.8.8 please? :)

@otoolep, OK! The current master is under major refactoring, but the latest stable it is 0.8.8 as I could find. Can you share building process specific for 0.8.8 please? :)

@mikhailov : https://github.com/influxdb/influxdb/blob/v0.8.8/docs/contributing.md

@otoolep brilliant, thanks!

Nothing to do here, building works fine now.


Resolve remaining issues with ifix

We have a few remaining questions we need to resolve:

What should the syntactic form of ifix be?

Should it be saturated?

i.e. ifix f :: k -> * vs ifix f a :: *

Should it be a binder?

We could bind the argument of the pattern functor, and also the argument if it's a saturated form.

Are there problems with preserving the normalization status of "multi-argment" pattern functors when we encode them into our unary version? (Not sure if I got that right, hopefully @effectfully can elaborate)

I believe the current state of the syntax is:

The Haskell implementation has a saturated, non-binder ifix
The Agda formalization has a non-staturated, non-binder ifix
Rebecca's proposed formalization has a saturated, binder ifix

Should it be saturated?
i.e. ifix f :: k -> * vs ifix f a :: *

I think we agree it should saturated?

Should it be a binder?
We could bind the argument of the pattern functor, and also the argument if it's a saturated form.

I tried that and it was a pain. That's what I posted in the PR that adds ifix:
There are three cases here:

fix binds no variables
fix binds the pat variable
fix binds the pat and the arg variables

I tried to do as fewer changes as possible, so my first attempt was (2). This resulted in code like this:
        TyIFix a tn ty arg -> do
            clonedArg <- clone what arg
            addBound (tn ^. unique)
            TyIFix a <$> clone what tn <*> clone what ty <*> pure clonedArg

I.e. we need to clone arg before cloning pat, because the latter requires adding a bound variable to scope which is not bound in arg. That's not too much trouble, but is not nice.
Then in type rules we need to interleave function application with explicit substitution.  This is how the unwrap rule looks like:
-- [infer| term : ifix n vPat vArg]
-- -----------------------------------------------------------
-- [infer| unwrap term : NORM (([ifix n vPat / n] vPat) vArg)]

I consider

-- [infer| term : ifix vPat vArg]
-- ---------------------------------------------------
-- [infer| unwrap term : NORM (vPat (ifix vPat) vArg)]

much more readable. And why would we bind one variable and not the other?
I haven't tried (3), but it has to be very bloated: two variables each of which has a name and a type. And every single function that matches on TyFix has to handle two binders. This is just boilerplate.
However there is another reason why I prefer not to make Fix a binder: because it's not a binder in all of the several dozens of Agda files I've produced so far. It has been a constant pain for me to convert something I wrote in Agda to PLC due to the need to perform that binding twist in my head. The same applies to type rules: I write them pretending fix is not a binder and then adapt to fix-as-a-binder setting. I think fix being not a binder is easier to reason about, especially when fix binds two variables
So with respect to implementation, making ifix a binder is just pain: it complicates code, it complicates rules, it complicates conversion from Agda (which is very important, because I have really complicated encodings there which would be very hard to write by hand without the Agda assistance), it impedes my reasoning. I have functions there like this:
-- > \(withSpine :: ((k -> *) -> *) -> k) (patF :: k -> k) ->
-- >     \(rec :: (k -> *) -> *) (spine :: k -> *) -> spine (patF (withSpine rec))
packPatternFunctorN ann k withSpine patF = do
    rec   <- freshTyName ann "rec"
    spine <- freshTyName ann "spine"
    let star  = Type ann
        (~~>) = KindArrow ann

    return
        . TyLam ann rec ((k ~~> star) ~~> star)
        . TyLam ann spine (k ~~> star)
        . TyApp ann (TyVar ann spine)
        . TyApp ann patF
        . withSpine
        . TyApp ann
        $ TyVar ann rec

And this is just one function, I have several functions like that. Originally, I kept everything on the Plutus Core side, but that resulted in huge unreadable terms, so I decided to pass some arguments via Haskell lambdas rather than Plutus Core lambdas, so in the function above I'm mixing the two levels and pass some stuff via Haskell lambdas and some via Plutus Core lambdas. It's already complicated, but at least I only move lambdas back and forth between the source and the target language, so it's pretty mechanical. But adding implicit scope extensions would be a nightmare, because then I also need to keep track of those extensions in my head and move back and forth between scope extensions and lambda as well.
I've described the reasons not to make ifix a binder, what are the reasons to make it a binder?

Are there problems with preserving the normalization status of "multi-argment" pattern functors when we encode them into our unary version? (Not sure if I got that right, hopefully @effectfully can elaborate)

There are. I'm solving them. Docs (including an explanation of what the problem is) are on the way.

I believe the current state of the syntax is:

I believe the same.

@psygnisfive made a well motivated argument thatmu should be a binder which as I understand it is because as we want to minimise connectives being dependent on each other.
G !- B => K
G, r :: K -> *, t :: K !- A => *
--------------------------------
G !- ifix(B; r,t.A) => *

However, this still needs to refer to the -> so this cannot be achieved completely.
I have an argument for the opposite: We can take a 'logical framework' approach. The lambda and application are 'meta' and the other stuff (pi and mu) are object level/language specific. We have one binder - lambda, and we define all other constructs using this one binder.
We can define mu as:
μ1 : ∀{φ K} → φ ⊢⋆ ((K ⇒ *) ⇒ K ⇒ *) ⇒ K ⇒ *

and, indeed, we could do the same for pi:
Π : ∀ {Φ K} → Φ ⊢⋆ (K ⇒ *) ⇒ *

I think all the versions should be equivalent, so I don't think it is a huge problem. Also, I think the iso- vs. equi- recursive type discussion is orthogonal, the fact we could have this discussion about pi instead of mu adds to this.

I'm not sure I really understand the motivation for wanting to reduce how much mu depends on ->. Is the idea that we could use a binder-mu in a language without higher kinds? Much like we can have forall without higher kinds, whereas James' binder-free version of forall requires them?
Of course, as James points out, we can't actually get to that point with ifix, because we still have a kind arrow even in the binder version. Does that negate all the benefit, or is it still helpful to go part way?

Here is another argument in favour of not making ifix a binder: often I separate definitions of pattern functors from definitions of actual data types. And sometimes pattern functors are computed rather than just written down. For example, in one of our docs we have
treeForestF =
    \(a :: *) (rec :: (* -> * -> *) -> *) (tag :: * -> * -> *) -> all (r :: *).
        tag ((a -> rec forestTag -> r) -> r)                  -- `case` over `tree`
            (r -> (rec treeTag -> rec forestTag -> r) -> r))  -- `case` over `forest`

Here treeForestF, being applied to an a, computes to a pattern functor. And it's not just for convenience, it's a really reusable and meaningful part of the encoding of the tree/forest family, because later we have
tree   = \(a :: *) -> ifix (treeForestF a) treeTag
forest = \(a :: *) -> ifix (treeForestF a) forestTag

How to do the same in the ifix-as-a-binder setting?

Okay, here is another bunch of ifix-related documentation, this time implementation-specific. It also describes the denormalization problem, a possible (already implemented) solution and enumerates pros and cons of this approach (and the old one with non-normalized types). Do not read the actual code there, only the docs.

@michaelpj in generally, it's preferable to have connectives be "pure", i.e. not defined in terms of one another, b/c it makes type systems more modular. the proofs of various properties aren't too tightly intertwined. in the case of ifix, i think there's no real avoiding it, however, so we might as well just go all out and make it a fully saturated non-binder

@effectfully I've read what you wrote about the denormalization problem. I'm not sure I'm convinced that it's a problem we need to solve. As I understand it, the problem is that a user of the Plutus Core library (i.e. someone programmatically generating Plutus Core) might feed in a normalized n-ary pattern functor and get out a type that is non-normalized.
But as someone generating Plutus Core programmatically, it doesn't seem that important to me to preserve normalization in this way. For one thing, tracking whether or not I'm preserving normalization is a pain (to be in this situation you are assuming I already know and care whether the pattern functor is normalized!). For another thing, if I actually do care about the types in my program being normalized, I can just normalize them afterwards!
So I think either:

The user cares about types being normalized. They can just normalize them afterwards (and will probably want to do so regardless, to be sure).
The user does not care about types being normalized. No problem.

So my vote is that we shouldn't care and we should just do the "semantic" packing.

@michaelpj, I completely agree with everything.
Some thoughts:

it's still good to have this "denormalization" thing being documented (including the motivation of why we're not preserving normalization when we can), because it's kind of a "surprise"
it's still good to be able not to add additional redexes and not to remove supplied ones, because I actually debug things by looking at Plutus Core from time to time

So if people think it's fine, I'm going to keep the docs and the "syntactic" packing, but only use "semantic" packing for production.

Yes, we should certainly document it.
I don't see why you want to keep the "syntactic" packing, though? It's not clear to me that it's easier to debug. Plus then we have two implementations that have to both be correct.

It's not clear to me that it's easier to debug.

Right.

Plus then we have two implementations that have to both be correct.

I want to check them against each other using property testing.
Well, I guess you're right, syntactic packing only increases maintenance burden, so I'll remove it (after a little bit more thinking just in case).

re: plutus/language-plutus-core/stdlib/Language/PlutusCore/StdLib/Type.hs
[Note: Denormalization]
I cannot read lines 218--225, despite the claim on the next line that it is "pretty readable (once you know how to read it)". Can you explain to me how to read it?
255-256: "Then the question is whether it's possible to preserve redexes in user-written types and not to
produce new ones while encoding the types." Why is this question of interest?

Here is another argument in favour of not making ifix a binder: often I separate definitions of pattern functors from definitions of actual data types. And sometimes pattern functors are computed rather than just written down. For example, in one of our docs we have
treeForestF =
    \(a :: *) (rec :: (* -> * -> *) -> *) (tag :: * -> * -> *) -> all (r :: *).
        tag ((a -> rec forestTag -> r) -> r)                  -- `case` over `tree`
            (r -> (rec treeTag -> rec forestTag -> r) -> r))  -- `case` over `forest`

Here treeForestF, being applied to an a, computes to a pattern functor. And it's not just for convenience, it's really a reusable and meaningful part of the encoding of the tree/forest family, because later we have
tree   = \(a :: *) -> ifix (treeForestF a) treeTag
forest = \(a :: *) -> ifix (treeForestF a) forestTag

How to do the same in the ifix-as-a-binder setting?

Wouldn't it just be the same, only now we pass the two arguments to the saturated form of ifix rather than use function application?

@wadler,

I cannot read lines 218--225, despite the claim on the next line that it is "pretty readable (once you know how to read it)". Can you explain to me how to read it?

Added a couple of more notes. Inlining them here:
{- Note [Generic fix]
Now that we know how to pack n-ary functors into 1-ary ones
(see [Packing n-ary pattern functors semantically]), only a few tiny steps remain to get the generic

    fix :: (k -> k) -> k

from just

    ifix :: ((i -> *) -> i -> *) -> i -> *

Having @pat :: k -> k@ we can pack it as

    toPat1 withSpine patN :: ((k -> *) -> *) -> (k -> *) -> *

(where 'withSpine' is constructed automatically from 'k' on the Haskell side) and we can apply
'ifix' to this 1-ary pattern functor and get

    ifix (toPat1 withSpine patN) :: (k -> *) -> *

It only remains to turn something of kind @(k -> *) -> *@ into something of kind @*@, i.e. to define
a type function of kind @((k -> *) -> *) -><EMAIL_ADDRESS>But we already have such a function: 'withSpine',
so the final encoding is

    fix = \(patN :: k -> k) -> withSpine (ifix (toPat1 withSpine patN))

The meaning of 'withSpine' here is the same as we've seen before: we use it to pack @n@ type
arguments as a single CPS-encoded spine and pass it to some function.

Summarizing, 'fix' receives an n-ary pattern functor and @n@ type arguments, the pattern functor
gets packed as a 1-ary one, the type arguments get packed into a single CPS-encoded spine and
'ifix' gets applied to the 1-ary pattern functor and the spine.

Read next: Note [Encoded InterList].
-}

{- Note [Encoded InterList]
Let's now look at an example.

Recall that the pattern functor of 'interlist' is

    interlistF =
        \(interlist :: * -> * -> *) (a :: *) (b :: *) ->
            all (r :: *). r -> (a -> b -> interlist b a -> r) -> r

We can apply generic 'fix' (see Note [Generic fix]) to this pattern functor directly:

    fix interlistF :: * -> *

which after eta-expansion and some reductions becomes

    \(a :: *) -> withSpine (ifix (toPat1 withSpine interlistF)) a

(as per Note [Generic fix]) which after some more reductions becomes

    -- Two type arguments that the data type receives and the 'ifix' primitive.
    \(a :: *) (b -> *) -> ifix
        -- The variable responsible for recursion and the variable representing a CPS-encoded spine
        -- of two elements. Note that the kind of the argument that the variable responsible for
        -- recursion receives is the same as the kind of 'spine', i.e. we always instantiate
        -- recursion at some spine.
        (\(rec :: ((* -> * -> *) -> *) -> *) -> \(spine :: (* -> * -> *) -> *) ->
            -- 'spine' unpacks a CPS-encoded spine and passes all its elements to a continuation.
            spine
              -- The 'interlistF' pattern functor given above applied to a function that receives
              -- two type arguments, packs them as a CPS-encoded spine and passes the spine to the
              -- variable responsible for recursion.
              (interlistF (\(a :: *) (b :: *) -> rec (\(dat :: * -> * -> *) -> dat a b)))
        )
        -- The two type arguments packed as a CPS-encoded spine.
        (\(dat :: * -> * -> *) -> dat a b)

We've elaborated the encoding on example, but there is a problem to consider here.
Read next: Note [Denormalization]
-}

255-256: "Then the question is whether it's possible to preserve redexes in user-written types and not to
produce new ones while encoding the types." Why is this question of interest?

Changed this part of the note to
In Plutus Core we have two
modes for type checking:

1. off-chain, type normalization is allowed
2. on-chain, type normalization is not allowed and types must already be normalized

Thus, we do care about whether types are normalized or not. In the compilation pipeline we just
explicitly normalize types whenever normalized types are required, but since this module belongs
to a library it better be general and not rely on particular details of downstream code.

Preserving properties of user-written code is generally a good idea while transforming it,
so we also do not want to remove redexes from user-written code and thus we can't just normalize
everything in sight to overcome the denormalization problem.

Then the question is whether it's possible to preserve redexes in user-written types and not to
produce new ones while encoding the types. And the answer is "yes, but it's too costly".

Wouldn't it just be the same, only now we pass the two arguments to the saturated form of ifix rather than use function application?

There are two dichotomies here:

whether ifix is saturated or not
whether ifix is a binder or not

What @psygnisfive  proposes is (True, True).
What I propose and what is implemented is (True, False).
What @jmchapman implemented is (False, False) (I think, it's me who proposed to use this in the formalization. Sorry, James!)
So in the implementation ifix is already saturated and we do not use function application there. My comment is about the other dichotomy.

Thanks for the clarification. But I still don't understand the issue. What becomes hard when ifix is a binder that is not hard when ifix is not a binder?

PS: I'm happy with the other responses!

Thanks for the clarification. But I still don't understand the issue. What becomes hard when ifix is a binder that is not hard when ifix is not a binder?

Now that I think about it, it's not hard, just wordy. Taking my example from the above
treeForestF =
    \(a :: *) (rec :: (* -> * -> *) -> *) (tag :: * -> * -> *) -> all (r :: *).
        tag ((a -> rec forestTag -> r) -> r)
            (r -> (rec treeTag -> rec forestTag -> r) -> r))

tree   = \(a :: *) -> ifix (treeForestF a) treeTag
forest = \(a :: *) -> ifix (treeForestF a) forestTag

in the ifix-as-a-binder setting we can leave treeForestF the same and define tree and forest as
tree   = \(a :: *) -> ifix (rec :: (* -> * -> *) -> *) (tag :: * -> * -> *) (treeForestF a rec tag) treeTag
forest = \(a :: *) -> ifix (rec :: (* -> * -> *) -> *) (tag :: * -> * -> *) (treeForestF a rec tag) forestTag

So in order to be able to compute a pattern functor, you have to bind two type variables using ifix and then pass them to the function returning a pattern functor. Just some inconvenience, but not a big deal. Not as important as my objections from the second message in this thread.

It looks like we're pretty settled on ifix.


chore: add useProductPrice unit tests

Why:
Closes: #ID
Describe your changes
Checklist before requesting a review

[ ] My code follows the code style of this project.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[ ] I've updated docs/changelog/changelog_en.md and docs/changelog/changelog_de.md. If I'm a non-German speaker, I've still updated the file with the English version as a placeholder.

@maxiroellplenty test: is a valid prefix.


Overlay breaking

Searching and loading the line fails to show up on overlay. Breaks overlay. Other shabads which work require refreshing the overlay page to work.
Potentially related to #303 or maybe has something to do with setting mainline and nextline to same id:
{"level":30,"time":1573940923220,"pid":6216,"hostname":"BhajneetX1Carbon","msg":"Setting Line ID to VVYJ","v":1}
{"level":30,"time":1573940923219,"pid":6216,"hostname":"BhajneetX1Carbon","msg":"Setting Shabad ID to 83K","v":1}
{"level":30,"time":1573940923221,"pid":6216,"hostname":"BhajneetX1Carbon","msg":"Setting mainLineId to VVYJ","v":1}
{"level":30,"time":1573940923222,"pid":6216,"hostname":"BhajneetX1Carbon","msg":"Setting nextLineId to VVYJ","v":1}

still broke?


add unarchiver example

Hi Matt, thanks for the great work!
I noticed there's no Unarchive function in v4 yet, and the documentation is incomplete and hard to follow, so I wrote a small program that invokes your extractor API to automatically identify the format and unarchive tarballs.
Please check the code and point out where I need to improve. It can serve as an example for users who seek a quick solution from this project.

Thanks for this; I like the idea of posting an example instead of prescribing one way to do it in the code itself.
I have a few suggestions on how to tweak this but I will have to come back and write those up later.
I suppose this might resolve #311.

Does this example take into account #391?

Does this example take into account #391?

In fact, my code doesn't retain the file/dir permissions. So I guess it will "solve" #391.

Gotcha. This is a good start, but I think file mode bits (permissions, etc) are important, so we might need to include those in a proper example.

I have updated the code with necessary changes to address the directory permission issue, and I also added code to handle symlinks.

Thank you! However, I don't think we should be providing an actual program, just an example in the docs.
I'm actually about the archive this repository, but we can continue this discussion over in mholt/archives if you would like.


Error

Service deploy but bot not working

Even after changing different servers

Hello, did you follow the Configure the Webhook instructions?


Merge of PRs, including some fixes to bugs that were introduced by them

I'll be testing out this PR and then create release/2.1.0 in preparation for new version release.
UPDATE: One of the PRs is not totally backwards compatible and I completely forgot (despite author have warned me). Instead of releasing a minor version (2.1.0) with the other PRs, I'm leaning towards merging #101 into the party and releasing 3.0.0 instead.
PRs included:

https://github.com/jseidl/hass-magic_areas/pull/118
https://github.com/jseidl/hass-magic_areas/pull/98
https://github.com/jseidl/hass-magic_areas/pull/99
https://github.com/jseidl/hass-magic_areas/pull/92

Huge thanks to @sti0 @KTibow and @caphm, you folks rock!

Still testing on my live setup and fixing whatever shows up until everything is working as expected.
Cover groups seems to be working but I haven't used it for control yet, will do in the morning.
Still investigating if the schema change on config flow will affect has_feature(). Will revise everything on the weekend, merge this PR and start working towards merging #101 into the develop branch.

Confirmed the #118 PR broke all integrations ability to pull config. Will be working on fixing it on this pr.

Looks like features are pulling the configuration from the new schema now. Will run on my main setup for a few days for testing.

Renamed the branch. PR is now #125


Trumbowyg plugins..

that i need help how i can load plugins ? for example upload ?

You just need to import trumbowyg/dist/plugins/upload/trumbowyg.upload.js as a dependency in the resources/assets/admin/js/admin.js file, and it should works as expected.
I close because it's not an issue 🙂


dep upload / dep download alias for SCP

Instead of running rsync/scp, it would be useful to create a dep upload to upload and dep download to download files the the application folder (eg dep download shared/backup.sql /some /folder and dep upload myfile.txt current/public).
That allows to upload arbitrary files without having to create custom tasks.

Rough implementation:
desc('Download file');
task('download', function () {

    $options = [
        'release' => '{{release_path}}/',
        'current' => '{{deploy_path}}/current/',
        'shared' => '{{deploy_path}}/shared/',
        'home' => '~/',
        'root' => '',
    ];
    $prefix = askChoice('Download from which folder?', $options, 'current');
    $prefix = $options[$prefix];

    $source = ask('What is the source filename?');
    $destination = ask('What is the local destination?') ?: $source;

    if (!$source) {
        throw new \RuntimeException('Source is required!');
    }
    if(askConfirmation("Download {$prefix}{$source} -> $destination ?")) {
        download("{$prefix}{$source}", $destination);
        echo "Done!";
    }

});

desc('Upload file');
task('upload', function () {

    $source = ask('What is the local filename?');

    $options = [
        'release' => '{{release_path}}/',
        'current' => '{{deploy_path}}/current/',
        'shared' => '{{deploy_path}}/shared/',
        'home' => '~/',
        'root' => '',
    ];
    $prefix = askChoice('Upload to which folder?', $options, 'current');
    $prefix = $options[$prefix];

    $destination = ask('What is the remote filename?') ?: $source;

    if (!$source) {
        throw new \RuntimeException('Source is required!');
    }
    if(askConfirmation("Upload {$source} -> {$prefix}$destination ?")) {
        upload($source, "{$prefix}$destination");
        echo "Done!";
    }

});```

Closing this issue as inactive. If you still have this problem, please open a new issue.

Not sure why this feature request was closed as inactive when it required input from the repository owner, and the user had done most of the work to build it out themselves.

Those tasks are bought and implemented in this issue issue I’m not the standard of deployer with


[BUG] Failed Model Registration: ModelUploader does not add model_content_size_in_bytes field to model config file

What is the bug?
ModelUploader does not add model_content_size_in_bytes field to model config when register_model is invoked. This leads to the following error in deployment stage: java.lang.NullPointerException: Cannot invoke "java.lang.Long.longValue()" because "size" is null.
How can one reproduce the bug?
Steps to reproduce the behavior:

Use SentenceTransformer to initiate huggingface model and zip it
Register the model
Look at the log shown when run ./opensearch  or try generate embedding using that model id

What is the expected behavior?
No error when registering the model using the local file
What is your host/environment?

OS: Linux
Version: 2

Do you have any screenshots?

Do you have any additional context?
N/A

PR merged. Closing the issue.


Android - autoComplete="tel" doesn't work with Google Accounts

Description
We want users to register by autofilling their phone number.
We have the following code snippet for that;
<TextInput   placeholder="Mobile number"   autoComplete="tel"   textContentType="telephoneNumber"   importantForAutofill="yes" />
It works seamlessly in the emulator. When we tried it on a physical device, even though our phone number is linked to our Google account, we didn't see any suggestions.
Later, we discovered that the app was using the number from the SIM card menu in the device settings.
Our expectation is that this input should be filled with our number on Google account. Because there are also settings related to autofill under the Google menu, and these settings are turned on. Are we expecting something wrong?
Steps to reproduce

Insert the input.
Compile the app for release.
Open the app on your physical device, you won't see any suggestions.
Go to device settings, fill in the number under the SIM card menu.
When you reopen the app, the number you entered there will be suggested.

React Native Version
0.71.10
Affected Platforms
Runtime - Android
Output of npx react-native info
System:
    OS: macOS 13.6
    CPU: (8) arm64 Apple M1
    Memory: 104.81 MB / 8.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 18.18.2 - ~/.nvm/versions/node/v18.18.2/bin/node
    Yarn: 1.22.21 - ~/.nvm/versions/node/v18.18.2/bin/yarn
    npm: 9.8.1 - ~/.nvm/versions/node/v18.18.2/bin/npm
    Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
  Managers:
    CocoaPods: 1.13.0 - /opt/homebrew/bin/pod
  SDKs:
    iOS SDK:
      Platforms: DriverKit 22.2, iOS 16.2, macOS 13.1, tvOS 16.1, watchOS 9.1
    Android SDK: Not Found
  IDEs:
    Android Studio: 2022.3 AI-223.8836.35.2231.10811636
    Xcode: 14.2/14C18 - /usr/bin/xcodebuild
  Languages:
    Java: <IP_ADDRESS> - /usr/bin/javac
  npmPackages:
    @react-native-community/cli: Not Found
    react: 18.2.0 => 18.2.0
    react-native: 0.71.10 => 0.71.10
    react-native-macos: Not Found
  npmGlobalPackages:
    *react-native*: Not Found

Stacktrace or Logs
No crash

Reproducer
https://github.com/melihcaliskanhnc
Screenshots and Videos

This is most likely unrelated to React Native. We just invoke view.setAutofillHints(hints) with the hint you provide (tel in this case). We don't do any other manipulation.
https://github.com/facebook/react-native/blob/f322dc7a84eb72370910f6933d0a4fa7780f49bc/packages/react-native/ReactAndroid/src/main/java/com/facebook/react/views/textinput/ReactTextInputManager.java#L448
So if you try to create a native apps without React Native will behave the same.
You can read more about it here: https://developer.android.com/guide/topics/text/autofill-optimize


PhpInputStream and '0' chunks

PhpInputStream::read doesn't add "0" chunks to $this->cache, because the first condition of the if evaluates to false.

This would need a minimal test: as-is, the issue is a bit cryptic

I think reproducible by below code.
<?php
use Zend\Diactoros\PhpInputStream;

$stream = new PhpInputStream('data://,a');
$stream->read(1); // read 'a'
$stream->read(1); // read 'EOF'
var_dump($stream->getContents()); // string(1) "a"

$stream = new PhpInputStream('data://,0');
$stream->read(1); // read '0' - not cached
$stream->read(1); // read 'EOF'
var_dump($stream->getContents()); // string(0) ""


Notifications | BCeID user not receiving in-app notification for Director Approval of PVR |2|

https://trello.com/c/uNSEUKqF/1094-notifications-bceid-user-not-receiving-in-app-notification-for-director-approval-of-pvr-2
Issue:
In the test env, a BCeID user does not receive an in-app notification when the Director approves a PVR transaction despite having the receive in-app notification setting selected for "Director Approval"
Note: The BCeID user also does not receive a PVR notification when the transaction is completed using the Historical Data Entry tool

 Notifications | BCeID user not receiving in-app notification for Director Approval of PVR |2|


AMD module

Please, add support for AMD modules.

Yes, this would be very appreciated!

Next release is built with webpack and will expose darkroom as a valid npm or amd module.
See #79.


Stack unwinding can now unwind beyond (optionally nested) exception handlers for selected architectures.

This is a replacement for the proof of concept #1495.
Stack unwinding can now unwind beyond (optionally nested) exception handlers (# ).

ARMv6-M: Report Exception / Fault description, and Unwind the registers and next frames.
ARMv7-M: Also decodes details about HardFault, UsageFault, BusFault, and MemManageFault.
ARMv7-A, Armv8-M, Armv8-A, RISC-V: Not implemented - requires architecture specific implementations, but the module contains default trait implementations that allow the existing functionality for unwind to work as before, i.e. unwind all frames to the first signal handler encountered.

See the white arrows below for new information available:

Note for reviewer: I don't understand why Run CI / Check docs are failing ... the error message doesn't make sense. Can you point me in the right direction please?

Note for reviewer: I don't understand why Run CI / Check docs is failing ... the error message doesn't make sense. Can you point me in the right direction please?

Well, RegisterRole is private and you are linking to it in public docs :) make RegisterRole public and you're good :)

Note for reviewer: I don't understand why Run CI / Check docs is failing ... the error message doesn't make sense. Can you point me in the right direction please?

Well, RegisterRole is private and you are linking to it in public docs :) make RegisterRole public and you're good :)

@Yatekii But it is a pub enum ...???

@Yatekii But it is a pub enum ...???

That does not mean it's parent module is too =)

For sure 1. should be the chosen solution. But you do not necessarily have to make the entire module public. It suffices if you publicly re-export the private item :)


receive and mark error responses for flux code within VSCode

The LSP server will currently make a diagnostics notification to the client  on save and on change events. VSCode should only need to support handling those notifications.


Trying to get in touch regarding a security issue

Hey there!
I belong to an open source security research community, and a member (@ranjit-git) has found an issue, but doesn’t know the best way to disclose it.
If not a hassle, might you kindly add a SECURITY.md file with an email, or another contact method? GitHub recommends this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.
Thank you for your consideration, and I look forward to hearing from you!
(cc @huntr-helper)

Email me at “ npm view yadeep maintainers.email”.  Thanks!
PR welcome for adding a SECURITY.md

@DABH - thanks for your follow-up!
Just for reference, the report can be found here:
https://huntr.dev/bounties/de65cf7d-675a-4515-b731-81fda23cd5e7/
It is private and only accessible to maintainers with repository write permissions. Let me know if you have any further questions!

Also created PR:
https://github.com/winstonjs/winston/pull/2024

@wbt
I just made a comment in the report

I opened a separate issue for tracking this feature request.  I’m leaving this issue closed though, along the lines of @wbt ‘s thinking.  I phrased the new issue in a way that I think it could be addressed in Winston (the file transport code), if anyone wants to take on the work.

@ranjit-git Thanks.  Same comment as above - unless other maintainer object, I think having the discussion about what to set a default value to would be OK in public.
To co-maintainers: if you want details but don't want to give personal info to the hosting platform there, you can email me or just wait for the public post.

Can you plz change the report status in Huntr accordingly?

OK, done


[CIT-564] Improve error reporting on software update

Proposed changes

Every software operation is logged in a file that reports the status/stdout/stderr all the plugin actions triggered by the agent
If one of this action fails the error is reported to the cloud admin user with a link to the file
The files are stored into /var/log/tedge/agent
Only the latest versions of these logs are kept (the 5 latest software updates, the latest software list)

Types of changes
What types of changes does your code introduce to thin-edge.io?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
[ ] Documentation Update (if none of the other choices apply)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[x] I have read the CONTRIBUTING doc
[x] I have signed the CLA (in all commits with git commit -s)
[x] I ran cargo fmt as mentioned in CODING_GUIDELINES
[ ] I used cargo clippy as mentioned in CODING_GUIDELINES
[x] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

We also need to make sure there are no system test relying on any of the old behaviour.

Codecov Report

Merging #443 (9f7ba56) into main (d1e57dd) will increase coverage by 0.56%.
The diff coverage is 75.46%.

@@            Coverage Diff             @@
##             main     #443      +/-   ##
==========================================
+ Coverage   62.45%   63.01%   +0.56%
==========================================
  Files         107      110       +3
  Lines        6749     6930     +181
==========================================
+ Hits         4215     4367     +152
- Misses       2534     2563      +29

Impacted Files
Coverage Δ

sm/tedge_agent/src/agent.rs
0.00% <0.00%> (ø)

sm/tedge_agent/src/main.rs
0.00% <0.00%> (ø)

sm/plugin_sm/src/plugin_manager.rs
43.68% <12.50%> (-6.92%)
:arrow_down:

sm/plugin_sm/src/plugin.rs
52.63% <52.17%> (-2.07%)
:arrow_down:

sm/plugin_sm/src/log_file.rs
75.00% <75.00%> (ø)

sm/tedge_agent/src/operation_logs.rs
94.79% <94.79%> (ø)

sm/plugin_sm/src/logged_command.rs
96.05% <96.05%> (ø)

sm/json_sm/src/lib.rs
98.52% <100.00%> (+<0.01%)
:arrow_up:

sm/json_sm/src/messages.rs
85.77% <100.00%> (+1.86%)
:arrow_up:

sm/plugin_sm/tests/plugin.rs
98.93% <100.00%> (+0.09%)
:arrow_up:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1e57dd...9f7ba56. Read the comment docs.


Update PTB download and add note on VRAM

PTB is no longer present in pytorch. Replaced with a repository that had identical files.
Added a note on needed VRAM.
(great result btw - still SOTA AFAIK)

Thank you for sending interesting results but I'm sorry I'm not sure the reason of these results because I also think your code and expectation seem right.
I have only one reason for generated example.
In small temperature such as 0.001, generated examples contain much special token .
I think the reason is that its frequency in training data is high.
In other words, the model generated a sequence (deeply) based on a frequency in the small temperature case.

Well, in my mine, temperature=1 is expected to be the most accurate as it corresponds to the optimized distribution. Runs for that value were also included above but the tokens seem rather disconnected. In the extreme of temperature->0, ofc only the most likely token is generated (which can make sense for unknown/eos/and). For that reason, I tested a range of values for a sweet spot and included the two extremes.
I wonder if I have a conceptual misunderstanding with perplexity implying language structure.


feature request: native ES module support

Is there a way to assist on this? Would also love to get ES module support!
Currently blocking / complicating our efforts to introduce a modern module bundler (like esbuild or Vite).


fix: Update python version in terraform to keep consistent

Upgrade python to v3.12 in terraform to be the same version in the GitHub pipeline workflow yamls.
We got a deployment error when merged pr #1517 https://github.com/bcgov/nr-forests-access-management/actions/runs/10275482241, the Api Gateway is not reachable. It's caused by a python version mismatch issue. After manually set the lambda python version to 3.12, the Api Gateway works again.

Hi Ian, I wonder if you know what this "version: 3.8" means here? It's in all three docker compose files @ianliuwk1019

Hi Ian, I wonder if you know what this "version: 3.8" means here? It's in all three docker compose files @ianliuwk1019

Hm... I don't think I know the reason. They were probably inherit from (copy-paste) the first docker-compose file, but I don't know why it was 3.8. I think we could remove this "version" to avoid confusion, the docker docs says it is not being used.


Feature: update helm chart by exposing the metrics port to the internal network

PR Checklist
PR Structure

[x] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
[x] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
[x] This PR's title starts with name of package that is most changed in the PR, ex.
paymentservice.stellar, or all or doc if the changes are broad or impact many
packages.

Thoroughness

[x] This PR adds tests for the most critical parts of the new functionality or fixes.

What
[TODO: Short statement about what is changing.]
Why
[TODO: Why this change is being made. Include any context required to understand the why.]
Known limitations
[TODO or N/A]

Reference Server Preview is available here:https://anchor-ref-pr610.previews.kube001.services.stellar-ops.com/SEP Server Preview is available here:https://anchor-sep-pr610.previews.kube001.services.stellar-ops.com/


[Question] Send a local response without closing the connection

Title: Send a local response without closing the connection
Description:
I have written a custom http filter for web socket connections to rate limit based on the number of frames sent through a connection. The filter basically publishes metadata needed for rate limiting and analytics through a gRPC bidi stream. I have a requirement to send a local reply from the filter when the connection is rate limited.
eg: Message throttled out due to subscription level throttling limit reached.
But since this is a websocket connection, I don't want to close the connection. Just send local responses without closing the connection. Envoy ext_auth and ratelimit filters use sendLocalReply() but it closes the connection. Is my requirement possible with the current envoy implementation in filter level without modifying envoy source ? Thanks

cc. @alyssawilk @snowp

I think I need a bit more information about what you're trying to do here.
If you're proxying websocket, you're forwarding headers and streaming body upstrema, then proxying headers and streaming websocket data from upstream.
sendLocalReply sends headers and body and end stream downstream.  There's not much point keeping the connection open once you've sent end stream as the upstream can't send more data.  Also having sendLocalReply send headers isn't going to play well with upstream sending response headers.    Are you just wanting to send some websocket payload downstream?   That might be doable but there will be some complications.

@alyssawilk Yeah, keeping the connection open after sendLocalReply() doesn't make any sense that way. So how do I send some websocket payload downstream ? By calling encodeData() from a filter ? If it's doable with my current knowledge about envoy (implemented some http filters with grpc async client), would like to give it a try. Thanks


let the user specify figure dpi when plotting

when using a DataArray plot function, it is already possible to specify the figure size and aspect ratio.
I think it would make sense to also be able to specify the dpi, e.g. x.plot(dpi=109). It would save one call to plt.figure().

Does pandas support setting DPI like this?

Apparently not.
https://stackoverflow.com/questions/48227792/how-to-change-dpi-of-my-pandas-dataframe-plot


Chapter 7

Hi Adrian,
Some problems in the chapter.
Please close the issue once they are solved and tell me what's been done.
Dataframes versions and casting
Using
### Dataframes version:	0.10.1
path_to_ratings  = "../chapter7_data/ratings.dat.txt"
ratings = CSV.read(path_to_ratings, header = false);

UserID | ProfileID | Rating
-- | -- | --
Nullable{Int64}(1) | Nullable{Int64}(133) | Nullable{Int64}(8)
Nullable{Int64}(1) | Nullable{Int64}(720) | Nullable{Int64}(6)

If I cast it manually then it works
ratings[:UserID] = Array{Int}(ratings[:UserID])
ratings[:ProfileID] = Array{Int}(ratings[:ProfileID])
ratings[:Rating] = Array{Int}(ratings[:Rating])
head(ratings, 3)

UserID | ProfileID | Rating
-- | -- | --
1 | 133 | 8
1 | 720 | 6
1 | 971 | 10

results differ
Training and train-test partitions need seeds to make results reproducible
0-element Array{Int64,1}

intersect(test_profile_ids, recommendations_to_profiles(recommend(recommender, 100, 100, [1:profile_counter...])))
0-element Array{Int64,1}

Hi David,
It should be DataFrames 0.11 - we can pretty much give up on DataFrames 0.10, it's been superseded.
I need to go over the initial chapters and update everything for DataFrames 0.11. DataFrames 0.11 no longer uses Nullables but Missings.

I have updated the DataFrames package but it updates to 0.10.1.
I have also removed it and installed it again and it's not working.
Probably it's worth waiting a bit to get this "automatically managed" by Pkg.add.
I guess Julia 0.7 will have this working properly.
julia> Pkg.rm("DataFrames")
INFO: Removing DataFrames (unregistered)

julia> Pkg.add("DataFrames")
INFO: Installing DataFrames v0.10.1
INFO: Building SpecialFunctions
INFO: Package database updated

Hi @davidbp
It seems @essenciary has made the changes and updated in the code files.
You may check and give an update. You may reopen in case any problem.
Closed.


Trip planning refactors (multi trip prep)

Overview
This PR includes the following proposed change(s):

{List all the changes, if possible add the linked issue/ticket #}

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Refactoring / Documentation
[ ] Version change

if your change is a breaking change, please add breaking change label to this PR
How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Screenshots
Please add any relevant UI screenshots if applicable.
Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation
[ ] New and existing unit tests pass locally with my changes

Closed the last one due to git getting confused between local and remote, and not wanting to do git surgery.


Containerisation over gym-http-api

#34 has not yet been merged, but represents the most-recent work on the environment. This pull request looks at testing the environment from within a docker container and using the gym-http-api setup.
Implements #38.

The steps to reproduce the test are:
Build and deploy the environment container:
docker-compose build
docker-compose up

Build and submit a random-agent container:
cd random_agent_submission
make build
pip install -r requirements.txt
./test_container.py

We'll want to be able to support multiple challenges. In terms of docker networking, let's rename the environment container to nztc:5000 instead of environment:5000.

I've tested a submission using a random agent and the round trip is working:

Tracking down the problem of the Error: None bug. This is to do with the way that the evalai command line tool fetches the credentials in order to be able to push container images to AWS.
Older versions of the evalai platform expect a "Token" attribute in the incoming request, for example you can manually fetch the AWS credentials using:
 http GET http://<IP_ADDRESS>:8000/api/challenges/phases/12/participant_team/aws/credentials/ Authorization:'Token XXXXXXXX'

The new versions of the platform expect a variant where "Token" is replaced with "Bearer", so they work with:
 http GET http://<IP_ADDRESS>:8000/api/challenges/phases/12/participant_team/aws/credentials/ Authorization:'Bearer XXXXXXXX'


Upgrade com.puppycrawl.tools:checkstyle 10.9.3 -> 10.10.0

This PR contains the following updates:

Package
Type
Update
Change

com.puppycrawl.tools:checkstyle (source)
build
minor
10.9.3 -> 10.10.0

[ ] If you want to rebase/retry this PR, check this box

Suggested commit message:
Upgrade Checkstyle 10.9.3 -> 10.10.0

See:
- https://checkstyle.sourceforge.io/releasenotes.html
- https://github.com/checkstyle/checkstyle/releases/tag/checkstyle-10.10.0
- https://github.com/checkstyle/checkstyle/compare/checkstyle-10.9.3...checkstyle-10.10.0


Apply go fmt

Which problem is this PR solving?

Short description of the changes

I found that ALL_SRC does not include some folders in the Makefile, which results in the format of some files that cannot be detected by make fmt. Not sure if this is acceptable.
Signed-off-by: Zhou Hao<EMAIL_ADDRESS>

We do not want to format or lint auto-generated files.

updated, PTAL.

This still includes mocks, which are auto-generated.

Done.


[ROCm] Update Tensorflow ROCm to build with hipclang compiler

This PR updates the Tensorflow master branch to build with ROCm hipclang compiler.

CC @whchung @deven-amd

/cc @nvining-work

gentle ping


Remove some crufty AppVeyor configuration

The crt-static feature should be long since stable as well as incremental
compilation.

Thanks @alexcrichton


implement optional pooling

This change adds:

the ability to connect either a single connection or a pool
the ability to acquire connections from a pool
termination of all connections when a pool is closed
configurable max and min connection counts

fixes https://github.com/edgedb/edgedb-go/issues/19
After

Before

I remove background go-routine from pool implementation.
The background goroutine only piped one channel into another and served to lock the "isClosed" pool state. In almost all real world scenarios this design would result in the background goroutine waking up, doing almost nothing, then sleeping again every time a connection was released. The overhead off waking a goroutine is nontrivial, so removing this background goroutine and using a read/write lock to guard the isClosed state improved the acquire/release performance by ~10x.


Replace float16 conversion intrinsic.

The float16 conversion intrinsic is not available on Sandy Bridge CPUs. Since the Sandy Bridge is part of the hardware compatibility, the float16 conversion needs to be replaced.

s = sra(x, 31) // Get sign.
s = sll(s, 15) // Sign bit for float-16
x = sll(x, 1) // strip sign bit.
x = max(x, ) // 1 higher than float16-denormal
x = min(x, ) // float16-infinite
x = sub(x, 112 * 2**24) // Convert bias, will not under or overflow.
x = srl(x, ) // truncate mantissa to float16
x = or(x, s) // Add sign.
x = packs_epi32(x, ) // pack 16 bits values, how does that work with saturation?
store(x, &mem) // Store as float16

If we movemask m we can check if one of the elements is denormal. And potentially take a slow path to calculate a new mantissa.
The biggest performance problem is shifting each mantissa. Since SSE/AVX does not have a shift by vector instruction (arrived in AVX2). To shift by vector we need to extract, shift and blend.


[ENG-2715] Remove Prereg landing page and campaign info

Purpose
Removes now unneeded prereg campaign code.
Changes

removes landing pages, campaign logic, tests, permissions for prereg campaign
redirects /prereg/ to the new workflow

QA Notes
Please make verification statements inspired by your code and what your code touches.

Verify that redirect works
Verify

What are the areas of risk?
Any concerns/considerations/questions that development raised?
Documentation
Tech task, no new docs.
Side Effects
None that I know of.
Ticket
https://openscience.atlassian.net/browse/ENG-2715

A few files I found with some lingering references:
Some tests still using the Prereg Challenge schema: here, here, here, here, here, here, and here. Some of these may require a separate ticket to clean up the test data that is populating registration responses to work with the OSF Prereg template.
This command should probably count "OSF Prereg" schemas if it's still relevant
This template still has references to %if campaign == "prereg":
Also, can you create a (low-priority?) follow-up ticket to remove permissions associated with the PreReg challenge?


USER + Specific tag

Is it possible to pull all the images from a "user" that have a specific tag?

I have tried the lowercase tag, but it still results 0 images. the code structure is same with above, any suggestion?


record 2 end-to-end test skeleton

Here is a list of feedback that we got during the tutorial:

common functionality tests across multiple pages e.g. login, branding, navbar
support of multiple schemas
doc for

running individual cases (for developer to run their tests)
running it as a suite on travisci

In the makefile, force all the dependencies to be there, otherwise, fail.
expand the test suites to run on multiple browsers
rename tables to a "test case" reusing the same generic test function.

@chiragsanghvi can you check those in the above list that have been done. Maybe we can generate different issues for the rest.


HDE-2744 added support of security params for searchById interaction

When we hit fhir server to fetch data of a resource using single id, in that case we are not considering security params. I have added code changes to consider security params while querying over db.

Discussed with Chris that Imran is working on https://icanbwell.atlassian.net/wiki/spaces/ENTARCH/pages/3909747022/Technical+Design+for+supporting+ids+from+multiple+systems+in+FHIR+server and it will fix this issue. So, closing this one.


Wrapped line unwrapped and "line exceeded maximum width"

I have an issue with a wrapped line getting unwrapped and then being too long:
lazy_static! {
    static ref INTERNER: Mutex<StringInterner<usize, RandomState>> =
        Mutex::new(StringInterner::new());
}

This static ref thing gets reformatted to one line, which is then too long.
I did rustup component add rustfmt-preview --toolchain nightly just today.
Maybe related to #2067 ?

rustfmt does not respect the width of indent inside lazy_static!:
https://github.com/rust-lang-nursery/rustfmt/blob/518ece227223cd6fc6d808c893b4404f450261f2/src/macros.rs#L1297
We should replace block_indent with block_left, and add a test in lazy_static.rs:
- let nested_shape = shape.block_indent(context.config.tab_spaces());
+ let nested_shape = shape.block_left(context.config.tab_spaces());

I would like to work at this one as my first issue.

@SvenKnabe great! https://github.com/rust-lang-nursery/rustfmt/issues/2610#issuecomment-379967321 gives instructions for how to fix this, let us know if you have any questions once you get stuck in.

@nrc the value of block_left will return an Option<Shape> where block_indent returns only a Shape. What would be the best approach to handle the Option inside of format_lazy_static?

@dsrees We often use ? to handle this kind of stuff in rustfmt. When we want to continue formatting even when there is no width left, we use .unwrap_or and retry in different way (e.g. use newline).
Anyhow, this issue is fixed via #2681, so I am going to close this. Thanks for your interests in contributing to rustfmt!


Bundler test suite hangs with simplecov

When adding simplecov to the test suite (through adding require 'simplecov' to bin/rspec and require 'simplecov' and SimpleCov.starttospec/spec_helper.rb), certain test cases hang indefinitely.
Hanging on fetching dependencies with a mirrored source:
$ bin/rspec spec/realworld/gemfile_source_header_spec.rb:4
Run options:
  include {:focused=>true, :locations=>{"./spec/realworld/gemfile_source_header_spec.rb"=>[4]}}
  exclude {:rubygems_master=>true, :rubygems=>"=< 2.5.0", :ruby=>"=< 2.2.3", :realworld=>true, :sudo=>true}

fetching dependencies with a mirrored source
/path/bundler/tmp/gems/base/gems/sinatra-1.2.7/lib/sinatra/base.rb:1258: warning: assigned but unused variable - e
.........Indefinite hanging here

Regular output (without simplecov) for reference:
$ bin/rspec spec/realworld/gemfile_source_header_spec.rb:4
Run options:
  include {:focused=>true, :locations=>{"./spec/realworld/gemfile_source_header_spec.rb"=>[4]}}
  exclude {:rubygems_master=>true, :rubygems=>"=< 2.5.0", :ruby=>"=< 2.2.3", :realworld=>true, :sudo=>true}

fetching dependencies with a mirrored source
/path/bundler/tmp/gems/base/gems/sinatra-1.2.7/lib/sinatra/base.rb:1258: warning: assigned but unused variable - e
[2015-12-29 01:13:30] INFO  WEBrick 1.3.1
[2015-12-29 01:13:30] INFO  ruby 2.2.3 (2015-08-18) [x86_64-darwin14]
[2015-12-29 01:13:30] INFO  WEBrick::HTTPServer#start: pid=7268 port=21453
[2015-12-29 01:13:31] INFO  going to shutdown ...
[2015-12-29 01:13:32] INFO  WEBrick::HTTPServer#start done.
  sets the 'X-Gemfile-Source' header and bundles successfully

Retried examples: 0

Finished in 4.19 seconds (files took 0.4488 seconds to load)
1 example, 0 failures

@RochesterinNYC our tests do horrible, horrible things... most tests shell out to run bundle in a subshell, some tests monkeypatch net/http, some tests literally spin up webrick in a thread and then run bundle in a subshell to make requests against it. I would mostly expect that simplecov has no real way to keep track of all of that and produce useful output :/
We have very few unit tests (the ones we have today are in spec/lib), but it would be awesome to have simplecov coverage for the unit tests! Is there a way to have travis runs include simplecov and display the coverage results?

Hm, yeah, after some experimentation, simplecov doesn't seem to play too well with Popen which we use for running a lot of the bundle commands in the test suite. Since the Simplecov.start is what starts tracking of code execution, then that bit would need to be put somewhere in the bundler app code (not really feasible) instead of the spec_helper if it were to track execution in all the subprocesses.
Similar issues @ colszowka/simplecov#346 and colszowka/simplecov#414
Using simplecov just for the unit tests might be viable. However, I don't see a spec/lib directory on the master branch? Is it on a different branch or something?

Ahh, sorry, I meant spec/bundler. Tests in that subdirectory map to the files inside lib/bundler.

Essentially resolved by #4224. Code Climate's test coverage engine is basically a wrapper around simplecov.


Wait to make sure xrootd starts up and dump log file in case of failure

The stashcache test set failed, but that was expected: https://travis-ci.com/github/opensciencegrid/osg-test/jobs/370485486. I'm going to merge.


Cursor arrow remains visible

Is there an option to make the cursor arrow disappear? Currently using 2.0.0-beta2 on macOS Monterey, with XBOX controller via bluetooth.

+1

+1


MD MSU MD+ audio in GSDK games

Hi, Is it possible and how can i implement the MD MSU MD+ audio in GSDK games?

Just include msu driver sources or binary in your code, execute driver code
to initialize cd hardware. Then you can use msu registers for playback
control
Пн, 1 нояб. 2021 г. в 21:50, PlaymanBR @.***>:

Hi, Is it possible and how can i implement the MD MSU MD+ audio in GSDK
games?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/krikzz/msu-md/issues/8, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGU2CTGCOVZ2OB4U24KBH33UJ3VRNANCNFSM5HEZIG2Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Hi, thanks for the reply.
We are players who dreamed of having arcade versions like SNK / KOF 9x
series, Marvel fighting and Capcom / Street Fighter Alpha games series for
genesis.
Please, could you know the work of my brazilian fighting game development
team for genesis?
I am responsible for the development of the audio area.
We are also developing tools that speed up sprite production and game
physics.
Here's our youtube channel for you to check it out.
https://www.youtube.com/playlist?list=PLO-5vY5uaWRt8VUzMrCrAj_BeCtK3PDya
We would be very happy if you could leave us comments and mention our
projects in your portfolio.
Thank you very much!! MSU has opened up new avenues for sega genesis
lovers.
Em sáb., 6 de nov. de 2021 às 05:39, krikzz @.***>
escreveu:

Just include msu driver sources or binary in your code, execute driver code
to initialize cd hardware. Then you can use msu registers for playback
control
Пн, 1 нояб. 2021 г. в 21:50, PlaymanBR @.***>:

Hi, Is it possible and how can i implement the MD MSU MD+ audio in GSDK
games?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/krikzz/msu-md/issues/8, or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AGU2CTGCOVZ2OB4U24KBH33UJ3VRNANCNFSM5HEZIG2Q
.
Triage notifications on the go with GitHub Mobile for iOS
<
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
<
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/krikzz/msu-md/issues/8#issuecomment-962419281, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AWJXHL4PG3S26TWKOVNUTBLUKTSUDANCNFSM5HEZIG2Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Binary env variables?

I would like to store binary strings like this: https://stackoverflow.com/a/46458707/1049833 - but it doesn't seem to be supported by shdotenv because "using without quotes is not allowed".
Is there any way around this?

There is no workaround at this time.
The reason shdotenv does not support $'...'  because it was not standardized in POSIX. But it was standardized in POSIX.1-2024, published on June 14, 2024. Also, many shells already support it, and dash, which is the biggest concern, will be supported in the next version.
$'...' implementation is acceptable. However, it will take some time as I do not have enough time at the moment. Pull Requests are wellcome.


Fixed bugs relating to generation of service file for gRPC

Purpose
Fixes service file generation for Empty type request. Also fixes service file generation for client streaming and bi directional streaming.
Fixes: Partly fixes #22778
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #22794 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #22794   +/-   ##
=======================================
  Coverage   14.59%   14.59%
=======================================
  Files          51       51
  Lines        1411     1411
  Branches      219      219
=======================================
  Hits          206      206
  Misses       1189     1189
  Partials       16       16

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9d6ceb...eb45ed8. Read the comment docs.


#18 fix: call now deprecated for parsers and performs as introduced parse_data method

@rapIsKal sorry for waiting so long, and thank you for contributing. But now there is merge conflict with your request. Please, could you fix it?

@rapIsKal sorry for waiting so long, and thank you for contributing. But now there is merge conflict with your request. Please, could you fix it?

Done

@rapIsKal thank you


Feature #18: Retry FileDownloads

Having trouble debugging some weird test issues, so I figure I'll make a PR and two-birds-one-stone the problem along with the code review.
Addresses #18 by manually triggering re-downloads of files in the event that FileDownload's streaming requests trigger one of several HTTP status codes in response. Additionally, this PR adds two new configuration values (yay, more config variables) for customizing this retry behavior: fileDownloadMaxRetries and fileDownloadRetryAfterMSec.
To test, modify the bulk-data-server repo to track the number of attempts made to a file download before succeeding. I'd push a branch but I don't have push permissions now so for the moment forgive the awkward git diff formatting:
@@ -292,7 +299,7 @@ class ExportManager
 const ABORT_CONTROLLERS = new Map();
+///////////////////////////////////
+// TEMP
+const TRIES_PER_FILE = new Map();
+// TEMP
+///////////////////////////////////
+

@@ -910,6 +917,25 @@ class ExportManager
         const shouldGzip     = (/\bgzip\b/.test(acceptEncoding));

+        ///////////////////////////////////
+        // TEMP
+        // Fail on first try
+        const numTries = TRIES_PER_FILE.get(req.params.file.split(".")[1]) ?? 0
+        console.log("NUM TRIES: ", numTries)
+        console.log("req.params.file.split('.')[1]: ", req.params.file.split(".")[1])
+        console.log(TRIES_PER_FILE)
+        TRIES_PER_FILE.set(req.params.file.split(".")[1], numTries + 1)
+        console.log(numTries <= 1)
+        console.log(numTries <= 1)
+        if (numTries <= 1) {
+            console.log('IN FAIL STATE')
+            return res.set("x-debugging-header", "someValue")
+                .status(500)
+                .end('')
+        }
+        // TEMP
+        ///////////////////////////////////
+
         // set the response headers
         res.set({
             "Content-Type": exportTypes[this.outputFormat].contentType,

Things to consider:

Why are tests failing? Specifically, the attachment-download related test seems to be failing with the addition of this new retry logic. I've been looking at the file for a bit too long and can't seem to puzzle out why that is, so I'm hoping to get another pair of eyes on that.
Do we want to combine fileDownloadRetryAfterMSec with retryAfterMSec?

Arguments against: the latter is supposed to only be used when bulk server's dont respond with a retry-after header, we don't want to make the responsibility of this variable ambiguous. Plus, the rate at which we retry for status updates might resonably differ from the rate at which we retry failed requests.
Arguments for: aren't we adding too many config variables? The meaning of this variable is pretty similar across both instances, and we probably don't have too many use cases where retrying wait time for status updates is too far removed

We could get rid of these specific variables altogether and instead expect users to configure the request option directly.

Argument for: Minimizes the # of config variables. Improves consistency between retry's handled manually by us and automatically by Got.
Argument against: It's a little awkward to expect users of the client to understand the config logic of Got. Additionally, Got doesn't have a concept of a fixed retryAfterMSec – it supports a calculateDelay function. Our config's are .js files, so users could define them inline, but it does feel cumbersome to have users specify a delay fn instead of a number.

Now that we know (hopefully) why the test was failing, we know that this is doable. Unfortunately, this is messy and would require a long discussion and some strategic decisions, so lets begin that by splitting the problem in multiple topics and try to agree on them. Here is the first one:
The client makes several types of requests:

Authorization requests to get (or renew) a token - Do we need to (does it make sense to) retry?
Bulk Data Kick-off - Do we need to (does it make sense to) retry?
Status polling - built-in retry logic following the bulk data spec. Not to be confused with what we are trying to do here!
Bulk Data download (as stream) - Use custom retry because got's retry does not work on streams
Attachments download - Use got's built-in retry capabilities

My intuition here is to:

Not retry the kick-off because it is the first request in the sequence and can be reviewed and retried manually if needed.
Same for the token request. Perhaps any error should be fatal in this case.

What do you think?

I agree with all your intuitions! To repeat:

(No Action) Authorization requests to get (or renew) a token - if there is an error it should probably be fatal. Any errors that would be related to token-expiration should be handled by the BulkDataClient.ts itself, since any this.request call invokes this.getAccessToken, which requests a new token if the current one is expired.
(No Action) Bulk Data Kick-off - I agree, this can be done manually by the user if there is an error.
(No Action) Status polling - the built-in retry logic following the bulk data spec is all we need here.
(Already Done 🤞) Bulk Data download (as stream) - Agreed and relieved that you too see the issues with got's stream-retry logic. Sad though – it would be nice to take advantage of if it worked.
(To-Fix) Attachments download - Use got's built-in retry capabilities, which aren't working as expected right now.

Now that we agree on which requests should actually retry, did you have next steps in mind?

OK, Yes, multiple next steps :) Next question:
We have 2 separate retries to handle. Let's call them got and custom for simplicity. The user should not have to know that these are different things. Ideally they should be configured in a single place and behave the same.

Option 1 - Use requests.retry for config that is also used by the custom retry method
Option 2 - Use custom config variables which are also used to override requests.retry variables

If we chose option 1, it means that we have to:

TODO: Override retry.limit to 1 for kick-off and auth requests
TODO: Explain in comments and docs that retry options are ignored for kick-off and auth requests, even though the requests config variable is supposed to affect every request.
Make sure the custom retry uses settings from requests.retry (which should be easily doable)

If we chose option 2, it means that we have to:

Duplicate settings that are otherwise provided in requests.retry
Make sure our custom settings override requests.retry wherever it makes sense

What do you think?
P.S.:

We probably don't need to provide custom calculateDelay (and fileDownloadRetryAfterMSec) and can use the one from got. Since we will only retry the file downloads, the single most useful variable that we need is retry.limit, and the built-in backoff from got should be good enough.
Given the previous comment, regardless of the chosen option we may have to add some code to override/ignore retry settings passed to kick-off and auth.

Personally, I lean towards Option 2 because it's more direct – if people want to retry file downloads, they can read through the options and README and find the fileDownloadMaxRetries option (or fileDownloadRetries, I'm open to renaming it).  If we were enabling wider retry logic across other types of requests, I'd lean towards Option 1 since we'd be affecting all requests, but we're really honing in on retrying in two limited scenarios. Pointing people to a generic retry option, only to explain that it only affects fileDownload requests, seems confusing… What do you think?
If we do go with Option 2, I'll make sure that the order of arguments are such that fileDownloadRetries overrides other retry options.
In either case, as far as not having access to calculateDelay, I'm sad but I agree it's not worth the effort right now. If users want to use a long delay, we can encourage them to set retries high; the built-in exponential backoff should help with that.

I agree, we can move the manual shouldRetry check outside of the wrapping if-statement. Good catch, I'll do that now.

I am not a big fan of either option and it is quite hard to decide

I could not agree more ^ 😆


question: Equivalent of StatusCode method

Hello,
is there a StatusCode method Equivalent? (return StatusCode(StatusCodes.Status304NotModified))
Just want to return in some cases the status 304.
Currently i am doing it with
HttpContext.Response.HttpContext.Items[0] = null;
HttpContext.Response.StatusCode = StatusCodes.Status304NotModified;

based on HttpResponseExtensions

you could write an extension method like this:
public static class EndpointExtensions
{
    public static Task SendStatusCode(this IEndpoint ep, int statusCode, CancellationToken ct = default)
    {
        ep.HttpContext.MarkResponseStart();
        ep.HttpContext.Response.StatusCode = statusCode;
        return ep.HttpContext.Response.StartAsync(ct);
    }
}

and use it like this:
public override async Task HandleAsync(Request r, CancellationToken ct)
{
    await this.SendStatusCode(403);
}


Interface to add valid TLDs in Hostname validator

I recently used this validator but found it did not support .car TLD. The company I work for recently purchased such a domain and we needed to allow valid email addresses with that TLD.
I needed to extend the existing validator and also extend the EmailAddress validator to be able to reach the TLD list and then could attach the new validator in Apigility and add a list of TLDs, which were pushed onto the array in Hostname validator.
It would have been quite useful to just have a public function in Hostname validator to add to the validTlds array. That is what I am proposing to do myself.

Originally posted by @peterkeatingie at https://github.com/zendframework/zend-validator/issues/117

There should be no need to extend the EmailAddress validator in this case; construct an instance of your extended Hostname validator and supply it to the EmailAddress instance through configuration option hostnameValidator or using the setHostnameValidator method.
Thanks to gTLDs the built-in TLD list falls out of sync with reality much more quickly than it used to when this validator was initially designed.  As far as I'm aware there is no formalized process for keeping this list up-to-date; if there isn't it would be worthwhile to formulate one.

Originally posted by @adamlundrigan at https://github.com/zendframework/zend-validator/issues/117#issuecomment-249675369

From what I see, this issue is connected with #27. In both, there is a discussion of where to keep the list of TLD.

This issue should no longer be relevant - The list of gTLDs is checked once per week and updated when necessary.


feat: Add y-axis label support to chart rendering

Add y-axis label support for improved chart readability
Overview:
This PR introduces y-axis label support to the charting library. Previously, users could view data values by hovering over items like the line, but there was no y-axis label to provide immediate context. This caused confusion, particularly when comparing multiple data points.
Key Changes:

Added y-axis label functionality for all relevant chart types.
Ensured that the labels dynamically adjust based on the scale and chart data.
Hovering over data points still displays values for detailed examination, but now with corresponding y-axis context.

Why this is important:
Charts with unlabeled axes can mislead users or make interpretation harder, especially when users aren't familiar with the specific dataset. This update makes charts more intuitive and user-friendly by ensuring all axes are clearly labeled.

Even the Anthropic marketing on the homepage shows they use an yAxis. Why would the quickstart not?


fix: use proper state to verify attestations

Motivation
Right now we always use head state to verify attestations
Description
We should only use head block state if:

head block slot and attestation slot are in the same fork
and head block state includes committees of target epoch

otherwise we should use head state dialed to target epoch. Notice that with maxSkippedSlot=32 we'll always use head state unless it's at epoch boundary
Also add some metrics to monitor slot distances in attestation
Closes #5498 Closes #4831
TODOs

[x] Unit test
[x] Deploy and monitor on a test mainnet node

Tested for some days on a mainnet node lodestar_gossip_attestation_use_head_block_state_dialed_to_target_epoch_count shows no value. Ready for review.

:tada: This PR is included in v1.9.0 :tada:


Arrow flight CI test failing on master

Describe the bug
Arrow flight CI test failing on master
Example:
https://github.com/apache/arrow-rs/actions/runs/11354119801/job/31580643040

running 3 tests
test tests::test_execute_update ... FAILED
test tests::test_select ... FAILED
test tests::test_auth ... FAILED

failures:

---- tests::test_execute_update stdout ----
testing uds client
Client finished!
=======
testing http client
Client finished!
=======
testing https client
thread 'tests::test_execute_update' panicked at arrow-flight/examples/flight_sql_server.rs:834:65:
called `Result::unwrap()` on an `Err` value: IpcError("Can't handshake status: Unknown, message: \"transport error\", details: [], metadata: MetadataMap { headers: {} }")
stack backtrace:

To Reproduce
RunC CI
Expected behavior
Should pass
Additional context

First started happening with 04fa369aa7b6c2c980909943b40abc264e873494 which is unrelated to flight
I suspect something changed in one of the underlying libraries

Starting to look at this one

@viirya perhaps you could take a look, as it appears you added these in https://github.com/apache/arrow-rs/pull/3388

I looked into these certificates a bit -- they appear to be copied from the tonic repo. I also tried my hand at regenerating them and failed to get the tests to pass. I will try my hand again either later today or tomorrow if no one beats me to it

Hmm, I forgot where I got these certificates...
I will try taking a look if @alamb doesn't fix it.

I think I've got this fixed with #6585, a review would be appreciated :)

label_issue.py automatically added labels {'arrow'} from #6572

label_issue.py automatically added labels {'arrow-flight'} from #6585


Mongoid 6 conflicts

The check should really run once. Maybe take a dependency on https://github.com/mongoid/mongoid-compatibility and fix using that? We also really need mongoid versions in the test matrix - that repo has a good example on how to do that.

@dblock added mongoid versions test using mongoid-compatibility. Please Let me know if I missed something.

Bump @saumyamehta17

@dblock
Hi Daniel
Sorry for delay in reply, I am bit occupied with my office work. Honestly, I am new to open source contribution and might be asking obvious (dumb) questions/doubts :)
What I understood:

I need to add for support mongoid 6
Also mongoid-compatibility can be added to check multiple versions of mongoid, which adds method to check versions. So, version_spec.rb does not required.
rspec.rb module Mongoid code not needed now.
I need to add all versions of mongoid in travis.yml which will do depenedency tests.

Please let me know if there is any gap.
Thank you for being patient.

No worries, I'm here to help!
You got it right. Just keep iterating and I'll try to make things clearer as we make progress. Hang in there.

Looks like there're real failures with Mongoid 5, trying to use classes that don't exist in that version.
I think instead of creating separate gemfiles you want something like what we do in https://github.com/mongoid/mongoid-compatibility.  Maybe copy from there?

I've added the Danger token to Travis-CI, so that should be working now.

Danger is now working, if you want to enable it as part of this PR it needs a CHANGELOG file. Usually I got back a little and try to dig up history from Rubygems, but you can also do this later. Otherwise this build is green, see my comments above. Lets finish it?


tell how each of us got started with VS Code

Use this

VS Code is the most popular IDE today. Why?

to tell how each of us got started using VS Code & why
(this can also be a natural place to talk a bit about the background of each of us, and to point out that we are not VS Code developers but rather "VS Code enthusiasts")

discussed, will do


bug: ManifestWork objects not getting removed automatically

Describe the bug
I am opening this for @namasl , who reported the following.

ManifestWorks are not getting removed automatically, and so the WEC Deployments stay active. This is possibly related to why editing the Placement on the WEC from "wantSingletonReportedState: true" to "false" will go from one to multiple WEC deployments, but doing the reverse from "false" to "true" does not remove any deployments (as in #1580).

Steps To Reproduce
See Nick.
Expected Behavior
The ManifestWork objects are implementation details that users should not have to see or manipulate.
Additional Context
No response

@MikeSpreitzer this seems a duplicate of https://github.com/kubestellar/kubestellar/issues/1596 , which has more context.

See #1596 instead of this.


KP-6009 Add a pipeline that runs unit tests on a self-hosted runner

Run a unit testing pipeline on pull requests and merges to the main branch.
The old Makefile was ditched, because unlike I assumed earlier, using virtualenv in the pipeline was not easy. As our runners are planned to be ephemeral, i.e. destroyed after a single run, it does not matter if we install the requirements system-wide. Thus the Makefile did not offer an easy way to keep local and pipeline testing methods in sync, so it was removed altogether.
A this moment, running a pipeline requires someone to launch a runner manually: hosting and setting up new runners automatically will be done in KP-5973.

Thanks Anni, looks good!


MetaAttack will OOM if PyTorch>1.9

Hi jin,
we find that MetaAttack will face the OOM problem if the PyTorch version is higher than 1.9. Can you fix this problem?
For example, we attack polblogs under 20% ptb rate in one tesla-v100 (32GB), and then the cuda will OOM. We retry this in another Linux server and it reports the same error, but if we turn the pytorch version into 1.8.2, everything would be ok.
Thanks

Hi!
I write an issue in #128, which seems to be helpful in avoiding OOM in MetaAttack.
I didn't check the solution thoroughly on different devices.
Please let me know if it could be also helpful to you.
Thanks.

I've tried your solution, and it works perfectly. Good job!